* flatbuffers Rust reflection: replace num with num-traits
num crate is a wrapper over num-traits and a few other crates, that
reexports the APIs from all of them. We only need num-traits.
Signed-off-by: Marcin Radomski <dextero@google.com>
* Rust reflection: drop dependency on stdint crate
We only use it to get intmax_t for deriving alignment, which is an alias
for `core::ffi::c_long` [1]. We can use that directly instead.
[1] https://docs.rs/stdint/1.0.0/stdint/type.intmax_t.html
Signed-off-by: Marcin Radomski <dextero@google.com>
* Rust reflection: drop dependency on escape_string crate
It's used to format a string used for debugging only, so we might as
well use the builtin Debug representation of a string.
Signed-off-by: Marcin Radomski <dextero@google.com>
* Rust codegen: add derives on generated bitflags
Otherwise it limits the use of structs generated for reflection.fbs
in Rust reflection API.
Signed-off-by: Marcin Radomski <dextero@google.com>
* Rust flatbuffers: update bitflags dependency to 2.8
Signed-off-by: Marcin Radomski <dextero@google.com>
* Rust codegen: use bitflags v2 API for converting from bits
from_bits_unchecked was replaced with safe from_bits_retain.
Signed-off-by: Marcin Radomski <dextero@google.com>
* Regenerate Rust code after idl change
Signed-off-by: Marcin Radomski <dextero@google.com>
* Regenerate reflection_generated.rs
With flatc --rust ../../../reflection/reflection.fbs
Signed-off-by: Marcin Radomski <dextero@google.com>
* ts/BUILD.bazel: add missing import
Found by Buildifire presubmit:
Function "sh_binary" is not global anymore and needs to be loaded from
"@rules_shell//shell:sh_binary.bzl".
Signed-off-by: Marcin Radomski <dextero@google.com>
* Update expected value in generated_code_debug_prints_correctly test
In bitflags v2, the debug string representation of enum values is
different than it was in v1:
Blue -> Color(Blue)
(empty) -> LongEnum(0x0)
This change adjusts the expected test value.
Signed-off-by: Marcin Radomski <dextero@google.com>
* Fix tests build on Swift 5.8
grpc-swift 1.4.1 depends on swift-nio-ssl 2.14.0+ [1]. swift-nio-ssl 2.29.1
published on 2025-01-30, introduced some code [2] that uses a "switch
expression syntax" supported since Swift 5.9 [3]. Attempts to compile it with
Swift 5.8 cause build errors.
swift-nio-ssl project doesn't seem to support Swift 5.8. A commit from
2024-10-29 removes a "deprecated reference to a Swift 5.8 pipeline" [4].
swift-nio-ssl 2.29.0 is the last version that can be compiled with Swift
5.8. This commit pins it to that exact version.
[1] 66e27d7e84/Package.swift (L33)
[2] 3cb4d5ad12 (diff-bc1db1321ff689c2819245dcce1a3080554f0fc13f81b8d326c97e7d42717c8fR54)
[3] https://github.com/swiftlang/swift-evolution/blob/main/proposals/0380-if-switch-expressions.md
[4] 8a6b89d9a4
---------
Signed-off-by: Marcin Radomski <dextero@google.com>
Co-authored-by: Marcin Radomski <dextero@google.com>
* #Rust Create a crate for reflection
* #Rust Add a crate for reflection tests and helper to access schema
* #Rust Get root table of a buffer and access field with schema
* #Rust Add 'Struct' struct and corresponding getter
* #Rust Add functions of getting any table/struct field value as integer/float/string
* #Rust Add setters for scalar fields
* #Rust Add setter for string fields
* #Rust Add getter for Table/Vector fields
* #Rust Add buffer verification
* Add a 'SafeBuffer' struct which provides safe methods for reflection
It verifies buffer against schema during construction and provides all the unsafe getters in lib.rs in a safe way
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Update release script to update Rust version (it still needs to be published after)
* Also update rust while I'm at it
Co-authored-by: Casper Neo <cneo@google.com>
* [Rust] Add length checks to arrays and vectors.
The previous behavior allowed for out of bounds access in
the public API (#7011).
* bump semver and test warning
Co-authored-by: Casper Neo <cneo@google.com>
* fix for rust build
* Rust: Implement Serialize on generated types
For debugging convenience it is really handy to be able to dump out
types as another format (ie: json). For example, if we are logging a
type to a structured logging system, or even printing it out in a
structured way to the console.
Right now we handle this by shelling out to `flatc` which is not ideal;
by implementing Serialize on the generated types we can use any of the
Serializer-implementing packages for our structured debug output.
* clang-format
* Make the flatbuffers Rust crate only have an optional dependency on the `serde` packages.
* fix warning
* fix rust test build
* Oh yeah this needs to be initialized
* fix toml syntax
* code review feedback
* rebuild test data
* benchmark many vtables
* Rust: Store written_table rev-positions sorted.
The previous implementation was slow if there were too many tables.
Asymototically when inserting the n^th vtable: The old implementation
took O(n) lookup steps and O(1) insertion. The new implementation is
O(log n) lookup and O(n) insertion. This might be improved further by
using a balanced btree.
Benchmarking, create_many_tables is 7.5x faster (on my laptop):
// Simple vector cache
test create_many_tables ... bench: 728,875 ns/iter (+/- 12,279) = 44 MB/s
// Sorted vector cache
test create_many_tables ... bench: 97,843 ns/iter (+/- 4,430) = 334 MB/s
* Fix lints
Co-authored-by: Casper Neo <cneo@google.com>
* Introduce new_from_vec in Rust (also fix formatting)
Also, rename `new_with_capacity` to `with_capacity` to match
how `Vec` does it.
* bump rust version
* mut_finished_buffer
Co-authored-by: Casper Neo <cneo@google.com>
* Fix Miri flag passing and bump Rust version.
* Fix Miri problems from Arrays PR.
SafeSliceAccess was removed for Arrays. It's kind of unsound.
It has two properties:
1. EndianSafe
2. Alignment 1
We only need 1. in create_vector_direct to memcpy data.
We both 1. and 2. for accessing things with slices as buffers are built on &[u8]
which is unaligned. Conditional compilation implements
SafeSliceAccess for >1byte scalars (like f32) on LittleEndian machines
which is wrong since they don't satisfy 2.
This UB is still accessible for Vectors (though not exercised our
tests) as it implements SafeSliceAccess. I'll fix this later by
splitting SafeSliceAccess into its 2 properties.
Co-authored-by: Casper Neo <cneo@google.com>
* Mark endian_scalar as unsafe.
Also
- removed the deprecated flexbuffer slice from example
- fixed some cargo warnings
* Assertions and read_scalar made unsafe
* Clippy lints
* Add to Safety
Co-authored-by: Casper Neo <cneo@google.com>
* Add support for fixed size arrays
* clang-format
* Update rust image to 1.51 to support const generics
* Handle correctly big endian
* Add fuzz tests and clean code
* Add struct fuzz test and optimize struct arrays for api
* Bump flatbuffers crate version
* Fix miri problems by assuming alignment is 1 in rust
* Removed is_aligned fn from rust verifier.
* Add back is_aligned, but make it w.r.t. buffer[0]
* touch unused variable
* touch unused variable
* +nightly
* Move Rust miri testing into its own docker
* fix bash
* missing one endian conversion
* fix endianness2
* format stuff
Co-authored-by: Casper Neo <cneo@google.com>
* Adds shared strings and tests for shared strings
* Adds resets on string_map
* Moved shared strings to use vector instead of hashmap
* Addresses all the issues
* Resolves some comments
* Updated comments and fixed a fundemental type error.
* bump rust flatbuffers semver
* Initial commit with verifier, need to clean up
* Verifier tested. Needs clean up and refactoring.
* Display for InvalidFlatbuffer and better errors for strings
* SimpleToVerify, some refactoring
* Combined VerifierType TableAccessorFuncBody into FollowType
* scrub todos
* Update Rust get_root functions.
There are 6 variants, with verifier options, default verifier options
and no verification "fast".
* Rename root fns
* inline
* Update to use thiserror
* fix for bad compiler
* improve error formatting
* Replace multiply with saturating_multiply
* saturating adds too
* Add docs disclaiming experimental verification system
Co-authored-by: Casper Neo <cneo@google.com>
