Commit Graph

9 Commits

Author SHA1 Message Date
Raphael Taylor-Davies
374f8fb5fb Rust soundness fixes (#7518)
* Rust soundness fixes

* Second pass

* Make init_from_table unsafe

* Remove SafeSliceAccess

* Clippy

* Remove create_vector_of_strings

* More clippy

* Remove deprecated root type accessors

* More soundness fixes

* Fix EndianScalar for bool

* Add TriviallyTransmutable

* Add debug assertions

* Review comments

* Review feedback
2022-09-29 09:58:49 -04:00
Bogdan Opanchuk
750dde7669 Make flatc generate Rust files not requiring std (#7273)
* Set an explicit 2018 edition for Rust tests

* Replace all `std` usage with `core` and `alloc` in Rust code generator

* Update the generated files

* Make Rust tests actually use no_std when the corresponding feature is enabled
2022-04-26 21:40:03 -04:00
Casper
35e2cac6eb Store vtables sorted in Rust builder (#6765)
* benchmark many vtables

* Rust: Store written_table rev-positions sorted.

The previous implementation was slow if there were too many tables.

Asymototically when inserting the n^th vtable: The old implementation
took O(n) lookup steps and O(1) insertion. The new implementation is
O(log n) lookup and O(n) insertion. This might be improved further by
using a balanced btree.

Benchmarking, create_many_tables is 7.5x faster (on my laptop):

// Simple vector cache
test create_many_tables ... bench: 728,875 ns/iter (+/- 12,279) = 44 MB/s

// Sorted vector cache
test create_many_tables ... bench: 97,843 ns/iter (+/- 4,430) = 334 MB/s

* Fix lints

Co-authored-by: Casper Neo <cneo@google.com>
2021-08-03 12:31:45 -07:00
Casper
12e341e4f8 Rework how Rust generated files are laid out (#6731)
* Refactored Rust Generated code into a module directory.

Each symbol will be generated into one file and then
imported into a module. This breaks the "out_dir"
pattern where some users would generate code in their
target/ directory. Also, these objects are best used
in their own module. It will be hard for users to share
their own module structure with flatbuffers namespaces.

There may be solutions to these drawbacks but that should
be discussed. I don't want to overengineer here.

* shadow error

* try fix .bat file

* fix .bat 2

* Restore accidentally deleted files

* Fixed some DONOTSUBMITs and made Rust outdir pattern use symlinks.

* fixed binary files

* git clang format

* make generated onefiles not public and fix .bat

* reduced diff with master in generate_code.sh

* fix shadowed variable

* add object api flags to .bat

* space

* Removed extern crate and extra &

* use statement

* more clippy lints

* format

* Undo extern crate -> use change, it actually matters to our tests

Co-authored-by: Casper Neo <cneo@google.com>
2021-07-22 19:02:28 -04:00
Colin
4174c10e7a [rust] Genericize flexbuffer reader (#6450)
* feature/rust-tokio-bytes added feature name for tokio-bytes

* Added flexbuffer implementation, TODO: typecast to avoid recurse

* Converted codebase to utilize FlexBuffer implementation, need to resolve deserialization issues

* Added todo for lifetime issue, may use &'de [u8] for deserializer instead of current method

* Added proper &[u8] implementation

* Removed unused struct

* Added experimental fix to get_slice

* Added experimental fix to get_slice

* Avoided lifetime issues via ref structs, need to check if this hurts peformance

* Updated deserializer implementation to allow for borrowed data from Reader struct

* Fixed bug with str

* Removed unnecessary generic parameter

* Added unsafe to avoid lifetime complaints, current tests pass, need to review alternatives to unsafe

* Opinionated: Removed bytes crate as this implementation could be done in a separate crate

* Cleaned up flatbuffer

* Fixed sample / example

* Resolved PR feedback, need to resolve issues with tests

* Cleaned up FlexBuffer trait to be an auto impl

* Removed TODO

* Reverted Deserializer to only support &'de [u8]

* Cleaned up / renamed function for clarification

* Renamed FlexBuffer -> InternalBuffer for clarification on it's purpose

* Fixed issue with key bytes

* resolved issues with broken tests, confirming this is a breaking change

* Removed FIXME that's solved by splitting String and Key variants

* Implemented associated types approach

* Fixed backward slice logic

* Fixed MapReader compile error

* Added from_buffer for deserialization, removed  function since it's only needed for deserialization

* Removed dead code

* Cleaned up buffer, removed AsRef in favor of Deref

* Renamed Buffer::as_str -> Buffer::buffer_str

* Minor cleanup

* Updated documentation, need to fix tests

* Removed unnecessary &

* Removed unused lifetime

* removed unnecessary as_ref

* Minor optimization wrap-up

* resolved issue with Clone

* Added test to verify no deep-copy

* Added  for optimization

* Updated to use empty fn instead of default

* Updated comments / test name - plus the 0.3.0 version bump

* comment
2021-02-16 08:04:48 -05:00
Casper
8be05f6bd4 Rust Flexbuffers (#5669)
* Cargo clippy lints

* more lints

* more lints

* Restored a doc comment

* Comment on float eps-eq and adjusted casting

* Rust Flexbuffers

* more serde tests, removed some unsafe

* Redid serde to be map-like and Reader is Display

* Moved iter from Reader to VectorReader

* Serious quickcheck + bugs

* wvo api

* Made types smaller for a reasonable speedup

* redid reading in a way that's a bit faster.

Profiling shows the rust slowdown as building +10%, reading +20%

* src/bin are developer binaries in rust

* Root and Map width are not packed

* key null check is debug only + doc changes

* BuilderOptions

* Documentation

* Documentation

* Moved tests to rust_usage_test

* Moved rust flexbuffers samples to Flatbuffers/samples

* Fixed RustTest

* Fixed for Rust 1.37.0

* Upgraded to rust 1_40_0

* fixed a little-endian-only feature in a test

* 1.40.0

* fixed some benchmarks for bigendian

* Updated .bat file

* misspelling

* Gold Flexbuffer test.

* Serialize,Deserialize, std::error::Error for Errors.

* Undo rustfmt in integration_test.rs

* from_slice instead of from_vec

* Added comments to unsafe blocks

* expanded on comment

* bump

Co-authored-by: CasperN <cneo@google.com>
2020-05-07 14:11:26 -07:00
Max Burke
6da1cf79d9 [rust] Add use declarations to Rust-generated bindings for imported FB definitions (#5645)
* Bugfix for Rust generation of union fields named with language keywords

Looking at ParseField, it appears that in the case of unions, an extra field with a `UnionTypeFieldSuffix` is added to the type definition, however, if the name of this field is a keyword in the target language, it isn't escaped.

For example, if generating code for rust for a union field named `type`, flatc will generate a (non-keyword escaped) field named `type_type` for this hidden union field, and one (keyword escaped) called `type_` for the actual union contents.

When the union accessors are generated, they refer to this `type_type` field, but they will escape it mistakenly, generating code like this:

```
  #[inline]
  #[allow(non_snake_case)]
  pub fn type__as_int(&self) -> Option<Int<'a>> {
    if self.type__type() == Type::Int {
      self.type_().map(|u| Int::init_from_table(u))
    } else {
      None
    }
  }
```

Which will fail to build because the field is called `self.type_type()`, not `self.type__type()`.

* [Rust] Add crate-relative use statements for FBS includes.

At present if a flatbuffer description includes a reference to a type in
another file, the generated Rust code needs to be hand-modified to add
the appropriate `use` statements.

This assumes that the dependencies are built into the same crate, which
I think is a reasonable assumption?

* Revert "[Rust] Add crate-relative use statements for FBS includes."

This reverts commit d554d79fec.

* Add updated generated test files.

* Fixing Rust test harness to handle new includes.

Test binaries need to add references to generated code that's
transitively included.

This also has the knock-on in that this code (which is referenced by
include directives directly in the flatbuffer schema files) also needs
to be generated, hence the changes to generate_code.sh.

* Test harnesses expect test data to be checked in.

Put include_test2 files into the same directory as the include_test2
schema definition.

Update all code generation scripts (forgot the batch file from last
time).

Path updates in Rust test.

* Include updated generated code

* Address comments raised in PR

* Fix failing Rust tests.

* Previous merge clobbered this branch change.

* Add updated imports to benchmarks.

* Clarifying comment per PR request

* Update documentation comments per feedback

* Remove non-Rust generated files for include tests, per feedback from @rw/@aardappel

* Broken code generation batch file

* Fix typo

* Add TODO for tidying up use declaration traversal sometime in the future

* Update test files.
2020-01-19 14:47:28 -08:00
bspeice
3968d00568 [Rust] Don't use inner attributes for allow (#5212)
* Don't use inner attributes for `allow`
Messes with being able to easily include elsewhere

* Regenerate tests

* No-op to retrigger CI

* Add the rest of the `allow` attributes
2019-03-09 12:58:27 -08:00
Robert
3c54fd964b Port FlatBuffers to Rust (#4898)
This is a port of FlatBuffers to Rust. It provides code generation and a
runtime library derived from the C++ implementation. It utilizes the
Rust type system to provide safe and fast traversal of FlatBuffers data.

There are 188 tests, including many fuzz tests of roundtrips for various
serialization scenarios. Initial benchmarks indicate that the canonical
example payload can be written in ~700ns, and traversed in ~100ns.

Rustaceans may be interested in the Follow, Push, and SafeSliceAccess
traits. These traits lift traversals, reads, writes, and slice accesses
into the type system, providing abstraction with no runtime penalty.
2018-09-02 18:26:55 -07:00