mirror of
https://github.com/google/flatbuffers.git
synced 2026-11-09 13:45:09 +00:00
Bigfoot
This commit is contained in:
@@ -168,6 +168,12 @@ list of `FILES...`.
|
||||
std::string from Flatbuffers, but (char* + length). This allows efficient
|
||||
construction of custom string types, including zero-copy construction.
|
||||
|
||||
- `--cpp-vector-type T` : Set object API vector type (default std::vector).
|
||||
T must be a template taking a single element type argument and support
|
||||
resize(), reserve(), size(), data(), operator[], emplace_back() and
|
||||
begin()/end(), matching the subset of std::vector's interface generated
|
||||
code relies on.
|
||||
|
||||
- `--no-cpp-direct-copy` : Don't generate direct copy methods for C++
|
||||
object-based API.
|
||||
|
||||
|
||||
@@ -57,7 +57,7 @@ a `char *` array, which you pass to `GetMonster()`.
|
||||
|
||||
```cpp
|
||||
#include "flatbuffers/flatbuffers.h"
|
||||
#include "monster_test_generated.h"
|
||||
#include "monster_test_generate.h"
|
||||
#include <iostream> // C++ header file for printing
|
||||
#include <fstream> // C++ header file for file access
|
||||
|
||||
@@ -242,6 +242,42 @@ provide the following functions to aide in the serialization process:
|
||||
}
|
||||
```
|
||||
|
||||
- `bigfoot_ref_wrapper("type")` (on a struct) together with `bigfoot_ref("type")`
|
||||
(on a field of that struct type): Bigfoot-specific sugar for a
|
||||
UUID-addressed reference to another native type. `bigfoot_ref_wrapper`
|
||||
marks a struct as a reference-wrapper template (e.g. Bigfoot's
|
||||
`HardReference`/`SoftReference`); `bigfoot_ref` on a field of that type
|
||||
names the referenced native type, e.g. `bigfoot_ref: "::Bigfoot::AssetA"`
|
||||
on a `HardReference`-typed field with `bigfoot_ref_wrapper:
|
||||
"::Bigfoot::HardReference"` produces the field's native type
|
||||
`::Bigfoot::HardReference<::Bigfoot::AssetA>` (auto-deriving a
|
||||
`native_type_pack_name` of `HardReferenceAssetA`, same short-name
|
||||
convention as `native_type_pack_name`). Unlike a hand-written
|
||||
`native_type`, `bigfoot_ref` also emits, directly into the generated
|
||||
header:
|
||||
- a forward declaration of the referenced type (so the referencing
|
||||
schema's generated header never needs that type's real definition -
|
||||
only the wrapper template's constructor and a `GetUUID()` accessor
|
||||
are used, and those don't require the referenced type to be
|
||||
complete), and
|
||||
- `inline` `Pack<Name>`/`UnPack<Name>` definitions for it (so no
|
||||
hand-written implementation is required anywhere).
|
||||
|
||||
```cpp
|
||||
struct HardReference (bigfoot_ref_wrapper: "::Bigfoot::HardReference") {
|
||||
uuid: UUID;
|
||||
}
|
||||
|
||||
table AssetB {
|
||||
ref_a: HardReference (bigfoot_ref: "::Bigfoot::AssetA");
|
||||
}
|
||||
```
|
||||
|
||||
is enough on its own - no forward declaration of `::Bigfoot::AssetA` and
|
||||
no `flatbuffers::Pack/UnPackHardReferenceAssetA` implementation need to
|
||||
be written by hand. `bigfoot_ref` is only valid on fields whose type is a
|
||||
struct (or vector of structs) that declares `bigfoot_ref_wrapper`.
|
||||
|
||||
- `native_type("type")` (on a table): Tables can also be represented with
|
||||
native types. For example, the following schema:
|
||||
|
||||
@@ -337,6 +373,27 @@ constructor in the following format: `custom_str_class(const char *, size_t)`.
|
||||
Please note that the character array is not guaranteed to be NULL terminated,
|
||||
you should always use the provided size to determine end of string.
|
||||
|
||||
## Using different vector type
|
||||
|
||||
By default the object tree's vector fields are built out of `std::vector`,
|
||||
but you can influence this either globally (using the `--cpp-vector-type`
|
||||
argument to `flatc`) or per field using the `cpp_vector_type` attribute, to
|
||||
use any other vector-like template type (e.g. `eastl::vector`).
|
||||
|
||||
The type must be a template taking a single element type argument
|
||||
(`my_vector<T>`), and must support the following member functions:
|
||||
`resize()`, `reserve()`, `size()`, `data()`, `operator[]`, `emplace_back()`,
|
||||
and `begin()`/`end()`. This matches the subset of `std::vector`'s interface
|
||||
that generated code relies on.
|
||||
|
||||
Note that unlike `std::vector<bool>`, the custom vector type must not use a
|
||||
bit-packed specialization for `bool` elements, since generated code accesses
|
||||
`data()` as a contiguous `bool` array.
|
||||
|
||||
As with custom string types, the header defining the custom vector type is
|
||||
not automatically included; use `--cpp-include` to add the necessary
|
||||
`#include`.
|
||||
|
||||
## Reflection (& Resizing)
|
||||
|
||||
There is experimental support for reflection in FlatBuffers, allowing you to
|
||||
|
||||
@@ -97,61 +97,6 @@ convenient accessors for all fields, e.g. `hp()`, `mana()`, etc:
|
||||
|
||||
*Note: That we never stored a `mana` value, so it will return the default.*
|
||||
|
||||
## Fallible API and Custom Allocators
|
||||
|
||||
Every `FlatBufferBuilder` method that may allocate has a `try_*` counterpart
|
||||
(e.g. `try_create_string`, `try_push`, `try_finish`) that returns
|
||||
`Result<T, A::Error>` instead of panicking. This is useful when allocation
|
||||
failures must be handled gracefully: for example in `no_std` environments or
|
||||
with fixed-capacity buffers.
|
||||
|
||||
The existing panicking methods are unchanged and remain the simplest option
|
||||
when using the default allocator.
|
||||
|
||||
#### Custom allocators
|
||||
|
||||
Implement the `Allocator` trait and pass it to `FlatBufferBuilder::new_in()`:
|
||||
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.rs}
|
||||
use flatbuffers::{Allocator, FlatBufferBuilder};
|
||||
|
||||
struct MyAllocator { /* ... */ }
|
||||
|
||||
unsafe impl Allocator for MyAllocator {
|
||||
type Error = MyError;
|
||||
fn grow_downwards(&mut self) -> Result<(), Self::Error> { /* ... */ }
|
||||
fn len(&self) -> usize { /* ... */ }
|
||||
}
|
||||
|
||||
let alloc = MyAllocator::new(/* ... */);
|
||||
let mut builder = FlatBufferBuilder::new_in(alloc);
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The built-in `DefaultAllocator` uses `Vec<u8>` and sets `Error = Infallible`,
|
||||
so the `try_*` methods on a default builder can never fail.
|
||||
|
||||
#### Example: building a buffer with error propagation
|
||||
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.rs}
|
||||
fn build<A: flatbuffers::Allocator>(
|
||||
builder: &mut FlatBufferBuilder<A>,
|
||||
) -> Result<(), A::Error> {
|
||||
let name = builder.try_create_string("Orc")?;
|
||||
let inventory = builder.try_create_vector(&[0u8, 1, 2, 3, 4])?;
|
||||
|
||||
let table_start = builder.start_table();
|
||||
builder.try_push_slot_always(Monster::VT_NAME, name)?;
|
||||
builder.try_push_slot_always(Monster::VT_INVENTORY, inventory)?;
|
||||
builder.try_push_slot(Monster::VT_HP, 80i16, 100)?;
|
||||
let root = builder.try_end_table(table_start)?;
|
||||
|
||||
builder.try_finish(root, None)?;
|
||||
Ok(())
|
||||
}
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
See the `FlatBufferBuilder` rustdoc for the full list of `try_*` methods.
|
||||
|
||||
## Direct memory access
|
||||
|
||||
As you can see from the above examples, all elements in a buffer are
|
||||
@@ -260,47 +205,6 @@ And example of usage, for the time being, can be found in
|
||||
- Safe getters in [SafeBuffer](https://docs.rs/flatbuffers-reflection/latest/flatbuffers_reflection/struct.SafeBuffer.html),
|
||||
which does verification when constructed so you can use it for any data source
|
||||
|
||||
## Buffer pre allocation in a latency-sensitive context
|
||||
|
||||
In latency-sensitive applications, dynamic memory allocations can introduce unpredictable latency spikes. The `FlatBufferBuilder` internally uses several `Vec`s that may reallocate during serialization:
|
||||
|
||||
- The backing buffer for the FlatBuffer data
|
||||
- `field_locs` for tracking field locations within tables
|
||||
- `written_vtable_revpos` for deduplicating vtables
|
||||
- `strings_pool` for shared string interning
|
||||
|
||||
To avoid allocations during serialization, you can preallocate all internal vectors upfront using the `with_internal_capacity` constructor:
|
||||
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.rs}
|
||||
// Preallocate: 1KB buffer, 8 field locations, 16 vtables, 32 shared strings
|
||||
let mut builder = FlatBufferBuilder::with_internal_capacity(1024, 8, 16, 32);
|
||||
|
||||
// All subsequent operations will not allocate (if capacities are sufficient)
|
||||
let name = builder.create_shared_string("MyMonster");
|
||||
// ... build your FlatBuffer ...
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
There are three variants available:
|
||||
|
||||
- `with_internal_capacity(size, field_locs, vtables, strings)` - Creates a new builder with all capacities preallocated
|
||||
- `from_vec_with_internal_capacity(buffer, field_locs, vtables, strings)` - Reuses an existing `Vec<u8>` as the backing buffer
|
||||
- `new_in_with_internal_capacity(allocator, field_locs, vtables, strings)` - Uses a custom `Allocator` with preallocated internal vecs
|
||||
|
||||
When combined with `reset()`, you can reuse the same builder across multiple serializations without any allocations after the initial setup:
|
||||
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.rs}
|
||||
let mut builder = FlatBufferBuilder::with_internal_capacity(1024, 8, 16, 32);
|
||||
|
||||
loop {
|
||||
// Build a FlatBuffer (allocation-free if capacities are sufficient)
|
||||
let data = build_message(&mut builder);
|
||||
send(data);
|
||||
|
||||
// Reset for reuse - clears state but retains allocated capacity
|
||||
builder.reset();
|
||||
}
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
## Useful tools created by others
|
||||
|
||||
* [flatc-rust](https://github.com/frol/flatc-rust) - FlatBuffers compiler
|
||||
|
||||
@@ -427,6 +427,9 @@ Current understood attributes:
|
||||
it won't be accessible anymore by newer code. Note that if you deprecate a
|
||||
field that was previous required, old code may fail to validate new data (when
|
||||
using the optional verifier).
|
||||
|
||||
### `required`
|
||||
|
||||
- `required` (on a non-scalar table field): this field must always be set. By
|
||||
default, fields do not need to be present in the binary. This is desirable, as
|
||||
it helps with forwards/backwards compatibility, and flexibility of data
|
||||
|
||||
@@ -26,7 +26,7 @@ Simple mutation | Yes | Yes | Yes | Yes | No | No
|
||||
Reflection | Yes | No | No | No | No | No | No | Basic | No | No | No | No | No
|
||||
Buffer verifier | Yes | No | No | No | No | No | No | Yes | No | No | No | No | Yes
|
||||
Native Object API | Yes | No | Yes | Yes | Yes | Yes | Yes | No | No | Yes | No | No | Yes
|
||||
Optional Scalars | Yes | Yes | Yes | Yes | No | Yes | Yes | Yes | No | No | Yes | Yes | Yes
|
||||
Optional Scalars | Yes | Yes | Yes | No | No | Yes | Yes | Yes | No | No | Yes | Yes | Yes
|
||||
Flexbuffers | Yes | Yes | ? | ? | ? | ? | ? | ? | ? | ? | ? | Yes | Yes
|
||||
Testing: basic | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | ? | Yes | Yes | Yes | Yes
|
||||
Testing: fuzz | Yes | No | No | Yes | Yes | No | No | No | ? | No | No | Yes | No
|
||||
|
||||
Reference in New Issue
Block a user