Artyom Palvelev
5eb190f000
In the FlatBuffers binary wire format specification, every scalar field is
aligned to its own byte size (`1`, `2`, `4`, or `8` bytes), regardless of the
host CPU architecture. Both the `flatc` schema compiler (when computing table
`max_align` and struct field offsets) and the C++ `FlatBufferBuilder::Align` /
`PushElement` implementation align 64-bit scalars (`u64`, `i64`, `f64`) to 8
bytes on all architectures.
However, in the Rust `flatbuffers` crate (`rust/flatbuffers/src/push.rs`), the
`Push` trait provides a default implementation:
```rust
fn alignment() -> PushAlignment {
PushAlignment::new(align_of::<Self::Output>())
}
```
And `impl_push_for_endian_scalar!` did not override `Push::alignment()`,
inheriting `core::mem::align_of::<T>()` for all primitive scalar types (`bool`,
`u8`, `i8`, `u16`, `i16`, `u32`, `i32`, `u64`, `i64`, `f32`, `f64`).
On 64-bit targets (`x86_64`, `aarch64`) and 32-bit ARM (`armv7`),
`align_of::<u64>() == 8`, so `align_of::<T>()` happens to equal `size_of::<T>()`.
On 32-bit x86 (`i686`, System V i386 ABI), however, the host ABI alignment of
64-bit scalars (`u64`, `i64`, `f64`) is **4 bytes** (`align_of::<u64>() == 4`),
while their FlatBuffer wire-format size and required wire alignment is **8
bytes** (`size_of::<u64>() == 8`).
As a consequence, on 32-bit x86 (`i686`):
1. `FlatBufferBuilder::push::<u64 / i64 / f64>` only aligns the builder's cursor
and tracks `min_align` to 4 bytes instead of 8 bytes when writing 64-bit
table fields or vectors of 64-bit scalars.
2. When a table containing a 64-bit scalar field (`uint64`, `int64`, `double`)
is serialized on `i686`, the field can be placed at a buffer offset that is
4-byte aligned but not 8-byte aligned.
3. When the resulting FlatBuffer is verified by C++ `flatbuffers::Verifier`
(which checks `VerifyField<uint64_t>` with `sizeof(uint64_t) == 8` alignment
when `FLATBUFFERS_TRACK_VERIFIER_BUFFER_SIZE` / alignment checks are active)
or transferred to a 64-bit reader, verification fails or unaligned 64-bit
reads occur.
Fix `impl_push_for_endian_scalar!` to explicitly implement `Push::alignment()`
using `size_of::<$ty>()` so that all endian scalar types always use their
wire-format byte size as their FlatBuffer alignment across all target
architectures.
FlatBuffers
FlatBuffers is a cross platform serialization library architected for maximum memory efficiency. It allows you to directly access serialized data without parsing/unpacking it first, while still having great forwards/backwards compatibility.
Quick Start
-
Build the compiler for flatbuffers (
flatc)Use
cmaketo create the build files for your platform and then perform the compilation (Linux example).cmake -G "Unix Makefiles" make -j -
Define your flatbuffer schema (
.fbs)Write the schema to define the data you want to serialize. See monster.fbs for an example.
-
Generate code for your language(s)
Use the
flatccompiler to take your schema and generate language-specific code:./flatc --cpp --rust monster.fbsWhich generates
monster_generated.handmonster_generated.rsfiles. -
Serialize data
Use the generated code, as well as the
FlatBufferBuilderto construct your serialized buffer. (C++example) -
Transmit/store/save Buffer
Use your serialized buffer however you want. Send it to someone, save it for later, etc...
-
Read the data
Use the generated accessors to read the data from the serialized buffer.
It doesn't need to be the same language/schema version, FlatBuffers ensures the data is readable across languages and schema versions. See the
Rustexample reading the data written byC++.
Documentation
Go to our landing page to browse our documentation.
Supported operating systems
- Windows
- macOS
- Linux
- Android
- And any others with a recent C++ compiler (C++ 11 and newer)
Supported programming languages
Code generation and runtime libraries for many popular languages.
- C
- C++ - snapcraft.io
- C# - nuget.org
- Dart - pub.dev
- Go - go.dev
- Java - Maven
- JavaScript - NPM
- Kotlin
- Lobster
- Lua
- PHP
- Python - PyPI
- Rust - crates.io
- Swift - swiftpackageindex
- TypeScript - NPM
- Nim
Versioning
FlatBuffers does not follow traditional SemVer versioning (see rationale) but rather uses a format of the date of the release.
Contribution
- FlatBuffers Issues Tracker to submit an issue.
- stackoverflow.com with
flatbufferstag for any questions regarding FlatBuffers.
To contribute to this project, see CONTRIBUTING.
Community
Security
Please see our Security Policy for reporting vulnerabilities.
Licensing
Flatbuffers is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.