* fix: correct operator precedence in ForAllFields reverse iteration
The expression `size() - i + 1` evaluates as `(size() - i) + 1` due to
left-to-right associativity, producing an out-of-bounds index when
reverse=true. For a vector of size N, the first iteration (i=0) accesses
index N+1, which is 2 past the last valid index.
Changed to `size() - (i + 1)` to match the correct implementation
already present in bfbs_gen.h:192.
Bug: CWE-125 (Out-of-bounds Read), CWE-783 (Operator Precedence Error)
* test: add ForAllFieldsReverseTest for reverse iteration correctness
Verify that ForAllFields with reverse=true iterates fields in
descending ID order. Tests both Stat (3 fields) and Monster
(many fields with non-sequential definition order) tables.
---------
Co-authored-by: Tulgaa <tulgaa.kek@gmail.com>
* Add fallible try_* API for FlatBufferBuilder
This is to support error propagation from Allocator trait. The Allocator
grow_downwards() method returns Result<(), Self::Error>, but
FlatBufferBuilder panics via .expect() when allocation fails instead of
propagating the error.
* Add rust fallible API docs
A blob is an array of bytes and has no intrinsic alignment (i.e. the
alignment is 1). The alignment of the existing flexbuffers blob is
solely affected by the width of the integer needed to store the blob's
size: that integer's width becomes the alignment of the blob.
The proposed AlignedBlob function here piggybacks on this effect and
simply uses a user-defined alignment for the width of the integer that
stores the blob's size; this automatically imparts that same alignment
on the blob itself. (The width is bounded below by the actual width
needed to store the blob's size.)
The ability to control the alignment of a blob is important for use
cases in which the blob itself stores structured data that we want to
access without further copies (e.g. other flatbuffer messages).
String default values parsed from .fbs schemas are un-escaped by the IDL
parser (e.g., \x22 becomes a raw " byte), but code generators embed these
raw values directly into generated source code string literals. This allows
specially crafted .fbs files to break out of string literals and inject
arbitrary code into generated C++, Rust, TypeScript, and Swift source.
Fix by adding EscapeCodeGenString() helper that re-escapes string content
before embedding, and applying it to all 7 affected injection points across
5 code generators (C++, Rust, TypeScript, Swift, FBS).
Resolves the TODO comments in idl_gen_cpp.cpp and idl_gen_rust.cpp.
* fix(flatbuffers): use manual impl Default for struct object types
* fix: handle bool and float zero literals in struct object Default impl
* fix: regenerate all test bindings with generate_code.py
* fix: data type check on swift build
* fix: test large array on struct and enum
When building a FlexBuffer using the Builder and adding data that exceeds the default buffer size (2048 bytes), in _newOffset() a larger buffer is created, but never used. This results in a RangeError.
Resolve by actually replacing the too small with the new larger buffer. Add a test that verifies this by adding multiple large strings to a vector.
There is a typo in the schema.md file in documentation. An empty 'required' section was added right before the 'required' item in the middle of the attributes list. It even appears in the table of content, which might confuse readers, making it look like following attributes might be required while they are not. You can notice the issue there: https://flatbuffers.dev/schema/#attributes.
I did check that `mkdocs serve -f mkdocs.yml` does produce the expected output (the same attributes list without that extra empty `required` section in the middle) with my changes.
* [C#] Add GetBytes methods for fixed arrays
I wanted to direct access to fixed array bytes. I made some changes to the idl generator to create GetBytes functions following the same naming conventions used for vectors of scalar types. There was not a 'Length' field present to bound the existing index accessor so I added that too.
+ Add generic GetBytes for fixed length arrays of scalar types
+ Implement conditional compilation for ENABLE_SPAN_T:
- ENABLE_SPAN_T: Returns `Span<T>` using `MemoryMarshal.Cast<byte, T>()` as needed.
- Else: Returns `ArraySegment<byte>?` for raw byte access
+ Added tests reusing arrays_test.fbs definitions
+ Added const int Length field to support existing index based accessors
* [C#] Sync generated code for after adding GetBytes methods for fixed arrays
---------
Co-authored-by: Björn Harrtell <bjornharrtell@users.noreply.github.com>
* Adds tests for Rust single file mode
All existing tests only compile Rust code using --rust-module-root-file.
* Adds standalone include tests for Rust
The imports for these tests have been moved to their own file, as the existing intergration_test.rs file hides compilation issues from code generation due to symbols brought into scope outside of the generated code (e.g. `extern crate alloc`).
* Declare alloc crate in every Rust namespace
When performing code generation within a single file, extern crate alloc needs to be delcared to bring alloc into scope within every inner namespace.
* Regenerates generated schemas
The write_vtable() function's comment claimed to "fill the WIP vtable
with zeros" but make_space() only reserves memory without initializing
it. When using custom allocators with non-zeroed buffers, unset vtable
field entries would contain garbage instead of zero (which indicates
"use default value").
This fix explicitly zeros the vtable memory after reserving space,
matching the C++ implementation's buf_.fill_big() behavior.
Added regression test using a garbage-filled allocator (0xAA) that
verifies vtable entries for unset fields are properly zeroed.
Fixes#8894
* Add --ts-undefined-for-optionals command line option
# Details
- Fixes#7656
- Added a new `--ts-undefined-for-optionals` command line option for `flatc`.
- If enabled, generated TypeScript code uses `undefined` for optional fields rather than `null`.
* Also add TS generated test files
* Run `sh scripts/clang-format-git.sh`
* also add tests/ts/lalala-options.ts to the repo
* move new tests to tests/ts/optional_values dir
* add tests/ts/optional_values/optional_values_generated.cjs to the repo
* reuse existing optional_scalars.fbs and add new test
* add comma
* sh scripts/clang-format-git.sh
* remove comma
* sh scripts/clang-format-git.sh
* trying things
* sh scripts/clang-format-git.sh
* done
* address feedback
* sh scripts/clang-format-git.sh
* run `sh scripts/clang-format-git.sh`
* remove uneeded `eslint-disable @typescript-eslint/no-namespace` line
---------
Co-authored-by: José Luis Millán <jmillan@aliax.net>
* fix for https://github.com/google/flatbuffers/issues/8759
__vector_as_array<T> calling ByteBuffer.ToArray<T> with the length in bytes by multiplying len with ByteBuffer.Sizeof<T> and FlatBuffersExampleTests extended to call GetVectorOfLongsArray/GetVectorOfDoublesArray which failed without the fix
* first try to repair build-dotnet-windows
* syntax error fixed
* Update solution creation command in build workflow
add --format sln to the dotnet new command, maybe it is currently creating a .slnx instead?
* Fix generate_code script path
* [Python] Make StartVector public
Make StartVector vector public since it is already being used in
generated code
* [Python] Improve vector creation for Python API
Makes Python API for vectors cleaner like Rust and Swift
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
* chore(idl): Check for case insensitive keywords
Most languages are not affected by this change. In PHP, some names such
as Bool cannot be used because it is a reserved keyword for to the bool
data type. The field `keywords_casing` in the configs enables checking
all characters in lowercase against every keyword. This should be safe
as flatbuffers does not allow non-ASCII characters in its grammar.
* chore: Fix formatting to follow google's coding style for enums
* chore: Extract convert case to lower when CaseInsensitive
---------
Co-authored-by: Justin Davis <jtdavis777@gmail.com>
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Remove invalid dependency on FLATBUFFERS_GENERATE_HEADERS_SCHEMAS
add_dependencies() is for targets.
CMake 4.2.0 fails because of this (it shouldn't crash though, but that's another topic). See https://gitlab.kitware.com/cmake/cmake/-/issues/27415
* Use FLATC_TARGET
---------
Co-authored-by: Justin Davis <jtdavis777@gmail.com>
* generate mutable union accessors
* add test
* Revert "add test"
This reverts commit 45e352b18f.
* update file
* formatter got in the way
* merge conflicts
* updated genned code
* manually fix code gen bc I can't figure out why this file won't code gen
---------
Co-authored-by: Wouter van Oortmerssen <aardappel@gmail.com>
Add missing bounds checking to ByteVector before slice
operations in the Go FlatBuffers implementation. Relative offset and
vector length are now checked against the buffer size. Instead of
panicking, the code now returns nil. Regression test added.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
Co-authored-by: Justin Davis <jtdavis777@gmail.com>
* arrays of enums with no value for 0 now throw errors
* move setting key field outside struct check
* set to default instead of required
* unsure of why these bfbs files have changed at this time, checking them in to run the pipelines.
* remove known bad test
* flatc builds and seems to work, some of the extra targets are having linker errors
* fix build system
* pipeline failures
* un-rename files
* refactor to use unique_ptr
* typo
* rm make_unique, add comments
* fix cmake
---------
Co-authored-by: Wouter van Oortmerssen <aardappel@gmail.com>
Improves unpack performance for vectors by allowing the compiler
to vectorize flatbuffers::Vector to std::vec::Vec conversions,
using the unstable trusted_len feature.
Internally, enables an optimization in src/alloc/vec/spec_extend.rs:
Previously, unpacking a flatbuffers::Vector called
SpecExtend::extend_desugared fallback, which inhibits vectorization
(due to a branch before every element move). Declaring TrustedLen
allows SpecExtend::extend_trusted, which LLVM can often vectorize
into a memcpy.
For [ubyte] vectors in particular, this turns a rather expensive
loop of 'mov BYTE PTR [rax+r13*1], bpl' into a `call memcpy`.
* add deser support for enum type
* update generated files
* remove deser generator when bitflag enable
* add deser test
* Restore the Rust editions version
* Remove unnecessary modifications
* Fixes checks for serde features in flexbuffers crate
* Removes unused MapReaderIndexer use statement
* Fixes warning about nightly cfg usage
Enabling a cfg attribute through cargo::rustc-cfg in build.rs should be coupled with a cargo::rust-check-cfg value so that the compiler knows about the custom cfg. See: https://doc.rust-lang.org/rustc/check-cfg/cargo-specifics.html#cargorustc-check-cfg-for-buildrsbuild-script.
* Migrates usage of deprecated float constants
This update fixes a compiler warning from use of the old constants.
Constants like EPSILON are now directly on the float primitives (e.g. f32::EPSILON) rather than in the f32 module (std::f32::EPSILON).
The new constants have existed since 1.43.0, which appears to be below the MSRV for the flatbuffers crate.
* Fixes incorrect key in flatbuffers Cargo.toml
The old code was using package.rust, which triggered a warning about an unused key:
warning: flatbuffers/rust/flatbuffers/Cargo.toml: unused manifest key: package.rust
The correct key for specifying MSRV is rust-version. See: https://doc.rust-lang.org/cargo/reference/rust-version.html#rust-version.
---------
Co-authored-by: Wouter van Oortmerssen <aardappel@gmail.com>
* [Java] Use Table's fully qualified path
When a table's name is called `Table`, the Java bindings generated result in an error due to there being 2 Tables. This fixes the issue by fully qualifyng the flatbuffers Table import.
* Update codegen
* Update generated Java code
---------
Co-authored-by: Neville Dipale <neville@urbanlogiq.com>
* [Go] Write required string fields into the buffer when using Object API
In C++, CreateX allows to write the default "" value of a required
string, when the string is not explicitly set. Object API Pack method
uses this implementation of CreateX.
However, in go, despite whether the field is optional or required, it is
always checked against empty string allowing to create messages that
fail to pass the verifier. This commits partially reverts #7719 when the
string field is required.
* Add test for serializing required string fields using Object API
* Update generated code
The Monster schema contains a key string field. For historical
convenience reasons, string keys are assumed required.
* fix CScript string.compare
Because the default compare sorting rules of c# string are different from those of cpp, the binary file exported by flatc -b cannot use LookupByKey
* run generate_code.py
---------
Android libraries include <android/api-level.h> which defines the __ANDROID_API__ symbol even when targeting non-Android platforms and not using Android's libc. This updates the FLATBUFFERS_LOCALE_INDEPENDENT ifdef to check for __ANDROID__ before checking the Android API level.
Removes an extra check from the __Fuchsia__ branch. Fuchsia's libc does not support locales or the locale independent entry points.
Updates the Android API check to check for an API level >= 26 instead of 21. This matches the Android header file's availability macros and Bionic documentation:
https://android.googlesource.com/platform/bionic/+/HEAD/docs/status.md
This tried to generate from a directories "MyGame/Sample/"
for a empty path_ in M, MyGame & MyGame/Sample.
Which is incorrect since we want to start with the first
kPathSeparator `/` and not position 1.
We can't use macos-latest-large in non paid GitHub account but we can
use macos-15-intel in public repositories. If we use macos-15-intel,
we can run CI jobs for Intel macOS in fork repositories.
* Fixes to make SizeVerifier work.
In particular change all the places in the Flatbuffers library
and generated code that were using `Verifier` to instead use
`VerifierTemplate<TrackBufferSize>` and wrap them all inside
`template <bool TrackBufferSize = false>`.
Also add unit tests for SizeVerifier.
* Format using `sh scripts/clang-format-git.sh`
* Use `B` rather than `TrackBufferSize` for the name of the template parameter.
* Update generated files.
Implements InlineArrays which allow us to use Flatbuffers arrays within
Structs natively, and also implements FlatbufferVectors as a secondary API
when using mutable Structs
Fixes mutations within fixed sizes arrays
Adds tests and remove inout and mutating from generated objects in favor of borrowing
---------
Co-authored-by: Wouter van Oortmerssen <aardappel@gmail.com>
* Support native_type for tables when using the C++ object API.
If native_type is specified on a table:
- No object API struct type is generated.
- The object API refers to the table by its native_type.
- UnPack and Create<TableName> methods are declared but not defined; as they
must be user-provided.
* Add tests for native_type on tables.
* Add documentation for native_type on tables.
Per the definition of --gen-compare: only generate comparators for object API
generated structs. Types annoated with native_type must define their own
comparators if `--gen-compare` is enabled.
Also enables --gen-compare for native_type_test and fixes the test by adding a
comparator for the Native::Vector3D type.
* Implements FlatbuffersVector in swift
Implements FlatbuffersVector which confirms to RandomAccessCollection,
this would give us semi-native sugary syntax to all the arrays in swift port.
This work will also be the foundation of using arrays in swift
* Fix failing tests for Swift
Previously: X::Pack forwarded to CreateX.
Now: CreateX will forward to X::Pack.
This is a step toward enabling using native types for tables when using the
object API. When defining a native table, the user will be able to define a
custom X::Pack method (which is more consistent with the existing native_type
functionality for structs). By reversing the order of the dependencies, CreateX
can continue to be auto-generated and will use the custom X::Pack method when
overriden for native_type tables.
* Prevent `make_span` from working with vectors and arrays of pointers
* support `make_structs_span` for little-endian
* fix build: add the required parentheses
---------
Co-authored-by: Wouter van Oortmerssen <aardappel@gmail.com>
Treat flatbuffers' definition of npm_typescript as a dev dependency, in order to avoid conflicts when consuming flatbuffers in a repo that also depends on aspect_rules_ts.
* Correction of bug inside ToArray<T> methods
Avoid allocating too large buffers (len is expressed in bytes, not in Ts).
Added validation to ensure len is a multiple of SizeOf<T>() before converting to array.
* Update ByteBuffer.cs
* Refactor ToArray and ToArrayPadded methods
I understand from failed test that pos, len, padLeft and padRight are expressed in Ts
* Refactor ToArray and ToArrayPadded methods
* Final correction
All functions parameters expressed in bytes for homogeneity
Tests run:
- UNSAFE_BYTEBUFFER=true/ENABLE_SPAN_T=true: passed
- UNSAFE_BYTEBUFFER=true/ENABLE_SPAN_T=false: passed
- UNSAFE_BYTEBUFFER=false/ENABLE_SPAN_T=false: passed
- UNSAFE_BYTEBUFFER=false/ENABLE_SPAN_T=true: configuration forbidden by compilation
Correction of FlatBuffers.Test.csproj to allow UNSAFE_BYTEBUFFER/ENABLE_SPAN_T tests
Correction of FlatBuffersExampleTests.cs: I think the test was not run because it could not pass (to be reviewed carefully)
* Added ToSizedArrayPadded(int padLeft, int padRight) + ToArrayPadded(pos, len, padLeft, padRight) to the byteBuffers.
This is for API completion and to avoid unnecessary copy when framing my packets. I needed this to create a flat buffer with space in front of it for header / metadata.
* Fix indentation
---------
Co-authored-by: Björn Harrtell <bjornharrtell@users.noreply.github.com>
Co-authored-by: Wouter van Oortmerssen <aardappel@gmail.com>
There are a couple instances where the ByteBuffer's Span property was accessed in a loop.
+ Extracted the use of the property outside of the loop to save a few cpu cycles.
Access to the allocator's internal buffer isn't exposed as a ReadOnlySpan<byte> from the ByteBuffer
or the FlatBufferBuilder.
+ Added a few convenience functions to access the buffer using a ReadOnlySpan<byte>.
There are a few cases where built in Span extensions can be used to run optimized code.
+ Added the use of Span.Fill() and ReadOnlySpan.SequenceCompareTo to replace existing loops.
Co-authored-by: Björn Harrtell <bjornharrtell@users.noreply.github.com>
Co-authored-by: Wouter van Oortmerssen <aardappel@gmail.com>
new_p is a local addr but is owned now by slice_
thus the life time does not end at the end of the function
Co-authored-by: Wouter van Oortmerssen <aardappel@gmail.com>
* fix(go/grpc): avoid panic on short FlatBuffers input
The gRPC codec read the root UOffsetT without checking input size. On
buffers shorter than SizeUOffsetT, GetUint32 touched data[3] and the
process panics.
Add a simple length check and validate the root offset stays within the
buffer. Return clear errors (insufficient data / invalid root offset)
instead of panicking.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
* fix(go/grpc): avoid signed overflow in offset
Keep the bounds check in the unsigned domain (UOffsetT) to avoid
signedness pitfalls.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
* chore: clarify comment regarding offset
A full FlatBuffer structure would be:
- uoffset_t: root table offset (4 bytes)
- soffset_t: vtable offset in root table (4 bytes)
- uint16_t: vtable size (2 bytes)
- uint16_t: table size (2 bytes)
In total 12 bytes. We are only validating the data length
before trying to read the uoffset_t, not the full structure.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
---------
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
Co-authored-by: Derek Bailey <derekbailey@google.com>
Our CI is broken and this is the error:
```
FAILURE: Build failed with an exception.
* What went wrong:
Gradle requires JVM 17 or later to run. Your build is currently configured to use JVM 11.
```
So updating our java-versions to the latest stable version which is 21 apparently.
Makes the return type of `static getFullyQualifiedName()` be a string literal instead of just the string type
Update tests
Co-authored-by: Björn Harrtell <bjornharrtell@users.noreply.github.com>
* [Python] Sync PythonTest.sh flags with generate_code.py
* [Python] Update generated code to latest flatc version for tests
* [Python] Fix test support for numpy newer than 2.0.0
* [Python] Remove unused variable
* [Python] Fix __eq__ for numpy arrays
* [Python] Run clang-format over the entire file
* Developers intro how to contribute
* Fix Rust code generation for Rust edition 2024
The errors look like:
```
warning[E0133]: call to unsafe function `fbs::flatbuffers::emplace_scalar` is unsafe and requires unsafe block
warning[E0133]: call to unsafe function `fbs::flatbuffers::follow_cast_ref` is unsafe and requires unsafe block
warning[E0133]: call to unsafe function `fbs::flatbuffers::Follow::follow` is unsafe and requires unsafe block
warning[E0133]: call to unsafe function `fbs::flatbuffers::read_scalar_at` is unsafe and requires unsafe block
warning[E0133]: call to unsafe function `fbs::flatbuffers::root_unchecked` is unsafe and requires unsafe block
warning[E0133]: call to unsafe function `fbs::flatbuffers::size_prefixed_root_unchecked` is unsafe and requires unsafe block
warning[E0133]: call to unsafe function `fbs::flatbuffers::Table::<'a>::new` is unsafe and requires unsafe block
warning[E0133]: call to unsafe function `std::slice::from_raw_parts` is unsafe and requires unsafe block
```
* Update goldens
Ran `goldens/generate_goldens.py`
* Regenerate code files
Ran `scripts/generate_code.py`
This commit significantly improves the developer experience for the Python Object-Based API by overhauling the generated `__init__` method for `T`-suffixed classes.
Previously, `T` objects had to be instantiated with an empty constructor, and their fields had to be populated manually one by one. This was verbose and not idiomatic Python.
This change modifies the Python code generator (`GenInitialize`) to produce `__init__` methods that are:
1. **Keyword-Argument-Friendly**: The constructor now accepts all table/struct fields as keyword arguments, allowing for concise, single-line object creation.
2. **Fully Typed**: The signature of the `__init__` method is now annotated with Python type hints. This provides immediate benefits for static analysis tools (like Mypy) and IDEs, enabling better autocompletion and type checking.
3. **Correctly Optional**: The generator now correctly wraps types in `Optional[...]` if their default value is `None`. This applies to strings, vectors, and other nullable fields, ensuring strict type safety.
The new approach remains **fully backward-compatible**, as all arguments have default values. Existing code that uses the empty constructor will continue to work without modification.
#### Example of a Generated `__init__`
**Before:**
```python
class KeyValueT(object):
def __init__(self):
self.key = None # type: str
self.value = None # type: str
```
**After:**
```python
class KeyValueT(object):
def __init__(self, key: Optional[str] = None, value: Optional[str] = None):
self.key = key
self.value = value
```
#### Example of User Code
**Before:**
```python
# Old, verbose way
kv = KeyValueT()
kv.key = "instrument"
kv.value = "EUR/USD"
```
**After:**
```python
# New, Pythonic way
kv = KeyValueT(key="instrument", value="EUR/USD")
```
Using the : syntax leads to non member attributes.
> If an attribute is defined in the class body with a type annotation
> but with no assigned value, a type checker should assume this is a non-member attribute
```
class Pet(Enum):
genus: str # Non-member attribute
species: str # Non-member attribute
CAT = 1 # Member attribute
DOG = 2 # Member attribute
```
https://typing.python.org/en/latest/spec/enums.html#defining-members
This prevents the include of the type defined in the pyi,
otherwise this leads to error message like this:
error: Name XYZ already defined (possibly by an import) [no-redef]
* [Python] Use correct type for str with None
Otherwise mypy will correctly flag code like this
def __init__(self):
self.fooBar = None # type: Optional[str]
error: Incompatible types in assignment (expression has type "None", variable has type "str")
* [Python] Make list type optional as they can contain None
Fixes an issue where exports were using incorrect relative paths for
>=3 namespace levels. This is fixed by making the starting range of the
namespace components relative to the amount of components.
Co-authored-by: Björn Harrtell <bjornharrtell@users.noreply.github.com>
Adding support for windows requires the code generations
to add a compiler statement to completely ignore GRPC code
generation on windows
Cleanup the project to use the main Package.swift to run tests
instead of having it separate and includes the imports for GRPC
within it.
Adds windows swift ci
* clang-format
* [Python] Replace . with _ in grpc filename suffix
Having filenames with . like `file.fb.grcp`
is not great for Python. Since dots are used for namespaces.
Replacing all of them with _ eg suffix `foo.bar.baz` will become
`foo_bar_baz`.
Restoring the previous default `_fb` suffix.
* [Python] Use namespace in path
This fixes a regression introduced with:
fb9afbafc7
And generates the grpc file in the namespace folder again.
* Sync commandline docs with web docs
* Offical Swift port for FlexBuffers
This is the offical port for FlexBuffers within
swift, and it introcudes a Common Module where code
is shared between flatbuffers and flexbuffers.
Writing most supported values like maps, vectors,
nil and scalars into a flexbuffer buffer. And includes
tests to verify that its similar to cpp
* Reading a flexbuffer
Implementing reading from a flexbuffer, enabling
most of the buffers features, like most types, maps, vectors,
typedvectors, and fixedtypedvectors.
Currently, if an offset/object cant be read we default to a swift
nil instead of the default flexbuffers 'null' with all values.
* Fixes bazel breaking due to new project structure
Address warnings within the library
* Adds comment on why we added the code & properly enforce the amout of bytes needed
For Vector or Array of structures the dereference operator of an
iterator returns the pointer to the structure. However, IndirectHelper,
which is used in the implementation of this operator, is instantiated
in the way that the IndirectHelper::Read returns structure by value.
This is because, Vector and Array instantiate IndirectHelper with
const T*, but VectorIterator instantiates IndirectHelper with T. There
are three IndirectHelper template definition: first for T, second for
Offset<T> and the last one for const T*. Those have different
IndirectHelper:Read implementations and (more importantly) return type.
This is the reason of mismatch in VectorIterator::operator* between
return type declaration and what was exactly returned.
That is, for Array<T,...> where T is scalar the VectorIterator is
instantiated as VectorIterator<T, T>, dereference operator returns T
and its implementation uses IndirectHelper<T> which Read function
returns T.
When T is not scalar, then VectorIterator is instantiated as
VectorIterator<T, const T *>, dereference operator returns const T * and
its implementation uses IndirectHelper<T> which Read function returns T.
The fix is done as follows:
* implement type trait is_specialization_of_Offset and
is_specialization_of_Offset64,
* change partial specialization of IndirectHelper with const T * that
it is instantiated by T and enabled only if T is not scalar and not
specialization of Offset or Offset64,
* remove type differentiation (due to scalar) from Array..
The above makes the IndirectHelper able to correctly instantiate itself
basing only on T. Thus, the instantiation in VectorIterator correctly
instantiate IndirectHelper::Read function, especially the return type.
Fixes access to union members when generating code with options "--cpp-field-case-style upper" and "--gen-object-api"
Co-authored-by: Wouter van Oortmerssen <aardappel@gmail.com>
* Adds new API to reduce memory copying within swift
Adds new storage container _InternalByteBuffer which
will be holding the data that will be created within the swift
lib, however reading data will be redirected to ByteBuffer, which
should be able to handle all types of data that swift provide without
the need to copy the data itself. This is due to holding a reference to
the data.
Replaces assumingMemoryBinding with bindMemory which is safer
Adds function that provides access to a UnsafeBufferPointer for
scalars and NativeStructs within swift
Updates docs
Suppress compilation warnings by replacing var with let
Using overflow operators within swift to improve performance
Adds tests for GRPC message creation from a retained _InternalByteBuffer
* flatbuffers Rust reflection: replace num with num-traits
num crate is a wrapper over num-traits and a few other crates, that
reexports the APIs from all of them. We only need num-traits.
Signed-off-by: Marcin Radomski <dextero@google.com>
* Rust reflection: drop dependency on stdint crate
We only use it to get intmax_t for deriving alignment, which is an alias
for `core::ffi::c_long` [1]. We can use that directly instead.
[1] https://docs.rs/stdint/1.0.0/stdint/type.intmax_t.html
Signed-off-by: Marcin Radomski <dextero@google.com>
* Rust reflection: drop dependency on escape_string crate
It's used to format a string used for debugging only, so we might as
well use the builtin Debug representation of a string.
Signed-off-by: Marcin Radomski <dextero@google.com>
* Rust codegen: add derives on generated bitflags
Otherwise it limits the use of structs generated for reflection.fbs
in Rust reflection API.
Signed-off-by: Marcin Radomski <dextero@google.com>
* Rust flatbuffers: update bitflags dependency to 2.8
Signed-off-by: Marcin Radomski <dextero@google.com>
* Rust codegen: use bitflags v2 API for converting from bits
from_bits_unchecked was replaced with safe from_bits_retain.
Signed-off-by: Marcin Radomski <dextero@google.com>
* Regenerate Rust code after idl change
Signed-off-by: Marcin Radomski <dextero@google.com>
* Regenerate reflection_generated.rs
With flatc --rust ../../../reflection/reflection.fbs
Signed-off-by: Marcin Radomski <dextero@google.com>
* ts/BUILD.bazel: add missing import
Found by Buildifire presubmit:
Function "sh_binary" is not global anymore and needs to be loaded from
"@rules_shell//shell:sh_binary.bzl".
Signed-off-by: Marcin Radomski <dextero@google.com>
* Update expected value in generated_code_debug_prints_correctly test
In bitflags v2, the debug string representation of enum values is
different than it was in v1:
Blue -> Color(Blue)
(empty) -> LongEnum(0x0)
This change adjusts the expected test value.
Signed-off-by: Marcin Radomski <dextero@google.com>
* Fix tests build on Swift 5.8
grpc-swift 1.4.1 depends on swift-nio-ssl 2.14.0+ [1]. swift-nio-ssl 2.29.1
published on 2025-01-30, introduced some code [2] that uses a "switch
expression syntax" supported since Swift 5.9 [3]. Attempts to compile it with
Swift 5.8 cause build errors.
swift-nio-ssl project doesn't seem to support Swift 5.8. A commit from
2024-10-29 removes a "deprecated reference to a Swift 5.8 pipeline" [4].
swift-nio-ssl 2.29.0 is the last version that can be compiled with Swift
5.8. This commit pins it to that exact version.
[1] 66e27d7e84/Package.swift (L33)
[2] 3cb4d5ad12 (diff-bc1db1321ff689c2819245dcce1a3080554f0fc13f81b8d326c97e7d42717c8fR54)
[3] https://github.com/swiftlang/swift-evolution/blob/main/proposals/0380-if-switch-expressions.md
[4] 8a6b89d9a4
---------
Signed-off-by: Marcin Radomski <dextero@google.com>
Co-authored-by: Marcin Radomski <dextero@google.com>
This setup is much simpler than calling Bazel from within Bazel
and making sure files and flags are set up correctly.
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Add missing file to filegroup for bazel integration tests
Fixup after a9257b6963.
* Align versions in bazel_respository_test_dir with root
* Update XCode version to 15.2
This is the oldest available version.
* Add WORKSPACE.bzlmod
* Add support for Bazel 7
* Add support for Bazel 8 in CI
* Restrict visibility of exported file
* Align npm_translate_lock attrs
* Remove defs_bzl_filename attr
* Align root_package with pnpm-lock.yaml location
Use a symlink to avoid copying the file.
* Update Buildkite Bazel CI
Restructure presubmit.yml to support matrix.
* Remove testing Ubuntu 18.04
The available LLVM 6.0 is too old to support std::filesystem.
* Add testing in Ubuntu 22.04
* Use Bazel version 6.5.0 in integration test
We need to copy the .fbs files into the package used for .bfbs files.
This is necessary as flatc doesn't provide support to specify the full
output file name for an .fbs file in a different folder.
I tried OUTPUT_FILE env var but this doesn't seem to be honored by
flatc.
Transitive headers like array.h have not been available in the runtime_cc target causing the build to fail. Adding all public headers to make sure transitive headers of flatbuffers.h are available.
* #Rust Create a crate for reflection
* #Rust Add a crate for reflection tests and helper to access schema
* #Rust Get root table of a buffer and access field with schema
* #Rust Add 'Struct' struct and corresponding getter
* #Rust Add functions of getting any table/struct field value as integer/float/string
* #Rust Add setters for scalar fields
* #Rust Add setter for string fields
* #Rust Add getter for Table/Vector fields
* #Rust Add buffer verification
* Add a 'SafeBuffer' struct which provides safe methods for reflection
It verifies buffer against schema during construction and provides all the unsafe getters in lib.rs in a safe way
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Improves vectors performance and adds a benchmark to vectors of offsets in swift
Improves performance for all arrays and for loops
Uses a tuple instead of allocating a struct each time we start iterating over fieldloc
Updates benchmark library
* Fixing swift Wasm ci
* Update grpc-core version
io.grpc:grpc-core package in version 1.36.0 contains multiple [CVE's](https://mvnrepository.com/artifact/io.grpc/grpc-core/1.36.0).
Bump grpc-core version to latest 1.68.0 version to mitigate potential vulnerabilities.
* Update grpc version to 1.67.1
grpc was mistakenly released to maven under version 1.68.0 whenever a real release was done for version 1.67.1 [1]. The mistake was fixed later.
[1] https://github.com/grpc/grpc-java/releases
Fix CVE-2022-25647
The package com.google.code.gson:gson before 2.8.9 is vulnerable to Deserialization of Untrusted Data via the writeReplace() method in internal classes, which may lead to denial of service attacks.
Bump up version of the gson package.
https://github.com/advisories/GHSA-4jrv-ppp4-jm57
This allows this type to meet the requirements of e.g.
std::ranges::range, which is necessary for it to work with the
std::span range constructor, or the "non-legacy" constructor for
Chromium's base::span.
Bug: none
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Update NativeObject.swift
Correct the word.
* Update ByteBuffer.swift
Type parameter does not existing, remove it.
* Update ByteBuffer.swift
Correct the word.
In case when flatbuffers are being used along with other project that defines "max" preprocessor macro, the ::max() in FLATBUFFERS_MAX_BUFFER_SIZE and FLATBUFFERS_MAX_64_BUFFER_SIZE is incorrectly being expanded to the macro. Adding parentheses enforces function-like interpretation.
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Move `namer.h` and `idl_namer.h` to `include/codegen` so they can be reused from `grpc` dirqectory.
* [gRPC] Update the Python generator to produce typed handlers and Python stubs if requested.
* [gRPC] Document the newly added compiler flags.
This allows enums to be type check with mypy.
They will still behave like ints ->
> IntEnum is the same as Enum,
> but its members are also integers and can be used anywhere
> that an integer can be used.
> If any integer operation is performed with an IntEnum member,
> the resulting value loses its enumeration status.
https://docs.python.org/3/library/enum.html#enum.IntEnum
Only if the --python-typing flag is set.
* [Python] Generate `.pyi` stub files when `--python-typing` is on.
To support this change, the following modifications were made:
- added a new option to disable `numpy` helpers generation;
- added a new flag to control the target Python version:
`--python-version` can be one of the following:
- `0.x.x` – compatible with any Python version;
- `2.x.x` – compatible with Python 2;
- `3.x.x` – compatible with Python 3.
- added codegen utilities for Python;
- added a note that the generated .py file is empty.
* [C++] Update the validator to skip structs in namespaces other than the current one.
Updates copyright from 2023 to 2024 & formats code - updates formatting rules
Updates CI to run with swift 5.8
Adds wasmer & updates command to run carton as a swift plugin
Update bazelci to also accept swift 5.8
Adds swift 5.10 to the test matrix
* [Python] Generate `.pyi` stub files when `--python-typing` is on.
To support this change, the following modifications were made:
- added a new option to disable `numpy` helpers generation;
- added a new flag to control the target Python version:
`--python-version` can be one of the following:
- `0.x.x` – compatible with any Python version;
- `2.x.x` – compatible with Python 2;
- `3.x.x` – compatible with Python 3.
- added codegen utilities for Python;
- added a note that the generated .py file is empty.
* [Python] Update Bazel build rules.
* [Python] Update Bazel build rules.
* [Python] Run buildifier on BUILD.bazel files.
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
This will allow the code to be compiled with `-Wfloat-equal`
as this would result in the folowing warning/error:
vendor/flatbuffers/include/flatbuffers/base.h:465:69:
error: comparing floating point with == or != is unsafe [-Werror,-Wfloat-equal]
template<typename T> inline bool IsTheSameAs(T e, T def) { return e == def; }
But the way it is used in flatbuffers it is ok to compare floating
points with ==.
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Fixes Bazel issues for windows and ci
Fetching boringssl within the flatbuffers repository, to patch the issues
of not being able to upgrade to Xcode 14.3 due to buildkite throwing
errors. The patch was inspired by the tenserflow patch
https://github.com/tensorflow/tensorflow/issues/60191#issuecomment-1496073147
Removes references of swift from the windows pipeline for bazel
Sets github actions to use xcode 14.3 for kotlin and sets the macOS
build for intel cpus.
* Update build.yml
Remove comment that is not relevant any longer.
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Update build.yml to ubuntu-24.04
Apparently g++-13 was removed from the ubuntu-22.04 runners.
We also don't have enterprise runners at 24.04 yet, so just use the free ones for now until we get support for those. CI builds might take longer now.
* Update build.yml
Downgrade to g++12 and revert change to using normal runners
* Update build.yml
Go back to ubuntu-24.04 and update both gcc and clang to their latest versions according to [this](https://github.com/actions/runner-images/blob/main/images/ubuntu/Ubuntu2404-Readme.md?plain=1#L16-L20).
* Update build.yml
Go back to g++13 for now, as we get some exotic warning in g++14 for newer C++ standards.
* Update build.yml
Fix the other issues with `macos-latest` going to arm: https://github.com/actions/runner-images/tree/main?tab=readme-ov-file#available-images and that Swift wasn't installed in the ubuntu-24.04 by default.
* Update build.yml
Disable Kotlin MacOs CI
The new options are:
- `--grpc-filename-suffix` controls the suffix of the generated files;
- `--grpc-use-system-headers` controls the type of C++ includes generated;
- `--grpc-search-path` controls the directory that contains gRPC runtime;
- `--grpc-additional-header` allows to provide additional dependencies for the generated code.
* [Python] Fix various codegen problems.
This includes:
- escaping keywords happens **after** converting the case:
- currently, `table ClassT` generate `class = Class()` which is invalid Python;
- imports in `one_file` mode use the filename rather than the type name when resolving module names;
- use `filename_suffix` instead of the hardcoded `_generated` one;
- generate empty files if no structs or enums are available. This makes the set of output files more predictable for Bazel.
* [Python] Fix various codegen problems.
This includes:
- escaping keywords happens **after** converting the case:
- currently, `table ClassT` generate `class = Class()` which is invalid Python;
- imports in `one_file` mode use the filename rather than the type name when resolving module names;
- use `filename_suffix` instead of the hardcoded `_generated` one;
- generate empty files if no structs or enums are available. This makes the set of output files more predictable for Bazel.
When flatbuffers is being used from a project that has no use for
JavaScript, users encounter an error similar to the following:
ERROR: Skipping '@com_github_google_flatbuffers//:flatbuffers': error loading package '@com_github_google_flatbuffers//': Unable to find package for @npm//:defs.bzl: The repository '@npm' could not be resolved: Repository '@npm' is not defined.
WARNING: Target pattern parsing failed.
ERROR: error loading package '@com_github_google_flatbuffers//': Unable to find package for @npm//:defs.bzl: The repository '@npm' could not be resolved: Repository '@npm' is not defined.
INFO: Elapsed time: 0.023s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
currently loading: @com_github_google_flatbuffers//
That's not ideal. Users that only care about C++ for example
shouldn't be forced to deal with rules_js and friends.
This patch attempts to fix that by moving the rules_js-specific things
into the `ts` and `tests/ts` directories. This should allow
non-JavaScript projects to ignore rules_js and friends completely.
Here I basically followed the `rules_foo` example from rules_js:
https://github.com/aspect-build/rules_js/tree/main/e2e/rules_foo
The idea is that flatbuffers has its own npm dependencies regardless
of what other projects may have. This means we should not force the
user to import flatbuffers's npm dependencies. The new
`ts/repositories.bzl` file is used by dependents to import
flatbuffers's dependencies. They can still import their own
dependencies. This cleanup allowed me to move all
JavaScript-specific stuff from the top-level directory into
subdirectories.
There should be no changes in this patch in terms of functionality.
It's just a refactor of the rules_js call sites. Users will have to
add a call to the function in `ts/repositories.bzl` in their own
`WORKSPACE` file. They can use
`tests/ts/bazel_repository_test/WORKSPACE` as an example.
Co-authored-by: Derek Bailey <derekbailey@google.com>
If a schema contains a message named e.g. FooT and a message named Foo
while the Object API suffix is T, then two classes with colliding names
will be generated. This scenario will produce a C++ compiler error, but
it's confusing.
This patch moves the error to the compiler, allowing the user to more
readily act to correct the issue.
Co-authored-by: Michael Beardsworth <beardsworth@intrinsic.ai>
Fix "One Definition Rule" violation when using flatbuffers::Verifier with
FLATBUFFERS_TRACK_VERIFIER_BUFFER_SIZE defined in some compilation units
and not defined in other compilation units.
The fix is to make Verifier a template class, with a boolean template
parameter replacing the "#ifdef" conditionals; to rename it as
VerifierTemplate; and then to use "#ifdef" only for a "using" declaration
that defines the original name Verifier an an alias for the instantiated
template. In this way, even if FLATBUFFERS_TRACK_VERIFIER_BUFFER_SIZE is
defined in some compilation units and not in others, as long as clients
only reference flatbuffers::Verifier in .cc files, not header files, there
will be no ODR violation, since the only part whose definition varies is the
"using" declaration, which does not have external linkage.
There is still some possibility of clients creating ODR violations
if the client header files (rather than .cc files) reference
flatbuffers::Verifier. To avoid that, this change also deprecates
FLATBUFFERS_TRACK_VERIFIER_BUFFER_SIZE, and instead introduces
flatbuffers::SizeVerifier as a public name for the template instance with
the boolean parameter set to true, so that clients don't need to define
the macro at all.
* Reproduce the error in a unit test
Reproduces #8200
* Overload KeyCompareWithValue to work for string-like objects
This fixes#8200.
* Extra tests
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
It appears the upgrade to xcode 14.3 broke the macos build on builkite.
The last good build was using xcode 14.2, so go back to this version
until the issue is resolved.
When building with make, it was failing for me because the target grpctext doesn't exist. I strongly assume this was meant to be grpctest.
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Add version of push which takes ContiguousBytes
* Ensure overloads aren't ambiguous
* Add version of createVector
* Add version of push which takes ContiguousBytes
* Ensure overloads aren't ambiguous
* Add version of createVector
* Add similar conditional to other use of ContiguousBytes
* Attempt CI fix
* Use memcpy instead of copyMemory
memcpy is faster in tests
* Add testContiguousBytes
* Add benchmarks
* Add version of createVector
* Add benchmarks
* Update push to copy memory
Since we don't care about endianness, we can simply memcpy the array of scalars
* Remove function and benchmarks
Since we don't care about endianness, a FixedWidthInteger version of createVector isn't needed
* Improve naming
* Add doc comment
* Fix C/C++ Create<Type>Direct with sorted vectors
If a struct has a key the vector has to be sorted. To sort the vector
you can't use "const".
* Changes due to code review
* Improve code readability
* Add generate of JSON schema to string to lib
* option indent_step is supported
* Remove unused variables
* Fix break in test
* Fix style to be consistent with rest of the code
* [TS] Fix reserved words as arguments (#6955)
* [TS] Fix generation of reserved words in object api (#7106)
* [TS] Fix generation of object api
* [TS] Fix MakeCamel -> ConvertCase
* [C#] Fix collision of field name and type name
* [TS] Add test for struct of struct of struct
* Update generated files
* Add missing files
* [TS] Fix query of null/undefined fields in object api
* Generate only files for comiled fbs (not for dependend ones)
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Update build.yml
Use our enterprise runners
* Make a default runs-on
* Update build.yml
Use the latest 64-core runners
* Update build.yml
Fix windows runner that don't have visual studios
* Update build.yml
use windows-2019 as the 2022 doesn't seem to have visual studios installed
* Update build.yml
Upgrade to gcc 13 and clang 15
* switch to __is_trivially_copyable
* fix cmake issue and warning about sign comparison
* Use libc++ for C++23 on clang for now
* Use libc++ for C++23 on clang for now
* exclude clang+15 for C++13 builds
Addresses a warning on xcode 15 regarding copying a pointer without safeguards
Address PR comments regarding initializing buffers with flag
Adds a test case for copying unaligned buffers
Formatting code
`buildifier` was complaining as follows:
#### :bazel: buildifier: found 2 lint issues in your WORKSPACE, BUILD and *.bzl files
<pre><code>tests/ts/bazel_repository_test_dir/BUILD:3:1: <a href="https://github.com/bazelbuild/buildtools/blob/master/WARNINGS.md#out-of-order-load">out-of-order-load</a>: Load statement is out of its lexicographical order.
ts/BUILD.bazel:2:1: <a href="https://github.com/bazelbuild/buildtools/blob/master/WARNINGS.md#out-of-order-load">out-of-order-load</a>: Load statement is out of its lexicographical order.</pre></code>
This can be fixed locally like so:
$ buildifier -lint fix $(git ls-files | grep -e '/BUILD.bazel$' -e '/BUILD$' -e '\<WORKSPACE$')
I also took this opportunity to fix one of the filenames.
I accidentally introduced these errors in #8078.
If you used flatbuffers_ts_library with gen_reflections = True then it
attempted to use the flat-file compiler rather than flatc itself.
Co-authored-by: Derek Bailey <derekbailey@google.com>
* [Kotlin] Small optimizations and benchmark on deserialization
* [Kotlin] Remove redudant assign() method (use init() instead)
* [Kotlin] Fix benchmark run after change in flatbuffers-java deps
Commit 6e214c3a49 fixes Kotlin build,
but makes the kotlin-benchmark plugin misses the java classes at
runtime, causing NotClassFoundError. The alternative to solve the issue
is to read java's pom.xml to get the latest java version and use it
as dependency. With that we avoid compilation errors on a new version and
keep benchmark plugin happy.
By using specific jar version to use java's runtime on the benchmark
module we let CI break when new versions are released. So we are using
source directly instead
* [Kotlin] Introduction to Kotlin Multiplaform
The first implementation of the Kotlin code generation was made years
ago at the time Kotlin Multiplaform was not stable and Kotlin is mostly
used on JVM-based targets. For this reason the generated code uses java
based runtime.
That design decision comes with many drawbacks, leaving the code
generated more java-like and making it impossible to use more advanced
features of the Kotlin language.
In this change we are adding two parts: A pure, multi-plaform, Kotlin
runtime and a new code generator to accompany it.
* [Kotlin] Remove scalar sign cast from code generation
Now that we have a new runtime the accepts unsigned types, we don't
need to code generate casting back and from signed scalars. This
MR removes this from both code generations and adds the necessary
API to the runtime.
* [Kotlin] Use offset on public API to represent buffer position
Currently, kotlin was following Java's approach of representing objects,
vectors, tables as "Int" (the position of it in the buffer). This change
replaces naked Int with Offset<T>, offering a type-safe API. So,
instead of
fun Table.createTable(b: FlatBufferBuilder, subTable: Int)
We will have
fun Table.createTable(b: FlatBufferBuilder, subTable: Offset<SubTable>)
Making impossible to accidentally switch parameters.
The performance should be similar to use Int as we are using value
class for Offset and ArrayOffset, which most of the time translate to
Int in the bytecode.
* [Kotlin] Add builder for tables
Add builder constructor to make create of table more ergonomic.
For example the movie sample for the test set could be written as:
Movie.createMovie(fbb,
mainCharacterType = Character_.MuLan,
mainCharacter = att) {
charactersType = charsType
this.characters = characters
}
instead of:
Movie.startMovie(fbb)
Movie.addMainCharacterType(fbb, Character_.MuLan)
Movie.addMainCharacter(fbb, att as Offset<Any>)
Movie.addCharactersType(fbb, charsType)
Movie.addCharacters(fbb, charsVec)
Movie.endMovie(fbb)
* [Kotlin] Move enum types to value class
Moving to flatbuffer enums to value class adds type safety for parameters
with minimum to no performance impact.
* [Kotlin] Simplify Union parameters to avoid naked casting
Just a small change on the APIs that receive union as parameters,
creating a typealias UnionOffset to avoid using Offset<Any>. To "convert"
an table offset to an union, one just call Offset.toUnion().
* [Kotlin] Apply clang-format on kotlin code generators
* [Kotlin] Update kotlin generator to follow official naming conventions
Updating directory, package and enum naming to follow Kotlin official
convention.
https://kotlinlang.org/docs/coding-conventions.html#naming-rules
* [Kotlin] Add fixes to improve performance
1 - Add benchmark comparing serialization between Java & Kotlin
2 - ReadWriteBuffer does not auto-grow (thus avoid check size in every op)
3 - Add specialized add functions on FlatBufferBuilder to avoid boxing
offsets.
4 - Remove a few Kotlin syntax sugar that generated performance penalties.
* [Kotlin] Remove builder from Kotlin KMP and add some optimizations
to avoid boxing of Offset classes
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
The test was not actually invoking the bazel that was downloaded with
the `http_file` rule. I failed to add `executable = True` to the
`http_file` call. This caused the test to ignore that bazel binary and
went to the next one on the system.
This patch fixes the issue by adding the missing attribute. Also, this
patch changes the check in the test to make sure that the downloaded
file is indeed executable.
* First working hack of adding 64-bit. Don't judge :)
* Made vector_downward work on 64 bit types
* vector_downward uses size_t, added offset64 to reflection
* cleaned up adding offset64 in parser
* Add C++ testing skeleton for 64-bit
* working test for CreateVector64
* working >2 GiB buffers
* support for large strings
* simplified CreateString<> to just provide the offset type
* generalize CreateVector template
* update test_64.afb due to upstream format change
* Added Vector64 type, which is just an alias for vector ATM
* Switch to Offset64 for Vector64
* Update for reflection bfbs output change
* Starting to add support for vector64 type in C++
* made a generic CreateVector that can handle different offsets and vector types
* Support for 32-vector with 64-addressing
* Vector64 basic builder + tests working
* basic support for json vector64 support
* renamed fields in test_64bit.fbs to better reflect their use
* working C++ vector64 builder
* Apply --annotate-sparse-vector to 64-bit tests
* Enable Vector64 for --annotate-sparse-vectors
* Merged from upstream
* Add `near_string` field for testing 32-bit offsets alongside
* keep track of where the 32-bit and 64-bit regions are for flatbufferbuilder
* move template<> outside class body for GCC
* update run.sh to build and run tests
* basic assertion for adding 64-bit offset at the wrong time
* started to separate `FlatBufferBuilder` into two classes, 1 64-bit aware, the other not
* add test for nested flatbuffer vector64, fix bug in alignment of big vectors
* fixed CreateDirect method by iterating by Offset64 first
* internal refactoring of flatbufferbuilder
* block not supported languages in the parser from using 64-bit
* evolution tests for adding a vector64 field
* conformity tests for adding/removing offset64 attributes
* ensure test is for a big buffer
* add parser error tests for `offset64` and `vector64` attributes
* add missing static that GCC only complains about
* remove stdint-uintn.h header that gets automatically added
* move 64-bit CalculateOffset internal
* fixed return size of EndVector
* various fixes on windows
* add SizeT to vector_downward
* minimze range of size changes in vector and builder
* reworked how tracking if 64-offsets are added
* Add ReturnT to EndVector
* small cleanups
* remove need for second Array definition
* combine IndirectHelpers into one definition
* started support for vector of struct
* Support for 32/64-vectors of structs + Offset64
* small cleanups
* add verification for vector64
* add sized prefix for 64-bit buffers
* add fuzzer for 64-bit
* add example of adding many vectors using a wrapper table
* run the new -bfbs-gen-embed logic on the 64-bit tests
* remove run.sh and fix cmakelist issue
* fixed bazel rules
* fixed some PR comments
* add 64-bit tests to cmakelist
* Add binary schema reflection
* remove not-used parameter
* move logic from object API to base API
* forward declare
* remove duplicate code gen that was stompping on the edits
* reduce to just typedef generation
* fixed bazel rules to not stomp
* more bazel fixes to support additional generated files
* Migrate from rules_nodejs to rules_js/rules_ts (take 2)
This is the second version of patch #7923. The first version got
reverted because bazel query was failing:
$ bazel --nosystem_rc --nohome_rc query tests(set('//...')) except tests(attr("tags", "manual", set('//...')))
ERROR: Traceback (most recent call last):
File "/workdir/tests/ts/bazel_repository_test_dir/BUILD", line 6, column 22, in <toplevel>
npm_link_all_packages(name = "node_modules")
File "/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/ec321eb2cc2d0f8f91b676b6d4c66c29/external/npm/defs.bzl", line 188, column 13, in npm_link_all_packages
fail(msg)
Error in fail: The npm_link_all_packages() macro loaded from @npm//:defs.bzl and called in bazel package 'tests/ts/bazel_repository_test_dir' may only be called in bazel packages that correspond to the pnpm root package '' and pnpm workspace projects ''
This was happening because the `.bazelrc` file only added
`--deleted_packages` to the `build` command. We also need it for the
`query` command. This second version of the patch fixes that.
Original commit message:
This patch migrates the current use of rules_nodejs to the new rules_js.
rules_js is the intended replacement of rules_nodejs as per this note:
https://github.com/aspect-build/rules_js#relationship-to-rules_nodejs
> rules_js is an alternative to the build_bazel_rules_nodejs Bazel module
> and accompanying npm packages hosted in
> https://github.com/bazelbuild/rules_nodejs, which is now
> unmaintained. All users are recommended to use rules_js instead.
There are a few notable changes in this patch:
1. The `flatbuffer_ts_library` macro no longer accepts a `package_name`
attribute. This is because rules_js appears to manage the import
naming of dependencies via top-level `npm_link_package` targets.
Users will have to migrate.
2. I added a few more arguments to `flatbuffer_library_public()`. These
helped with exposing esbuild to `ts/compile_flat_file.sh`.
3. I pinned the version of `typescript` in `package.json` so that
rules_ts can download the exact same version. rules_ts doesn't know
what to do if the version isn't exact.
4. Since rules_js uses the pnpm locking mechanism, we now have a
`pnpm-lock.yaml` file instead of a yarn lock file.
4. I added bazel targets for a few of the existing tests in `tests/ts`.
They can be run with `bazel test //test/ts:all`. Since there is no
flexbuffers bazel target, I did not add a bazel target for the
corresponding test.
5. I added a separate workspace in `tests/ts/bazel_repository_test_dir/`
to validate that the flatbuffers code can be imported as an external
repository. You can run the test with
`bazel test //test/ts:bazel_repository_test`. For this to work, I
needed to expose a non-trivial chunk of the flatbuffers code to the
test. I achieved this through some recursive `distribution`
filegroups. This is inspired by rules_python's workspace tests.
I did not do anything special to validate that the `gen_reflections`
parameter works the same. This patch doesn't change anything about
the TypeScript generation.
As a side note: I am not an expert with rules_js. This patch is my
attempt based on my limited understanding of the rule set.
Fixes#7817
* Fix the query
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Start using pnpm
* Add @npm
* get more stuff set up
* Get the analysis phase passing.
* Get esbuild working?
* Get it compiling?
$ bazel build //tests/ts/...
* Try to get the test working
* test is passing
* Get the other tests working
* clarify comment
* clean up a bit
* Try to add another test
* Add another test
* clean up more
* remove unused reference
* Add e2e test
* Get more of the test working
* add lock file
* Get test working on its own
* Get e2e test passing
* fix infinite recursion
* Add comments
* clean up some more
* clean up more again
* Source typescript version from package.json
* run buildifier
* lint
* Fix unset `extra_env`
* Incorporate feedback
* run buildifier
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
absl::string_view is uses std::string_view when available. It already checks if std::string_view is available in the earlier code.
It should only use absl::string_view implementation.
Co-authored-by: Derek Bailey <derekbailey@google.com>
* optionally generate type prefixes and suffixes for python code
* fix codegen error when qualified name is empty
* WIP: Python typing
* more progress towards python typing
* Further iterate on Python generated code typing
* clang-format
* Regenerate code
* add documentation for Python type annotations option
* generate code with Python type annotations
* handle forward references
* clang-format
This change allows user to decode binary with given schema to JSON
representation when schema defines union with struct.
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Fix C/C++ Create<Type>Direct with sorted vectors
If a struct has a key the vector has to be sorted. To sort the vector
you can't use "const".
* Changes due to code review
* Improve code readability
* Add generate of JSON schema to string to lib
* option indent_step is supported
* Remove unused variables
* Fix break in test
* Fix style to be consistent with rest of the code
* [TS] Fix reserved words as arguments (#6955)
* [TS] Fix generation of reserved words in object api (#7106)
* [TS] Fix generation of object api
* [TS] Fix MakeCamel -> ConvertCase
* [C#] Fix collision of field name and type name
* [TS] Add test for struct of struct of struct
* Update generated files
* Add missing files
* [TS] Fix query of null/undefined fields in object api
* Add .Net verfier
* Add some fuzz tests for .Net
* Remove additional files
* Fix .net test
* Changes due to PR
* Fix generated files
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
PyPI has three capital letters. See the front page of the service:
https://pypi.org/
"The Python Package Index (PyPI) ..."
Update the Python link under "Supported programming languages"
Co-authored-by: Michael Le <michael.le647@gmail.com>
ToCamelCase(input, true) converts first char to upper case, but
ToCamelCase(input, false) keeps the case of the first char. We are
changing its behavior to force a lower case.
Co-authored-by: Derek Bailey <derekbailey@google.com>
Shorten the PR staleness from 6 months to 3 weeks + 1 week notice. PRs become much harder to deal with the old they become due to merge conflicts and divergence.
Updated to stale@v7.0.0
The distributions for C++ and Python include the generated reflection
bindings but are currently missing from the other language packages.
This will bring the Java package generated for releases closer to
feature parity with the C++ and Python release artifacts.
* TS/JS: Export object based classes on entry
Along with the non object ones, for consistency. This is a regression
introduced recently.
Before:
`export { UpdateSettingsRequest } from './worker/update-settings-request.js';`
Now:
`export { UpdateSettingsRequest, UpdateSettingsRequestT } from './worker/update-settings-request.js';`
* only export object based classes for structs
Enums are not elegible.
---------
Co-authored-by: Björn Harrtell <bjornharrtell@users.noreply.github.com>
Co-authored-by: Derek Bailey <derekbailey@google.com>
* [Android] fixed build after decomission of jcenter
JCenter[1] has been removed and now is failing android build. This
change updates the configuration to remove this and few other warnings.
1 - https://developer.android.com/studio/build/jcenter-migration
* [Kotlin] fix build for latest gradle version 8.0.1
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
The generation of the library interface supplied by this function only
works within the same directory as that the target was defined. By
adding a custom target named GENERATE_<TARGET> now also interface files
will be generated by making a target dependend on the generate target.
Example:
/CMakeLists.txt
set(MY_INCL_SRC ${CMAKE_CURRENT_SOURCE_DIR}/fbs/my_incl.fbs)
flatbuffers_generate_headers(TARGET my_incl
SCHEMAS ${MY_INCL_SRC})
add_subdirectory(app)
/app/CMakeLists.txt
add_executable(app src/test.cpp)
target_link_libraries(app my_incl)
add_dependencies(app GENERATE_my_incl)
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Fix C/C++ Create<Type>Direct with sorted vectors
If a struct has a key the vector has to be sorted. To sort the vector
you can't use "const".
* Changes due to code review
* Improve code readability
* Add generate of JSON schema to string to lib
* option indent_step is supported
* Remove unused variables
* Fix break in test
* Fix style to be consistent with rest of the code
* [TS] Fix reserved words as arguments (#6955)
* [TS] Fix generation of reserved words in object api (#7106)
* [TS] Fix generation of object api
* [TS] Fix MakeCamel -> ConvertCase
* [C#] Fix collision of field name and type name
* [TS] Add test for struct of struct of struct
* Update generated files
* Add missing files
* [TS] Fix query of null/undefined fields in object api
* Fix collision if field name is equal to table name and used as key in an array
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Add code generator for proto files
* Update
* Add --proto to script
* Remove cmt
* Move proto parsing logic into else block to share same set up logic for code_generator
* Remove IsValidCodeGenerator
If flatbuffers is built in C++11 mode, but there is a recent version of
absl which requires C++14, the build will fail.
Cf https://github.com/MapServer/MapServer/issues/6822 for the use case
that triggered this.
* Parsing from proto should keep field ID. (fixes#7645)
* Fix failed tests
* Fix windows warning
* Improve attribute generation in proto to fbs
* Check if id is used twice. fix Some clang-format problems
* Test if fake id can solve the test problem
* Validate proto file in proto -> fbs generation.
* Fix error messages
* Ignore id in union
* Add keep proto id for legacy and check gap flag have been added. Reserved id will be checked.
* Add needed flags
* unit tests
* fix fromat problem. fix comments and error messages.
* clear
* More unit tests
* Fix windows build
* Fix include problems
* Fake commit to invoke rebuild
* Fix buzel build
* Fix some issues
* Fix comments, fix return value and sort for android NDK
* Fix return type
* Break down big function
* Place todo
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
The current detection method fails on GCC 12.2 with -std=c++20 because
the __cpp_lib_span macro is undefined.
As per https://en.cppreference.com/w/cpp/utility/feature_test ,
__cpp_lib_span requires including either <version> or <span>.
Since both these headers were added in C++20, checking for C++20 is
sufficient (and simpler than using the library feature-test macro).
Signed-off-by: Bernie Innocenti <bernie@codewiz.org>
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Refactor to use CodeGenerator interface.
- Move code to its own header file to be included in flatc_main.cpp
- Refactor code to use CodeGenerator interface for all languages
* Format all files
* remove lua code generator since it doesn't support bfbs generator
* Update CMakeLists file with new idl_gen_*.cpp and idl_gen_*.h files
* Add idl_gen_swift header file
* Add idl_gen_swift header file and update bazel file
* Remove CodeGenerator interface for idl_gen_text.*. Remove comments and extern declaration
* Reorder header and implementation files in CMakeLists.txt
* Add idl_gen_* header files to implementation files
* Update CMakeLists and remove unused import
Co-authored-by: Derek Bailey <derekbailey@google.com>
* add unit tests for support struct as key
* make changes to parser and add helper function to generate comparator for struct
* implement
* add more unit tests
* format
* just a test
* test done
* rerun generator
* restore build file
* address comment
* format
* rebase
* rebase
* add more unit tests
* rerun generator
* address some comments
* address comment
* update
* format
* address comment
Co-authored-by: Wen Sun <sunwen@google.com>
Co-authored-by: Derek Bailey <derekbailey@google.com>
* [TS/JS] Entry point per namespace
* Fix handling of outputpath and array_test
* Attempt to fix generate_code
* Fix cwd for ts in generate_code
* Attempt to fixup bazel and some docs
* Add --ts-flat-files to bazel build to get bundle
* Move to DEFAULT_FLATC_TS_ARGS
* Attempt to add esbuild
* Attempt to use npm instead
* Remove futile attempt to add esbuild
* Attempt to as bazel esbuild
* Shuffle
* Upgrade bazel deps
* Revert failed attempts to get bazel working
* Ignore flatc tests for now
* Add esbuild dependency
* `package.json` Include esbuild
* `WORKSPACE` Add fetching esbuild binary
* Update WORKSPACE
* Unfreeze Lockfile
* Update WORKSPACE
* Update BUILD.bazel
* Rework to suggest instead of running external bundler
* Add esbuild generation to test script
* Prelim bundle test
* Run test JavaScriptTest from flatbuffers 1.x
* Deps upgrade
* Clang format fix
* Revert bazel changes
* Fix newline
* Generate with type declarations
* Handle "empty" root namespace
* Adjust tests for typescript_keywords.ts
* Separate test procedure for old node resolution module output
* Fix rel path for root level re-exports
* Bazel support for esbuild-based flatc
Unfortunately, we lose typing information because the new esbuild method
of generating single files does not generate type information.
The method used here is a bit hack-ish because it relies on parsing the
console output of flatc to figure out what to do.
* Try to fix bazel build for when node isn't present on host
* Auto formatting fixes
* Fix missing generated code
Co-authored-by: Derek Bailey <derekbailey@google.com>
Co-authored-by: James Kuszmaul <jabukuszmaul+collab@gmail.com>
* Fix binary output different in different platform, due to the nan serialization
* Add check generated code on windows ci
* Remove resdundant script
* Fix eof, and check script
* Minor bug in gen code script
* Fix windows script, remove redundant scripts
* Undelete redundante codes
* Fix github action
* Ignore eof generate grpc
Co-authored-by: Derek Bailey <derekbailey@google.com>
Comparing short strings, small integers, and Booleans by identity
(memory address) can work due to optimizations in the Python
interpreter, but it is neither formally correct nor reliable. Use
equality comparisons instead.
Co-authored-by: Derek Bailey <derekbailey@google.com>
It is deprecated in favour of importlib and slated for removal in Python
3.12. Since the return value of imp.find_module('numpy') is unused, the
only effect of calling this function is to raise an ImportError when
numpy is not available; importing numpy directly is already sufficient
to do this.
The imp package is still used in python/flatbuffers/compat.py, but only
on Python 2, where it is not deprecated and will not be removed.
Co-authored-by: Derek Bailey <derekbailey@google.com>
* [TS]: Fix vtable creation for consecutive required fileds (#7739)
* handle feedback
* comment the schema
* comment change in builder.ts
* [TS]: builder, Fix requiredField()
Verifty that the field is present in the vtable.
* restore monsterdata binary file
Co-authored-by: Derek Bailey <derekbailey@google.com>
* [Kotlin] Control the generation of reflection with --reflect-names.
Tested:
```
$ cmake -G "Unix Makefiles" && make && ./tests/flatc/main.py
...
KotlinTests.EnumValAttributes
[PASSED]
KotlinTests.EnumValAttributes_ReflectNames
[PASSED]
KotlinTests: 2 of 2 passsed
...
35 of 35 tests passed
```
* [Kotlin] Fix SampleBinary by converting Byte to UByte for ubyte fields.
* [Kotlin] Annotate all generated classes with kotlin.ExperimentalUnsignedTypes.
Since flatbuffers is using calendar versioning and does not provide
any ABI stability guarantees, use the complete version as SOVERSION
for the shared library rather than just the major component. This
prevents breaking reverse dependencies on incompatible upgrades.
Fixes#7759
* Fix [C#] Object API - Invalid Property Name used in UnPackTo for union fieldhttps://github.com/google/flatbuffers/issues/7750, also fixes invalid Code generated in WriteJson for Unions named Value.
* Test added: new schema union_value_collision.fbs with a Union named Value and a union field named value. The generated C# code now compiles when NetTest.bat. The Code generated with an older flatc.exe didn't compile because of a mismatch of the property name (Value vs. Value_).
* branch was not up-to-date with master
* BASE_OPTS + CPP_OPTS removed and union_value_collision_generated.h deleted
Co-authored-by: Derek Bailey <derekbailey@google.com>
Add the --no-minmax-values flag to prevent flatc from generating C++
enums with MIN and MAX enumerated values that otherwise would be set
to the inclusive lower and upper bound respectively of the enum.
This command-line flag is needed to avoid collisions when an enum that
is being ported to FlatBuffers already has a MIN or MAX enumerated
value.
It is also needed to work around a long-standing problem with
magic_enum that causes magic_enum to not see enumerated values that
are not unique. For example, if FlatBuffers sets MIN = FOO and MAX =
BAR, MIN and FOO share the same underlying value so they are not
unique. The same is true of MAX and BAR. This prevents magic_enum
from converting FOO and BAR to and from strings as well as causing
magic_enum to return a count of enumerated values that is two fewer
than it should be.
Co-authored-by: Paul Serice <paul@serice.net>
* [Kotlin] Only generate nullable return types if the field is not required
* [Kotlin] Fix generated code formatting according to kotlin style guide
Co-authored-by: Derek Bailey <derekbailey@google.com>
Co-authored-by: Paulo Pinheiro <paulovictor.pinheiro@gmail.com>
To make it simple to map between a union field and its union type
field we are adding a pointer to FieldDef to point to each other. For
all other types the pointer will be nullptr.
Co-authored-by: Derek Bailey <derekbailey@google.com>
#7451 caused getFullyQualifiedName to return a name with underscores,
not periods. Because the fully qualified name is a property of
FlatBuffers, not the language being codegen'd for, it should use
periods. Fixes#7564.
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Add clang-tidy, fix some bugpron problems.
* Fix more issues
* Fix some more issues :))
* Minimal pr to just add clang-tidy
Co-authored-by: Derek Bailey <derekbailey@google.com>
* create job to build linux and run unit test on s390x
* update
* update
* update
* update
* update
* print out machine type
* create regression test to build a big endian arch and run unit tests daily
* rename and schedule run on pr merged and on request
* udpate
Co-authored-by: Wen Sun <sunwen@google.com>
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Add Ref.AsStringBytes to flatbuffers.flexbuffers Python API
* Append Bytes to AsStringBytes return value
Co-authored-by: Jared Junyoung Lim <jaredlim@google.com>
* update unit test and generated file to test is extra endianswap can help resolve issue
* remove EndianScalar wrapper from Get method
* remove endianscalar wrapper
* update
* update
* use Array instead
* clang format
* address error
* clang
* update
* manually generate
* Move Nim to completed language
* Add swift link
* address comments
* update unit test
* address comment
* address comment
* regenerate file
* use auto instead of size_t
* use uint32_t instead
* update
* format
* delete extra whitespace
Co-authored-by: Wen Sun <sunwen@google.com>
Co-authored-by: Derek Bailey <derekbailey@google.com>
Without the change build fails on weekly `gcc-13` snapshots as:
In file included from /build/flatbuffers/tests/reflection_test.cpp:1:
tests/reflection_test.h:9:57: error: 'uint8_t' has not been declared
9 | void ReflectionTest(const std::string& tests_data_path, uint8_t *flatbuf, size_t length);
| ^~~~~~~
* Update release script to update Rust version (it still needs to be published after)
* Also update rust while I'm at it
Co-authored-by: Casper Neo <cneo@google.com>
* Add --go-module-name flag to support generating code for go modules
* Rename echo example folder
* Grammar
* Update readme for go-echo example
* Update readme for go-echo example
* Re-enable go modules after test is done
* Add support for key lookup for tables in Go
* Run clang format
* Run go fmt on tests
* Remove TODO in tests
* Update LookupByKey API
* Update LookupByKey API
* Don't use resolvePointer in expectEq
* Use generated getters instead of reading values directly from buffer
* Fix typo
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Fix go generator undefined Package, also throw exception in specific examples.
* Add test for go generator import problem
* Add new version of generated go file. Fix conflict.
* Add executable permission to generate_code.py script.
* Improve test quality, remove unwanted generated files, better naming
* Fix comments
* clang format
Co-authored-by: Derek Bailey <derekbailey@google.com>
The BytesConsumed function uses the `cursor_` to determine how many
bytes have been consumed by the parser, in case the user of the Parser
object wants to step over the parsed flatbuffer that is embedded in some
larger string. However, the `cursor_` is always one token ahead, so that
it can determine how to consume it. It points at the token that is about
to be consumed, which is ahead of the last byte consumed.
For example, if you had a string containing these two json objects and
parsed them...
"{\"key\":\"value\"},{\"key\":\"value\"}"
...then the `cursor_` would be pointing at the comma between the two
tables. If you were to hold a pointer to the beginning of the string and
add `BytesConsumed()` to it like so:
const char* json = // ...
parser.ParseJson(json);
json += parser.BytesConsumed();
then the pointer would skip over the comma, which is not the expected
behavior. It should only consume the table itself.
The solution is simple: Just hold onto a previous cursor location and
use that for the `BytesConsumed()` call. The previous cursor location
just needs to be set to the cursor_ location each time the cursor_ is
about to be updated. This will result in `BytesConsumed()` returning
the correct number of bytes without the off-by-one-token error.
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Fix C/C++ Create<Type>Direct with sorted vectors
If a struct has a key the vector has to be sorted. To sort the vector
you can't use "const".
* Changes due to code review
* Improve code readability
* Add generate of JSON schema to string to lib
* option indent_step is supported
* Remove unused variables
* Fix break in test
* Fix style to be consistent with rest of the code
* [TS] Fix reserved words as arguments (#6955)
* [TS] Fix generation of reserved words in object api (#7106)
* [TS] Fix generation of object api
* [TS] Fix MakeCamel -> ConvertCase
* [C#] Fix collision of field name and type name
* [TS] Add test for struct of struct of struct
* Update generated files
* Add missing files
* [TS] Fix query of null/undefined fields in object api
* [C#] Fix collision of member if enum name is "Value"
* Fix due to style guide
Co-authored-by: Derek Bailey <derekbailey@google.com>
* add support for using array of scalar as key field
* update cmakelist and test.cpp to include the tests
* update bazel rule
* address comments
* clang format
* delete comment
* delete comment
* address the rest of the commnets
* address comments
* update naming in test file
* format build file
* buildifier
* make keycomparelessthan call keycomparewithvalue
* update to use flatbuffer array instead of raw pointer
* clang
* format
* revert format
* revert format
* update
* run generate_code.py
* run code generator
* revert changes by generate_code.py
* fist run make flatc and then run generate_code.py
Co-authored-by: Wen Sun <sunwen@google.com>
* Fix C/C++ Create<Type>Direct with sorted vectors
If a struct has a key the vector has to be sorted. To sort the vector
you can't use "const".
* Changes due to code review
* Improve code readability
* Add generate of JSON schema to string to lib
* option indent_step is supported
* Remove unused variables
* Fix break in test
* Fix style to be consistent with rest of the code
* [TS] Fix reserved words as arguments (#6955)
* [TS] Fix generation of reserved words in object api (#7106)
* [TS] Fix generation of object api
* [TS] Fix MakeCamel -> ConvertCase
* [C#] Fix collision of field name and type name
* [TS] Add test for struct of struct of struct
* Update generated files
* Add missing files
* [TS] Fix query of null/undefined fields in object api
* Put documentation to bfbs if it is not empty
* Fix monster test bfbs reference files
* Fix generated monster test files
Why they are different when generating it with linux and windows executable?
* Fix import problem in dart generated files. (fixes#7609).
* Fix naming.
* Fix minor changes in generated files.
* Add some tests. Fix minor problems.
* Fix minor format problem plus import alias issue.
* Minor fix in dart code generator, remove java from examples
* remove java and go generated files
* Fix dart tests.
* Fix spell problem.
* Remove excessive tests :))
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Temporary fix for SLSA generators
Sigstore made a breaking change as part of their recent GA announcement. We need a temporary fix to avoid builder failure (see slsa-framework/slsa-github-generator#1163)
/cc @asraa
* Update build.yml
+/-inf were not being handled, and so invalid typescript was being
generated when a float/double had an infinite default value. NaN was
being handled correctly.
Co-authored-by: Derek Bailey <derekbailey@google.com>
Co-authored-by: Casper <casperneo@uchicago.edu>
* Add support for proto 3 map to fbs gen
* Run clang-format
* Update proto golden test
* Rename variables
* Remove iostream
* Remove iostream
* Run clang format
Co-authored-by: Derek Bailey <derekbailey@google.com>
Previously when parsing a JSON representation of a Flatbuffer, the
parser required that the input string contain one and only one root
table. This change adds a flag that removes that requirement, so that
if a Flatbuffer table is embedded in some larger string the parser will
simply stop parsing once it reaches the end of the root table, and does
not validate that it has reached the end of the string.
This change also adds a BytesConsumed function, which returns the number
of bytes the parser consumed. This is useful if the table embedded in
some larger string that is being parsed, and that outer parser needs to
know how many bytes the table was so that it can step over it.
* [TS] Add support for fixed length arrays on Typescript (#5864) (#7021)
* Typescript / Javascript don't have fixed arrays but it is important to support these languages for compatibility.
* Generated TS code checks the length of the given array and do truncating / padding to conform to the schema.
* Supports the both standard API and Object Based API.
* Added a test.
Co-authored-by: Mehmet Baker <mehmet.baker@zerodensity.tv>
Signed-off-by: Bulent Vural <bulent.vural@zerodensity.tv>
Signed-off-by: Bülent Vural <bulent.vural@zerodensity.tv>
* Formatting & readability fixes on idl_gen_ts.cpp
Signed-off-by: Bülent Vural <bulent.vural@zerodensity.tv>
* Added array_test_complex.bfbs
Signed-off-by: Bülent Vural <bulent.vural@zerodensity.tv>
* TS arrays_test_complex: Remove bfbs and use fbs directly
Signed-off-by: Bülent Vural <bulent.vural@zerodensity.tv>
Signed-off-by: Bülent Vural <bulent.vural@zerodensity.tv>
* feat: Fixed the issue with nested unions relying on InitFromBuf.
Problem: Issue #7569
Nested Unions were broken with the introduction of parsing buffers with an initial encoding offset.
Fix:
Revert the InitFromBuf method to the previous version and introduction of InitFromPackedBuf that allows
users to read types from packed buffers applying the offset automatically.
Test:
Added in TestNestedUnionTables to test the encoding and decoding ability using a nested table with a
union field.
* fix: Uncommented generate code command
Create a new `release.yml` for defining automatic publishing to package managers on releases.
Adds the NPM publishing steps as it is the most straightforward.
* Bfbs Nim Generator
* Remove commented out tests
* add missing line to idl.h
* Commit python reflection changes
* Commit python reflection changes and move tests
* Remove default string addition
* Move tests to python file
* Fix element size check when element is table
* remove whitespace changes
* add element_type docs and commit further to namer and remove kkeep
* Bfbs Nim Generator
* Remove commented out tests
* add missing line to idl.h
* Commit python reflection changes
* Commit python reflection changes and move tests
* Remove default string addition
* Move tests to python file
* Fix element size check when element is table
* remove whitespace changes
* add element_type docs and commit further to namer and remove kkeep
* remove unused variables
* added tests to ci
* added tests to ci
* fixes
* Added reflection type Field, Variable to namer
* Moved reflection namer impl to bfbsnamer
* Remove whitespace at end of line
* Added nim to generated code
* Revert whitespace removal
Co-authored-by: Derek Bailey <derekbailey@google.com>
As Java does not support unsigned integer types, the value types
are "rounded up" (an uint32 is represented as a long) but persisted
correctly (an uint32 is persisted as 4 bytes).
This CL makes a cast operation explicit so that the compiler
does not throw warning messages.
Co-authored-by: Dominic Battre <battre@chromium.org>
Co-authored-by: Derek Bailey <derekbailey@google.com>
The MyGame/Example/LongEnum.java class did not compile because
Java expects an "L" suffix for literals of type long.
This CL fixes the code generation to include such a suffix.
Co-authored-by: Dominic Battre <battre@chromium.org>
on a subset of platforms. The test calls are guarded via #ifndef
FLATBUFFERS_NO_FILE_TEST.
Embedders of flatbuffers that rely on -Werror,-Wunused-function
compiler flags (like chromium) complain about the exsitence of these
tests in the anonymous namespace.
This CL guards the test definitions as well (not just the test
calls).
Co-authored-by: Dominic Battre <battre@chromium.org>
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Add support for metadata attributes for enum values (#7567)
* Fix path lookup in flatc test
* Try a fix for Windows paths
* Convert path to string to fix Windows error
* feat: Added support for fixed sized arrays to python
Problem:
We encountered that using fixed arrays from C++ to python that python would
not read those arrays correctly due to no size information being encoded in the byte
array itself.
Fix:
Encode the sizes within the generated python file during code generation.
Specfically we add GetArrayAsNumpy to the python version of table, which takes as input
the length of the vector. When generating the python message files we include this length
from the VectorType().fixed_length.
* fix: added digit support for camel case to snake case conversion
Problem:
When including a number in the message name we would encounter cases where SnakeCase would
not add the appropirate breaks. e.g. Int32Stamped -> int_32stamped rather than int_32_stamped.
Fix:
To fix this we can add the condition that we check if the current character is not lower and
not a digit, that we check if the previous character was a lower or digit. If it was a lower
or digit then we add the break.
* fix: Array support for structures
Problem:
The python generated code for handling non-struct and struct vectors
and arrays was inconsistent. The calls to populate the obj api was
creating incorrect code.
Solution:
To fix this the VectorOfStruct and VectorOfNonStruct was rewritten
to handle array cases and bring the two methods in line which each
other.
Testing:
PythonTesting.sh now correctly runs and generates the code for
array_test.fbs.
Minor modifications were done on the test to use the new index
accessor for struct arrays and the script correctly sources the
location of the python code.
* chore: clang format changes
* Added code generated by scripts/generate_code. Modified GetArrayOfNonStruct slightly
to allow for function overloading allowing the user to get a single element of an array
or the whole array.
* Added new_line parameter to OffsetPrefix to allow optional new lines to be added.
This allows us to use the GenIndents method that automatically adds new lines instead.
* Reupload of generated code from the scripts/generate_code.py
* Removed new line in GetVectorAsNumpy.
* Updated Array lengths to use Length methods where possible. Added fallthrough for GenTypePointer. Added digit check to CamelToSnake method. Added and modified tests for ToSnakeCase and CamelToSnake.
* Added range check on the getter methods for vector and array types. Renamed == as is for python
* [C++] Add a failing unit test for #7516 (Rare bad buffer content alignment if sizeof(T) != alignof(T))
* [C++] Fix final buffer alignment when using an array of structs
* A struct can have an arbitrary size and therefore sizeof(struct) == alignof(struct)
does not hold anymore as for value primitives.
* This patch fixes this by introducing alignment parameters to various
CreateVector*/StartVector calls.
* Closes#7516
* Start of mvn-ification of the test
* move to right locations
* Update the IO done in the test to read from resources / write to temp folders
* Add github workflow attempt to mvn test it instead of JavaTest.sh
* Pin the Kotlin benchmark's symlink for /java to the right location
* Inline equality assertions and format JavaTest.java
* fix android gradle source directory
Co-authored-by: Derek Bailey <derekbailey@google.com>
Untyped fixed vectors are not supported in FlexBuffers. There
was an assert to check for it, but on java, asserts are optional. This
change converts the assertion into a runtime exception.
Fixes#7358
Co-authored-by: Derek Bailey <derekbailey@google.com>
* C++: Add option to skip verifying nested flatbuffers
Additionally, add an options struct to the verifier for those
who prefer designated initializers to default arguments. The former
constructor is defined in terms of the latter because in old c++,
having default values for members removes list initialization, making
defining constructors in the other way a lot more challenging to write.
* fixes
* fmt
* formatting, and remove an argument
* fix
Co-authored-by: Casper Neo <cneo@google.com>
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Add timing command to cmakelist
* Start to split test.cpp. Move flexbuffers tests
* Move monster related tests to own file
* Move parser related tests to own file
* Move json related tests to own file
* Move proto related tests to own file
* moved more functions to parser test
* moved more functions to parser test2
* move monster extra tests to monster test
* move evolution tests to own file
* move reflection tests to own file
* move util tests to own file
* rehomed various tests
* move optional scalars test to own file:
* rehome more tests
* move fuzz tests. Got rid of global test_data_path
* fixes for CI failures
* add bazel files
This fixes two problems:
1. The project could be build when using own tags which does not follow the pattern v<major>.<minor>.<patch>.
2. The case "tag points to the commit" will be handled correctly by setting VERSION_COMMIT to 0.
This commit resolves https://github.com/google/flatbuffers/issues/7472
Co-authored-by: Axel Sommerfeldt <axel.sommerfeldt@gmail.com>
Added (for compiler versions that support it):
-Wmissing-declarations
-Wzero-as-null-pointer-constant
Then, fixes to problems identified by the extra warnings
Tested only on GCC 9.4.0
Adjusted the CPP code generator to output nullptr where appropriate,
to satisfy -Wzero-as-null-pointer-constant
Added a lot of 'static' declarations in front of functions,
to satisfy -Wmissing-declarations,
and wrap static function defs in anonymous namespaces.
There are advantages to both anonymous namespaces and static,
it seems that marking a function as static will not publish the name in
the symbol table at all, thus giving the linker less work to do.
With a change introduce in 385dda5c3785ed8d6a35868bc169f07e40e889087fd2edc66,
flatc was not able to emit code for Kotlin if a namespace is specified
and the folders do not exist. The fix create folders if neded.
Additional changes are introduced in gradle files to bring more visibility
to the error messages.
Co-authored-by: Derek Bailey <derekbailey@google.com>
* [C++] Rename template parameter U in make_span of stl_emulation.h
CPPRESTSDK defines a U macro therefore, calls to U() are problematic.
Rename U to W to avoid conflict.
* [C++] Make typenames of span_convertable and make_span more expressive
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Vector of Tables equality
* support nullptr and fix for not being able to use auto in lambda
* use different std::equal overload
* use flatbuffers::unique_ptr
* go back to auto and clang-format fix
Change config.escape_keywords to AfterConvertingCase.
It avoids unecessay escaping since the generated native
structs have fields starting with a uppercase letter
and Go's keywords start with lowercase letters.
Co-authored-by: Derek Bailey <derekbailey@google.com>
* [golang] Add support for text parsing with json struct tags
Add struct tags to Go native structs generated when object API is used.
This allows to use the same JSON file as for C++ text
parsing(i.e. snake_case vs CamelCase) and thus enabling text parsing
for Go(when using object API).
* [golang] Add test for text parsing
Added small test to check and demonstrate text parsing in Go.
Also, ran clang-format on cpp file changes.
* grpc/compiler: Respect filename suffix and extension during code generation
grpc compiler is not respecting filename suffix and extension passed to
flatc CLI. This causes compiler to spit out incorrect code, which then
cannot be compiled without modification.
Following patch fixes the problem.
Note, I ended up removing some code introduced #6954 ("Have grpc include
file with correct filename-suffix given to flatc") in favour of keeping
sanity of the generator code.
Signed-off-by: Aman Priyadarshi <aman.eureka@gmail.com>
* tests: Add filename-suffix and filename-ext test files
* Test 1: Filename extension changed to "hpp".
* Test 2: Filename suffix changed to "_suffix".
* Test 3: Filename extension changed to "hpp" and suffix changed to "_suffix"
Signed-off-by: Aman Priyadarshi <aman.eureka@gmail.com>
* CMake find_package fixes (#7323)
Rename FlatbuffersConfigVersion.cmake to match CMake project name
* CMake find_package fixes (#7323)
Rename FlatBuffersTargets to match CMake project name
* grpc/compiler: Respect filename suffix and extension during code generation
grpc compiler is not respecting filename suffix and extension passed to
flatc CLI. This causes compiler to spit out incorrect code, which then
cannot be compiled without modification.
Following patch fixes the problem.
Note, I ended up removing some code introduced #6954 ("Have grpc include
file with correct filename-suffix given to flatc") in favour of keeping
sanity of the generator code.
Signed-off-by: Aman Priyadarshi <aman.eureka@gmail.com>
* tests: Add filename-suffix and filename-ext test files
* Test 1: Filename extension changed to "hpp".
* Test 2: Filename suffix changed to "_suffix".
* Test 3: Filename extension changed to "hpp" and suffix changed to "_suffix"
Signed-off-by: Aman Priyadarshi <aman.eureka@gmail.com>
Introduce a MSVC_LIKE variable in the CMake scripts, set that variable to
true only if the compiler is either MSVC or tries to emulate the MSVC
command line, and test that variable when setting compiler arguments.
Tested with cmake .. -G Ninja -DCMAKE_C_COMPILER:PATH="clang-cl.exe" -DCMAKE_CXX_COMPILER:PATH="clang-cl.exe" -DFLATBUFFERS_BUILD_TESTS=OFF -DFLATBUFFERS_BUILD_CPP17=ON
Co-authored-by: Kaiyi Li <kaiyili@google.com>
* Move reflection_ts_fbs into a separate directory (#7342)
Right now, reflection_ts_fbs target is in reflection/BUILD.bazel.
This is not ideal because reflection:reflection_fbs_schema is referenced
from :flatc in the root. Thus, for any Bazel projects that want to
include flatbuffers, they need to include npm / yarn_install and nodejs
support all because reflection/BUILD.bazel loads typescript.bzl and that
requires all TypeScript things.
This PR separated that target into a different subdirectory, freeing
root BUILD.bazel from that dependency.
* Minor improvements to typescript.bzl
* Uses @...// dependencies so that flatbuffers can actually
be used as an external repo (had forgotten to upstream this earlier).
* Allows using flatbuffer_ts_library to generate reflection schemas
in the same way that flatbuffer_cc_library does (but default it
to off to avoid behavioral changes).
* Pass through a package_name attribute to flatbuffer_ts_library to
allow non-relative imports of generated typescript code.
Co-authored-by: Liu Liu <i@liuliu.me>
Presently flatc generates a placeholder for the unused synchronous varient of client-side streaming gRPCs which includes the name of an unused parameter in the method.
To avoid warnings, comment out the parameter name (in a manner similar to other gRPCS).
The logic to manage generating typescript in a single file was
generating self imports and unused local names, which triggered some
linters.
This resolves#7191
There was a step in the compilation process where benchmark data is
downloaded before starting the kotlin compilation process.
Since we are not running benchmark on CI anymore, we remove the
dependency. To run benchmarks the download task needs to be executed
manually.
* Implement optional scalars for JSON
* Add optional scalars JSON test
* Extend JSON optional scalars test to test without defaults
* Fix optional scalars in JSON for binary schema
Co-authored-by: Caleb Zulawski <caleb.zulawski@caci.com>
Right now, reflection_ts_fbs target is in reflection/BUILD.bazel.
This is not ideal because reflection:reflection_fbs_schema is referenced
from :flatc in the root. Thus, for any Bazel projects that want to
include flatbuffers, they need to include npm / yarn_install and nodejs
support all because reflection/BUILD.bazel loads typescript.bzl and that
requires all TypeScript things.
This PR separated that target into a different subdirectory, freeing
root BUILD.bazel from that dependency.
* Implement optional scalars for Python
* Use == for integer comparison, remove empty line
* Fix optional type hint
Co-authored-by: Caleb Zulawski <caleb.zulawski@caci.com>
Fixes#7320.
I realize that Rust doesn't really follow the Namer convention, since
it uses Field case for methods... that's a future problem.
Co-authored-by: Casper Neo <cneo@google.com>
* Add explicit return types to lobster generated code
* Add support for optional fields.
Convert to bool explicitly from int8 to match type signature
Fix whitespace
I think these changes reflect the current state, but I found it hard to
track the state of some of the planned work. Hence why it'd be good to
have it documented :)
I also expanded some sections that I found misleading, as somebody
familiar with Rust and FlatBuffers separately. Parts of these pages seem
to be aimed at people familiar with FlatBuffers (ie via the other
documentation pages) but not each language, which I'm trying to
preserve. However, Rust does some things differently, and as somebody
with expectations about how typical Rust APIs work the discussion of
threading made me wonder what was different.
std::span lacks these; make the flatbuffers STL emulation and tests
match. This fixes a compile error in C++20 mode when using std::span.
Bug: chromium:1284275
* Set an explicit 2018 edition for Rust tests
* Replace all `std` usage with `core` and `alloc` in Rust code generator
* Update the generated files
* Make Rust tests actually use no_std when the corresponding feature is enabled
* Keep the underlying storage capacity when clearing the FlatBufferBuilder. Gives a significant performance boost for serialisation of many small messages.
* Use Googles Swift benchmark library for more consistent results and dynamic number of iterations, simplification of tests as result.
Co-authored-by: Joakim Hassila <hassila@users.noreply.github.com>
Since CreateVectorOfStrings() takes a templated container, make sure that
the default template deduction from just an initializer list will
still work.
Signed-off-by: Henner Zeller <hzeller@google.com>
* [Java] Fix key lookup returning null clashing with default value
A field with key attribute must always be written on the message so it
can be looked up by key. There is a edge case where inserting a key
field with same value as default would prevent it to be written on
the message and later cannot be found when searched by key.
* [Kotlin] Fix key lookup returning null clashing with default value
A field with key attribute must always be written on the message so it
can be looked up by key. There is a edge case where inserting a key
field with same value as default would prevent it to be written on
the message and later cannot be found when searched by key.
Co-authored-by: Derek Bailey <derekbailey@google.com>
A field with key attribute must always be written on the message so it
can be looked up by key. There is a edge case where inserting a key
field with same value as default would prevent it to be written on
the message and later cannot be found when searched by key.
Any string type that is supported by CreateString(), e.g.
const char* or string_view will now also work.
Signed-off-by: Henner Zeller <hzeller@google.com>
* [Kotlin] Update gradle to 7.4.1 and simplify config files.
* [Kotlin] Add wrapper-validation-action to Kotlin Linux
* [Kotlin] Remove benchmark actions to reduce CI time
* [Kotlin] Move CI js test to Linux action, to increase Mac action speed
* [Kotlin] Generate gradle wrapper in order to be validate
Gradle wrapper from 3.3 to 4.0 are not verifiable because those files
were dynamically generated by Gradle in a non-reproducible way.
So they are now regenerated and will be checked using gitlab action:
gradle/wrapper-validation-action@v1
* Adds implementation flag for swift
Forces internal flag when using @_implementationOnly in swift
Fixes access type for verifier functions & encoder functions
Updates generated code
* Addresses PR comments & adds a code gen dir within the swift tests
* Adds test case for no-include
* Fixes code gen script
Removes prefix
* Started to migrate to target_compile_options
* combined compile options together. Added Mac CI builds
* remove arm build (not supported). Fixed old-style-casts
* moved to using a ProjectConfig interface library to specify options
* remove the explicit CMAKE_CXX_STANDARD
* code gen flexbuffer verifier
* remove verify nested flexbuffers from flexbuffers
* made function static, and placed higher in file
* moved function to own header
* Typo in flatc options (warning-as-errors instead of warnings-as-errors)
* VerifySizePrefixed (reflection::Schema) and GetAnySizePrefixedRoot added
* some review comments
* more review comments
* Fix for https://github.com/google/flatbuffers/issues/7209
* Fixes [C++] flatc generates invalid Code for union field accessors, when --cpp-field-case-style is used #7210
* Add overloads to Add/Put for ArraySegment and IntPtr
In order to allow using code to reduce memory allocations, add overloads to ByteBuffer's and FlatBuffersBuilder's Put/Add methods that take ArraySegment<T> or IntPtr respectively.
Also, adaptions to the c# code generator in flatc to emit corresponding CreateVectorBlock() overloads
* Add missing files generated with generate_code.py
The previous commit changed the C# code generate, but didn't contain the updated generated test files.
* Incorporate review findings
(1) Adhere to 80 characters limit.
(2) In FlatBufferBuilder.Add(IntPtr,int), move zero length check topmost and add sanity check against negative input
* Started applying Namer to Java.
- Java didn't previously have keyword escaping
- Added prefixes and suffixes to the Namer methods
- TODO: migrate previous namer applications to using pre/suffixes
- Java methods / functions are interesting, it's mostly camel case
except when it involves a struct/enum name. That section is Keep case
- I changed the casing for some internal arguments/variables. This
violates the "don't change genfiles" rule that I've been using but it
shouldn't break user code.
- LegacyJavaMethod2 is interesting. Basically, Java has a "mixed" case
convention where it's camel case, except for the type/variant name
itself, which is keep case. So a type foo_bar would become getfoo_bar
instead of getFooBar.
* small fix
* Namer for Namespaces
* removed unused parameter, add const everywhere
* Remove unused argument
* More unused args
* Use mutable reference out parameters
* Made more strings const and inlined const empty strings
* remove do not submit
Co-authored-by: Casper Neo <cneo@google.com>
* Fix C/C++ Create<Type>Direct with sorted vectors
If a struct has a key the vector has to be sorted. To sort the vector
you can't use "const".
* Changes due to code review
* Improve code readability
* Add generate of JSON schema to string to lib
* option indent_step is supported
* Remove unused variables
* Fix break in test
* Fix style to be consistent with rest of the code
* [TS] Fix reserved words as arguments (#6955)
* [TS] Fix generation of reserved words in object api (#7106)
* [TS] Fix generation of object api
* [TS] Fix MakeCamel -> ConvertCase
* [C#] Fix collision of field name and type name
* [TS] Add test for struct of struct of struct
* Update generated files
* Add missing files
* [TS] Fix query of null/undefined fields in object api
* Add example for type field name collision
* [swift] Add bazel configuration for Swift
This change adds a simple bazel BUILD file for consuming the Swift
support. This also bumps the platforms bazel repo to fix support for M1s
and bazel 5.1+ https://github.com/bazelbuild/bazel/issues/15099#issuecomment-1075368289
The rules_swift inclusion here must happen before gRPC to ensure we
don't pull in an older version.
* Add CC=clang which is a requirement for Swift on Linux
* Add Swift to PATH
* Apply Namer to Dart.
- Also refactor idl_gen_dart a bit
- to use more const and references
- out parameters should be the last argument
* Add keyword test
* minor fixes
* fix merge
* extra 's'
* move dart keyord into dart dir
* Address comments
* Use $ for escaping keywords
* Outparameters for namespace_map
* Escape dollar in toString
* Escape dollar in toString2
* Use UpperCamelCase for types and variants
* try to fix ToString
* namer Type fixes
* Remove path prefixing in imports
* gen code
Co-authored-by: Casper Neo <cneo@google.com>
* Handle invalid root offset
* Handle vtable offset invalidation
* Added script generator. Add more cases through vtable ref table size
* review responses
* vtable offset validation
* Moved padding insertion to the end. Tests invalid field lenghts
* table offsets validated. Added type after field
* validate string length
* add todo
* invalid vector length
* invalid structs
* general cleanup
* reworded invalid offsets
* example for vector of structs
* invalid vector of tables
* invalid vector of strings
* invalid vector of scalars
* vector of unions
* validate union type value
* invalid vector union type values
* Fix C/C++ Create<Type>Direct with sorted vectors
If a struct has a key the vector has to be sorted. To sort the vector
you can't use "const".
* Changes due to code review
* Improve code readability
* Add generate of JSON schema to string to lib
* option indent_step is supported
* Remove unused variables
* Fix break in test
* Fix style to be consistent with rest of the code
* [TS] Fix reserved words as arguments (#6955)
* [TS] Fix generation of reserved words in object api (#7106)
* [TS] Fix generation of object api
* [TS] Fix MakeCamel -> ConvertCase
* [TS] Add test for struct of struct of struct
* Update generated files
* Add missing files
* [TS] Fix query of null/undefined fields in object api
* Typo in flatc options (warning-as-errors instead of warnings-as-errors)
* VerifySizePrefixed (reflection::Schema) and GetAnySizePrefixedRoot added
* some review comments
* more review comments
* Apply Namer to Lua bfbs code gen
* refactor namer into IdlNamer to keep idl includes separate
* remove commented out code
* added bfbs_namer
* remove Enum case
* add to bazel
* Allow suppression of including <optional> via macro
* Combine preprocessor conditions
* Make inclusion of header <optional> opt-in
* Make inclusion of <optional> opt-out via macro
* Auto-detect C++17+ when defining macro
* Enclose macro expression in parentheses
* Check value of macro, not whether it's defined
* Fix parentheses
* Don't define macro with expression using other macros
The preprocessor doesn't seem to like it.
* Fix parentheses
* Define and use Namer overloads
* more
* NamespacedType needs a public string-like overload
* Move Rust FieldOffsetName to Namer LegacyRustFieldOffsetName
* Started swift
* More Namer updates
* Remove more usage of the variable 'name' which was semantically overloaded
* unshadow varible
* Make ptr to bool explicit
Co-authored-by: Casper Neo <cneo@google.com>
* Define and use Namer overloads
* more
* NamespacedType needs a public string-like overload
* Move Rust FieldOffsetName to Namer LegacyRustFieldOffsetName
Co-authored-by: Casper Neo <cneo@google.com>
The headline here is adding a flatbuffer_ts_library rule for generating
typescript code in bazel. This entails some non-trivial other changes,
but ideally none are user-visible.
In particular:
* Added a --ts-flat-file flag that generates a single *_generated.ts
file instead of separate files for each typescript type. This makes
bazel much happier.
* Import the bazel rules_nodejs stuff needed to support building
typescript in bazel
* Move flatbuffers.ts to index.ts because I wasn't sure how to make
bazel comprehend the "main" attribute of the package.json. Happy
to take another stab at figuring that out if really needed.
* Fix another couple keyword escaping spots in typescript...
* Refactor out a class from Rust Codegen
* Convert GenerateRustModuleRootFile
* git-clang-format
* unused variable
* parenthesis
* update BUILD file
* buildifier
* Delete bfbs_gen_rust.h
* Delete bfbs_gen_rust.cpp
* Addressed some comments
* Namer::EnumVariant
* Remove do not submit; Add Namespace vector overload
* Unshadow variable
* removed redundant variables
* Apply Namer to Python
* Use more variables a bit
* Apply const a bunch
* More variables
* Fix ObjectTypes
* git clang format
* small thing
* Simplified code around nested flatbuffers
* Make more methods const.
* Python files are kKeep case
* Address DO NOT SUBMIT in SaveType
* ensure dir exists before saving files
* clang format
Co-authored-by: Casper Neo <cneo@google.com>
* Refactor out a class from Rust Codegen
* Convert GenerateRustModuleRootFile
* git-clang-format
* unused variable
* parenthesis
* update BUILD file
* buildifier
* Delete bfbs_gen_rust.h
* Delete bfbs_gen_rust.cpp
* Addressed some comments
* Namer::EnumVariant
* Remove do not submit; Add Namespace vector overload
* Unshadow variable
* removed redundant variables
* Apply Namer to Python
* Use more variables a bit
* Apply const a bunch
* More variables
* Fix ObjectTypes
* git clang format
* small thing
* Simplified code around nested flatbuffers
* Make more methods const.
* Python files are kKeep case
* Address DO NOT SUBMIT in SaveType
* ensure dir exists before saving files
* fix space
Co-authored-by: Casper Neo <cneo@google.com>
* Add Object to the list of reserved keywords.
* Properly escape keywords in type names.
* Properly escape keywords in enum names.
* Properly escape keywords in enum field names.
* Fix 64-bit default numeric enum values in typescript
If you had a default value that wasn't a valid enum value (e.g., a zero
if you used a bit_flag setting, like you get with AdvancedFeatures
in reflection.fbs), we weren't using BigInt.
* Run generate_code.py
* [DART] Handle deprecated fields & invalid enum defaults
* Update .NET test
* [Rust] Add length checks to arrays and vectors.
The previous behavior allowed for out of bounds access in
the public API (#7011).
* bump semver and test warning
Co-authored-by: Casper Neo <cneo@google.com>
* Unified name case conversion to single method
* Convert bfbs_gen to use ConvertCase
* convert rust to use ConvertCase
* Convert idl_parser to use ConvertCase
* Convert MakeScreamingCamel to ConvertCase
* Replaced MakeCamel with ConvertCase
* minor fixes
I'm not seeing the reason why we didn't attempt to support transitive
dependencies for flatbuffer_cc_library, and the current setup makes
having to propagate new dependencies to all of their recursive
dependents obnoxious.
* Fix C/C++ Create<Type>Direct with sorted vectors
If a struct has a key the vector has to be sorted. To sort the vector
you can't use "const".
* Changes due to code review
* Improve code readability
* Add generate of JSON schema to string to lib
* option indent_step is supported
* Remove unused variables
* Fix break in test
* Fix style to be consistent with rest of the code
* [TS] Fix reserved words as arguments (#6955)
* [TS] Fix generation of reserved words in object api (#7106)
* [go] always write required types
* support optional scalars in go
* generate optional_scalars and monster_test
* restore original behavior for non-optional scalars
* add tests
* Change Rust generated file defaults
After #6731, flatc changed its default behavior
for generating rust code to fix some importing issues.
This was a breaking change which invlidated the patch release,
`flatc 2.0.5` (#7081). This PR reverses the default so we can
release a patch update. However, does break Rust users who work at
HEAD.
* Bump flatc patch version (2.0.6)
Co-authored-by: Casper Neo <cneo@google.com>
We already have the reflection.fbs file and the flatbuffers
python language support.
Adding this feature would give the python developers the
ability to parse the flatbuffers schema and write some tools.
Co-authored-by: Derek Bailey <derekbailey@google.com>
* fix for rust build
* Rust: Implement Serialize on generated types
For debugging convenience it is really handy to be able to dump out
types as another format (ie: json). For example, if we are logging a
type to a structured logging system, or even printing it out in a
structured way to the console.
Right now we handle this by shelling out to `flatc` which is not ideal;
by implementing Serialize on the generated types we can use any of the
Serializer-implementing packages for our structured debug output.
* clang-format
* Make the flatbuffers Rust crate only have an optional dependency on the `serde` packages.
* fix warning
* fix rust test build
* Oh yeah this needs to be initialized
* fix toml syntax
* code review feedback
* rebuild test data
Augment the C++ generator to emit a C++ copy constructor and a by-value
copy assignment operator. This is enabled by default when the C++
standard is C++11 or later. These additional functions are only emitted
for objects that need it, typically tables containing other tables.
These new functions are declared in the object table type and are
defined as inline functions after table declarations.
When these new functions are declared, a user-defined
explicitly-defaulted default constructor and move constructor are also
emitted.
The copy assignment operator uses the copy-and-swap idiom to provide
strong exception safety, at the expense of keeping 2 full table copies
in memory temporarily.
fixes#5783
* [ts] Builder incorrectly serializing empty strings
The builder was returning an offset of zero for empty strings. This is
leading to flatbuffers which fail verification in other languages, such
as Rust.
* tests expect 0 offset for null or undefined strings
* Emit include for bfbs-gen-embed
* Use python3 explicitly
* bump min python version to 3.6
* Sort find_package for python
* try casting Path to string
* cast WindowsPath to string to please CI
* stringify the wrong thing
* another stringify path
* Moved the CMake check for Clang *after* the one for MSVC, as clang-cl
matches both Clang and MSVC.
* Removed /MP definition - controlled by CMake
* Added CPP define for _CRT_SECURE_NO_WARNINGS so clang can see it
(it seems it doesnt pick up the pragma in util.cpp)
* BuildFlatbuffers.cmake: add verbose on build
* BuildFlatbuffers.cmake: properly add *.fb.* files with --grpc argument
When "--grpc" argument is provided as an extra flag, resulting grpc files
should be added as part of the interface library.
This prevent adding .fb.cc files manually to the build.
V2: fix dependency on grpc files
* Add --warnings-as-errors to flatc compiler.
With this option set, flatc will return an error on parsing if
any warnings occurred.
* Add unit test for opts.warnings_as_errors.
* Change explicit option setting to default.
https://github.com/tensorflow/tflite-micro makes use of flatbuffers with
a variety of DSP toolchains.
Without the change from this PR, we can get a double-promotion warning
with some of these DSP toolchains:
```
flatbuffers/include/flatbuffers/util.h:104:11: error: implicit conversion increases floating-point precision: 'std::numeric_limits<float>::_Ty' (aka 'float') to 'double' [-Werror,-Wdouble-promotion]
T eps = std::numeric_limits<float>::epsilon();
~~~ ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
iomanip isn't available for our DSP. Luckily, we compile with FLATBUFFERS_PREFER_PRINTF, so moved the #include directive there.
ctype.h has to be included explicilty for tolower() and toupper().
Parsing as bytes produces buffers that are unsafe to access unless passed thru a verifier,
whereas users could reasonably assume that any JSON parsed without errors is safe to access.
Users that still have legacy JSON files with such bytes in it will get a helpful error point them
to the option to turn on to have it work again.
This change checks if the current source directory is a git repository.
If this is not checked the git command picks the commit hash from
parent directory. e.g. when tarball is extracted in a packaging repository.
* Enable --gen-onefile in Python
Made it possible to generate all python code in one file. Modified
py_test.py so that it can switch between the multi-file code and
the one-file code.
Updated PythonTest.sh and py_test.py so that the multi-file code
and the one-file code can be tested based on the same test code.
* Sync with google/flatbuffers
* Add --gen-onefile to generate_code.py
* Make idl_parser deterministic
Some golden tests even exercise this [logic](df2df21ec1/tests/prototest/test.golden (L8)). Let's make the parser fully stable not depending on the implementation of std::sort
* Retry the ci
* Fix C/C++ Create<Type>Direct with sorted vectors
If a struct has a key the vector has to be sorted. To sort the vector
you can't use "const".
* Changes due to code review
* Improve code readability
* Add generate of JSON schema to string to lib
* option indent_step is supported
* Remove unused variables
* Fix break in test
* Fix style to be consistent with rest of the code
* [TS] Fix reserved words as arguments (#6955)
* initial hack to get new Lua generator into flatc
* Starting to output enum defs for Lua
* Continue to work on table generation for Lua
* Finished basic getter access for Lua
* Added ability to get object by index
* Finished struct builder
* aliased reflection to r
* finish table builder generation
* register requiring files
* better generated header info
* Tying up loose ends
* Updated reflection to handle struct padding
* Addd type sizes to reflection
* Fixed some vector indirect issues
* Lua tests passed
* Misc cleanup
* ci fixes 1
* ci fixes 2
* renaming
* up size of type sizes
* manually ran clang-format-11 -i src/idl_parser.cpp
* fixed some windows casting
* remove stupid auto import
* more static_casting
* remove std
* update other build environments
* remove scoped enums
* replaced std::to_string with NumToString
* more win fixes
* more win fixes
* replaced old lua with new
* removed auto import
* review responses
* more style fixes
* refactor bfbs_gen_len to use code +=
* added consts
* fix lambda capture for windows
* remove unused return type
When generating code with --grpc, --cpp and using filename-suffix, the generated grpc files where not including the correct header that had the filename-suffix. As a suffix, they used the default "_generated".
Free functions for these were used to get the suffix. FlatBufFile had such methods, but also needed to be into its base File and use these.
- grpc generated files include the correct message header.
- grpc generated files also have the suffix
- grpc generated cc file does not include initial message header
* removed test/generate_code.{sh|bat}
remove c++0x from generate_code.py
added check generate code script in python
add windows specific call
added flags to generate_code.py
Set c++-0x on BUILD_LEGACY
Skip generating monster_extra if requested
* added option to skip monster extra
* add conditional to skip 2010 check gen
Restore flatbuffers::FlatBufferBuilder::kFileIdentifierLength, which was a documented part of the public API, but the identifier was lost during the refactoring effected by comment commit 6c8c291559.
* Added Google benchmarks (and gtests)
* Separate benchmark CMakeLists.txt to its own file
* Move output directory to target just flatbenchmark
* Reduced from encoding 210ns -> 188ns
* store size_ as uoffset_t
* fixed windows c4267 warning
Detected instability when built `flatbuffers-2.0.0` on `gcc-12`:
[ 75%] Building CXX object CMakeFiles/flattests.dir/tests/test_builder.cpp.o
.../c++/12.0.0/bits/shared_ptr_base.h:397:45: error: 'size' may be used uninitialized [-Werror=maybe-uninitialized]
397 | explicit _Sp_ebo_helper(_Tp&& __tp) : _M_tp(std::move(__tp)) { }
| ^~~~~~~~~~~~~~~~~~~~~~
In file included from flatbuffers/tests/test_builder.cpp:1:
flatbuffers/tests/test_builder.h: In function 'void builder_move_assign_after_releaseraw_test(Builder) [with Builder = flatbuffers::FlatBufferBuilder]':
flatbuffers/tests/test_builder.h:63:10: note: 'size' was declared here
63 | size_t size, offset;
| ^~~~
...
In file included from flatbuffers/tests/test_builder.cpp:1:
flatbuffers/tests/test_builder.h: In function 'void builder_move_assign_after_releaseraw_test(Builder) [with Builder = GrpcLikeMessageBuilder]':
flatbuffers/tests/test_builder.h:63:10: note: 'size' was declared here
63 | size_t size, offset;
| ^~~~
cc1plus: all warnings being treated as errors
Here is the relevant bit of test:
template<class Builder>
void builder_move_assign_after_releaseraw_test(Builder b1) {
auto root_offset1 = populate1(b1);
b1.Finish(root_offset1);
size_t size, offset;
std::shared_ptr<uint8_t> raw(
b1.ReleaseRaw(size, offset), [size](uint8_t *ptr) {
flatbuffers::DefaultAllocator::dealloc(ptr, size);
});
Note how `b1.ReleaseRaw(size, offset)` is expected to populate `size`
and `[size](uint8_t *ptr) {` captures the result. But both are parameters
to the same function call and thus evaluation order is unspecified.
* chore: make flatc artifacts from CI executable
* chore: prepare dart 2.0.0 release
* refactor: update description in pubspec.yaml to make pub.dev happy
"The package description is too long.
Search engines display only the first part of the description. Try to keep the value of the description field in your package's pubspec.yaml file between 60 and 180 characters."
* Added Google benchmarks (and gtests)
* Default building benchmarks to OFF as it requires c++11
* Separate benchmark CMakeLists.txt to its own file
* Move output directory to target just flatbenchmark
* split flatbuffers.h into separate files
* wrong variable in cmakelists for android
* readded two accidentally deleted includes
* created buffer.h and moved buffer related things over
* Keep methods with struct name and switch them to default
This PR can help fix the following two issues:
1): A set of simplified API (without struct name) was added in
https://github.com/google/flatbuffers/pull/6336. It causes name
conflict when merging all generated python file into a single one
(the primary usage senario in Google).
2): Flatbuffers 2.0 generates absolute import path, which may cause
name space conflicts. See more details in
https://github.com/google/flatbuffers/issues/5840.
The solution for both is to generate the merged Python code, similar
C++. The merged code will not contain the simplied API, but only the
method with struct name. For issue (1), it will mimic the exactly
usage pattern for Google internal. For issue (2), users can generate
the merged flatbuffer code, without worrying about the imports.
The above idea will be implemented in the following steps:
Step 1 (this PR): revert changes in https://github.com/google/flatbuffers/pull/6336
that set the simplified API as default. Remove statements that the
original API will be deprecated, and reset the original API as default.
Step 2 (the following PR): create a flag to generate the merged code.
The Simplified API will be removed from the merged code, otherwise it
will cause name conflict.
* Update the generated sample code
* Update the generated example code
* Reverst the changes of GetRootAs
* Update examples from grpc/example/generate.sh
It seems like `--conform` already works for vectors of unions - there is
just a spurious check that prevents it from running. Fixes#6882
Also, if schemas do not conform, `flatc` no longer prints out the usage
(since the error is not due to bad usage). Fixes#6496
* CPP Default Value Generation in Mutators
If the mutator is for a value that is compatible with having a default value, then the single parameter becomes a default parameter. With this, a value can be mutated to it's default value without storing the default value, as that will be stored with the mutate function.
Fixed Casting When Generating Default for Enum Value
Added support for typecasting an int default value into the correct enum type in the default parameter. This fixed the issue of trying to use set a strongly typed enum parameter to an int which fails type checking.
Fixed Boolean Edge Case
Boolean types generate 0 != 0 when generating the underlying type which appears to be unique to the boolean type so it is now checked and the proper default value generated. It may be beneficial to check if it is instead an enum type, however the seeming edge case nature is why boolean was chosen to be checked.
Updated Generated Files
Regenerated the auto generated files to reflect the new changes.
Updated Remaining Files
Should fix auto generated header files that were not updated.
* Unified Repeated Code
Relocated identical append code to outside of conditional. Also changed 'casted' default value name from FIELD to INTERFACE to more accurately describe it.
* Moved Field Name Outside Conditional
Removed duplicate _{{FIELD_NAME}} and moved to unified append.
Finished documenting flatbuffersbuilder
Replaces swift 5.5 with 5.2 packages
Starts building the tutorial for xcode 13
Finishes building the tutorial for xcode 13
Removes docc files from old swift versions
Updates swift style guide
* added --bfbs-builtins
* update generate_code.bat
* forgot the .
* updated checking scripts
* added bypass for the monster_test.bfbs and arrays_test.bfbs diff issue
* removed check on windows for now
Updates the `files` globs in package.json to include subdirectories in order to also include the flexbuffers directory in the published npm package.
Updates .gitignore to include the tsc-generated JS files.
* Update idl_gen_csharp.cpp
Change csharp generator to use "global::" for it's qualifying_start_ to disambiguate namespaces
* regenerate testing files
regenerate testing files
* Missed TableInC.cs
updated with global prefix
* Remove "global::" from qualifying_start_ for csharp generator
* C# global alias
* Tests and docs for --cs-global-alias
Add tests for --cs-global-alias to demonstrate use case for why it's needed.
Add documentation to Compiler.md
* Add also to help text
Add also to help text
* Test to make sure optional enum is written properly
* Handle optional enum codegen: when cast optional enum add `?`
* Run `tests/generate_code.sh` to generate code from schema
* Fix type casting in case of CreateXXXTypeVector
* Reason why vector's type is not optional
Change the FlatBufferBuilder's methods to accept std::vector parameters
with non-default allocator, by adding another template parameter to
them. This should make using the builder slightly more convenient, as
one won't need to manually pass data() and size() separately.
* Changes to support binary schema file loading and parsing (flatc)
- parser.reset() is also called if binary schema file is given
- code flow changed to not try to load a binary schema as textual schema
* Removed unneeded braces
This allows building FlatBuffers with gcc on macOS:
- avoid linking libc++ when not using clang
- look at compiler first, then OS-specific options
Signed-off-by: Rafal Kapuscik <rkapuscik@antmicro.com>
* [C++] Add mutable version of LookupByKey and test
This adds an overload of LookupByKey to allow lookup in sorted Vectors to
return a mutable instance of the object (or nullptr if not found).
* Fix naming
* Revert "avoiding even more NoSuchMethod exceptions (#6729)"
This reverts commit 6fb2c90d9e.
* Revert "avoiding more NoSuchMethod exceptions (#6671)"
This reverts commit 752c7b576d.
* Revert "avoiding NoSuchMethod exception (#6658)"
This reverts commit 813d3632ec.
* Use Java 8 for Kotlin Linux builds to verify
* flattests_cpp17 doesn't compile with Visual Studio 2017: warning C4100: 'indent': unreferenced formal parameter
stringify_util.h(127): error C2220: warning treated as error - no 'object' file generated
stringify_util.h(127): warning C4100: 'indent': unreferenced formal parameter
stringify_util.h(85): warning C4100: 'indent': unreferenced formal parameter
stringify_util.h(85): warning C4100: 'fbs': unreferenced formal parameter
* [C++] Add GetMutableSizePrefixedRoot() and generate a GetMutableSizePrefixed function
When using the mutable API together with size prefixed buffers these functions should be present.
* clang-format
* Cleanup branch for PR
Revert "flattests_cpp17 doesn't compile with Visual Studio 2017: warning C4100: 'indent': unreferenced formal parameter"
This reverts commit a92055203e.
* Remove dead code in idl_gen_rust
* Use 2space indentation in mod.rs
* use In/DecrementIdentValue in idl_gen_rust
Fix some whitespace too in generated code.
* make default fn 2space ident
* More 2space formatting
* git clang format
* make vs2015 happy
Co-authored-by: Casper Neo <cneo@google.com>
* Handle keywords in C#
C# should handle keywords similarly to other languages by appending an underscore after the keyword.
* Fix clang-format for idl_gen_csharp.cpp
* Fix spacing clang-format issue
* Make CSharp use the command line parameter for file name extension and suffix
* Default file name now has _generated in it,
so we need to rename all the generated test files
tests/namespace_test/NamespaceA/TableInC.* are no
longer used, so these can be deleted
* Fix C# testing projects so that they reference
the new _generated.cs files
* Revert "Fix C# testing projects so that they reference"
This reverts commit cf62b9c5d4.
* Revert "Default file name now has _generated in it,"
This reverts commit 042aa81f10.
* Revert "Make CSharp use the command line parameter for file name extension and suffix"
This reverts commit ba8cd06646.
* Make CSharp use the command line parameter for file name extension and suffix
* Fixup clang tidy
* Fix more clang-tidy
With the changes introduced on #6729, #6671, #6658 it seems that using java
compiler version 8 is no longer possible. We are adjusting our CI build for
Kotlin to use Java 11 as compiler, while still emiting 1.8 bytecode.
The Kotlin CI action is also being breakdown into two: Mac & Linux.
Kotlin mac will test ios & mac builds while Linux will test js and JVM.
This change will improve build times for Kotlin on CI, which is currently
taking long times.
Causes generated code to check for existing pointers in an object (when using the object based api) and use UnPackTo(existingPointer) rather than just using UnPack() to replace the pointer. This is a performance issue when unpacking to existing objects when using large buffer fields (e.g. image frames)
Co-authored-by: RobWauson <robwauson@gmail.com>
* benchmark many vtables
* Rust: Store written_table rev-positions sorted.
The previous implementation was slow if there were too many tables.
Asymototically when inserting the n^th vtable: The old implementation
took O(n) lookup steps and O(1) insertion. The new implementation is
O(log n) lookup and O(n) insertion. This might be improved further by
using a balanced btree.
Benchmarking, create_many_tables is 7.5x faster (on my laptop):
// Simple vector cache
test create_many_tables ... bench: 728,875 ns/iter (+/- 12,279) = 44 MB/s
// Sorted vector cache
test create_many_tables ... bench: 97,843 ns/iter (+/- 4,430) = 334 MB/s
* Fix lints
Co-authored-by: Casper Neo <cneo@google.com>
* Fix C/C++ Create<Type>Direct with sorted vectors
If a struct has a key the vector has to be sorted. To sort the vector
you can't use "const".
* Changes due to code review
* Improve code readability
* Add generate of JSON schema to string to lib
* option indent_step is supported
* Remove unused variables
* Fix break in test
* Fix style to be consistent with rest of the code
* [C#] Fix field name struct name collision (#6744)
* [C#] add union constructor utility.
* add union utility test.
* std::tolower -> CharToLower
* use std::transform instead
* rebase to master. and regenerate test code
* Refactored Rust Generated code into a module directory.
Each symbol will be generated into one file and then
imported into a module. This breaks the "out_dir"
pattern where some users would generate code in their
target/ directory. Also, these objects are best used
in their own module. It will be hard for users to share
their own module structure with flatbuffers namespaces.
There may be solutions to these drawbacks but that should
be discussed. I don't want to overengineer here.
* shadow error
* try fix .bat file
* fix .bat 2
* Restore accidentally deleted files
* Fixed some DONOTSUBMITs and made Rust outdir pattern use symlinks.
* fixed binary files
* git clang format
* make generated onefiles not public and fix .bat
* reduced diff with master in generate_code.sh
* fix shadowed variable
* add object api flags to .bat
* space
* Removed extern crate and extra &
* use statement
* more clippy lints
* format
* Undo extern crate -> use change, it actually matters to our tests
Co-authored-by: Casper Neo <cneo@google.com>
* Dart - change Builder "lowFinish()" to "buffer" and "finish()" to not void return
Aligning the API with other languages, e.g. c++ and allowing custom use-cases to avoid creating a Uint8List
* Dart - change builder.buffer to check that finish() was already called
* Dart - builder - move !finished assertion to _prepare() which is run from all other functions
* Remove ubuntu1604 from presubmit.yml
Ubuntu 16.04 is end-of-life, we're going to remove it from Bazel CI.
If you like you can add testing on `ubuntu2004` platform which we also support.
* Add ubuntu2004 to presubmit.yml
* Fixup CPP documentation's markdown errors
Note that I couldn't get the ~~~{.cpp} method to work,
so I switched to using ```cpp which did work.
* Fixup C++ docs, typo in repeated code example
Co-authored-by: Paul Harris <paulharris@computer.org>
* Make --bfbs-filenames default to location of first schema file.
Make RelativeToProjectRoot always work, applying "../" where needed. This is
needed for backwards compatibility. The first input file may be deeper in some
directory than the other files. Now, there will always be a declaration
file.
* documentation
* clang format
Co-authored-by: Casper Neo <cneo@google.com>
* Add comments for properties
* regenerated test files
* Reverted changes in testfiles generated with generate_code.bat on windows
* fixed formatting in util.cpp because of failed .travis/format_check.sh
* removed the use of tuple
* util.cpp: Fixed format
* idl_gen_json_schema.cpp: removed extra comma in deprecated fields
* Corrected monster_test.schema.json
* idl_gen_json_schema: Inlined Trim function
* Fixed format
* PrepareDescription(): Avoid unnecessary string copy
* fixed formatting
* Improve generated comparisons for tables
Previously, the generated code called std::unique_ptr<>::operator== on
non-scalar members, which would be correct only if the member was null
on both sides. After this CL, comparison is done on the pointed-to
values (including using a default-constructed value if the pointer is
null).
* Don't equate null Tables with all defaults
Also removes the cost of default-constructing tables for comparisons.
* Regenerate code
* fix formatting
* Replace filenames in reflection with filenames+includes.
This is needed for some use cases and may be just useful metadata.
* deser files_included_per_file_
* check project_root
* fix bazel
* git clang format
Co-authored-by: Casper Neo <cneo@google.com>
* Add the file a symbol is declared in to Reflection
If we move a code-generator to depend on Reflection,
it may need to know which file something was declared in
to properly name generated files.
* Doc comments in reflection, and more precise tests
* Add --project-root flag to flatc, normalize declaraion_file to this root
* fix --project-root stuff
* posixpath
* fix scripts
* format
* rename --project-root to --bfbs-filenames
Also, make it optional, rather than defaulting to `./`, if its not
specified, then don't serialize the filenames.
* bfbs generation
* fix some tests
* uncomment a thing
* add to project root directory conditionally
* fix
* git clang format
* Added help description and removed != nullptr
* "
* Remove accidental change to docs
* Remove accidental change to docs
* Pool strings
Co-authored-by: Casper Neo <cneo@google.com>
* dart test scripts - generate with `--gen-object-api`
* Dart object API, pack and unpack methods (WIP)
* Dart flatc - extract Builder code to separate functions to reuse it in Pack()
* Dart flatc - use builder field-building implementation in pack()
* Dart flatc - add pack() as an intance method of the "T" class
* Dart object API - make inner fields unpacked as well
* Dart object API - use pack() when collecting field offsets
* Dart object API - use packOnce() for fields that are structs or vectors of structs
* Dart object API - remove obsolete union support TODO
* Dart object API - minor review changes, test and fixes
* Dart object API - revert packOnce() - not supported by other object API implementations
* Dart object API - update docs
* update dart generated code in tests/ to fix CI failure on ./scripts/check-generated-code.sh
* Dart flatc - fix compilation for old MSVC and c++0x
* flatc --cpp-field-case option to permit camel-case field names in C++
* fixed option name; cleaned up tabs
* formatting fixed to conform to CI
* resolved comments
* fixed white space indentation
* per PR comments
* rename snake case option to unchanged for clarity, per PR comments
* cleanup of unchanged case option in C++ codegen, per PR 6669 comments
* incorporated PR feedback from vglavnyy
* cleaned up to pass Travis CI / clang format
* bumped PR to retry transient CI failure
* bumped PR to retry transient CI failure
* bump PR
* assert union type field name length > suffix, per PR 6669 comments
FlexBuffers are used by custom op handlers for TFLite. Using the optimized path for ReadUInt64 in FlexBuffers causes a crash in models using custom ops on win32 build. This change fixes the problem by using unoptimized implementation of ReadUInt64 for win32.
* [C++] Removed most heap allocations in builder
* Updated API docs to indicate heap usage
* Override assertion for heap allocation in parser
* Cleaned up implemenations, enable heap alloc for tests
* Generalized CreateVectorOfStrings
* remove cmake option for heap alloc. reverted two heap removals
* Only use scratch space for vector of strings
* Override Windows SCL warning
* Changed std::transform to for loop to avoid MSCV warnings
* switched to const iterators
* Replaced iterator with for loop
* remove std::to_string in test to be compatible
* [C++] Removed most heap allocations in builder
* Updated API docs to indicate heap usage
* Override assertion for heap allocation in parser
* Cleaned up implemenations, enable heap alloc for tests
* Generalized CreateVectorOfStrings
* remove cmake option for heap alloc. reverted two heap removals
* Only use scratch space for vector of strings
* Override Windows SCL warning
* Changed std::transform to for loop to avoid MSCV warnings
* switched to const iterators
* [idl_parser] Check structs and enums do not clash in a namespace
Uses fully qualified names to check for clashes within a given namespace whether explicitly defined or in the global namespace.
* [idl_parser] Move type name clash check to ParseEnum and ParseDecl
Change point at which parsing error is returned to ensure error is caught more generally. This change means that the error is returned after parsing the entirety of the offending duplicate rather than at the start when parsing it's name.
* [idl_parser] Add single and multi file type name clash tests
Adds a selection of tests for valid single file schemas with types that have the same name but are in different namespaces.
Adds a test for an a valid schema that spans two files with two types that have the same name but are in different namespaces.
Adds a test for an an invalid schema that spans two files with two types that have the same name and are in the same namespace.
* Implement Serialize for flexbuffer::Reader
* bump version
* Remove use of experimantal or-patterns
* Remove more use of experimantal or-patterns
Co-authored-by: Casper Neo <cneo@google.com>
This commit fixes two serious issues connected with reverse iterators:
1. Vector's rbegin/rend was incompatible with std::reverse_iterator::base();
2. Array's rend() was incorrect, it based on end() instead of begin().
This adds basic support for different Lua versions.
For Lua 5.2 and Lua 5.3, both the Bit32 and Compat53 Lua modules must be
installed for it to work. You can typically get these on Linux using
apt install lua-compat53 lua-bit32
For Lua 5.4, it should work as is, as it is a clean superset of Lua 5.3,
which is what the original Lua Flatbuffers supported.
LuaJIT as installed by apt doesn't include the COMPAT mode for using
Lua5.2 features. So this change just makes the flatbuffer code use a
common implementation.
* [Lua] Add LuaJIT support
Here is the output of LuaTest.sh:
Run with LuaJIT:
built 100 512-byte flatbuffers in 0.16sec: 0.63/msec, 0.31MB/sec
built 1000 512-byte flatbuffers in 0.08sec: 12.06/msec, 5.89MB/sec
built 10000 512-byte flatbuffers in 0.80sec: 12.44/msec, 6.07MB/sec
built 10000 512-byte flatbuffers in 0.33sec: 30.58/msec, 14.93MB/sec
traversed 100 592-byte flatbuffers in 0.04sec: 2.51/msec, 1.42MB/sec
traversed 1000 592-byte flatbuffers in 0.03sec: 31.52/msec, 17.79MB/sec
traversed 10000 592-byte flatbuffers in 0.21sec: 48.77/msec, 27.53MB/sec
Run with Lua 5.3:
built 100 512-byte flatbuffers in 0.02sec: 5.44/msec, 2.66MB/sec
built 1000 512-byte flatbuffers in 0.17sec: 5.74/msec, 2.80MB/sec
built 10000 512-byte flatbuffers in 1.75sec: 5.72/msec, 2.79MB/sec
built 10000 512-byte flatbuffers in 1.38sec: 7.26/msec, 3.55MB/sec
traversed 100 592-byte flatbuffers in 0.00sec: 27.64/msec, 15.60MB/sec
traversed 1000 592-byte flatbuffers in 0.03sec: 30.46/msec, 17.20MB/sec
traversed 10000 592-byte flatbuffers in 0.34sec: 29.62/msec, 16.72MB/sec
* [Lua] Better usage description
* update according to the review
* Introduce new_from_vec in Rust (also fix formatting)
Also, rename `new_with_capacity` to `with_capacity` to match
how `Vec` does it.
* bump rust version
* mut_finished_buffer
Co-authored-by: Casper Neo <cneo@google.com>
* Add advance feature indicators to reflection
* deserialize too
* model advanced features as bitflags
* use uint64_t instead of AdvancedFeatures
* git clang format
* initialize advanced_features_
* remove whitespace
Co-authored-by: Casper Neo <cneo@google.com>
* Fix Miri flag passing and bump Rust version.
* Fix Miri problems from Arrays PR.
SafeSliceAccess was removed for Arrays. It's kind of unsound.
It has two properties:
1. EndianSafe
2. Alignment 1
We only need 1. in create_vector_direct to memcpy data.
We both 1. and 2. for accessing things with slices as buffers are built on &[u8]
which is unaligned. Conditional compilation implements
SafeSliceAccess for >1byte scalars (like f32) on LittleEndian machines
which is wrong since they don't satisfy 2.
This UB is still accessible for Vectors (though not exercised our
tests) as it implements SafeSliceAccess. I'll fix this later by
splitting SafeSliceAccess into its 2 properties.
Co-authored-by: Casper Neo <cneo@google.com>
* Mark endian_scalar as unsafe.
Also
- removed the deprecated flexbuffer slice from example
- fixed some cargo warnings
* Assertions and read_scalar made unsafe
* Clippy lints
* Add to Safety
Co-authored-by: Casper Neo <cneo@google.com>
* Add support for fixed size arrays
* clang-format
* Update rust image to 1.51 to support const generics
* Handle correctly big endian
* Add fuzz tests and clean code
* Add struct fuzz test and optimize struct arrays for api
* Bump flatbuffers crate version
Kotlinx.benchmark project just abandoned bintray and moving to maven central,
removing the artifacts from bintray and causing missing dependencies in our build.
So we are updating the dependency to point to the new version on maven central.
More information in:
53ee45d0d9 (diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5)
* Add Build TS to CI jobs
* Add npm compile step
* Fix syntax
* Add required code gen for TS
* Exit on failure
* Make TS code gen identical to test run
* Remove duplicate TS code gen artifacts
* Remove bad gitignore
* Forgot parts of generate_code and make sure same settings for test run
* Don't forget about the bat
* Try and fix flatc args
* Another attempt to fix args
* Fix typo
* Commit up to date code gen
* Another attempt to fix sh/bat
* Move -o param to after -I
* Commit missing code gen file
* Fix grpc code gen and test
* Another grpc code gen fix
* Rework to not use ts folder
* Fix env vars in bat and make less churn
* Move TS code gen to dedicated folder
* Fix transpilation output folder and module paths
* Fixes to code gen
* Include generated js
* Moved ts code gen
* Remove test ts code gen folder
* disable clippy
* Vector of enum default
* swift and tests
* git clang format
* Rewrite enum parser checks
* Remove Voids from more_defaults
* vector enum swift
* remove vector accessor from swift
* clang format
Co-authored-by: Casper Neo <cneo@google.com>
* [C++] #6501 - Problem when mapping a native type multiple times
- idl.h:
added "native_type_pack_name"
- flatbuffers.h:
added CreateVectorOfNativeStructs variants which receive a pointer to the serialization function
- idl_gen_cpp.cpp:
adapted code generation in case "native_type_pack_name" attribute is present
- extended tests & docs; improved surrounding native_type docs a little
* integrated review feedback
* [idl_parser] Add kTokenNumericConstant token
This commit adds the new token for correct parsing of signed numeric constants.
Before this expressions `-nan` or `-inf` were treated as kTokenStringConstant.
This was ambiguous if a real string field parsed.
For example, `{ "text_field" : -name }` was accepted by the parser as valid JSON object.
Related oss-fuzz issue: 6200301176619008
* Add additional positive tests fo 'inf' and 'nan' as identifiers
* Rebase to HEAD
* Move processing of signed constants to ParseSingleValue method.
* Add missed `--cpp-static-reflection` (#6324) to pass CI
* Remove `flatbuffers.pc` from repository to unblock CI (#6455).
Probably the generated flatbuffers.pc should not be a part of repo.
* Fix FieldIdentifierTest()
ARM64EC is a new ARM64 ABI designed by Microsoft to support x64 application emulation on ARM64 CPUs. When compiling for ARM64EC, both the _M_X64 and _M_ARM64EC macros are defined. However, that causes problem in compiling this file, because the __movsb intrinsic, which is lowered to rep movsb, is not supported on ARM64, so the optimization for native x64 should be disabled for ARM64EC.
* [C++17] Add compile-time reflection for fields.
Included in this commit is the following:
- The C++ generator has been modified so that,
when in C++17 mode, it will emit Table and
Struct field traits that can be used at com-
pile time as a form of static reflection. This
includes field types, field names, and a tuple
of field getter results.
- Diffs to the cpp17 generated files. No other
generated files are affected.
- A unit test that also serves as an example. It
demonstrates how to use the full power of this
reflection to implement a full recursive
JSON-like stringifier for Flatbuffers types,
but without needing any runtime access to the
*.fbs definition files; the computation is
done using only static reflection.
Tested on Linux with gcc 10.2.0.
Fixes#6285.
* Fix int-conversion warning on MSVC.
* Try to fix std::to_string ambiguity on MSVC.
* Fix clang-format diffs.
* Fix more clang-format diffs.
* Fix last clang-format diff.
* Enable C++17 build/test for VC 19 platform in CI.
* Forgot to add value to cmake command line variable.
* Various fixes/changes in response to @vglavnyy's feedback.
* Replace "fields pack" with index-based getters.
* Fix MSVC error.
* Fix clang-format diffs.
* getter_for method returns result instead of address-of-getter.
* Next round of reviewer suggestions.
* Use type instead of hardcoded struct name.
* Fix clang-format diff.
* Add test for FieldType since it is not used in the stringify test.
* Add fields_number field to Traits struct.
* Add --cpp-static-reflection flag and put those features behind it.
* Fix clang-format diffs.
* Remove <tuple> include.
This commit disable JSON parsing for an incomplete scheme if JSON object is embedded into one file with the scheme.
This should improve the quality of OSS-Fuzz inputs for the parser_fuzzer target.
* [idl_gen] Delete ts::GenPrefixedImport()
I don't know what this is for, but it's the only piece of code external
to idl_parser.cpp that expects the key of Parser::included_files_ to be
a path. And it appears to be unused.
* [idl_parser] Track included files by hash
Parser::included_files_ is a map whose main purpose is to keep track of
which files have already been parsed in order to protect against
multiple inclusion. Its key is the path that the file was found at
during parsing (or, if it's an in-memory file, just its name).
This commit changes the key to be the 64 bit FNV-1a hash of the file's
name (just the name, not the complete path) xor'd with the hash of the
file's contents (unless it's an in-memory file, then we only hash the
name.)
This allows multiple include protection to function even in the face of
unique per-file include paths (fixes#6425).
* Ran tests/generate_code.sh
CI told me to do it.
* Gracefullt handle case where source_filename == nullptr
I just learned source_filename might also be null. In that case, we
should exclude it from the hash instead of hashing a zero length
string, just like we exclude source when it is null.
Presumably nameless files will never be included (they can't, can they?)
so this doesn't really matter, but I think it's prettier anyways.
Refactored python grpc code gen
Adds example server & client + fixes ci
Fixes generated code
Making sure we encode the reply string as utf8
Adds Readme details to clarify issue regarding encoding when python is sending/receiving
* Remove debug code
This was added for testing in the recent genericize PR for flexbuffer Reader.
* Added alloc tests -> MapReader::{is_empty, index_key, len}
* Added , accessible through Deref to deprecation warning
This commit enables CMake to generate a flatbuffers.pc file on install.
pkg-config eases the utilization of software libraries by enabling the
developers of a library to define how the library should be used.
It can be used as a tool by build systems to find and manage dependencies,
this is notably the default Meson behavior and optionally used by CMake.
Implemented server.go and half implemented client.go
Finishes implementation for greeter go example
Update grpc code for monster.fbs
Adds a couple of cpp methods
Adhere to gofmt standards
Adds a readme for issues with grpc
* feature/rust-tokio-bytes added feature name for tokio-bytes
* Added flexbuffer implementation, TODO: typecast to avoid recurse
* Converted codebase to utilize FlexBuffer implementation, need to resolve deserialization issues
* Added todo for lifetime issue, may use &'de [u8] for deserializer instead of current method
* Added proper &[u8] implementation
* Removed unused struct
* Added experimental fix to get_slice
* Added experimental fix to get_slice
* Avoided lifetime issues via ref structs, need to check if this hurts peformance
* Updated deserializer implementation to allow for borrowed data from Reader struct
* Fixed bug with str
* Removed unnecessary generic parameter
* Added unsafe to avoid lifetime complaints, current tests pass, need to review alternatives to unsafe
* Opinionated: Removed bytes crate as this implementation could be done in a separate crate
* Cleaned up flatbuffer
* Fixed sample / example
* Resolved PR feedback, need to resolve issues with tests
* Cleaned up FlexBuffer trait to be an auto impl
* Removed TODO
* Reverted Deserializer to only support &'de [u8]
* Cleaned up / renamed function for clarification
* Renamed FlexBuffer -> InternalBuffer for clarification on it's purpose
* Fixed issue with key bytes
* resolved issues with broken tests, confirming this is a breaking change
* Removed FIXME that's solved by splitting String and Key variants
* Implemented associated types approach
* Fixed backward slice logic
* Fixed MapReader compile error
* Added from_buffer for deserialization, removed function since it's only needed for deserialization
* Removed dead code
* Cleaned up buffer, removed AsRef in favor of Deref
* Renamed Buffer::as_str -> Buffer::buffer_str
* Minor cleanup
* Updated documentation, need to fix tests
* Removed unnecessary &
* Removed unused lifetime
* removed unnecessary as_ref
* Minor optimization wrap-up
* resolved issue with Clone
* Added test to verify no deep-copy
* Added for optimization
* Updated to use empty fn instead of default
* Updated comments / test name - plus the 0.3.0 version bump
* comment
* [Build, cmake] Add -Werror override option
This commit adds FLATBUFFERS_CXX_FLAGS cmake option.
This option allows override the -Werror flag (or any other flags).
Related issues: #6337, #5930
* Remove CMAKE_CXX_FLAGS replace option
* Fix C/C++ Create<Type>Direct with sorted vectors
If a struct has a key the vector has to be sorted. To sort the vector
you can't use "const".
* Changes due to code review
* Improve code readability
* Add generate of JSON schema to string to lib
* option indent_step is supported
* Remove unused variables
* Fix break in test
* Fix style to be consistent with rest of the code
* Fix double quotes for (u)int8 in json schema
* Fix reference file for JSON schema test
* Fix reference file for JSON schema test
* [idl_parser] Validate `force_align` on all possible paths
- added validation of `force_align` for `Vector`.
- added assertion to `flatbuffers::PreAlign` method
These changes should resolve following oss-fuzz issues:
- 6275816289861632
- 5713529908887552
- 4761242994606080
* size_t problem on Mac
* Fix review notes
Limit the length of the fuzzer input to 16384 characters to prevent timeout
in JSON parser (Vector of tables with key).
Related oss-fuzz issues:
- 5742497110294528
on the current Java Implementation.
The code dependencies related to JVM were removed and the project is able to target all available platforms.
The only requirement to implement to fully support a target is to implement functions described in `ByteArray.kt`.
Right now the code support JVM and native targets. JS port still missing, but just be trivial to introduce.
Currently, only the `jvm` and `macosX64` targets are enabled until we figure out how to enable tests on all
platforms on CI.
A submodule called "benchmark" is also introduced. It contains a series
of benchmarks comparing Java and Kotlin implementations of FlexBuffers and the UTF8 API.
Finally, this commit does not contain the scripts necessary to publish the artifacts. This will
be introduced at a later stage once the team has an agreement on how to rollout Kotlin releases.
* Remove a lot of redundancy from the Python generated code.
Update tutorial to reflect new Python generated code.
* Add aliases for newly deprecated Python generated methods.
This should help with backwards compatibility.
* Fix incorrect names in deprecated comments.
* remove inner attributes
* Added test for outdir in Rust
* add bin/outdir
* Moved outdir test to its own package and only run it if flatc is available
Co-authored-by: Casper Neo <cneo@google.com>
* Add codegen test for namespaced unions
* [Rust] Handle cross-namespace union use
* [Rust] Test namespace handling
* [Rust] Drop trailing whitespace in codegen
* [Rust] Set flags in generate_code.bat to match .sh
* [C#] Add additional namespace test file
* Define presence.
* Migrate to IsRequired and IsOptional methods
* moved stuff around
* Removed optional and required bools from FieldDef
* change assert to return error
* Fix tests.cpp
* MakeFieldPresence helper
* fmt
* old c++ compatibility stuff
Co-authored-by: Casper Neo <cneo@google.com>
* Apply NativeName before WrapInNameSpace in idl_gen_cpp.cpp
* remove actual_type argument from GetUnionElement -- it's always true
* Merge GetUnionElement's native_type and wrap_native argument -- they always have the same value.
* Use convenience method WrapNativeNameInNameSpace
* Remove wrap_namespace argument from GetUnionElement
* Move declaration closer to first use.
I'm also clarifying that while elements are not deduplicated, vectors
may contain more than one offset to the same value.
Signed-off-by: Juan Cruz Viotti <jv@jviotti.com>
This is an attempt to explain how FlatBuffers encodes union types as an
extra section in the "FlatBuffers internals" document.
Signed-off-by: Juan Cruz Viotti <jv@jviotti.com>
* TS/ES6 modules spike iteration 1
* Initial modularized dasherized output
* Remove obsoleted parts and namespace wrapping
* Use _flatbuffers_ prefix
* First part of imports logic
* Second part of imports logic
* Fix TS/JS code removal mixup
* Alias imported symbols if same name from different namespaces and some fixes
* Use star import for bare imports
* Fix messed up string concat
* var to const and remove not needed semi
* Remove some cases of ns prefixing
* Add missing space
* Cleanups
* Completed initial import tracking logic
* Compilable output
* Adjust TypeScriptTest and dependents to work
* Use local flatbuffers package for tests
* Refactor away use of any
* Remove obsolete imported_fileset and reexport_map
* Still need any and fix JavaScriptTest.sh
* Fix test runs out of the box
* Temp add generated files
* TypeScriptTest replaces JavaScriptTest and cleanups
* Also remove reference to JavaScriptTest in TestAll.sh
* Remove old generated ts/js files
* Remove use of --js in generate_code scripts
* idl_gen_js_ts to idl_gen_ts and removal of js gen
* Remove obsoleted options
* Fix obsolete ts test detection
* Tweak ts compilation be as strict as possible
* Remove jsdoc type annotation generation
* Generated test ts files
* Fix search and replace messup
* Regenerated ts test output
* Use CharToLower
* Use normal for loop
* Rework namespacedir
* Revert "Rework namespacedir"
This reverts commit 6f4eb0104ceeb86011bb076ebca901138c48e068.
* Revert "Use normal for loop"
This reverts commit 676b2135bfaa1853dfbb06c92b5c16a0d81bb13a.
* Revert "Use CharToLower"
This reverts commit 2d08648d0d72d0af201fad80d54cdc76412b35e9.
* Again do rework but correct
* Avoid runtime cast
* Fix test runs
* Also add npm install to get tsc
* Bump node test versions
* for range to std for loop
* Clang format
* Missed one clang format
* Move accessor to later
* Attempt to make windows version of TypeScriptTest
* Want to see the output
* Try to get newer node at appveyor
* Style changes
Added the new method LookupTableByName for searching symbols in parent namespaces.
This method speedup (x50 or greater) symbol resolution (enum or struct) in parent namespaces.
The speedup was measured without `table.empty()` guard condition.
This method should suppress fuzzer timeout issue without artificial limits for nested namespaces (https://oss-fuzz.com/testcase-detail/6244168439169024).
Additionally, this commit speedup (x5) the GetFullyQualifiedName method by removing unnecessary temporary std::string object.
* [codegen] Search for includes in the directory containg the current file
This matches the behavior of a C preprocessor, see:
https://gcc.gnu.org/onlinedocs/cpp/Search-Path.html
* Skip FileExists() when file is guaranteed to not exist
* end comment with period to match others
* Unset FieldDef.optional if its key
* removed StringKey table, just removed an extra "required, key"
* removed extra newline
Co-authored-by: Casper Neo <cneo@google.com>
* Fix miri problems by assuming alignment is 1 in rust
* Removed is_aligned fn from rust verifier.
* Add back is_aligned, but make it w.r.t. buffer[0]
* touch unused variable
* touch unused variable
* +nightly
* Move Rust miri testing into its own docker
* fix bash
* missing one endian conversion
* fix endianness2
* format stuff
Co-authored-by: Casper Neo <cneo@google.com>
For some reason, this fuzzer failed to load the binary schema file
when run on the `/clusterfuzz` server.
Issue: https://oss-fuzz.com/testcase-detail/6215075358703616
This issue doesn't reproduce locally with the latest oss-fuzz docker image.
* tests/GoTest.sh: Fix flags.Parse location to work on new go SDKs.
Calling flags.Parse() within init() races with other packages which register
flags in their init(), and in particular with the testing package itself. It is
more reliable to call flags.Parse() from a TestMain implementation.
See https://github.com/golang/go/issues/31859,
https://github.com/golang/go/issues/33869.
* .github: Enable build-go action in build.yaml workflow.
This commit fixes handling of default and NULL `key` fields in `Parser::ParseVector` (#5928).
The JSON generator updated. It outputs `key` fields even if the `--force-defaults` option is inactive.
Additional test cases for `key` added.
It errors with "Fatal error: Uncaught exception 'InvalidArgumentException' with message 'bad number for type byte.. in /home/runner/work/flatbuffers/flatbuffers/php/ByteBuffer.php:490" which I can't reproduce locally, and trying to fix it on CI runs into PHP's insane handling of numbers vs strings.
* [idl_parser] Improve stack overflow protection
Add stack overflow protection for Flexbuffer and nested Flatbuffer parsers.
Replaces the `Recurse()` method by the new ParseDepthGuard RAII class.
* Remove move operator from Parser.
It was wrong decision to add move ctor and assignment into Parser class.
These operators will make it extremely difficult to add constant or reference fields in the future.
* Remove ';' from definition of FLATBUFFERS_DELETE_FUNC
* Format code
* Make this PR compatible with MSVC2010 (it doesn't support inherited ctor)
* [idl_parser] Check the range of explicitly set field's id value
The explicitly set `id` attribute should be a non-negative value of the `voffset_t` type.
* Format FieldIdentifierTest()
* Adds shared strings and tests for shared strings
* Adds resets on string_map
* Moved shared strings to use vector instead of hashmap
* Addresses all the issues
* Resolves some comments
* Rebuild the way swift handles structs from scratch
* Updates docs, and sample binary
* Replaces InMemory to Mutable
* Migrates docs from inmemory
* use inline for some functions
* Renamed Mutable objects
* Updates documentation
The infinityf symbol is causing a conflict when building for cygwin. In
the cygwin math.h header there is also a symbol called infinityf. So
this patch is needed to be able to build the flatbuffer tests in a
cygwin environment.
* Add vectorNumElements attribute to Builder for simpler vector creation.
This adds a default to EndVector which should simplify its use.
* Update tutorial to reflect new default arg in Python EndVector.
* Remove optional argument to Python EndVector.
* Add generated files.
* Unset Builder.vectorNumElems when not in use.
* Added missing EndTable() call to VerifyObject()
VerifyObject called VerifyTableStart() but not EndTable(). This made Verifier::VerifyComplexity() increase depth_ with each table, not with the depth of tables.
https://groups.google.com/forum/#!topic/flatbuffers/OpxtW5UFAdg
* Added Check to VerifyAlignment
https://stackoverflow.com/questions/59376308/flatbuffers-verifier-returns-false-without-any-assertion-flatbuffers-debug-veri
* Add GetStringView (Convenience function to get string_view from a String returning an empty string_view on null pointer) like GetString, GetCstring
* flatc should warn, when an attribute is attached more than once.
flatc.exe -b duplicate.fbs
warning: duplicate.fbs(5, 36): warning: attribute already found: priority
duplicate.fbs:
namespace MyGame;
attribute "priority";
table Monster (priority:1, priority:2) {
}
root_type Monster;
* flatc should support --binary --schema with optional scalar fields.
This fixes 'error: Optional scalars are not yet supported in at least one the of the specified programming languages.' when calling flatc.exe --binary --schema with a schema containing optional scalars.
* Generate nullable properties in C# object-based API for optional scalars.
tests\generate_code.bat extended to test this.
Ran tests\generate_code.bat
Ran tests\Flatbuffers.Test\NetTest.bat
* %TEST_BASE_FLAGS% replaced with --gen-object-api in generate_code.bat, because only this is part of this PR. Added this same flag to generate_code.sh
* generate_code.bat and generate_code.sh changed to only test c# with object based api.
* Fix C/C++ Create<Type>Direct with sorted vectors
If a struct has a key the vector has to be sorted. To sort the vector
you can't use "const".
* Changes due to code review
* Improve code readability
* Add generate of JSON schema to string to lib
* option indent_step is supported
* Remove unused variables
* Fix break in test
* Fix style to be consistent with rest of the code
* Add option --no-warnings to inhibit all warnings
* Fix order of member initialization
* Add documentation for --no-warnings
This commit makes the names of fuzzing dictionaries the same as the target binary names.
Also it explicitly limits size of test inputs to prevent failures in `regex` and fuzzing time-outs.
* Updated comments and fixed a fundemental type error.
* bump rust flatbuffers semver
* Initial commit with verifier, need to clean up
* Verifier tested. Needs clean up and refactoring.
* Display for InvalidFlatbuffer and better errors for strings
* SimpleToVerify, some refactoring
* Combined VerifierType TableAccessorFuncBody into FollowType
* scrub todos
* Update Rust get_root functions.
There are 6 variants, with verifier options, default verifier options
and no verification "fast".
* Rename root fns
* inline
* Update to use thiserror
* fix for bad compiler
* improve error formatting
* Replace multiply with saturating_multiply
* saturating adds too
* Add docs disclaiming experimental verification system
Co-authored-by: Casper Neo <cneo@google.com>
It is useful to be able to call CreateSharedString with a string_view.
A string_view can be implicitly converted from a std::string or a const
char*. This means if string_view is available, we can use it instead of
both other functions and get all 3.
The flatbuffers::Parser::Parse() isn't an idempotent method for schema parsing.
This commit removes a wrong for-loop that tried to check the same schema twice.
Prior to this commit the default C++ code generator was `c++0x`.
A code generated with `c++0x` code-gen might have a vulnerability (undefined behavior) connected evolution of enums in a schema. This UB could break the backward compatibility if previously generated code casts an unknown enumerator to enum type that knows nothing about future enumerators added to the schema.
The main differences between `c++0x` and `c++11`:
- generated enums use explicitly declared underlying type;
- generated object-API tables don't declare default ctor() explicitly, instead of it default data member initializers are generated.
Please use `flatc --cpp-std c++0x` option for backward compatibility with old compilers.
- add a new method ParseJson to minimize failures during fuzzing
- add default (conditional) move-constructor for Parser
- add a new monster_fuzzer
- switch fuzzers to C++17 and `test/cpp17` generated code
* Add --require-explicit-ids to require explicit ids
We just got bit by a well intentioned developer forgetting that field
order by default is the field index. 3 people missed it in review.
I'm looking at ways to make it harder to mess up. We are requesting
that developers explicitly id all fields in tables. Automatic (opt in
for others) enforcement of this will help the effort succeed. This
patch adds a command line flag which lets the user require ids on all
fields in tables.
* Added docs to Compiler.md as well
* idl_gen_json_schema.cpp: Changed generation of array element types
#6175
* idl_gen_json_schema.cpp: Simplified indent generation as suggested by @vglavnyy
#6175
* Remove _POSIX_C_SOURCE and _XOPEN_SOURCE definitions when compiling on OpenBSD
* Only define _POSIX_C_SOURCE and _XOPEN_SOURCE for mingw/cygwin platforms
* Only define POSIX statements for mingw/cygwin/qnx platforms
* Added missing EndTable() call to VerifyObject()
VerifyObject called VerifyTableStart() but not EndTable(). This made Verifier::VerifyComplexity() increase depth_ with each table, not with the depth of tables.
https://groups.google.com/forum/#!topic/flatbuffers/OpxtW5UFAdg
* Added Check to VerifyAlignment
https://stackoverflow.com/questions/59376308/flatbuffers-verifier-returns-false-without-any-assertion-flatbuffers-debug-veri
* Add GetStringView (Convenience function to get string_view from a String returning an empty string_view on null pointer) like GetString, GetCstring
* flatc should warn, when an attribute is attached more than once.
flatc.exe -b duplicate.fbs
warning: duplicate.fbs(5, 36): warning: attribute already found: priority
duplicate.fbs:
namespace MyGame;
attribute "priority";
table Monster (priority:1, priority:2) {
}
root_type Monster;
* flatc should support --binary --schema with optional scalar fields.
This fixes 'error: Optional scalars are not yet supported in at least one the of the specified programming languages.' when calling flatc.exe --binary --schema with a schema containing optional scalars.
Versions of rules_go below 0.24 include platforms repository, which are lacking
definitions of os:macos, cpu:arm64 constraints.
This definitions are needed to properly select toolchains in latest
bazel.
Android build was dated, using the Android.mk approach. Current
project configuration on Android encourages the usage of CMake, so we
are updating the android project as an example on how to use either the
Java/Kotlin generate code or the native C++ one.
* Refactor idl_gen_rust to a ForAllX continuation pattern.
* Removed unneeded SetValue and updated sample rust gencode
* Make Rust flatbuffers print right
* Generated code and removed unnecessary trait constraint
* bumped rust version. Release required
* removed an unwrap in Rust Debug-print unions
* Tested formatting flatbuffers in rust.
* Set float precision in flaky debug-print test
* impl Debug for structs too
Co-authored-by: Casper Neo <cneo@google.com>
* Add test for optional enums in Kotlin
* Rename optional_scalars2.fbs into optional_scalars.fbs
Also updated all references in the project to point to
"optional_scalars.fbs" instead of "optional_scalars2.fbs".
* Updated SupportsAdvancedUnionFeatures to look out for string
* Mass refactoring to use BASE_TYPE helper functions.
Co-authored-by: Casper Neo <cneo@google.com>
* Rework enums in rust.
They're now a unit struct, rather than an enum. This is a
backwards incompatible change but the previous version had UB
and was also backwards incompatible so...
* Update and test sample rust flatbuffers
* Use bitflags crate to properly support rust enums.
Previously, the bitflags attribute was just ignored. This is a breaking change
as the bitflgs API is not like a normal rust enum (duh).
* variant_name() -> Option<_>
* repr transparent
* Reexport bitflags from flatbuffers
* Make bitflags constants CamelCase, matching normal enums
* Deprecate c-style associated enum constants
Co-authored-by: Casper Neo <cneo@google.com>
* Fix C/C++ Create<Type>Direct with sorted vectors
If a struct has a key the vector has to be sorted. To sort the vector
you can't use "const".
* Changes due to code review
* Improve code readability
* Add generate of JSON schema to string to lib
* option indent_step is supported
* Remove unused variables
* Fix break in test
* Fix style to be consistent with rest of the code
* Add option to not generate direct copy methods.
The direct copy methods generated by flatc utilize std::vector which isn't allowed on some embedded systems. Permit users of the compiler to not generate these methods so they don't have to be stubbed out.
* Update docs for no-cpp-direct-copy option.
* Fixed rust nested flatbuffers for tables other than self
* replaced lifetimes
* Use WrapInNameSpace and also update samples
Co-authored-by: Casper Neo <cneo@google.com>
Older versions of GCC (at least on ARM) complain about narrowing conversions and
overflows when setting the reference index (defined as an 11-bit signed integer
bitfield) to -1:
```
error: narrowing conversion of '-1' from 'int' to 'short unsigned int' inside { } [-Wnarrowing]
error: conversion from 'short unsigned int' to 'short unsigned int:11' changes value from '65535' to '2047'
```
This is due to the signedness of integer bitfields (not explicitly signed or
unsigned) being implementation-defined. This addresses this issue by explicitly
defining the signedness of all the bitfields in `flatbuffers::TypeCode` in C++.
* GRPC implementation for Typescript
* Fixes a couple of issues
* Finished implementing the typescript support for grpc
* Updated generated code
* Fixes CI
* Moved C++ to optional_scalars2 and added some tests.
Also deleted unused optional_scalars_generated.lobster
* Fixed whitespece in C++ gencode & fixed BUILD file
* Moved C++ onto optional_scalars2 in the .bat file
Co-authored-by: Casper Neo <cneo@google.com>
UnpackTo copies vector elements one-by-one, which can be very inefficient depending on the quality of the compiler optimizations performed. This change updates the operation for vectors of bytes that aren't enums to use 'std::copy', which is usually highly optimized.
vectors of types that are more than one byte can't be optimized in this way because of the endianness of the serialized bytes vs. the target architecture endianness.
vectors of enums can't be optimized because they are required to be static_cast into the appropriate enum type when stored in the vector.
vectors of bools can be optimized in most cases, but since the standard
allows std::vector<bool> template specialization for space-savings,
std::copy doesn't work on every implementation (looking at you
Microsoft). Thus, this optimization is skipped for vector<bool>.
For a specific example, this improves the latency of unpacking large buffers on the Hexagon DSP by about 10x.
Co-authored-by: Matthew Markwell <markwell@google.com>
* [C#] Fix and improve project files
* "net35" target included for Unity 5
* "net46" target included for Unity 2017
* "netstandard2.0" target included for Unity 2018 and general use
* "netstandard2.1" target included for Span<T> support
Included project properties for defining UNSAFE_BYTEBUFFER, BYTEBUFFER_NO_BOUNDS_CHECK, and ENABLE_SPAN_T conditional compilation symbols.
* Add documentation on building for C#
* Added missing EndTable() call to VerifyObject()
VerifyObject called VerifyTableStart() but not EndTable(). This made Verifier::VerifyComplexity() increase depth_ with each table, not with the depth of tables.
https://groups.google.com/forum/#!topic/flatbuffers/OpxtW5UFAdg
* Added Check to VerifyAlignment
https://stackoverflow.com/questions/59376308/flatbuffers-verifier-returns-false-without-any-assertion-flatbuffers-debug-veri
* Add GetStringView (Convenience function to get string_view from a String returning an empty string_view on null pointer) like GetString, GetCstring
* flatc should warn, when an attribute is attached more than once.
flatc.exe -b duplicate.fbs
warning: duplicate.fbs(5, 36): warning: attribute already found: priority
duplicate.fbs:
namespace MyGame;
attribute "priority";
table Monster (priority:1, priority:2) {
}
root_type Monster;
* [Python] Codegen SizeOf classmethod for structs
This codegens a `SizeOf()` classmethod for all structs since we can't
determine the size of a FlatBuffer generated struct from Python otherwise.
* [JS/TS] Codegen sizeOf static method for structs
This codegens a `sizeOf()` static method for all structs since we can't
determine the size of a FlatBuffer generated struct from JavaScript or
TypeScript otherwise.
Add static cast from float to double in flexbuffers.h to avoid implicit double promotion error. This error is surfacing during the tensorflow lite for microcontrollers build since we enabled -Werror and -Wdouble-promotion.
* ENABLE_SPAN_T doesn't require UNSAFE_BYTEBUFFER.
Change to ENABLE_SPAN_T that doesn't require UNSAFE_BYTEBUFFER.
* Selectable framework.
Changed target framework to allow selection of 2.0 or 2.1 (or higher)
* Added target framework version check.
* Add core test project.
* Added run on .Net Core.
* [C#] Fix and improve project files
* "net35" target included for Unity 5
* "net46" target included for Unity 2017
* "netstandard2.0" target included for Unity 2018 and general use
* "netstandard2.1" target included for Span<T> support
Included project properties for defining UNSAFE_BYTEBUFFER, BYTEBUFFER_NO_BOUNDS_CHECK, and ENABLE_SPAN_T conditional compiler constants.
* Add documentation on building for C#
This commit adds replacement of `::tolower` and `::toupper`.
Added CharToLower and CharToUpper routines reduce the number of cast operators
that required for correct usage of standard C/C++ `::tolower/toupper` routines.
This optionally generates a static `getFullyQualifiedName()` function to get
the fully-qualified name of a type in JavaScript and TypeScript in a similar
fashion to the C++ codegen.
Related with issue #6113.
`(old_buf_size & 0xC0000000) != 0` checks if we can duplicate old_buf_size
and still be under 2GB (by checking if bit 30 or 31 is 1). This doesn't
allow buffers larger than 1GB.
The strategy now is to allocate a buffer with the maximum array size
when we detect that we are overflowing the 2GB.
Also changed default buffer size to 1024.
* Adding FlexBuffers support for Dart language
* Introduce snapshot method.
* Fix docu
* Replacing extension methods with static methods in order to support older Dart version
* Improving code based on PR feedback. Mainly rename refactoring.
* Addressing all PR feedback which does not need clarification
* exchange dynamic type with Object
* Adds better API documentation.
[] operator throws a very descriptive exception in case of a bad key.
* Implementation of JavaScript FlexBuffers decoder
* implements JS FlexBuffers builder
* replacing _toF32 with Math.fround
* Introducing test for BigInt number
* Moving functions from BitWitdth & ValueType object into BitWidthUtil and ValueTypeUtil accordingly.
Removing defensive checks.
Introducing fix for large int numbers by converting them to BigInt type.
Introducing handling for BigInt type in `add` method.
Using TextEncoder and Decoder to handle string / utf8 conversion.
* rename variable
* Lets user turn deduplication strategies for strings, keys and vector of keys off while building FlexBuffer.
Implements quick sort and choses quick sort if the number of keys is bigger then 20.
Removes unnecessary dict lookups in BitWidthUtil helper functions
* make iwidth and uwidth computation simpler and faster
* Making redInt and readUint a bit faster and shim the BigInt / BigUint usage
* Fixing a bug in FlexBuffers JS, where offsets and lengths are stored and read as int and not as uint values.
* Fixing a bug in FlexBuffers Dart, where offset and length values are stored and read as int values instead of uint values
* Adding FlexBuffers support for Dart language
* Introduce snapshot method.
* Fix docu
* Replacing extension methods with static methods in order to support older Dart version
* Improving code based on PR feedback. Mainly rename refactoring.
* Addressing all PR feedback which does not need clarification
* exchange dynamic type with Object
* Adds better API documentation.
[] operator throws a very descriptive exception in case of a bad key.
* Implementation of JavaScript FlexBuffers decoder
* implements JS FlexBuffers builder
* replacing _toF32 with Math.fround
* Introducing test for BigInt number
* Moving functions from BitWitdth & ValueType object into BitWidthUtil and ValueTypeUtil accordingly.
Removing defensive checks.
Introducing fix for large int numbers by converting them to BigInt type.
Introducing handling for BigInt type in `add` method.
Using TextEncoder and Decoder to handle string / utf8 conversion.
* rename variable
* Lets user turn deduplication strategies for strings, keys and vector of keys off while building FlexBuffer.
Implements quick sort and choses quick sort if the number of keys is bigger then 20.
Removes unnecessary dict lookups in BitWidthUtil helper functions
* make iwidth and uwidth computation simpler and faster
* Making redInt and readUint a bit faster and shim the BigInt / BigUint usage
This commit performs refactoring (Extract Method) of the C++ code generator.
It extracts code generation of a table getter/mutator into separated methods.
* Attach JvmStatic annotation to each method of companion object
Kotlin does not have static accessor so companion object used instead of static.
It's so natural. But when use kotlin companion object methods on java it is very inconvenient.
```java
GeneratedClassByFlatBuffer.Companion.someMethod()
```
If use @JvmStatic annotation it can be shorten like below.
```java
GeneratedClassByFlatBuffer.someMethod()
```
* Formatting by Idea Google C++ style
* Add comments - Commit for missing cla
Signed-off-by: Yoon KyongSik <sam1287@gmail.com>
* Add comments - Commit for missing cla
Signed-off-by: Yoon KyongSik <sam1287@gmail.com>
* Reset code formatting except modified lines
Signed-off-by: Yoon KyongSik <sam1287@gmail.com>
* Pass missing flag to validateVersion method
Signed-off-by: Yoon KyongSik <sam1287@gmail.com>
* Add annotations to missing method in companion object
* addVector
* createVector
* endVector
* tableCreator
And also I tried add compiler option for generate annotation who don't like this operation.
Signed-off-by: Yoon KyongSik <sam1287@gmail.com>
* Modify unmatched option name in compiler usage
Signed-off-by: Yoon KyongSik <sam1287@gmail.com>
* Add missing operation for finishSizePrefixed and finishStructBuffer method.
Signed-off-by: Yoon KyongSik <sam1287@gmail.com>
* Add compiled example with --kotlin-gen-jvmstatic option.
Signed-off-by: Yoon KyongSik <sam1287@gmail.com>
* Fix Compiler.md
Change option name from --gen-jvm-static-annotation to --kotlin-gen-jvmstatic
Signed-off-by: Yoon KyongSik <sam1287@gmail.com>
* Use IDLOptions reference instead of bool parameter.
Signed-off-by: Yoon KyongSik <sam1287@gmail.com>
* Change option name - kotlin_gen_jvmstatic to gen_jvmstatic
Signed-off-by: Yoon KyongSik <sam1287@gmail.com>
* Use IDLOptions reference instead of bool parameter and missing process @JvmStatic as suffix.
Signed-off-by: Yoon KyongSik <sam1287@gmail.com>
* Add code generation for --gen-jvmstatic option
Signed-off-by: Yoon KyongSik <sam1287@gmail.com>
* Miss typo directory for including.
Signed-off-by: Yoon KyongSik <sam1287@gmail.com>
* Miss typo variable suffix for including.
Signed-off-by: Yoon KyongSik <sam1287@gmail.com>
* Fix camel case to snake case.
Signed-off-by: Yoon KyongSik <sam1287@gmail.com>
* Rollback generate code for gen_jvmstatic option.
Signed-off-by: Yoon KyongSik <sam1287@gmail.com>
* Delete generated test files.
Signed-off-by: Yoon KyongSik <sam1287@gmail.com>
* add missing new line at end of file.
Signed-off-by: Yoon KyongSik <sam1287@gmail.com>
* Remove generated test file by command line.
Signed-off-by: Yoon KyongSik <sam1287@gmail.com>
Co-authored-by: sam <sam@jennifersoft.com>
* Cleaned up .NET testing script for Mono
Cleaned up the .NET testing script to make it a little better. It
purposefully doesn't delete the .NET installer and SDk after running the
script, so that they can be used in subsequent invocations. This greatly
speeds up the script.
The downloaded files are ignored by git by default. They can be
explicitly cleaned up by runnning the clean script (clean.sh).
* Trying using older Version indicator
* Remove lins to monsterdata and reference it directly.
* Updated appveryor script to remove copying of files no longer needed
* Continue to update appveyor script to work. Disabled CS0169 in ByteBufferTest
* [TS] Fix four bugs with imported types in TypeScript.
* When a type had a vector of imported enums:
1) the enum type's file wasn't added to the generated code's list of
imports; and
2) the enum wasn't prefixed with the NS<hash> prefix and wasn't getting
resolved; but
3) non-enum types (ie, "flatbuffers.Offset") were getting the NS<hash>
prefix when they weren't.
* Also, type name prefixes weren't properly attributed with imported
structs in unions because the source definition passed to the typename
prefixing method was for the union, not for the location of the imported
struct.
* clang fmt
* Use of enum_def / struct_def for prefixing types needs to have the files
added to imported files when not generating all types.
* clang fmt
This PR attempts to switch namespace from public enum back to ordinary
concat with _ in Swift. This kept style similar with protobuf, but
different from other popular style guide in Swift.
This is needed because previously, when we do `public enum`, we don't
really know when to declare and when to extend (extension). With namespace
implementation in this PR, there is no such ambiguity.
Adds swift test code
Replaces if statments
Adds swift to supported languages for optionals
Moved std::string to auto
Adds nullable scalars support in object api
* [C++] Fix compiler error from deleted assignment operator (#6036)
The assignment operator of the `buf_` member is deleted, we cannot call it from the assignment operator of the `TableKeyComparator` struct.
=> Also delete the assignment operator of the `TableKeyComparator` struct (already private anyhow).
* [C++] Fix compiler error from deleted assignment operator (#6036) - fix extraneous semicolon
The assignment operator of the `buf_` member is deleted, we cannot call it from the assignment operator of the `TableKeyComparator` struct.
=> Also delete the assignment operator of the `TableKeyComparator` struct (already private anyhow).
* First draft of rust optionals
* Code cleanup around ftBool and ftVectorOfBool
* Tests for Rust optional scalars
* test bools too
Co-authored-by: Casper Neo <cneo@google.com>
This is the first step to upgrade grpc dependency to the latest version.
- Patch protobuf 3.6.1. and grpc 1.15.1 to fix build errors when using the latest Bazel version (3.4.1).
- Add grpc/tests:grpc_test. One can kick off tests in Bazel by calling `bazel test grpc/tests/...`.
- Add missing build targets in tests/BUILD in order to support grpc/tests/BUILD
* Perpares swift to take optional scalars + adds optional string helper method + disables linters in generated code
* Small fix for generated code
* Update grpc support to alpha 17 for swift
* Parser support for nullable scalars
* Use older C++ features
* use default element
* Add a test for json, flexbuffers, and null
* test comments and names
Co-authored-by: Casper Neo <cneo@google.com>
Dart schema compiler generated a static getter for enum values, which
always created a new map instance for its callers. See #5819.
Now it generates const map for better performance
and readability.
Previous FB import was based on the original early TS implementation
and did not take into accout NPM. There doesn't seem to be a use
for current implemenentation anymore, while NPM compatibility is
needed.
Co-authored-by: Kamil Rojewski <kamil.rojewski@gmail.com>
Added a method FlatBufferBuilder::createSharedString that
enable string sharing for building messages on java.
The shared pool will only contains strings inserted by
this methods.
* [Swift] FlatBuffers createMonster method doesn't treat struct properly
This PR fixed a issue where a struct is not treated properly when use
create inside.
A example would be the pos inside Monster. The createMonster method
takes an Offset for pos. However, FlatBuffersBuilder.add(struct:)
doesn't really take Offset argument. That means we don't really add a
struct at all for Monster.
It will show up as the pos never set.
This doesn't show up in FlatBuffersMonsterWriterTests.swift because it
implements its own createMonster method, which happens do the dance
properly (i.e. first call create(struct) and then immediately call
add:).
This PR modified the `add(pos:)` interface such that it takes the
UnsafeMutableRawPointer directly, calling `create(struct:)` under the hood.
I can add unit tests once the direction of this PR approved.
* Fix object api pack method codegen.
* Add unit tests that uses Monster.createMonster method to serialize.
* Updated sample_binary.swift
flatbuffers project currently depends on an old version of GRPC library and has known issues working with the latest version (issue #5836). Therefore, a patch file is created and put under bazel/ directory before supporting the latest GRPC version.
* Added code gen for evolution tests back in.
* General generate code and clang format
* Added code gen for evolution tests back in.
* General generate code and clang format
* reran generate_code.sh
* Added code gen for evolution tests back in.
* General generate code and clang format
* Added code gen for evolution tests back in.
* General generate code and clang format
* Enforce snake_case for schema field names
* Switched to basic for loop, Fixed warning message
* Fix doc comment warnings
Can't use doc comment "///" syntax on macros, that generates the following warning:
warning: unused doc comment
--> src\flexbuffer_type.rs:236:5
|
236 | /// returns true if and only if the flexbuffer type is `VectorFloat4`.
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ rustdoc does not generate documentation for macros
|
= help: to document an item produced by a macro, the macro must produce the documentation as part of its expansion
So switched to just use ordinary "//" comments on these to be warning free
* Upgrade num_enum 0.4.1 -> 0.5.0
* Remove unused and non-working usage of test crates
* Remove usage of abandoned debug_stub_derive crate
Which brought in old pre-v1 syn and quote crates.
This replaces it with just manual Debug trait implementation instead for the 2 cases
* Fixed refractoring issue in reflection/generate_code.sh. Also, mv deletes the original file, so I don't need to clean it up manually in that case.
* Fixed Dart Tests by removing code-gen for included files.
* Added code gen for evolution tests back in.
* General generate code and clang format
* Added evolution schema generation to .bat file
* Added code gen for evolution tests back in.
* General generate code and clang format
* Added evolution schema generation to .bat file
* reran generate_code.sh
* Removed wildcard from generate_code.bat that doesn't work
* Initialize memory when clear ByteBuffer
It seems (based on my limited understanding) that FlatBuffers requires
the writable area to be 0 initialized. However, we missed it when clear
the buffer to reinitialize it.
This PR fixed that by calling initialize (also fixed the typo)
explicitly.
* Update version to 0.5.3
* [TS] Use proper TypedArray in create*Vector
This commit adds TypeScript function overloads to create*Vector for
proper TypedArray types, effectively resolves#5373.
* Add @deprecated to old Uint8Array overloads
* Removes posix definition
stat and fseeko are not used. Tested on QNX (GCC 5.4.0) and MinGw
* Updates realpath to more modern (2008) version
* Removes unix specific headers
* Adds detection of strtoll_l as android API < 21 does not provide it
* Includes cstdlib and formatting
Removes cmakelist alterations as not necessary. Formatting not complete.
* Stdlib outside if
Stdlib is available on all platforms
* Fixes indentation
* Adds locale check to android build
* Adds missing brace
* full names
* Removes again, no clue
* Updates base to check for locale independent android
Cmake already checks for others, also MSC?
* Changes to test on available and not requested
* Fixes android bad xopen_version define
* Removes warning
In >= C++11 mode, generate default member initializers instead of a
default constructor.
The new code is semantically equivalent, but will allow aggregate
initialization in C++20.
This is a different take on #5951.
Kotlin code generation was producing wrong logic for accessors
of vector of union elements. This was shadowed by the fact[1] that asserts
in Kotlin are silently ignored unless the flag "-ea" is passed to the JVM.
The tests are also updated to enable asserts.
1 - https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/assert.html
* RFC: Add ExternalStorage for ByteBuffer in Swift implementation
This PR proposed one more API for ByteBuffer such that no copy is
required to parse FlatBuffers content. This API has limited use, but for
cases that you need to read part of the flatbuffers' data to decide
whether you want to parse / copy the full buffer out, it is useful.
* Use a variable rather than protocol.
* Revert grouping changes from the other PR.
* Add unit test to read from unowned UnsafePointer.
* Manifest changed.
* use correct language formatter for TypeScript examples
* fixes typo in JS/TS copied from PHP (apparently)
the variables are not named with a prefixed `$`
* fixes bizarre line breaks in markdown examples
* fixes snake case typo to fit JS/TS conventions
* makes example of Uint8Array usage explicit
* removes random extra lines between language blocks
* adds simple example for writing to file in node
* typo: flabuffers => flatbuffers
* adds (previously missing) code blocks to TypeScript code block
* adds context about where `monster_generated` comes from
to the uninitiated, a bit of help like this is welcome
* Moves addition to overflow addition in swift by using &+
Moves code to use Int instead of UInt32 & fixes functions
Updates swift performance to great
Updated version to 0.5.2
Updated swift package version to 5.2
Updated docker to swift 5.2
Removed all none & arithmetic operations
* Small refactoring
* updates monsterdata.json to be valid json
the same monster.json file was not valid json
* updates reference to monsterdata.json in docs to also be valid json
The default operations per run is 30 which would be consumed on the reprocessing already-marked issues. Greatly increased the operations per run to process all issues and pull request (~200 + 83 * 5).
Uses the Stale action (https://github.com/actions/stale) to help clean up older Issues and PR. Set to 6 months to mark as stale, and 14 days to close stale.
* [rust] Add force_defaults method FlatBufferBuilder
This works just like the same method already available in other
languages.
* Add binary format test for force_defaults
* [Java] Grow ArrayReadWriteBuf enough to match requested capacity.
Also handle requested capacity overflowing.
* [Java] Improve readability of ArrayReadWriteBuf.requestCapacity()
* [Java] prepare testBuilderGrowth() to fail again once FlexBuffersBuilder uses ByteBufferReadWriteBuf internally, add TestFail() as a better alternative to a plain "assert false"
* [Java] Revert some test changes and extract a small string that was used more than once; one could say three times, however, it might be worthy of a discussion if the third occasion also falls into this category, as it is an independent use case and thus would work in the same way even if the value changed.
Co-authored-by: Markus <markus@greenrobot>
* Add static asserts to ensure that reflection API arrays are kept in sync
* Move changes from generated file into source fbs file
* Rename enum value and regenerate reflection_generated.h
* Add comments to each entries of type sizes array
* Fixed refractoring issue in reflection/generate_code.sh. Also, mv deletes the original file, so I don't need to clean it up manually in that case.
* Ensuring test/generate_code.sh was ran
* Fixed Dart Tests by removing code-gen for included files.
* General cleanup of codebase.
* Fixed refractoring issue in reflection/generate_code.sh. Also, mv deletes the original file, so I don't need to clean it up manually in that case.
* Fixed Dart Tests by removing code-gen for included files.
* Fix issue #5906, Prohibit declaration of an array of pointers inside structs
- idl_parser.cpp: Prohibit declaration of an array of pointers inside structs
- idl_gen_cpp.cpp: Extract GenStructConstructor() method from GenStruct() to simplify future modification
- idl_gen_cpp.cpp: Add assert for checking of Array fields in structs on code-generation stage
* Fix the error 'unused local variable' in release build
* Fix: format the PR code according to coding rules
* Add test-case and fix review notes
* Fixed refractoring issue in reflection/generate_code.sh. Also, mv deletes the original file, so I don't need to clean it up manually in that case.
* Thread safe reads of Double and Floats from ByteBuffer
* Cargo clippy lints
* more lints
* more lints
* Restored a doc comment
* Comment on float eps-eq and adjusted casting
* Rust Flexbuffers
* more serde tests, removed some unsafe
* Redid serde to be map-like and Reader is Display
* Moved iter from Reader to VectorReader
* Serious quickcheck + bugs
* wvo api
* Made types smaller for a reasonable speedup
* redid reading in a way that's a bit faster.
Profiling shows the rust slowdown as building +10%, reading +20%
* src/bin are developer binaries in rust
* Root and Map width are not packed
* key null check is debug only + doc changes
* BuilderOptions
* Documentation
* Documentation
* Moved tests to rust_usage_test
* Moved rust flexbuffers samples to Flatbuffers/samples
* Fixed RustTest
* Fixed for Rust 1.37.0
* Upgraded to rust 1_40_0
* fixed a little-endian-only feature in a test
* 1.40.0
* fixed some benchmarks for bigendian
* Updated .bat file
* misspelling
* Gold Flexbuffer test.
* Serialize,Deserialize, std::error::Error for Errors.
* Undo rustfmt in integration_test.rs
* from_slice instead of from_vec
* Added comments to unsafe blocks
* expanded on comment
* bump
Co-authored-by: CasperN <cneo@google.com>
* Moves the code to use _vtablestorage
Rebuilt the test to confirm to the new API
Adds documentation + generates code for grpc
Reverts indentation
v0.4.0
Updated swift/readme.md
Updates VtableStorage to ensure space instead of reallocating each time
Fixes str count not being correct
* Fixes issue with boolean constant not being set + removes unused function
This change is for fix warning:
```
CMake Warning (dev) at /home/aoleksy/share/cmake-3.17/Modules/FindPackageHandleStandardArgs.cmake:272 (message):
The package name passed to `find_package_handle_standard_args`
(flatbuffers) does not match the name of the calling package (FlatBuffers).
This can lead to problems in calling code that expects `find_package`
result variables (e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
FindFlatBuffers.cmake:33 (find_package_handle_standard_args)
CMakeLists.txt:6 (find_package)
This warning is for project developers. Use -Wno-dev to suppress it.
```
The native include files tag is intended to be used together with the object based api. A client which does not use the object based api should have no need to know about the additional include files needed for the object based api. The generated flatbuffers code without object api enabled should compile without the additional header files.
* idl_gen_rust.cpp: Fixgoogle/flatbuffers#5849 (Option/required-aware
codegen for unions)
The generated code was assuming that an Option is always returned by the
union-getter method: however, this is only true if the field is not
marked as `(required)`.
* idl_gen_rust.cpp: flip conditional
* idl_gen_rust.cpp: Add comment, as requested in review
The code added is not covered by the integration test
This option was in an earlier revision of flatbuffers_generate_headers
but was lost while refactoring by mistake. We have it as a function
argument rather than one of the optional flags so that we can use it to
inform the rule where to place the generated headers, so that the file
location matches the prefix that will be added to the filenames in the
generated files.
Also fixed some of the documentation in the file.
* added basic code
* backup work
* got class property to work
* backup progress
* implementented fmt for creating code
* added docs for genFieldUtils
* back up work
* added base helper js func
* added union js code
* added unpackTo and base for pack
* added pack code
* added null check for packing struct list
* passes compile test
* fixed some spacing of generated functions
* added annotations for constructors
* added obj api unpack test
* tested pack to work
* merge branch
* separated js and ts test
* fixed union signature to include string
* fixed generator to support string union
* hardcoded fb builder name
* refactored struct vector creation
* work around createLong
* handle default value in constructor
* update typescript docs
* added notes about import flag
* fixed formatting stuffs
* undo TypescriptTest change
* refactored fmt
* updated generated code
* remove ignoring union_vector for js
* revert changes for .project
* revert changes for package.json
* don't generate js in ts test
* fixed android project file
* removed unused js function
* removed package-lock.json
* adjust createObjList to new signature
* changed regex to callback style
* fixed package.json
* used existing func for generating annotation
* changed ternary to !!
* added return type for lambda
* removed callback style for obj api generator
* fixed js file indentation
* removed unused header
* added tests for string only union
* handle string only union and refactor union conv func
* updated generated ts files
* renamed union conv func
* made js test create files like other languages
* removed union string only handling
* don't allow null in createObjectOffsetList
* updated generated ts code
* changed the line that triggers Windows build errors
* hopefully fix CI error
* [C#] Fix nested structs and arrays in Object API
The adds support for nested structs and fixed size arrays in the C#
Object API codegen which previously generated invalid code that wouldn't
compile.
- Nested structs would originally generate syntax errors due to adding an
additional `.` to separate fields.
- Fixed size arrays of nested structs would originally generate code for
the first field in the top most struct, and would lead to a compiler
error due to referencing undefined variables.
* [C#] fix nested structs and arrays of structs.
* fix nested structs + arrays
* add table support
* Cleanup code
Co-authored-by: mugisoba <mugisoba+github@icloud.com>
When either source or destination or both namespaces were empty, flatc
was generating incorrect TS code.
For example:
```
export namespace {
export import ObjectId = NS10770172024577249292..ObjectId;}
```
In this case the target namespace is empty, and so is the namespace
between the NSxxx placeholder and the target type.
The original implementation of map access is very naive:
- Encode String to UTF8 byte[]
- Creates a new KeyVector
- Performs a binary search to find the key
- return value
So every access to the Map there was useless allocations of Keys and KeyVector
and complete encoding of the search key, which for most comparisons would be wasteful.
This changes completely removes the use of KeyVector and compute the key
positions on the spot. Besides that, it compares keys codepoint-by-codepoint,
avoiding unnecessary allocations and reducing encoding for most cases.
Some benchmarks result in a 2.75x speedup.
* [CMake] : Add precompiled header support with FLATBUFFERS_ENABLE_PCH
FLATBUFFERS_ENABLE_PCH enable precompile headers support for 'flatbuffers' and 'flatc'. Default if OFF.
You need to set it to ON to make it work. 'cmake -DFLATBUFFERS_ENABLE_PCH=ON ..'
This will only work if CMake version is >= 3.16, because target_precompile_headers is used.
If CMake doesn't have target_precompile_headers, then nothing will happen.
This can speed up compilation time.
Precompiled header file for 'flatbuffers' is 'pch.h'
Precompiled header file for 'flatc' is 'flatc_pch.h'
* Enable FLATBUFFERS_ENABLE_PCH for VC 2017 and VS2019 builds
* [CMake]: Fix error in FLATBUFFERS_ENABLE_PCH description
* Add a function add_pch_to_target to avoid copy and pasting the same code like add_fsanitize_to_target
Move pch.h and flatc_pch.h from include/flatbuffers to include/flatbuffers/pch
Make flatc_pch.h depends on pch.h since it extend it
flatbuffers_generate_headers is a function that can be used to generate Flatbuffers headers and binary headers. It uses named argument flags to wrap all the flags relevant to C++ header generation that flatc uses. It generates an interface library that can be added as a dependency of other targets.
flatbuffers_generate_binary_files is a function that can be used to generate Flatbuffers binaries. It uses named argument flags to wrap all the flags relevant to C++ binary generation from JSON file that flatc uses. It generates an interface library that can be added as a dependency of other targets.
Number of elements on the stack shouldn't affect the calculation
of ElemWidth(). Variable 'start' needs to be subtracted from the
loop variable 'i' to make indexing zero-based.
There is an additional unit test to pack nested vectors. Size of
the packed buffer *without* this fix is 798 and only 664 bytes
*with* the fix.
For example:
include/flatbuffers/reflection.h:365:8: error: definition of implicit copy
constructor for 'pointer_inside_vector<flatbuffers::Table, unsigned char>'
is deprecated because it has a user-declared copy assignment operator
[-Werror,-Wdeprecated-copy]
void operator=(const pointer_inside_vector &piv);
^
It's unclear why the old code wanted to declare a public `operator=`
without defining it; that just seems like a misunderstanding of the C++03 idiom
for deleting a member function. And anyway, we don't *want* to delete the
assignment operator; these are polymorphic types that do not follow value
semantics and nobody should ever be trying to copy them. So the simplest fix
is just to go back to the Rule of Zero: remove the declaration of `operator=`
and let the compiler do what it wanted to do originally anyway.
"The best code is no code."
Also, update the generated .h files.
Fixes#5649.
* flatc should generate a 'Create…' method for tables with struct fields when also generating the object based api (C#)
https://stackoverflow.com/questions/60724317/flatc-should-generate-a-create-method-for-tables-with-struct-fields-when-al
* missing namespace fixed: C:\projects\flatbuffers\tests\namespace_test\NamespaceA\TableInFirstNS.cs(30,7): error CS0246: The type or namespace name 'StructInNestedNST' could not be found (are you missing a using directive or an assembly reference?) [C:\projects\flatbuffers\tests\FlatBuffers.Test\FlatBuffers.Test.csproj]
Co-authored-by: stefan301 <Stefan.Felkel@de.Zuken.com>
Including CPack without the additional packaging information creates unused project files, and can potentially cause issues for projects including Flatbuffers that also use CPack.
* Parser reject "nan(n)" string as it does with nan(n)
* Adjust scalar fuzzer to ignore '$schema' substrings
- Scalar fuzzer ignores '$schema' substrings at the input
- Added 'scalar_debug' target to simplify research of fuzzed cases
* Improve formatting of './tests/fuzzer/CMakeLists.txt'
Please delete this standard text once you've created your own description.
Make sure you include the names of the affected language(s) in your PR title.
This helps us get the correct maintainers to look at your issue.
If you make changes to any of the code generators (`src/idl_gen*`) be sure to
[build](https://google.github.io/flatbuffers/flatbuffers_guide_building.html) your project, as it will generate code based on the changes. If necessary
the code generation script can be directly run (`scripts/generate_code.py`),
requires Python3. This allows us to better see the effect of the PR.
If you make changes to any of the code generators, be sure to run
`cd tests && sh generate_code.sh` (or equivalent .bat) and include the generated
code changes in the PR. This allows us to better see the effect of the PR.
If your PR includes C++ code, please adhere to the Google C++ Style Guide,
If your PR includes C++ code, please adhere to the
[Google C++ Style Guide](https://google.github.io/styleguide/cppguide.html),
and don't forget we try to support older compilers (e.g. VS2010, GCC 4.6.3),
so only some C++11 support is available.
For any C++ changes, please make sure to run `sh src/clang-format-git.sh`
For any C++ changes, please make sure to run `sh scripts/clang-format-git.sh`
stale-issue-message:'This issue is stale because it has been open 6 months with no activity. Please comment or label `not-stale`, or this will be closed in 14 days.'
close-issue-message:'This issue was automatically closed due to no activity for 6 months plus the 14 day notice period.'
days-before-issue-stale:182# 6 months
days-before-issue-close:14# 2 weeks
exempt-issue-labels:not-stale
stale-pr-message:'This pull request is stale because it has been open 6 months with no activity. Please comment or label `not-stale`, or this will be closed in 14 days.'
close-pr-message:'This pull request was automatically closed due to no activity for 6 months plus the 14 day notice period.'
* flatc: `--grpc-callback-api` flag generates C++ gRPC Callback API server `CallbackService` skeletons AND client native callback/async stubs (unary + all streaming reactor forms) (opt-in, non-breaking, issue #8596).
* Swift - Adds new API to reduce memory copying within swift (#8484)
* Rust - Support Rust edition 2024 (#8638)
* [C++] - Use the Google Style for clang-format without exceptions (#8706)
* Migrate from rules_nodejs to rules_js/rules_ts (take 2) (#7928)
*`flat_buffers.dart`: mark const variable finals for internal Dart linters
* fixed some windows warnings (#7929)
* inject no long for FBS generation to remove logs in flattests (#7926)
* Revert "Migrate from rules_nodejs to rules_js/rules_ts (#7923)" (#7927)
* Migrate from rules_nodejs to rules_js/rules_ts (#7923)
* Only generate @kotlin.ExperimentalUnsigned annotation on create*Vector methods having an unsigned array type parameter. (#7881)
* additional check for absl::string_view availability (#7897)
* Optionally generate Python type annotations (#7858)
* Replace deprecated command with environment file (#7921)
* drop glibc from runtime dependencies (#7906)
* Make JSON supporting advanced union features (#7869)
* Allow to use functions from `BuildFlatBuffers.cmake` from a flatbuffers installation installed with CMake. (#7912)
* TS/JS: Use TypeError instead of Error when appropriate (#7910)
* Go: make generated code more compliant to "go fmt" (#7907)
* Support file_identifier in Go (#7904)
* Optionally generate type prefixes and suffixes for python code (#7857)
* Go: add test for FinishWithFileIdentifier (#7905)
* Fix go_sample.sh (#7903)
* [TS/JS] Upgrade dependencies (#7889)
* Add a FileWriter interface (#7821)
* TS/JS: Use minvalue from enum if not found (#7888)
* [CS] Verifier (#7850)
* README.md: PyPI case typo (#7880)
* Update go documentation link to point to root module (#7879)
* use Bool for flatbuffers bool instead of Byte (#7876)
* fix using null string in vector (#7872)
* Add `flatbuffers-64` branch to CI for pushes
* made changes to the rust docs so they would compile. new_with_capacity is deprecated should use with_capacity, get_root_as_monster should be root_as_monster (#7871)
* Adding comment for code clarification (#7856)
* ToCamelCase() when kLowerCamel now converts first char to lower. (#7838)
* Fix help output for --java-checkerframework (#7854)
* Update filename to README.md and improve formatting (#7855)
If you are interesting in contributing to the flatbuffers project, please take a second to read this document. Each language has it's own set of rules, that are defined in their respective formatter/linter documents.
# Notes
- Run the linter on the language you are working on before making a Pull Request.
- DONT format/lint the generated code.
# Languages
## C++
C++ uses `clang-format` as it's formatter. Run the following script `sh scripts/clang-format-git.sh`, and it should style the C++ code according to [google style guide](https://google.github.io/styleguide/cppguide.html).
## Swift
Swift uses swiftformat as it's formatter. Take a look at [how to install here](https://github.com/nicklockwood/SwiftFormat/blob/master/README.md#how-do-i-install-it). Run the following command `swiftformat --config swift.swiftformat .` in the root directory of the project
## Typescript
Typescript uses eslint as it's linter. Take a look at [how to install here](https://eslint.org/docs/user-guide/getting-started). Run the following command `eslint ts/** --ext .ts` in the root directory of the project
**FlatBuffers** is a cross platform serialization library architected for
maximum memory efficiency. It allows you to directly access serialized data without parsing/unpacking it first, while still having great forwards/backwards compatibility.
## Quick Start
1. Build the compiler for flatbuffers (`flatc`)
Use `cmake` to create the build files for your platform and then perform the compilation (Linux example).
```
cmake -G "Unix Makefiles"
make -j
```
2. Define your flatbuffer schema (`.fbs`)
Write the [schema](https://flatbuffers.dev/flatbuffers_guide_writing_schema.html) to define the data you want to serialize. See [monster.fbs](https://github.com/google/flatbuffers/blob/master/samples/monster.fbs) for an example.
3. Generate code for your language(s)
Use the `flatc` compiler to take your schema and generate language-specific code:
```
./flatc --cpp --rust monster.fbs
```
Which generates `monster_generated.h` and `monster_generated.rs` files.
4. Serialize data
Use the generated code, as well as the `FlatBufferBuilder` to construct your serialized buffer. ([`C++` example](https://github.com/google/flatbuffers/blob/master/samples/sample_binary.cpp#L24-L56))
5. Transmit/store/save Buffer
Use your serialized buffer however you want. Send it to someone, save it for later, etc...
6. Read the data
Use the generated accessors to read the data from the serialized buffer.
It doesn't need to be the same language/schema version, FlatBuffers ensures the data is readable across languages and schema versions. See the [`Rust` example](https://github.com/google/flatbuffers/blob/master/samples/sample_binary.rs#L92-L106) reading the data written by `C++`.
## Documentation
**Go to our [landing page][] to browse our documentation.**
## Supported operating systems
- Windows
- macOS
- Linux
- Android
- And any others with a recent C++ compiler (C++ 11 and newer)
## Supported programming languages
Code generation and runtime libraries for many popular languages.
1. C
1. C++ - [snapcraft.io](https://snapcraft.io/flatbuffers)
FlatBuffers does not follow traditional SemVer versioning (see [rationale](https://github.com/google/flatbuffers/wiki/Versioning)) but rather uses a format of the date of the release.
## Contribution
* [FlatBuffers Issues Tracker][] to submit an issue.
* [stackoverflow.com][] with [`flatbuffers` tag][] for any questions regarding FlatBuffers.
*To contribute to this project,* see [CONTRIBUTING][].
## Community
* [Discord Server](https:///discord.gg/6qgKs3R)
## Security
Please see our [Security Policy](SECURITY.md) for reporting vulnerabilities.
## Licensing
*Flatbuffers* is licensed under the Apache License, Version 2.0. See [LICENSE][] for the full license text.
# Workaround for https://github.com/conda/conda-build/issues/636
PYTHONIOENCODING:UTF-8
CONDA_INSTALL_LOCN:"C:\\Miniconda35-x64"
CMAKE_OPTIONS:""
CPP_TEST_OPTIONS:""
matrix:
- APPVEYOR_BUILD_WORKER_IMAGE:Visual Studio 2015
CMAKE_VS_VERSION:"10 2010"
CMAKE_OPTIONS:"-DFLATBUFFERS_BUILD_LEGACY=1"
CPP_TEST_OPTIONS:"--std-cpp c++0x"
MONSTER_EXTRA:"skip"
- APPVEYOR_BUILD_WORKER_IMAGE:Visual Studio 2015
CMAKE_VS_VERSION:"12 2013"
MONSTER_EXTRA:"skip"
- APPVEYOR_BUILD_WORKER_IMAGE:Visual Studio 2015
CMAKE_VS_VERSION:"14 2015"
MONSTER_EXTRA:""
- APPVEYOR_BUILD_WORKER_IMAGE:Visual Studio 2017
CMAKE_VS_VERSION:"15 2017"
MONSTER_EXTRA:""
- APPVEYOR_BUILD_WORKER_IMAGE:Visual Studio 2019
CMAKE_VS_VERSION:"16 2019"
MONSTER_EXTRA:""
platform:
- x86
- x64
configuration:
- Debug
- Release
before_build:
- set MONSTER_EXTRA=%MONSTER_EXTRA%
- cmake . -G"Visual Studio %CMAKE_VS_VERSION%" -DFLATBUFFERS_CODE_SANITIZE=1 %CMAKE_OPTIONS%
# This cuts down on a lot of noise generated by xamarin warnings.
- if exist "C:\Program Files (x86)\MSBuild\14.0\Microsoft.Common.targets\ImportAfter\Xamarin.Common.targets" del "C:\Program Files (x86)\MSBuild\14.0\Microsoft.Common.targets\ImportAfter\Xamarin.Common.targets"
- if exist "C:\Program Files (x86)\MSBuild\15.0\Microsoft.Common.targets\ImportAfter\Xamarin.Common.targets" del "C:\Program Files (x86)\MSBuild\15.0\Microsoft.Common.targets\ImportAfter\Xamarin.Common.targets"
- if exist "C:\Program Files (x86)\MSBuild\16.0\Microsoft.Common.targets\ImportAfter\Xamarin.Common.targets" del "C:\Program Files (x86)\MSBuild\16.0\Microsoft.Common.targets\ImportAfter\Xamarin.Common.targets"
build:
project:ALL_BUILD.vcxproj
verbosity:minimal
after_build:
- python conan/appveyor/install.py
- python conan/appveyor/build.py
install:
- set PATH=%CONDA_INSTALL_LOCN%;%CONDA_INSTALL_LOCN%\scripts;%PATH%;
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.