String default values parsed from .fbs schemas are un-escaped by the IDL
parser (e.g., \x22 becomes a raw " byte), but code generators embed these
raw values directly into generated source code string literals. This allows
specially crafted .fbs files to break out of string literals and inject
arbitrary code into generated C++, Rust, TypeScript, and Swift source.
Fix by adding EscapeCodeGenString() helper that re-escapes string content
before embedding, and applying it to all 7 affected injection points across
5 code generators (C++, Rust, TypeScript, Swift, FBS).
Resolves the TODO comments in idl_gen_cpp.cpp and idl_gen_rust.cpp.
* generate mutable union accessors
* add test
* Revert "add test"
This reverts commit 45e352b18f.
* update file
* formatter got in the way
* merge conflicts
* updated genned code
* manually fix code gen bc I can't figure out why this file won't code gen
---------
Co-authored-by: Wouter van Oortmerssen <aardappel@gmail.com>
* flatc builds and seems to work, some of the extra targets are having linker errors
* fix build system
* pipeline failures
* un-rename files
* refactor to use unique_ptr
* typo
* rm make_unique, add comments
* fix cmake
---------
Co-authored-by: Wouter van Oortmerssen <aardappel@gmail.com>
* Fixes to make SizeVerifier work.
In particular change all the places in the Flatbuffers library
and generated code that were using `Verifier` to instead use
`VerifierTemplate<TrackBufferSize>` and wrap them all inside
`template <bool TrackBufferSize = false>`.
Also add unit tests for SizeVerifier.
* Format using `sh scripts/clang-format-git.sh`
* Use `B` rather than `TrackBufferSize` for the name of the template parameter.
* Update generated files.
* Support native_type for tables when using the C++ object API.
If native_type is specified on a table:
- No object API struct type is generated.
- The object API refers to the table by its native_type.
- UnPack and Create<TableName> methods are declared but not defined; as they
must be user-provided.
* Add tests for native_type on tables.
* Add documentation for native_type on tables.
Per the definition of --gen-compare: only generate comparators for object API
generated structs. Types annoated with native_type must define their own
comparators if `--gen-compare` is enabled.
Also enables --gen-compare for native_type_test and fixes the test by adding a
comparator for the Native::Vector3D type.
Previously: X::Pack forwarded to CreateX.
Now: CreateX will forward to X::Pack.
This is a step toward enabling using native types for tables when using the
object API. When defining a native table, the user will be able to define a
custom X::Pack method (which is more consistent with the existing native_type
functionality for structs). By reversing the order of the dependencies, CreateX
can continue to be auto-generated and will use the custom X::Pack method when
overriden for native_type tables.
Fixes access to union members when generating code with options "--cpp-field-case-style upper" and "--gen-object-api"
Co-authored-by: Wouter van Oortmerssen <aardappel@gmail.com>
* [Python] Generate `.pyi` stub files when `--python-typing` is on.
To support this change, the following modifications were made:
- added a new option to disable `numpy` helpers generation;
- added a new flag to control the target Python version:
`--python-version` can be one of the following:
- `0.x.x` – compatible with any Python version;
- `2.x.x` – compatible with Python 2;
- `3.x.x` – compatible with Python 3.
- added codegen utilities for Python;
- added a note that the generated .py file is empty.
* [C++] Update the validator to skip structs in namespaces other than the current one.
If a schema contains a message named e.g. FooT and a message named Foo
while the Object API suffix is T, then two classes with colliding names
will be generated. This scenario will produce a C++ compiler error, but
it's confusing.
This patch moves the error to the compiler, allowing the user to more
readily act to correct the issue.
Co-authored-by: Michael Beardsworth <beardsworth@intrinsic.ai>
* Reproduce the error in a unit test
Reproduces #8200
* Overload KeyCompareWithValue to work for string-like objects
This fixes#8200.
* Extra tests
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
* First working hack of adding 64-bit. Don't judge :)
* Made vector_downward work on 64 bit types
* vector_downward uses size_t, added offset64 to reflection
* cleaned up adding offset64 in parser
* Add C++ testing skeleton for 64-bit
* working test for CreateVector64
* working >2 GiB buffers
* support for large strings
* simplified CreateString<> to just provide the offset type
* generalize CreateVector template
* update test_64.afb due to upstream format change
* Added Vector64 type, which is just an alias for vector ATM
* Switch to Offset64 for Vector64
* Update for reflection bfbs output change
* Starting to add support for vector64 type in C++
* made a generic CreateVector that can handle different offsets and vector types
* Support for 32-vector with 64-addressing
* Vector64 basic builder + tests working
* basic support for json vector64 support
* renamed fields in test_64bit.fbs to better reflect their use
* working C++ vector64 builder
* Apply --annotate-sparse-vector to 64-bit tests
* Enable Vector64 for --annotate-sparse-vectors
* Merged from upstream
* Add `near_string` field for testing 32-bit offsets alongside
* keep track of where the 32-bit and 64-bit regions are for flatbufferbuilder
* move template<> outside class body for GCC
* update run.sh to build and run tests
* basic assertion for adding 64-bit offset at the wrong time
* started to separate `FlatBufferBuilder` into two classes, 1 64-bit aware, the other not
* add test for nested flatbuffer vector64, fix bug in alignment of big vectors
* fixed CreateDirect method by iterating by Offset64 first
* internal refactoring of flatbufferbuilder
* block not supported languages in the parser from using 64-bit
* evolution tests for adding a vector64 field
* conformity tests for adding/removing offset64 attributes
* ensure test is for a big buffer
* add parser error tests for `offset64` and `vector64` attributes
* add missing static that GCC only complains about
* remove stdint-uintn.h header that gets automatically added
* move 64-bit CalculateOffset internal
* fixed return size of EndVector
* various fixes on windows
* add SizeT to vector_downward
* minimze range of size changes in vector and builder
* reworked how tracking if 64-offsets are added
* Add ReturnT to EndVector
* small cleanups
* remove need for second Array definition
* combine IndirectHelpers into one definition
* started support for vector of struct
* Support for 32/64-vectors of structs + Offset64
* small cleanups
* add verification for vector64
* add sized prefix for 64-bit buffers
* add fuzzer for 64-bit
* add example of adding many vectors using a wrapper table
* run the new -bfbs-gen-embed logic on the 64-bit tests
* remove run.sh and fix cmakelist issue
* fixed bazel rules
* fixed some PR comments
* add 64-bit tests to cmakelist
* Add binary schema reflection
* remove not-used parameter
* move logic from object API to base API
* forward declare
* remove duplicate code gen that was stompping on the edits
* reduce to just typedef generation
* fixed bazel rules to not stomp
* more bazel fixes to support additional generated files
* Refactor to use CodeGenerator interface.
- Move code to its own header file to be included in flatc_main.cpp
- Refactor code to use CodeGenerator interface for all languages
* Format all files
* remove lua code generator since it doesn't support bfbs generator
* Update CMakeLists file with new idl_gen_*.cpp and idl_gen_*.h files
* Add idl_gen_swift header file
* Add idl_gen_swift header file and update bazel file
* Remove CodeGenerator interface for idl_gen_text.*. Remove comments and extern declaration
* Reorder header and implementation files in CMakeLists.txt
* Add idl_gen_* header files to implementation files
* Update CMakeLists and remove unused import
Co-authored-by: Derek Bailey <derekbailey@google.com>
* add unit tests for support struct as key
* make changes to parser and add helper function to generate comparator for struct
* implement
* add more unit tests
* format
* just a test
* test done
* rerun generator
* restore build file
* address comment
* format
* rebase
* rebase
* add more unit tests
* rerun generator
* address some comments
* address comment
* update
* format
* address comment
Co-authored-by: Wen Sun <sunwen@google.com>
Co-authored-by: Derek Bailey <derekbailey@google.com>
Add the --no-minmax-values flag to prevent flatc from generating C++
enums with MIN and MAX enumerated values that otherwise would be set
to the inclusive lower and upper bound respectively of the enum.
This command-line flag is needed to avoid collisions when an enum that
is being ported to FlatBuffers already has a MIN or MAX enumerated
value.
It is also needed to work around a long-standing problem with
magic_enum that causes magic_enum to not see enumerated values that
are not unique. For example, if FlatBuffers sets MIN = FOO and MAX =
BAR, MIN and FOO share the same underlying value so they are not
unique. The same is true of MAX and BAR. This prevents magic_enum
from converting FOO and BAR to and from strings as well as causing
magic_enum to return a count of enumerated values that is two fewer
than it should be.
Co-authored-by: Paul Serice <paul@serice.net>
* update unit test and generated file to test is extra endianswap can help resolve issue
* remove EndianScalar wrapper from Get method
* remove endianscalar wrapper
* update
* update
* use Array instead
* clang format
* address error
* clang
* update
* manually generate
* Move Nim to completed language
* Add swift link
* address comments
* update unit test
* address comment
* address comment
* regenerate file
* use auto instead of size_t
* use uint32_t instead
* update
* format
* delete extra whitespace
Co-authored-by: Wen Sun <sunwen@google.com>
Co-authored-by: Derek Bailey <derekbailey@google.com>
* add support for using array of scalar as key field
* update cmakelist and test.cpp to include the tests
* update bazel rule
* address comments
* clang format
* delete comment
* delete comment
* address the rest of the commnets
* address comments
* update naming in test file
* format build file
* buildifier
* make keycomparelessthan call keycomparewithvalue
* update to use flatbuffer array instead of raw pointer
* clang
* format
* revert format
* revert format
* update
* run generate_code.py
* run code generator
* revert changes by generate_code.py
* fist run make flatc and then run generate_code.py
Co-authored-by: Wen Sun <sunwen@google.com>
* Vector of Tables equality
* support nullptr and fix for not being able to use auto in lambda
* use different std::equal overload
* use flatbuffers::unique_ptr
* go back to auto and clang-format fix
