For Vector or Array of structures the dereference operator of an
iterator returns the pointer to the structure. However, IndirectHelper,
which is used in the implementation of this operator, is instantiated
in the way that the IndirectHelper::Read returns structure by value.
This is because, Vector and Array instantiate IndirectHelper with
const T*, but VectorIterator instantiates IndirectHelper with T. There
are three IndirectHelper template definition: first for T, second for
Offset<T> and the last one for const T*. Those have different
IndirectHelper:Read implementations and (more importantly) return type.
This is the reason of mismatch in VectorIterator::operator* between
return type declaration and what was exactly returned.
That is, for Array<T,...> where T is scalar the VectorIterator is
instantiated as VectorIterator<T, T>, dereference operator returns T
and its implementation uses IndirectHelper<T> which Read function
returns T.
When T is not scalar, then VectorIterator is instantiated as
VectorIterator<T, const T *>, dereference operator returns const T * and
its implementation uses IndirectHelper<T> which Read function returns T.
The fix is done as follows:
* implement type trait is_specialization_of_Offset and
is_specialization_of_Offset64,
* change partial specialization of IndirectHelper with const T * that
it is instantiated by T and enabled only if T is not scalar and not
specialization of Offset or Offset64,
* remove type differentiation (due to scalar) from Array..
The above makes the IndirectHelper able to correctly instantiate itself
basing only on T. Thus, the instantiation in VectorIterator correctly
instantiate IndirectHelper::Read function, especially the return type.
* Update build.yml
Upgrade to gcc 13 and clang 15
* switch to __is_trivially_copyable
* fix cmake issue and warning about sign comparison
* Use libc++ for C++23 on clang for now
* Use libc++ for C++23 on clang for now
* exclude clang+15 for C++13 builds
* First working hack of adding 64-bit. Don't judge :)
* Made vector_downward work on 64 bit types
* vector_downward uses size_t, added offset64 to reflection
* cleaned up adding offset64 in parser
* Add C++ testing skeleton for 64-bit
* working test for CreateVector64
* working >2 GiB buffers
* support for large strings
* simplified CreateString<> to just provide the offset type
* generalize CreateVector template
* update test_64.afb due to upstream format change
* Added Vector64 type, which is just an alias for vector ATM
* Switch to Offset64 for Vector64
* Update for reflection bfbs output change
* Starting to add support for vector64 type in C++
* made a generic CreateVector that can handle different offsets and vector types
* Support for 32-vector with 64-addressing
* Vector64 basic builder + tests working
* basic support for json vector64 support
* renamed fields in test_64bit.fbs to better reflect their use
* working C++ vector64 builder
* Apply --annotate-sparse-vector to 64-bit tests
* Enable Vector64 for --annotate-sparse-vectors
* Merged from upstream
* Add `near_string` field for testing 32-bit offsets alongside
* keep track of where the 32-bit and 64-bit regions are for flatbufferbuilder
* move template<> outside class body for GCC
* update run.sh to build and run tests
* basic assertion for adding 64-bit offset at the wrong time
* started to separate `FlatBufferBuilder` into two classes, 1 64-bit aware, the other not
* add test for nested flatbuffer vector64, fix bug in alignment of big vectors
* fixed CreateDirect method by iterating by Offset64 first
* internal refactoring of flatbufferbuilder
* block not supported languages in the parser from using 64-bit
* evolution tests for adding a vector64 field
* conformity tests for adding/removing offset64 attributes
* ensure test is for a big buffer
* add parser error tests for `offset64` and `vector64` attributes
* add missing static that GCC only complains about
* remove stdint-uintn.h header that gets automatically added
* move 64-bit CalculateOffset internal
* fixed return size of EndVector
* various fixes on windows
* add SizeT to vector_downward
* minimze range of size changes in vector and builder
* reworked how tracking if 64-offsets are added
* Add ReturnT to EndVector
* small cleanups
* remove need for second Array definition
* combine IndirectHelpers into one definition
* started support for vector of struct
* Support for 32/64-vectors of structs + Offset64
* small cleanups
* add verification for vector64
* add sized prefix for 64-bit buffers
* add fuzzer for 64-bit
* add example of adding many vectors using a wrapper table
* run the new -bfbs-gen-embed logic on the 64-bit tests
* remove run.sh and fix cmakelist issue
* fixed bazel rules
* fixed some PR comments
* add 64-bit tests to cmakelist
* Add binary schema reflection
* remove not-used parameter
* move logic from object API to base API
* forward declare
* remove duplicate code gen that was stompping on the edits
* reduce to just typedef generation
* fixed bazel rules to not stomp
* more bazel fixes to support additional generated files
This change allows user to decode binary with given schema to JSON
representation when schema defines union with struct.
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Parsing from proto should keep field ID. (fixes#7645)
* Fix failed tests
* Fix windows warning
* Improve attribute generation in proto to fbs
* Check if id is used twice. fix Some clang-format problems
* Test if fake id can solve the test problem
* Validate proto file in proto -> fbs generation.
* Fix error messages
* Ignore id in union
* Add keep proto id for legacy and check gap flag have been added. Reserved id will be checked.
* Add needed flags
* unit tests
* fix fromat problem. fix comments and error messages.
* clear
* More unit tests
* Fix windows build
* Fix include problems
* Fake commit to invoke rebuild
* Fix buzel build
* Fix some issues
* Fix comments, fix return value and sort for android NDK
* Fix return type
* Break down big function
* Place todo
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
* add unit tests for support struct as key
* make changes to parser and add helper function to generate comparator for struct
* implement
* add more unit tests
* format
* just a test
* test done
* rerun generator
* restore build file
* address comment
* format
* rebase
* rebase
* add more unit tests
* rerun generator
* address some comments
* address comment
* update
* format
* address comment
Co-authored-by: Wen Sun <sunwen@google.com>
Co-authored-by: Derek Bailey <derekbailey@google.com>
The BytesConsumed function uses the `cursor_` to determine how many
bytes have been consumed by the parser, in case the user of the Parser
object wants to step over the parsed flatbuffer that is embedded in some
larger string. However, the `cursor_` is always one token ahead, so that
it can determine how to consume it. It points at the token that is about
to be consumed, which is ahead of the last byte consumed.
For example, if you had a string containing these two json objects and
parsed them...
"{\"key\":\"value\"},{\"key\":\"value\"}"
...then the `cursor_` would be pointing at the comma between the two
tables. If you were to hold a pointer to the beginning of the string and
add `BytesConsumed()` to it like so:
const char* json = // ...
parser.ParseJson(json);
json += parser.BytesConsumed();
then the pointer would skip over the comma, which is not the expected
behavior. It should only consume the table itself.
The solution is simple: Just hold onto a previous cursor location and
use that for the `BytesConsumed()` call. The previous cursor location
just needs to be set to the cursor_ location each time the cursor_ is
about to be updated. This will result in `BytesConsumed()` returning
the correct number of bytes without the off-by-one-token error.
Co-authored-by: Derek Bailey <derekbailey@google.com>
* add support for using array of scalar as key field
* update cmakelist and test.cpp to include the tests
* update bazel rule
* address comments
* clang format
* delete comment
* delete comment
* address the rest of the commnets
* address comments
* update naming in test file
* format build file
* buildifier
* make keycomparelessthan call keycomparewithvalue
* update to use flatbuffer array instead of raw pointer
* clang
* format
* revert format
* revert format
* update
* run generate_code.py
* run code generator
* revert changes by generate_code.py
* fist run make flatc and then run generate_code.py
Co-authored-by: Wen Sun <sunwen@google.com>
+/-inf were not being handled, and so invalid typescript was being
generated when a float/double had an infinite default value. NaN was
being handled correctly.
Co-authored-by: Derek Bailey <derekbailey@google.com>
Co-authored-by: Casper <casperneo@uchicago.edu>
Previously when parsing a JSON representation of a Flatbuffer, the
parser required that the input string contain one and only one root
table. This change adds a flag that removes that requirement, so that
if a Flatbuffer table is embedded in some larger string the parser will
simply stop parsing once it reaches the end of the root table, and does
not validate that it has reached the end of the string.
This change also adds a BytesConsumed function, which returns the number
of bytes the parser consumed. This is useful if the table embedded in
some larger string that is being parsed, and that outer parser needs to
know how many bytes the table was so that it can step over it.
on a subset of platforms. The test calls are guarded via #ifndef
FLATBUFFERS_NO_FILE_TEST.
Embedders of flatbuffers that rely on -Werror,-Wunused-function
compiler flags (like chromium) complain about the exsitence of these
tests in the anonymous namespace.
This CL guards the test definitions as well (not just the test
calls).
Co-authored-by: Dominic Battre <battre@chromium.org>
Co-authored-by: Derek Bailey <derekbailey@google.com>
* [C++] Add a failing unit test for #7516 (Rare bad buffer content alignment if sizeof(T) != alignof(T))
* [C++] Fix final buffer alignment when using an array of structs
* A struct can have an arbitrary size and therefore sizeof(struct) == alignof(struct)
does not hold anymore as for value primitives.
* This patch fixes this by introducing alignment parameters to various
CreateVector*/StartVector calls.
* Closes#7516
* C++: Add option to skip verifying nested flatbuffers
Additionally, add an options struct to the verifier for those
who prefer designated initializers to default arguments. The former
constructor is defined in terms of the latter because in old c++,
having default values for members removes list initialization, making
defining constructors in the other way a lot more challenging to write.
* fixes
* fmt
* formatting, and remove an argument
* fix
Co-authored-by: Casper Neo <cneo@google.com>
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Add timing command to cmakelist
* Start to split test.cpp. Move flexbuffers tests
* Move monster related tests to own file
* Move parser related tests to own file
* Move json related tests to own file
* Move proto related tests to own file
* moved more functions to parser test
* moved more functions to parser test2
* move monster extra tests to monster test
* move evolution tests to own file
* move reflection tests to own file
* move util tests to own file
* rehomed various tests
* move optional scalars test to own file:
* rehome more tests
* move fuzz tests. Got rid of global test_data_path
* fixes for CI failures
* add bazel files
Added (for compiler versions that support it):
-Wmissing-declarations
-Wzero-as-null-pointer-constant
Then, fixes to problems identified by the extra warnings
Tested only on GCC 9.4.0
Adjusted the CPP code generator to output nullptr where appropriate,
to satisfy -Wzero-as-null-pointer-constant
Added a lot of 'static' declarations in front of functions,
to satisfy -Wmissing-declarations,
and wrap static function defs in anonymous namespaces.
There are advantages to both anonymous namespaces and static,
it seems that marking a function as static will not publish the name in
the symbol table at all, thus giving the linker less work to do.
* Vector of Tables equality
* support nullptr and fix for not being able to use auto in lambda
* use different std::equal overload
* use flatbuffers::unique_ptr
* go back to auto and clang-format fix
* Implement optional scalars for JSON
* Add optional scalars JSON test
* Extend JSON optional scalars test to test without defaults
* Fix optional scalars in JSON for binary schema
Co-authored-by: Caleb Zulawski <caleb.zulawski@caci.com>
std::span lacks these; make the flatbuffers STL emulation and tests
match. This fixes a compile error in C++20 mode when using std::span.
Bug: chromium:1284275
Since CreateVectorOfStrings() takes a templated container, make sure that
the default template deduction from just an initializer list will
still work.
Signed-off-by: Henner Zeller <hzeller@google.com>
Any string type that is supported by CreateString(), e.g.
const char* or string_view will now also work.
Signed-off-by: Henner Zeller <hzeller@google.com>
* Started to migrate to target_compile_options
* combined compile options together. Added Mac CI builds
* remove arm build (not supported). Fixed old-style-casts
* moved to using a ProjectConfig interface library to specify options
* remove the explicit CMAKE_CXX_STANDARD
* Apply Namer to Dart.
- Also refactor idl_gen_dart a bit
- to use more const and references
- out parameters should be the last argument
* Add keyword test
* minor fixes
* fix merge
* extra 's'
* move dart keyord into dart dir
* Address comments
* Use $ for escaping keywords
* Outparameters for namespace_map
* Escape dollar in toString
* Escape dollar in toString2
* Use UpperCamelCase for types and variants
* try to fix ToString
* namer Type fixes
* Remove path prefixing in imports
* gen code
Co-authored-by: Casper Neo <cneo@google.com>
