Commit Graph

649 Commits

Author SHA1 Message Date
Max Burke
1d294a31b8 Implement Serialize on generated rust types (#7022)
* fix for rust build

* Rust: Implement Serialize on generated types

For debugging convenience it is really handy to be able to dump out
types as another format (ie: json). For example, if we are logging a
type to a structured logging system, or even printing it out in a
structured way to the console.

Right now we handle this by shelling out to `flatc` which is not ideal;
by implementing Serialize on the generated types we can use any of the
Serializer-implementing packages for our structured debug output.

* clang-format

* Make the flatbuffers Rust crate only have an optional dependency on the `serde` packages.

* fix warning

* fix rust test build

* Oh yeah this needs to be initialized

* fix toml syntax

* code review feedback

* rebuild test data
2022-01-30 19:29:18 -05:00
Derek Bailey
7089c9ecdd Convert flatbuffers_version_string to inline function (#7046)
* Skip conditional for python executable

* replaced flatbuffers_version_string with inline function

* use const char* instead of string
2022-01-29 19:42:56 -08:00
Justin T Conroy
19920db39f Include a SizePrefixed..HasIdentifier for c++. (#6871)
* Include a SizePrefixed..HasIdentifier for c++.

* Add updated generated code.
2022-01-28 20:52:32 -08:00
Jon Simantov
162ad7a37e Fix comment with line orders - Rust should be last. (#7037)
Fix for #6867.
2022-01-25 16:43:08 -08:00
Jon Simantov
a2d38fbb98 Add --warnings-as-errors to flatc compiler. (#7034)
* Add --warnings-as-errors to flatc compiler.

With this option set, flatc will return an error on parsing if
any warnings occurred.

* Add unit test for opts.warnings_as_errors.

* Change explicit option setting to default.
2022-01-25 14:01:16 -08:00
Danila Kutenin
d7b75417fc Make flatbuffer builder deterministic (#6993) 2022-01-14 15:37:27 -08:00
Wouter van Oortmerssen
a42e898979 Added verifier alignment checking to table fields (#7018) 2022-01-14 14:39:15 -08:00
Advait Jain
f28c2b2936 Avoid implicit conversion from float to double. (#7003)
https://github.com/tensorflow/tflite-micro makes use of flatbuffers with
a variety of DSP toolchains.

Without the change from this PR, we can get a double-promotion warning
with some of these DSP toolchains:
```
flatbuffers/include/flatbuffers/util.h:104:11: error: implicit conversion increases floating-point precision: 'std::numeric_limits<float>::_Ty' (aka 'float') to 'double' [-Werror,-Wdouble-promotion]
  T eps = std::numeric_limits<float>::epsilon();
    ~~~   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
2021-12-23 20:31:30 -06:00
Shlomi Regev
96f3cf690f Rearrange #include directives to pass to compilation with a particular DSP toolchain (#7000)
iomanip isn't available for our DSP. Luckily, we compile with FLATBUFFERS_PREFER_PRINTF, so moved the #include directive there.
ctype.h has to be included explicilty for tolower() and toupper().
2021-12-22 10:40:38 -06:00
Wouter van Oortmerssen
b8aaccee82 Disable parsing of nested_flatbuffers as bytes by default
Parsing as bytes produces buffers that are unsafe to access unless passed thru a verifier,
whereas users could reasonably assume that any JSON parsed without errors is safe to access.
Users that still have legacy JSON files with such bytes in it will get a helpful error point them
to the option to turn on to have it work again.
2021-12-15 10:41:29 -08:00
Wouter van Oortmerssen
d8b7041d7e Added alignment checking to FlexBuffers verifier 2021-12-14 11:20:23 -08:00
Wouter van Oortmerssen
5a0c3366c3 FlexBuffers verifier additionally checks for nesting 2021-12-14 11:20:23 -08:00
Wouter van Oortmerssen
f8148b8dad Made FlexBuffers reuse tracker track types 2021-12-14 11:20:23 -08:00
Derek Bailey
a22b1b6267 Prevent shadow with _{{FIELD_NAME}} (#6991) 2021-12-13 21:27:36 -08:00
Wouter van Oortmerssen
4264daadd2 FlexBuffers fuzzer fixes
- String dedup wasn't handling internal nulls correctly.
- Verifier wasn't recursing for certain types.
- Vector self-reference could create inf recursion.
2021-12-13 15:47:06 -08:00
Derek Bailey
3cbc120a08 Refractor Flatc Options (#6987)
* Moved error/warnings to bottom of std error

* Refactor flatc options

* pass program name to flatc

* extra -- in one option

* merge of upstream
2021-12-13 13:56:47 -08:00
Wouter van Oortmerssen
e367ca32ad Verifier for FlexBuffers (#6977)
* Verifier for FlexBuffers

* Verifier improvements & fuzzer
2021-12-10 14:59:08 -08:00
Derek Bailey
d0cede9c90 Bumping to Version 2.0.5 (#6967) 2021-12-09 10:31:57 -08:00
Derek Bailey
061d61f3f8 Lua Generator using IR. (#6940)
* initial hack to get new Lua generator into flatc

* Starting to output enum defs for Lua

* Continue to work on table generation for Lua

* Finished basic getter access for Lua

* Added ability to get object by index

* Finished struct builder

* aliased reflection to r

* finish table builder generation

* register requiring files

* better generated header info

* Tying up loose ends

* Updated reflection to handle struct padding

* Addd type sizes to reflection

* Fixed some vector indirect issues

* Lua tests passed

* Misc cleanup

* ci fixes 1

* ci fixes 2

* renaming

* up size of type sizes

* manually ran clang-format-11 -i src/idl_parser.cpp

* fixed some windows casting

* remove stupid auto import

* more static_casting

* remove std

* update other build environments

* remove scoped enums

* replaced std::to_string with NumToString

* more win fixes

* more win fixes

* replaced old lua with new

* removed auto import

* review responses

* more style fixes

* refactor bfbs_gen_len to use code +=

* added consts

* fix lambda capture for windows

* remove unused return type
2021-12-02 21:29:19 -08:00
Markus Junginger
cffe0c4546 FixedTypedVector: add const to ElementType() and size() (#6965) 2021-12-02 20:32:51 -08:00
Derek Bailey
e47dc0e465 Removed test/generate_code.{sh|bat} (#6873)
* removed test/generate_code.{sh|bat}

remove c++0x from generate_code.py

added check generate code script in python

add windows specific call

added flags to generate_code.py

Set c++-0x on BUILD_LEGACY

Skip generating monster_extra if requested

* added option to skip monster extra

* add conditional to skip 2010 check gen
2021-11-30 23:13:24 -08:00
engedy
4f7f6dc301 Restore FlatBufferBuilder::kFileIdentifierLength. (#6960)
Restore flatbuffers::FlatBufferBuilder::kFileIdentifierLength, which was a documented part of the public API, but the identifier was lost during the refactoring effected by comment commit 6c8c291559.
2021-11-30 23:06:19 -08:00
Wouter van Oortmerssen
0fadaf391d Enable verifier on nested_flatbuffers 2021-11-29 15:05:59 -08:00
Wouter van Oortmerssen
4d0e9a8706 Turn off nested FlatBuffers/FlexBuffers for the fuzzer
this is a temporary fix to allow the fuzzer to work until the real fix of adding a verifier for this is added.
2021-11-29 09:19:29 -08:00
Derek Bailey
9e4ca857b6 Vector Downward GetSize optimization (#6925)
* Added Google benchmarks (and gtests)

* Separate benchmark CMakeLists.txt to its own file

* Move output directory to target just flatbenchmark

* Reduced from encoding 210ns -> 188ns

* store size_ as uoffset_t

* fixed windows c4267 warning
2021-11-22 19:50:42 -08:00
jonathan-conder-sm
a2b99084b4 Fix flexbuffers clang-analyzer warning (#6947) 2021-11-22 13:37:36 -08:00
Derek Bailey
e89de4411d clang-all (#6941) 2021-11-19 17:01:48 -08:00
Wouter van Oortmerssen
6748c373be Removal of support for FLATBUFFERS_CPP98_STL (#6918)
* Removal of support for FLATBUFFERS_CPP98_STL

For context, see https://github.com/google/flatbuffers/issues/6704

* Also removed vector_data
2021-11-12 14:01:11 -08:00
Derek Bailey
3fab0c6ee4 move [[ ]] to calling sites (#6914) 2021-11-11 10:31:15 -08:00
Derek Bailey
6c8c291559 [C++] Split flatbuffers.h into separate files (#6868)
* split flatbuffers.h into separate files

* wrong variable in cmakelists for android

* readded two accidentally deleted includes

* created buffer.h and moved buffer related things over
2021-11-10 22:26:09 -08:00
Even Rouault
8433eb108c Typo fixes in comments of flatbuffers.h and flexbuffers.h (#6901) 2021-10-28 13:03:03 -07:00
xuyatian
e672dabfea [C++] Fix memory leak when using grpc (#6896) (#6897) 2021-10-18 11:24:52 -07:00
mustiikhalil
4c7a9c10d3 Adds JSON encoding to swift (#6874)
Updates generated code & removes unneeded protocol

Updates cpp to only generate code when flag is passed

Updates code gen script
2021-10-07 23:22:22 +02:00
Todd Hansen
6f7a57eaa0 [C#] Using 'global::' as qualifying_start_ within BaseGenerator (#6767)
* Update idl_gen_csharp.cpp

Change csharp generator to use "global::" for it's qualifying_start_ to disambiguate namespaces

* regenerate testing files

regenerate testing files

* Missed TableInC.cs

updated with global prefix

* Remove "global::" from qualifying_start_ for csharp generator

* C# global alias

* Tests and docs for --cs-global-alias

Add tests for --cs-global-alias to demonstrate use case for why it's needed.

Add documentation to Compiler.md

* Add also to help text

Add also to help text
2021-09-15 16:57:29 -05:00
Casper
0a3b017f09 Delete IDLOptions::lang (#6841)
Co-authored-by: Casper Neo <cneo@google.com>
2021-09-13 18:18:00 -04:00
Maksim Ivanov
1d063d87cf [C++] Let builder accept custom-alloc std::vector (#6814)
Change the FlatBufferBuilder's methods to accept std::vector parameters
with non-default allocator, by adding another template parameter to
them. This should make using the builder slightly more convenient, as
one won't need to manually pass data() and size() separately.
2021-09-08 13:25:06 -07:00
Tyler Denniston
74c3d7eba2 [C++] Add mutable version of LookupByKey and test (#6826)
* [C++] Add mutable version of LookupByKey and test

This adds an overload of LookupByKey to allow lookup in sorted Vectors to
return a mutable instance of the object (or nullptr if not found).

* Fix naming
2021-08-31 17:17:20 -04:00
Stefan F
b20c4d3aad [C++] Add GetMutableSizePrefixedRoot and generate GetMutableSizePrefixedXxx functions (#6815)
* flattests_cpp17 doesn't compile with Visual Studio 2017: warning C4100: 'indent': unreferenced formal parameter

stringify_util.h(127): error C2220: warning treated as error - no 'object' file generated
stringify_util.h(127): warning C4100: 'indent': unreferenced formal parameter
stringify_util.h(85): warning C4100: 'indent': unreferenced formal parameter
stringify_util.h(85): warning C4100: 'fbs': unreferenced formal parameter

* [C++] Add GetMutableSizePrefixedRoot() and generate a GetMutableSizePrefixed function
When using the mutable API together with size prefixed buffers these functions should be present.

* clang-format

* Cleanup branch for PR
Revert "flattests_cpp17 doesn't compile with Visual Studio 2017: warning C4100: 'indent': unreferenced formal parameter"

This reverts commit a92055203e.
2021-08-30 15:34:42 -07:00
mogemimi
775c6567d1 [C++] Fix compiler warning -Wredundant-parens in clang (#6761) 2021-08-02 13:40:17 -07:00
Esun Kim
e012054667 [C++] Use proper gRPC C++ API when using MemoryBuffer Slice (#6756)
* Use ByteBuffer

* Use Slice

* Upgrade gRPC to 1.39.0

* Update gRPC build script

* Reformat

* Added CMAKE_CXX_STANDARD=11 when building Abseil

* Enabled ABSL_ENABLE_INSTALL

* Added absl to grpctest deps
2021-07-29 10:41:51 -07:00
Casper
12e341e4f8 Rework how Rust generated files are laid out (#6731)
* Refactored Rust Generated code into a module directory.

Each symbol will be generated into one file and then
imported into a module. This breaks the "out_dir"
pattern where some users would generate code in their
target/ directory. Also, these objects are best used
in their own module. It will be hard for users to share
their own module structure with flatbuffers namespaces.

There may be solutions to these drawbacks but that should
be discussed. I don't want to overengineer here.

* shadow error

* try fix .bat file

* fix .bat 2

* Restore accidentally deleted files

* Fixed some DONOTSUBMITs and made Rust outdir pattern use symlinks.

* fixed binary files

* git clang format

* make generated onefiles not public and fix .bat

* reduced diff with master in generate_code.sh

* fix shadowed variable

* add object api flags to .bat

* space

* Removed extern crate and extra &

* use statement

* more clippy lints

* format

* Undo extern crate -> use change, it actually matters to our tests

Co-authored-by: Casper Neo <cneo@google.com>
2021-07-22 19:02:28 -04:00
Shlomi Regev
0980e39c9b flexbuffers: Add variant of Blob() that takes a key (#6730) 2021-07-12 09:42:55 -07:00
Daniil Nikolenko
bf3470c161 Fix UB in CreateVectorOfStructs in case of an empty vector: avoid calling memcpy in this case (#6726) 2021-07-08 13:40:53 -07:00
Vladimir Glavnyy
22498cf3a9 [C++] add make_span for Array<T,N> (#6663)
* [C++] Add make_span for Array<T,N> and Vector<T>

* [C++] Add tests for make_span<T,N>(Array<T,N>)

* [C++] Add span iterators

* [C++] Checked span iterator (MSVC)

* Fix review notes
2021-06-28 10:22:02 -07:00
pkasting
bd37e67ac0 [C++] Fix a -Wdeprecated-copy warning. (#6708)
* Fix incorrect assertion usage.

* Fix a -Wdeprecated-copy warning.

Bug: chromium:1221591

* Just remove constructors/assignment
2021-06-27 09:50:12 +07:00
Casper
06fd6d640c Replace filenames in reflection with filenames+includes. (#6703)
* Replace filenames in reflection with filenames+includes.

This is needed for some use cases and may be just useful metadata.

* deser files_included_per_file_

* check project_root

* fix bazel

* git clang format

Co-authored-by: Casper Neo <cneo@google.com>
2021-06-19 19:21:33 -04:00
Casper
c58ae94225 Add the file a symbol is declared in to Reflection (#6613)
* Add the file a symbol is declared in to Reflection

If we move a code-generator to depend on Reflection,
it may need to know which file something was declared in
to properly name generated files.

* Doc comments in reflection, and more precise tests

* Add --project-root flag to flatc, normalize declaraion_file to this root

* fix --project-root stuff

* posixpath

* fix scripts

* format

* rename --project-root to --bfbs-filenames

Also, make it optional, rather than defaulting to `./`, if its not
specified, then don't serialize the filenames.

* bfbs generation

* fix some tests

* uncomment a thing

* add  to project root directory conditionally

* fix

* git clang format

* Added help description and removed != nullptr

* "

* Remove accidental change to docs

* Remove accidental change to docs

* Pool strings

Co-authored-by: Casper Neo <cneo@google.com>
2021-06-17 11:50:04 -04:00
Huw Rogers
f069396d1b [C++] flatc --cpp-field-case-style option to permit camel-case field names in C++ (#6669)
* flatc --cpp-field-case option to permit camel-case field names in C++

* fixed option name; cleaned up tabs

* formatting fixed to conform to CI

* resolved comments

* fixed white space indentation

* per PR comments

* rename snake case option to unchanged for clarity, per PR comments

* cleanup of unchanged case option in C++ codegen, per PR 6669 comments

* incorporated PR feedback from vglavnyy

* cleaned up to pass Travis CI / clang format

* bumped PR to retry transient CI failure

* bumped PR to retry transient CI failure

* bump PR

* assert union type field name length > suffix, per PR 6669 comments
2021-06-08 11:23:05 -07:00
Jared Duke
093badb0a0 Use unoptimized path for ReadUInt64 for win32 build as the optimized path crashes. (#6681)
FlexBuffers are used by custom op handlers for TFLite. Using the optimized path for ReadUInt64 in FlexBuffers causes a crash in models using custom ops on win32 build. This change fixes the problem by using unoptimized implementation of ReadUInt64 for win32.
2021-06-04 13:27:47 -07:00
ofirm93
7c3e267e1e [Java] ObjectAPI implementation (#6521) (#6582)
* [Java] ObjectAPI implementation (#6521)

* [Java] ObjectAPI implementation (#6521)

* [Java] ObjectAPI implementation (#6521)

* [Java] ObjectAPI implementation (#6521)

* [Java] ObjectAPI implementation (#6521)

* [Java] ObjectAPI implementation (#6521)

* [Java] ObjectAPI implementation (#6521)

* [Java] ObjectAPI implementation (#6521)

* [Java] ObjectAPI implementation (#6521)

* [Java] ObjectAPI implementation (#6521)

* [Java] ObjectAPI implementation (#6521)

* [Java] ObjectAPI implementation (#6521)

* [Java] ObjectAPI implementation (#6521)

* [Java] ObjectAPI implementation (#6521)

* [Java] ObjectAPI implementation (#6521)

* [Java] ObjectAPI implementation (#6521)

* [Java] ObjectAPI implementation (#6521)
2021-06-03 14:17:40 -07:00