Commit Graph

983 Commits

Author SHA1 Message Date
Wouter van Oortmerssen
4d0e9a8706 Turn off nested FlatBuffers/FlexBuffers for the fuzzer
this is a temporary fix to allow the fuzzer to work until the real fix of adding a verifier for this is added.
2021-11-29 09:19:29 -08:00
Sergei Trofimovich
85b4effac6 test: fix undefined order of functio parameters. (#6946)
Detected instability when built `flatbuffers-2.0.0` on `gcc-12`:

    [ 75%] Building CXX object CMakeFiles/flattests.dir/tests/test_builder.cpp.o
    .../c++/12.0.0/bits/shared_ptr_base.h:397:45: error: 'size' may be used uninitialized [-Werror=maybe-uninitialized]
      397 |       explicit _Sp_ebo_helper(_Tp&& __tp) : _M_tp(std::move(__tp)) { }
          |                                             ^~~~~~~~~~~~~~~~~~~~~~
    In file included from flatbuffers/tests/test_builder.cpp:1:
    flatbuffers/tests/test_builder.h: In function 'void builder_move_assign_after_releaseraw_test(Builder) [with Builder = flatbuffers::FlatBufferBuilder]':
    flatbuffers/tests/test_builder.h:63:10: note: 'size' was declared here
       63 |   size_t size, offset;
          |          ^~~~
    ...
    In file included from flatbuffers/tests/test_builder.cpp:1:
    flatbuffers/tests/test_builder.h: In function 'void builder_move_assign_after_releaseraw_test(Builder) [with Builder = GrpcLikeMessageBuilder]':
    flatbuffers/tests/test_builder.h:63:10: note: 'size' was declared here
       63 |   size_t size, offset;
          |          ^~~~
    cc1plus: all warnings being treated as errors

Here is the relevant bit of test:

    template<class Builder>
    void builder_move_assign_after_releaseraw_test(Builder b1) {
      auto root_offset1 = populate1(b1);
      b1.Finish(root_offset1);
      size_t size, offset;
      std::shared_ptr<uint8_t> raw(
          b1.ReleaseRaw(size, offset), [size](uint8_t *ptr) {
            flatbuffers::DefaultAllocator::dealloc(ptr, size);
          });

Note how `b1.ReleaseRaw(size, offset)` is expected to populate `size`
and `[size](uint8_t *ptr) {` captures the result. But both are parameters
to the same function call and thus evaluation order is unspecified.
2021-11-22 12:14:31 -08:00
Derek Bailey
e89de4411d clang-all (#6941) 2021-11-19 17:01:48 -08:00
Taiju Tsuiki
587bbd49a7 [C++] Fix compile failure on Object API union construction for struct member (#6923)
* Add dedicated traits to Object API version of unions.

* Add suppression for unused parameters on unions of structs.
2021-11-18 10:55:11 -08:00
Catena cyber
d727579b6d fuzzers: fix typo in file path (#6931) 2021-11-17 14:43:56 -08:00
taroplus
927175ea20 [Java] lookup by byteArray is giving back wrong entry (#6915) 2021-11-15 11:16:45 -08:00
Wouter van Oortmerssen
6748c373be Removal of support for FLATBUFFERS_CPP98_STL (#6918)
* Removal of support for FLATBUFFERS_CPP98_STL

For context, see https://github.com/google/flatbuffers/issues/6704

* Also removed vector_data
2021-11-12 14:01:11 -08:00
Derek Bailey
6c8c291559 [C++] Split flatbuffers.h into separate files (#6868)
* split flatbuffers.h into separate files

* wrong variable in cmakelists for android

* readded two accidentally deleted includes

* created buffer.h and moved buffer related things over
2021-11-10 22:26:09 -08:00
lu-wang-g
fd4ff23da0 Keep methods with struct name and switch them to default (#6879)
* Keep methods with struct name and switch them to default

This PR can help fix the following two issues:
1): A set of simplified API (without struct name) was added in
https://github.com/google/flatbuffers/pull/6336. It causes name
conflict when merging all generated python file into a single one
(the primary usage senario in Google).
2): Flatbuffers 2.0 generates absolute import path, which may cause
name space conflicts. See more details in
https://github.com/google/flatbuffers/issues/5840.

The solution for both is to generate the merged Python code, similar
C++. The merged code will not contain the simplied API, but only the
method with struct name. For issue (1), it will mimic the exactly
usage pattern for Google internal. For issue (2), users can generate
the merged flatbuffer code, without worrying about the imports.

The above idea will be implemented in the following steps:
Step 1 (this PR): revert changes in https://github.com/google/flatbuffers/pull/6336
that set the simplified API as default. Remove statements that the
original API will be deprecated, and reset the original API as default.
Step 2 (the following PR): create a flag to generate the merged code.
The Simplified API will be removed from the merged code, otherwise it
will cause name conflict.

* Update the generated sample code

* Update the generated example code

* Reverst the changes of GetRootAs

* Update examples from grpc/example/generate.sh
2021-11-04 15:12:25 -07:00
Tom Miller
f8b69e3308 [Python] Verify EnumDef is not generated in GenUnionCreator (#6903)
* [Python] Verify EnumDef is not generated in GenUnionCreator

* update test output
2021-11-04 14:21:31 -07:00
mustiikhalil
4c7a9c10d3 Adds JSON encoding to swift (#6874)
Updates generated code & removes unneeded protocol

Updates cpp to only generate code when flag is passed

Updates code gen script
2021-10-07 23:22:22 +02:00
T Sprecher
45e5642e91 Default Arguments for Mutators C++ [Updated] (#6872)
* CPP Default Value Generation in Mutators

If the mutator is for a value that is compatible with having a default value, then the single parameter becomes a default parameter. With this, a value can be mutated to it's default value without storing the default value, as that will be stored with the mutate function.
Fixed Casting When Generating Default for Enum Value

Added support for typecasting an int default value into the correct enum type in the default parameter. This fixed the issue of trying to use set a strongly typed enum parameter to an int which fails type checking.
Fixed Boolean Edge Case

Boolean types generate 0 != 0 when generating the underlying type which appears to be unique to the boolean type so it is now checked and the proper default value generated. It may be beneficial to check if it is instead an enum type, however the seeming edge case nature is why boolean was chosen to be checked.
Updated Generated Files

Regenerated the auto generated files to reflect the new changes.
Updated Remaining Files

Should fix auto generated header files that were not updated.

* Unified Repeated Code

Relocated identical append code to outside of conditional. Also changed 'casted' default value name from FIELD to INTERFACE to more accurately describe it.

* Moved Field Name Outside Conditional

Removed duplicate _{{FIELD_NAME}} and moved to unified append.
2021-09-30 16:44:39 -07:00
Ivan Dlugos
b9d43a557c dart - add Packable interface (#6846) 2021-09-28 13:23:30 -07:00
mustiikhalil
f63c130c28 Improves documentation, and adding DocC (#6784)
Finished documenting flatbuffersbuilder

Replaces swift 5.5 with 5.2 packages

Starts building the tutorial for xcode 13

Finishes building the tutorial for xcode 13

Removes docc files from old swift versions

Updates swift style guide
2021-09-27 20:59:19 +02:00
Derek Bailey
e2b26ee19b Make and generate_code.sh generate same outputs (#6855)
* added --bfbs-builtins

* update generate_code.bat

* forgot the .

* updated checking scripts

* added bypass for the monster_test.bfbs and arrays_test.bfbs diff issue

* removed check on windows for now
2021-09-24 12:01:34 -07:00
Derek Bailey
354d97f6da fixed errant itemgroup element in C# test project (#6854)
errant save
2021-09-21 15:01:29 -07:00
Todd Hansen
6f7a57eaa0 [C#] Using 'global::' as qualifying_start_ within BaseGenerator (#6767)
* Update idl_gen_csharp.cpp

Change csharp generator to use "global::" for it's qualifying_start_ to disambiguate namespaces

* regenerate testing files

regenerate testing files

* Missed TableInC.cs

updated with global prefix

* Remove "global::" from qualifying_start_ for csharp generator

* C# global alias

* Tests and docs for --cs-global-alias

Add tests for --cs-global-alias to demonstrate use case for why it's needed.

Add documentation to Compiler.md

* Add also to help text

Add also to help text
2021-09-15 16:57:29 -05:00
Ivan Dlugos
90baa1444b Dart: binary lists (typed_data) (#6839)
* Dart - add eager mode to Uint8ListReader

* Dart - add Int8ListReader

* Dart - use binary reader where useful

* Dart - test binary list reader laziness

* Dart - update generated code
2021-09-15 08:50:57 -07:00
Artavazd Balaian
47d35f1053 BugFix: Optional enum when it is null (#6835)
* Test to make sure optional enum is written properly

* Handle optional enum codegen: when cast optional enum add `?`

* Run `tests/generate_code.sh` to generate code from schema

* Fix type casting in case of CreateXXXTypeVector

* Reason why vector's type is not optional
2021-09-10 12:55:46 -04:00
Derek Bailey
8fb8c2ce1d [C#] Use @ for keyword escaping (#6834) 2021-09-09 10:29:27 -07:00
Tyler Denniston
74c3d7eba2 [C++] Add mutable version of LookupByKey and test (#6826)
* [C++] Add mutable version of LookupByKey and test

This adds an overload of LookupByKey to allow lookup in sorted Vectors to
return a mutable instance of the object (or nullptr if not found).

* Fix naming
2021-08-31 17:17:20 -04:00
Stefan F
b20c4d3aad [C++] Add GetMutableSizePrefixedRoot and generate GetMutableSizePrefixedXxx functions (#6815)
* flattests_cpp17 doesn't compile with Visual Studio 2017: warning C4100: 'indent': unreferenced formal parameter

stringify_util.h(127): error C2220: warning treated as error - no 'object' file generated
stringify_util.h(127): warning C4100: 'indent': unreferenced formal parameter
stringify_util.h(85): warning C4100: 'indent': unreferenced formal parameter
stringify_util.h(85): warning C4100: 'fbs': unreferenced formal parameter

* [C++] Add GetMutableSizePrefixedRoot() and generate a GetMutableSizePrefixed function
When using the mutable API together with size prefixed buffers these functions should be present.

* clang-format

* Cleanup branch for PR
Revert "flattests_cpp17 doesn't compile with Visual Studio 2017: warning C4100: 'indent': unreferenced formal parameter"

This reverts commit a92055203e.
2021-08-30 15:34:42 -07:00
Stefan F
ae145293cc flattests_cpp17 doesn't compile with Visual Studio 2017: warning C4100: 'indent': unreferenced formal parameter (#6811)
stringify_util.h(127): error C2220: warning treated as error - no 'object' file generated
stringify_util.h(127): warning C4100: 'indent': unreferenced formal parameter
stringify_util.h(85): warning C4100: 'indent': unreferenced formal parameter
stringify_util.h(85): warning C4100: 'fbs': unreferenced formal parameter
2021-08-30 15:32:36 -07:00
Ivan Dlugos
e3c76a5cde Dart lints (#6808)
* Dart - add analysis options with the recommended lints

* Dart - generate code without linter issues

* Dart - remove linter issues in example and test code

* Dart - remove linter issues in lib code

* Dart - update generate code in /tests/
2021-08-30 12:54:59 -07:00
Casper
4b9123baff fix parser string=null (#6810)
Co-authored-by: Casper Neo <cneo@google.com>
2021-08-24 09:41:39 -04:00
Casper
f89e0b1a6c Refactor idl_gen_rust and Rust generated code to use consistent whitespace (#6809)
* Remove dead code in idl_gen_rust

* Use 2space indentation in mod.rs

* use In/DecrementIdentValue in idl_gen_rust

Fix some whitespace too in generated code.

* make default fn 2space ident

* More 2space formatting

* git clang format

* make vs2015 happy

Co-authored-by: Casper Neo <cneo@google.com>
2021-08-23 18:53:10 -04:00
Casper
273f6084e5 Fix Rust keyword inconsistency (#6794)
* Make Rust keyword escaping more consistent

* fix comment

* git clang format

Co-authored-by: Casper Neo <cneo@google.com>
2021-08-18 15:37:44 -04:00
Todd Hansen
4a0879458d [C#] Remove superfluous semicolon in C# structs (#6791)
Aims to resolve #6788

-Remove superfluous semicolon in C# structs
2021-08-17 11:02:57 -04:00
Todd Hansen
067dce6e79 [C#] Handle keywords (#6775)
* Handle keywords in C#

C# should handle keywords similarly to other languages by appending an underscore after the keyword.

* Fix clang-format for idl_gen_csharp.cpp

* Fix spacing clang-format issue
2021-08-16 14:09:21 -04:00
bakinovsky-m
5d77820b34 [C++] Using calculated fields_number for field_names size (#6769)
* using already defined "fields_number" in "field_names" size calculation

* tests

* forgotten test
2021-08-05 11:41:49 -07:00
RWauson
909ce970ae [C++] Use UnPackTo instead of UnPack if pointer exists (#6725)
Causes generated code to check for existing pointers in an object (when using the object based api) and use UnPackTo(existingPointer) rather than just using UnPack() to replace the pointer. This is a performance issue when unpacking to existing objects when using large buffer fields (e.g. image frames)

Co-authored-by: RobWauson <robwauson@gmail.com>
2021-08-05 11:40:03 -07:00
Casper
35e2cac6eb Store vtables sorted in Rust builder (#6765)
* benchmark many vtables

* Rust: Store written_table rev-positions sorted.

The previous implementation was slow if there were too many tables.

Asymototically when inserting the n^th vtable: The old implementation
took O(n) lookup steps and O(1) insertion. The new implementation is
O(log n) lookup and O(n) insertion. This might be improved further by
using a balanced btree.

Benchmarking, create_many_tables is 7.5x faster (on my laptop):

// Simple vector cache
test create_many_tables ... bench: 728,875 ns/iter (+/- 12,279) = 44 MB/s

// Sorted vector cache
test create_many_tables ... bench: 97,843 ns/iter (+/- 4,430) = 334 MB/s

* Fix lints

Co-authored-by: Casper Neo <cneo@google.com>
2021-08-03 12:31:45 -07:00
mogemimi
c39fc9dd9c [C++] Use nullptr instead of 0 in generated headers (#6762) 2021-08-02 13:41:15 -07:00
Ivan Dlugos
97d9527f6c Dart 2.0 release prep (#6759)
* Dart - pubspec.yaml shouldn't contain authors section anymore - it's unused

* Dart - update CHANGELOG.md

* Dart - update test and publish scripts
2021-08-02 13:31:36 -07:00
mugisoba
e77926f0ed [C#] Add Union Constructor Utility for ObjectAPI. (#6105)
* [C#] add union constructor utility.

* add union utility test.

* std::tolower -> CharToLower

* use std::transform instead

* rebase to master. and regenerate test code
2021-07-26 10:01:41 -07:00
Taiju Tsuiki
8be8a0a713 [C++] Fix union type handling on object api in C++17 (#6745) 2021-07-25 16:20:34 +07:00
Casper
12e341e4f8 Rework how Rust generated files are laid out (#6731)
* Refactored Rust Generated code into a module directory.

Each symbol will be generated into one file and then
imported into a module. This breaks the "out_dir"
pattern where some users would generate code in their
target/ directory. Also, these objects are best used
in their own module. It will be hard for users to share
their own module structure with flatbuffers namespaces.

There may be solutions to these drawbacks but that should
be discussed. I don't want to overengineer here.

* shadow error

* try fix .bat file

* fix .bat 2

* Restore accidentally deleted files

* Fixed some DONOTSUBMITs and made Rust outdir pattern use symlinks.

* fixed binary files

* git clang format

* make generated onefiles not public and fix .bat

* reduced diff with master in generate_code.sh

* fix shadowed variable

* add object api flags to .bat

* space

* Removed extern crate and extra &

* use statement

* more clippy lints

* format

* Undo extern crate -> use change, it actually matters to our tests

Co-authored-by: Casper Neo <cneo@google.com>
2021-07-22 19:02:28 -04:00
Ivan Dlugos
2bfc8e9f01 Dart - make writeString() argument non-nullable (#6737) 2021-07-20 07:54:48 -07:00
Casper
674a9f2aae fix lints in rust tests (#6743)
Co-authored-by: Casper Neo <cneo@google.com>
2021-07-19 18:47:20 -04:00
Ivan Dlugos
65700441d3 Dart - make vTable fixed size (expect the number of fields when creating) (#6735) 2021-07-19 11:27:01 -07:00
Ivan Dlugos
dd5bb55cad Dart - make vTable deduplication optional (#6734) 2021-07-15 15:50:12 -07:00
Ivan Dlugos
e73fab27d4 Dart - store empty vectors instead of NULL (#6728) 2021-07-09 12:44:40 -07:00
Ivan Dlugos
92ae532e43 Dart - finish/lowFinish/buffer changes (#6712)
* Dart - change Builder "lowFinish()" to "buffer" and "finish()" to not void return

Aligning the API with other languages, e.g. c++ and allowing custom use-cases to avoid creating a Uint8List

* Dart - change builder.buffer to check that finish() was already called

* Dart - builder - move !finished assertion to _prepare() which is run from all other functions
2021-07-09 11:45:45 -07:00
Ivan Dlugos
3f77dc9a0e Dart - unpack() must use eager list reader (#6723) 2021-07-09 09:55:34 -07:00
Ivan Dlugos
838c93b843 Dart - fix flex-builder compilation error (#6722)
* Dart - fix flex-builder compilation error

* DartTest.sh - run all tests
2021-07-09 09:53:27 -07:00
Vladimir Glavnyy
22498cf3a9 [C++] add make_span for Array<T,N> (#6663)
* [C++] Add make_span for Array<T,N> and Vector<T>

* [C++] Add tests for make_span<T,N>(Array<T,N>)

* [C++] Add span iterators

* [C++] Checked span iterator (MSVC)

* Fix review notes
2021-06-28 10:22:02 -07:00
Ivan Dlugos
a6ee335574 Dart null safety (#6696)
* Dart null-safety - prepare migration annotations in library files

* Dart null-safety - apply migration

* Dart null-safety - update flatc to generate valid code

* Dart null-safety - fix flatc generated code and adjust tests

* Dart null-safety - update example and the generated code in the tests folder

* Dart null safety - minor review changes

* Dart - apply `dartfmt -w .`
2021-06-22 08:54:57 -07:00
Casper
71d43f3be9 Make --bfbs-filenames default to location of first schema file. (#6705)
* Make --bfbs-filenames default to location of first schema file.

Make RelativeToProjectRoot always work, applying "../" where needed. This is
needed for backwards compatibility. The first input file may be deeper in some
directory than the other files. Now, there will always be a declaration
file.

* documentation

* clang format

Co-authored-by: Casper Neo <cneo@google.com>
2021-06-22 10:19:46 -04:00
schoetbi
c8db1ca5d4 Jsonschema add prop comments (#6617)
* Add comments for properties

* regenerated test files

* Reverted changes in testfiles generated with generate_code.bat on windows

* fixed formatting in util.cpp because of failed .travis/format_check.sh

* removed the use of tuple

* util.cpp: Fixed format

* idl_gen_json_schema.cpp: removed extra comma in deprecated fields

* Corrected monster_test.schema.json

* idl_gen_json_schema: Inlined Trim function

* Fixed format

* PrepareDescription(): Avoid unnecessary string copy

* fixed formatting
2021-06-21 11:42:07 -07:00
Mika Raento
962751a6ec Improve generated comparisons for tables (#6486)
* Improve generated comparisons for tables

Previously, the generated code called std::unique_ptr<>::operator== on
non-scalar members, which would be correct only if the member was null
on both sides. After this CL, comparison is done on the pointed-to
values (including using a default-constructed value if the pointer is
null).

* Don't equate null Tables with all defaults

Also removes the cost of default-constructing tables for comparisons.

* Regenerate code

* fix formatting
2021-06-21 11:37:56 -07:00