Commit Graph

887 Commits

Author SHA1 Message Date
Wouter van Oortmerssen
55686100aa Changed direct calls to strtod to use StringToNumber
StringToNumber will correctly use locale-insensitive functions
when available.

Change-Id: I6bde11039a541634186f8f791012af2eb0d86b8d
2020-01-09 11:48:09 -08:00
Michael Beardsworth
21b7061963 (Optionally) add an additional suffix namespace to generated fbs files. (#5698)
This change allows for the generation of fbs files (from proto) that
don't contain name collisions with the protobuf generated C++ code,
allowing both the proto and fbs message types to be linked into the same binary.
2020-01-06 10:00:59 -08:00
dreifachstein
35daaf83d3 [Java] Replace Table.UTF8_CHARSET with StandardCharsets.UTF_8 (#5696)
StandardCharsets.UTF_8 is already used in FlexBuffersBuilder.
2020-01-06 09:28:19 -08:00
David P. Sicilia
a5d9d0f7d3 [C++17] Add Traits class for Tables and Factory function within it. (#5678)
* Include flattests_cpp17 in unit tests when C++17 build is enabled.

* [C++17] Generate generic table factory function.

1. For each table, generate a convenient free-standing factory
   function that allows creating the table in a generic way by
   specifying only the type.  This is the first change in a series
   of changes to make Flatbuffers generated C++ code more friendly
   to code bases that make use of C++ template metaprogramming
   techniques to manage the serialization process.  Example:

     Before :(

       // The name of the Flatbuffers type (and namespace) must
       // be hard-coded when writing the factory function.
       auto monster = MyGame::Example::CreateMonster(fbb, ...);

     After  :)

       using type_to_create = MyGame::Example::Monster;
       // No namespace needed on CreateByTagType.
       auto monster = CreateByTagType((type_to_create*)nullptr,
                                      fbb, ...);

   This feature requires building with C++14 or greater, and thus
   it is guarded behind --cpp-std >= c++17 in the flatbuffers C++
   generator.

2. Fix a CMake bug to include C++17 unit tests in test suite.

* [C++17] Replace standalone variadic factory function with type_traits.

Add a `type_traits` to each table class.  This `type_traits` can be
populated with various compile-time info about the table.  Initially,
we have the Create* function and type, but is extensible in the future.

* Remove empty line and fix stale comments.

* Rename type_traits to Traits and move fwd declaration.

* Fix parameter evaluation order issue and use lambda for scope.
2020-01-02 10:12:14 -08:00
Max Burke
355dfd48d1 [rust] Make enum names public (#5690)
* Bugfix for Rust generation of union fields named with language keywords

Looking at ParseField, it appears that in the case of unions, an extra field with a `UnionTypeFieldSuffix` is added to the type definition, however, if the name of this field is a keyword in the target language, it isn't escaped.

For example, if generating code for rust for a union field named `type`, flatc will generate a (non-keyword escaped) field named `type_type` for this hidden union field, and one (keyword escaped) called `type_` for the actual union contents.

When the union accessors are generated, they refer to this `type_type` field, but they will escape it mistakenly, generating code like this:

```
  #[inline]
  #[allow(non_snake_case)]
  pub fn type__as_int(&self) -> Option<Int<'a>> {
    if self.type__type() == Type::Int {
      self.type_().map(|u| Int::init_from_table(u))
    } else {
      None
    }
  }
```

Which will fail to build because the field is called `self.type_type()`, not `self.type__type()`.

* [Rust] Add crate-relative use statements for FBS includes.

At present if a flatbuffer description includes a reference to a type in
another file, the generated Rust code needs to be hand-modified to add
the appropriate `use` statements.

This assumes that the dependencies are built into the same crate, which
I think is a reasonable assumption?

* Revert "[Rust] Add crate-relative use statements for FBS includes."

This reverts commit d554d79fec.

* Address comments raised in PR

* Update documentation comments per feedback

* Fix typo

* [rust] Make enum variant names public.

* Update generated test files

* Add test for public enum names
2019-12-31 10:28:58 -08:00
Michael Beardsworth
bcd58a159b Correct inverted logic around include prefixes. (#5689)
4d1a9f8d9e inverted the logic around
keeping the include prefix. This change fixes the error.
2019-12-30 10:34:01 -08:00
lu-wang-g
a2c12900aa Optimize Pack method using numpy (#5662)
Add the support to pack using numpy for scalar vectors when numpy is available.
2019-12-26 21:42:11 -05:00
Austin Schuh
901b89e733 [C++] Add Builder and Table typedefs (#5685)
* Add Builder and Table typedefs

This gives us a way to use templates to go from a builder to a table
and back again without having to pass both types in.

* Fix tests/cpp17/generated_cpp17/monster_test_generated.h
2019-12-26 14:56:46 -08:00
Vladimir Glavnyy
44bf719883 Add flatc '--cpp_std' switch (#5656)
* Add flatc '--cpp_std' switch and sandbox for C++17 code generator

- Added 'flac --cpp_std legacy' for compatibility with old compilers (VS2010);
- Added experimental switch 'flac --cpp_std c++17' for future development;
- Added C++17 sandbox test_cpp17.cpp;
- C++ code generator generates enums with explicit underlying type to avoid problems with the forward and backward schema compatibility;
- Adjusted CMakeLists.txt, CI and generate code scripts to support of introduced '--cpp_std';

* Fix --cpp_std values: c++0x, c++11, c++17

* Add 'cpp::CppStandard' enum

* Add testing engine into test_cpp17

* Rebase to upstream/master

* Set default '--cpp-std C++0x'

* Fix code generation (--cpp_std C++11) in CMakeLists.txt

- Fix dependency declaration of grpctest target

* Revert --cpp-std for the tests from explicit C++11 to flatc default value (C++0x)
2019-12-23 12:13:48 -08:00
Michael Beardsworth
13c05f4da3 Improve import handling for proto conversion (#5673)
* Keep include prefix when converting from proto.

This change preserves the include prefix when generating flatbuffers
from proto (with FBS_GEN_INCLUDES) defined.

* Improve handling of imports in proto conversion.

Previously, there was no runtime flag to make proto->fbs conversion keep
the import structure of a collection of files. This change makes proto
conversion respect the --no-gen-includes flag and skip the output of
"generated" symbols.
2019-12-23 08:50:29 -08:00
Matt Brubeck
aa75e5734b Make Rust constants public (#5659)
* Make Rust constants public

Otherwise they cannot be accessed by code that consumes the generated
bindings.

* Re-generate test code

* Add a test for enum constants
2019-12-17 12:18:59 -08:00
cryptocode
2790fee257 Add namespace qualification to union types (#5666) 2019-12-17 09:11:26 -08:00
Max Burke
eddebec1b6 Bugfix for Rust generation of union fields named with language keywords (#5592)
* Bugfix for Rust generation of union fields named with language keywords

Looking at ParseField, it appears that in the case of unions, an extra field with a `UnionTypeFieldSuffix` is added to the type definition, however, if the name of this field is a keyword in the target language, it isn't escaped.

For example, if generating code for rust for a union field named `type`, flatc will generate a (non-keyword escaped) field named `type_type` for this hidden union field, and one (keyword escaped) called `type_` for the actual union contents.

When the union accessors are generated, they refer to this `type_type` field, but they will escape it mistakenly, generating code like this:

```
  #[inline]
  #[allow(non_snake_case)]
  pub fn type__as_int(&self) -> Option<Int<'a>> {
    if self.type__type() == Type::Int {
      self.type_().map(|u| Int::init_from_table(u))
    } else {
      None
    }
  }
```

Which will fail to build because the field is called `self.type_type()`, not `self.type__type()`.

* [Rust] Add crate-relative use statements for FBS includes.

At present if a flatbuffer description includes a reference to a type in
another file, the generated Rust code needs to be hand-modified to add
the appropriate `use` statements.

This assumes that the dependencies are built into the same crate, which
I think is a reasonable assumption?

* Revert "[Rust] Add crate-relative use statements for FBS includes."

This reverts commit d554d79fec.

* Address comments raised in PR

* Update documentation comments per feedback

* Fix typo
2019-12-15 15:18:22 -08:00
cryptocode
a8e800bd7c Add --force-empty-vectors option (#5653)
The rationale for this option is that JSON clients typically want empty arrays (i.e [] in the JSON) instead of missing properties, but not empty strings when the value isn't set.
--force-empty is kept as-is, i.e. it will force both empty strings and vectors.

Closes #5652
2019-12-05 17:33:45 -08:00
Wouter van Oortmerssen
d7530ae961 Fixed enum min/max values not properly escaped.
Change-Id: I503fbfaff1d2579807ca71a07cca8363dff75e52
2019-12-05 14:28:36 -08:00
bttk
99d11e279f Split Bazel targets into multiple packages (#5640)
* Split Bazel targets into multiple packages

* Merge /include/BUILD back into /BUILD
2019-12-02 14:14:48 -08:00
lu-wang-g
65f8703572 Flatbuffers Python Object API (#5616)
* Flatbuffers Python Object API

Implement the logic to generate the Python object API that can
unpack the data from a buf class into an object class, and pack
the data of an object class to a buf class.

* Fix the build issues

Remove unused parameters and replace auto in the for-loop statement
with std::string to make it compatible with VS2010.

* Fix the build issues.

* Add support for Array type

Added logic to handle Array type in Python Object API. Updated the
generated code accordingly.

* Fix the old style casting from int to char

* Fixed another conversion from int to char

* Fixed the import for typing

Importing typing may cause errors when a machine do not have the
moduel typing installed. This PR fixes the issue by guarding
"import typing" with the "try/except" statement.

* Fix issue of iterating the vector of import list

* Update the generated examples using generate_code.sh

* Fix the import order for typing

The import list was stored in unordered_set, so that each generated
codes may have different import order. Therefore, it failed in the
consistency test where two generated copies need to have exactly the
same apperance.

* Optimize unpack using numpy

Use numpy to unpack vector whenever it is possible to improve unpack
performance.

Also, added codegen command for Python specificly in generate_code.sh,
because --no-includes cannot be turn on for Python.

* Fix the import order

* Update generate_code.bat for windows accordingly

* Replace error message with pass

Avoid printing error message for every Python2 users about typing.
Replace it with pass.
2019-12-02 14:11:28 -08:00
Tiago Silva
3c964e10ab [GO] Fix support for enums with underscores and Unions with imported members (#5600)
* Fix Enum Stringer method when Enum has underscores

Fixes #5481

* Fix go package importing itself when Union has imported members.
2019-11-26 12:25:36 -08:00
Vladimir Glavnyy
075e8d676b Simplify declarations of x-macro FLATBUFFERS_TD (#5638) 2019-11-25 12:54:59 -08:00
Malar Kannan
6beb9f49cb Support for python grpc - continuing the work from the pull request #4270 #4705 (#5613)
* Support for python grpc

* add few fixes

* Fixes build

* Fix python generator

* Add tests

* Fix grpc python test

* Fix tests and add incomplete python generator

* Fix python generator

* Add python generator methods

* Fix Appveyor build

* grpc python support v0.1

* Update tests

* update grpctest

* Remove duplicated code and fix a brace

* tests for flatbuffers grpc python

* Updated tests + removed SerializeToString, From String

* remove pickle import

* include missing files in ci - BUILD and generated test result
2019-11-14 16:58:35 -08:00
Derek Bailey
80988ea869 Removed idl_gen_general.cpp and move contents to code_generators.cpp (#5625) 2019-11-14 15:47:58 -08:00
Derek Bailey
adbcbba5d1 [C++, C#, Java] Separated C# and Java generators into their own classes (#5618)
* Cloned idl_gen_general.cpp to idl_gen_csharp.cpp and removed java references

* Java generator changes
2019-11-11 11:37:55 -08:00
Dmitry
cbbd6aca04 add check for root_type specified for json schema generation (#5622) 2019-11-11 11:33:27 -08:00
Derek Bailey
42c08cbca6 Ran src/clang-format-all.sh (#5617) 2019-11-11 09:29:37 -08:00
Wouter van Oortmerssen
33d5dd9bdd Improved pull request & clang-format instructions.
Change-Id: Ia439bcc61bac5788792244d10e573b1fba54b347
2019-11-07 14:37:33 -08:00
Derek Bailey
f0f0efe7b8 [C++] Refactor to conform to Google C++ style guide (#5608)
* Automatic refractor of C++ headers to Google C++ style guide

* Automatic refractor of C++ source to Google C++ style guide

* Automatic refractor of C++ tests to Google C++ style guide

* Fixed clang-format issues by running clang-format twice to correct itself. Kotlin was missing clang-format on after turning it off, so it was changed,
2019-11-07 12:22:54 -08:00
Derek Bailey
46ae3f80a6 [C++, Java, C#, TypeScript, JavaScript] Skip generation of mutable union types (#5599)
* Skip generation of mutable union types

* Removed C# and Java unit tests that mutated a Union type
2019-11-04 14:54:44 -08:00
Paulo Pinheiro
8526e12d73 [Kotlin] Fix union vector accessor after change in Java API (#5605)
Kolin uses java library as dependency, which changed the way it access union vector recently
(e365c502ff).

This changes updates kotlin code generation to match Java's changes.
2019-11-04 14:25:07 -08:00
Paulo Pinheiro
3c7b660d62 [flatc] Remove an always true condition for flexbuffers (#5604)
The condition was unnecessary and Detected by

PVS-Studio
V560 [CWE-571] A part of conditional expression is always true: !opts.use_flexbuffers. flatc.cpp 438
2019-11-04 14:23:15 -08:00
Lawrence Chan
964365ba69 [Go] Add UnPackTo functions (#5598) 2019-11-04 12:12:25 -08:00
iceboy
32254b7acd [Go] Object API support (#5339)
* start

* works for current usages!

* unpack: vector of struct

* optimize byte slice

* support nested struct

* support null table

* support struct

* support union

* update generated code

* grumble

* fix compiler warning

* update generated code

* wrap type in namespace

* bug

* wrap in namespace

* enum byte arrays

* generate struct for unions

* basic testing

* remove branching

* fix assert

* pack vector of fixed structs correctly

* omit null vectors

* Refactor Union Pack and UnPack methods

Remove append usage to increase code efficiency when dealing with large vectors

* generate goldens
2019-10-31 11:13:45 -07:00
Derek Bailey
480815447a C++ verifier for evolved union fields should return true (#5586) 2019-10-28 09:43:33 -07:00
Derek Bailey
8d5e424c65 Add ByteBuffer copy for vector of bytes in Java (#5587) 2019-10-28 09:30:31 -07:00
Jacek Surazski
8bfafc76de Java: Don't annotate vector-of-tables item getters with @nullable. (#5562)
* Annotate getters with @Pure when --java-checkerframework is specified.

Together with @Nullable, this allows users to use static analysis tools
like CheckerFramework to catch NPEs caused by unset fields.

* Don't annotate vector-of-tables item getters with @Nullable.

Since Flatbuffers don't support null items in vectors of tables.
2019-10-17 15:23:15 -07:00
Wouter van Oortmerssen
df3e8bf4a7 Fixed warnings generated by recent JSON sorting feature.
Change-Id: I6fd6283b616c7a39bb878b1610e4ddf6e208fa0a
2019-10-17 15:16:40 -07:00
tira-misu
5797540ed0 #5544 Fix of Array of table is not sorted if Create<type>Direct() is used (#5546)
* Fix C/C++ Create<Type>Direct with sorted vectors

If a struct has a key the vector has to be sorted. To sort the vector
you can't use "const".

* Changes due to code review

* Improve code readability
2019-10-17 14:22:21 -07:00
Wouter van Oortmerssen
32f47ad247 Fixed JSON parser not sorting vectors of tables/structs with key.
Change-Id: Iacc0c8513af80a736700e6cbaf513ebdf8e3ac89
2019-10-10 15:25:39 -07:00
Wouter van Oortmerssen
f1147f65bb Fixed Android STLPort related error.
Change-Id: I59ff072e526fc63b3215767a4d4a2a8944b65654
2019-09-30 12:12:36 -07:00
Jack Zhou
69d3fec488 Fix namespaced struct/field name collision detection (#5540) (#5545)
Changes the use of `LookupStruct` to `LookupCreateStruct` in
`ParseField` to also detect when collisions happen in namespaces.
2019-09-30 11:50:03 -07:00
Paulo Pinheiro
cfb4ecf6f0 [flac] Add FlexBuffers option for generating data (#5519)
Alongside --binary and --json, an additional
switch (--flexbuffers) can be passed to convert
data using FlexBuffers
2019-09-26 15:06:11 -07:00
mjvk
625338d095 Adds XOPEN_SOURCE for PATH_MAX and POSIX 1993 for stat (#5529)
* Adds XOPEN_SOURCE for PATH_MAX and POSIX 1993 for stat

These are the only two required extension for compilation of
flatbuffers using -std=c++11 instead of gnu++11.

* Sets _XOPEN_SOURCE to 600 and enable POSIX2001 for fseeko
2019-09-26 09:40:14 -07:00
Kulikov Alexey
e365c502ff Java: Added access object for vector of struct and vector of tables. (#5233)
* Java: Added access object for vector of struct and vector of tables.

* Java: Workarounds removed when accessing the union vector.
2019-09-23 09:22:43 -07:00
Wouter van Oortmerssen
917687c7a6 Fixed Reflection Verifier not handling vectors of unions.
Change-Id: Ie94386ff8e10fd2a964bd9155139b50953746a37
2019-09-16 17:48:54 -07:00
Wouter van Oortmerssen
f9277e691d Fixed GenerateText not handling vectors of unions.
Change-Id: Ie82abaf178495c4692e7d10be6b4a13f2fa1bee6
2019-09-16 14:44:14 -07:00
Kevin Fort
2706381eef Add element size parameter to __vector_as_arraysegment [c#] (#5512)
* Add element size parameter to __vector_as_arraysegment

Add element size parameter to __vector_as_arraysegment fixing issue where VectorAsBytes returns incorrect size span for multibyte element types.

* Update codegen

Update codegen and Table to return typed span.

* update test files

update test files
2019-09-10 17:07:46 -07:00
jaceksur
782b865c55 Annotate getters with @Pure when --java-checkerframework is specified. (#5510)
Together with @Nullable, this allows users to use static analysis tools
like CheckerFramework to catch NPEs caused by unset fields.
2019-09-10 10:01:27 -07:00
Edward
3bfc86eaff [Dart]fix: segment fault with empty namespace when generating dart file (#5507)
* fix: segment fault with empty namespace when generating dart file

* fix: compile err

* fix: compile err

* fix: imported file names

* fix test issues

* fix some language style issues
2019-09-09 14:40:33 -07:00
Jaemin Park
4b870aca98 [Javascript] Fix syntax error for signed enum (#5503)
* wrap quotes to enum name map to prevent syntax errorn when enum value is negative

* Add a test that covers signed enum case
2019-09-05 10:22:04 -07:00
Nathan Williams
d0e3870c0f [C#] Fix retrieving enumeration vectors as arrays (#5457)
* [C#] Fix retrieving enumeration vectors as arrays

* [C#] Don't generate CreateVectorBlock for enums
2019-09-03 14:10:54 -07:00
Wes McKinney
c1058a903b C++ IDL generation adds superfluous semicolon in GenTablePost, causing (#5483)
-Wextra-semi-stmt warning when building with Clang 8.0.0. Update generated
files for testing
2019-08-19 17:49:37 -07:00