Commit Graph

693 Commits

Author SHA1 Message Date
Max Burke
355dfd48d1 [rust] Make enum names public (#5690)
* Bugfix for Rust generation of union fields named with language keywords

Looking at ParseField, it appears that in the case of unions, an extra field with a `UnionTypeFieldSuffix` is added to the type definition, however, if the name of this field is a keyword in the target language, it isn't escaped.

For example, if generating code for rust for a union field named `type`, flatc will generate a (non-keyword escaped) field named `type_type` for this hidden union field, and one (keyword escaped) called `type_` for the actual union contents.

When the union accessors are generated, they refer to this `type_type` field, but they will escape it mistakenly, generating code like this:

```
  #[inline]
  #[allow(non_snake_case)]
  pub fn type__as_int(&self) -> Option<Int<'a>> {
    if self.type__type() == Type::Int {
      self.type_().map(|u| Int::init_from_table(u))
    } else {
      None
    }
  }
```

Which will fail to build because the field is called `self.type_type()`, not `self.type__type()`.

* [Rust] Add crate-relative use statements for FBS includes.

At present if a flatbuffer description includes a reference to a type in
another file, the generated Rust code needs to be hand-modified to add
the appropriate `use` statements.

This assumes that the dependencies are built into the same crate, which
I think is a reasonable assumption?

* Revert "[Rust] Add crate-relative use statements for FBS includes."

This reverts commit d554d79fec.

* Address comments raised in PR

* Update documentation comments per feedback

* Fix typo

* [rust] Make enum variant names public.

* Update generated test files

* Add test for public enum names
2019-12-31 10:28:58 -08:00
lu-wang-g
a2c12900aa Optimize Pack method using numpy (#5662)
Add the support to pack using numpy for scalar vectors when numpy is available.
2019-12-26 21:42:11 -05:00
Austin Schuh
901b89e733 [C++] Add Builder and Table typedefs (#5685)
* Add Builder and Table typedefs

This gives us a way to use templates to go from a builder to a table
and back again without having to pass both types in.

* Fix tests/cpp17/generated_cpp17/monster_test_generated.h
2019-12-26 14:56:46 -08:00
Austin Schuh
b4154405d1 Fix --incompatible_load_cc_rules_from_bzl (#5683)
Incompatible flag --incompatible_load_cc_rules_from_bzl will break
FlatBuffers once Bazel 1.2.1 is released.

Fixes #5671
2019-12-26 10:03:22 -08:00
Robert Winslow
7de668053f CI: New Docker tests for Python with numpy (#5677)
* New Docker tests for Python with numpy

* print numpy status in test suite
2019-12-24 02:14:55 -05:00
Wouter van Oortmerssen
c957550511 Removed test proto output.
Change-Id: Iaf64bec068d03dd1b75670e9a28dde7392ebddb5
2019-12-23 12:33:16 -08:00
Vladimir Glavnyy
44bf719883 Add flatc '--cpp_std' switch (#5656)
* Add flatc '--cpp_std' switch and sandbox for C++17 code generator

- Added 'flac --cpp_std legacy' for compatibility with old compilers (VS2010);
- Added experimental switch 'flac --cpp_std c++17' for future development;
- Added C++17 sandbox test_cpp17.cpp;
- C++ code generator generates enums with explicit underlying type to avoid problems with the forward and backward schema compatibility;
- Adjusted CMakeLists.txt, CI and generate code scripts to support of introduced '--cpp_std';

* Fix --cpp_std values: c++0x, c++11, c++17

* Add 'cpp::CppStandard' enum

* Add testing engine into test_cpp17

* Rebase to upstream/master

* Set default '--cpp-std C++0x'

* Fix code generation (--cpp_std C++11) in CMakeLists.txt

- Fix dependency declaration of grpctest target

* Revert --cpp-std for the tests from explicit C++11 to flatc default value (C++0x)
2019-12-23 12:13:48 -08:00
Wouter van Oortmerssen
3e8f15df90 Fix for FlexBuffers FBT_VECTOR_STRING size bit-width.
For details, test.cpp/FlexBuffersDeprecatedTest(), and also
https://github.com/google/flatbuffers/issues/5627

Change-Id: I6e86e1138a5777e31055cfa2f79276d44732efbc
2019-12-23 11:49:59 -08:00
Michael Beardsworth
13c05f4da3 Improve import handling for proto conversion (#5673)
* Keep include prefix when converting from proto.

This change preserves the include prefix when generating flatbuffers
from proto (with FBS_GEN_INCLUDES) defined.

* Improve handling of imports in proto conversion.

Previously, there was no runtime flag to make proto->fbs conversion keep
the import structure of a collection of files. This change makes proto
conversion respect the --no-gen-includes flag and skip the output of
"generated" symbols.
2019-12-23 08:50:29 -08:00
Matt Brubeck
aa75e5734b Make Rust constants public (#5659)
* Make Rust constants public

Otherwise they cannot be accessed by code that consumes the generated
bindings.

* Re-generate test code

* Add a test for enum constants
2019-12-17 12:18:59 -08:00
cryptocode
2790fee257 Add namespace qualification to union types (#5666) 2019-12-17 09:11:26 -08:00
FujiZ
030fee36ab wrap multiple statements in do {} while(!IsConstTrue(true)) (#5655) 2019-12-13 10:14:14 -08:00
bttk
99d11e279f Split Bazel targets into multiple packages (#5640)
* Split Bazel targets into multiple packages

* Merge /include/BUILD back into /BUILD
2019-12-02 14:14:48 -08:00
Google AutoFuzz Team
4fd8eb214b Remove a static_assert (#5643)
Having a static_assert on MSAN and ASAN prevents
the fuzzers from being used with different engines,
like TSAN, UBSAN, … but also with fuzzers that aren't
using MSAN/ASAN like afl for example.
2019-12-02 14:13:28 -08:00
lu-wang-g
65f8703572 Flatbuffers Python Object API (#5616)
* Flatbuffers Python Object API

Implement the logic to generate the Python object API that can
unpack the data from a buf class into an object class, and pack
the data of an object class to a buf class.

* Fix the build issues

Remove unused parameters and replace auto in the for-loop statement
with std::string to make it compatible with VS2010.

* Fix the build issues.

* Add support for Array type

Added logic to handle Array type in Python Object API. Updated the
generated code accordingly.

* Fix the old style casting from int to char

* Fixed another conversion from int to char

* Fixed the import for typing

Importing typing may cause errors when a machine do not have the
moduel typing installed. This PR fixes the issue by guarding
"import typing" with the "try/except" statement.

* Fix issue of iterating the vector of import list

* Update the generated examples using generate_code.sh

* Fix the import order for typing

The import list was stored in unordered_set, so that each generated
codes may have different import order. Therefore, it failed in the
consistency test where two generated copies need to have exactly the
same apperance.

* Optimize unpack using numpy

Use numpy to unpack vector whenever it is possible to improve unpack
performance.

Also, added codegen command for Python specificly in generate_code.sh,
because --no-includes cannot be turn on for Python.

* Fix the import order

* Update generate_code.bat for windows accordingly

* Replace error message with pass

Avoid printing error message for every Python2 users about typing.
Replace it with pass.
2019-12-02 14:11:28 -08:00
Tiago Silva
3c964e10ab [GO] Fix support for enums with underscores and Unions with imported members (#5600)
* Fix Enum Stringer method when Enum has underscores

Fixes #5481

* Fix go package importing itself when Union has imported members.
2019-11-26 12:25:36 -08:00
Vladimir Glavnyy
c3c32ec942 Fix ambiguity of a type deduction in TEST_EQ macro if arguments have enum class type. (#5630) 2019-11-25 12:56:47 -08:00
Derek Bailey
136d75fa65 Changed null checks in test. Removed verifier pointer usage (#5634) 2019-11-21 10:25:31 -08:00
Vladimir Glavnyy
091fa1fd1b Add testing of C++ with sanitizers (CI-Docker) (#5631)
* Add C++ build testing with clang and gcc

This adds Dockerfiles which test building flatc and the C++ library against clang
and gcc. See discussion at #5119.  It is derived from the Travis CI tooling.

The GRPC tests are failing due to #5099 so those are commented out.

These are run from the .travis.yml file rather than the tests/docker/languages
folder because the builds may each take longer than 30 minutes and were hitting
Travis timeouts.

Parallel builds and build caching attempt to keep the build times low.

* Add GCC 8.3 and Clang 7.0 with sanitizers into CI (based on #5130)

- Add a docker based on Debian Buster.
- Add C++ building scripts for the docker.
- Leak-sanitizer requires SYS_PTRACE.
2019-11-18 12:16:41 -08:00
Malar Kannan
6beb9f49cb Support for python grpc - continuing the work from the pull request #4270 #4705 (#5613)
* Support for python grpc

* add few fixes

* Fixes build

* Fix python generator

* Add tests

* Fix grpc python test

* Fix tests and add incomplete python generator

* Fix python generator

* Add python generator methods

* Fix Appveyor build

* grpc python support v0.1

* Update tests

* update grpctest

* Remove duplicated code and fix a brace

* tests for flatbuffers grpc python

* Updated tests + removed SerializeToString, From String

* remove pickle import

* include missing files in ci - BUILD and generated test result
2019-11-14 16:58:35 -08:00
Derek Bailey
0f2ff7eaa9 Lua cleanup (#5624) 2019-11-14 15:46:46 -08:00
Derek Bailey
dda095023d [C++] Adds basic schema evolution tests (#5611)
* Added basic schema evolution tests

* Add BUILD targets for evolution tests. Added to test/generate_code scripts

* Use vector.front() instead of vector.data()

* Added --scoped-enums option for evolution test
2019-11-14 15:44:18 -08:00
Derek Bailey
42c08cbca6 Ran src/clang-format-all.sh (#5617) 2019-11-11 09:29:37 -08:00
Derek Bailey
105dd528e9 Change monster_extra generation to use flatbuffers::unique_ptr (#5612) 2019-11-07 14:10:41 -08:00
Derek Bailey
f0f0efe7b8 [C++] Refactor to conform to Google C++ style guide (#5608)
* Automatic refractor of C++ headers to Google C++ style guide

* Automatic refractor of C++ source to Google C++ style guide

* Automatic refractor of C++ tests to Google C++ style guide

* Fixed clang-format issues by running clang-format twice to correct itself. Kotlin was missing clang-format on after turning it off, so it was changed,
2019-11-07 12:22:54 -08:00
Derek Bailey
46ae3f80a6 [C++, Java, C#, TypeScript, JavaScript] Skip generation of mutable union types (#5599)
* Skip generation of mutable union types

* Removed C# and Java unit tests that mutated a Union type
2019-11-04 14:54:44 -08:00
Derek Bailey
661bedd837 Add Lua FlatbufferBuilder Clean() method to enable reuseable builders (#5606) 2019-11-04 14:27:59 -08:00
Paulo Pinheiro
8526e12d73 [Kotlin] Fix union vector accessor after change in Java API (#5605)
Kolin uses java library as dependency, which changed the way it access union vector recently
(e365c502ff).

This changes updates kotlin code generation to match Java's changes.
2019-11-04 14:25:07 -08:00
Lawrence Chan
964365ba69 [Go] Add UnPackTo functions (#5598) 2019-11-04 12:12:25 -08:00
iceboy
32254b7acd [Go] Object API support (#5339)
* start

* works for current usages!

* unpack: vector of struct

* optimize byte slice

* support nested struct

* support null table

* support struct

* support union

* update generated code

* grumble

* fix compiler warning

* update generated code

* wrap type in namespace

* bug

* wrap in namespace

* enum byte arrays

* generate struct for unions

* basic testing

* remove branching

* fix assert

* pack vector of fixed structs correctly

* omit null vectors

* Refactor Union Pack and UnPack methods

Remove append usage to increase code efficiency when dealing with large vectors

* generate goldens
2019-10-31 11:13:45 -07:00
Mathias Svensson
521e255ad9 Rust: Add idiomatic iterator for Vector type (#5579)
* Rust: Add idiomatic iterator for Vector type

* Add comments explaining some implementation details
2019-10-28 14:19:47 -04:00
Derek Bailey
480815447a C++ verifier for evolved union fields should return true (#5586) 2019-10-28 09:43:33 -07:00
Derek Bailey
8d5e424c65 Add ByteBuffer copy for vector of bytes in Java (#5587) 2019-10-28 09:30:31 -07:00
Mathias Svensson
b4774d2354 Rust: Fix Copy and Clone impls for a few generic types (#5577)
* Rust: Fix Copy and Clone impls for a few generic types

* Add tests for Copy+Clone

* Wrap Copy+Clone checks in a #[test] function
2019-10-28 00:20:29 -04:00
Bharat Tak
e9d29c21a7 Python: Add forceDefaults opt to python Builder (#5564)
* Add forceDefaults opt to python Builder

* Add test functions for force_default option for python builder

* Simplify

* Add force default test for UOffsetTFlags
2019-10-17 15:25:05 -07:00
Wouter van Oortmerssen
df3e8bf4a7 Fixed warnings generated by recent JSON sorting feature.
Change-Id: I6fd6283b616c7a39bb878b1610e4ddf6e208fa0a
2019-10-17 15:16:40 -07:00
tira-misu
5797540ed0 #5544 Fix of Array of table is not sorted if Create<type>Direct() is used (#5546)
* Fix C/C++ Create<Type>Direct with sorted vectors

If a struct has a key the vector has to be sorted. To sort the vector
you can't use "const".

* Changes due to code review

* Improve code readability
2019-10-17 14:22:21 -07:00
Wouter van Oortmerssen
32f47ad247 Fixed JSON parser not sorting vectors of tables/structs with key.
Change-Id: Iacc0c8513af80a736700e6cbaf513ebdf8e3ac89
2019-10-10 15:25:39 -07:00
Paulo Pinheiro
d4cae0a623 Fix issue #5542 (#5543)
Empty objects that inherit from Sized would try to access internal
ByteBuffer when Sized::size was called. So we add a single byte in
the empty buffer, so when size() is called it would return 0
2019-09-30 14:05:19 -07:00
Robert Winslow
a92039687a Update Rust versions under test from 1.30.1 to 1.37.0 (#5538)
Fixes build errors from using newer crate versions with an old Rust version.
2019-09-26 14:31:11 -07:00
Paulo Pinheiro
3f8ce99c50 [FlexBuffers][Java] Add override Key::toString (#5533) 2019-09-25 14:14:39 -07:00
Paulo Pinheiro
0798b7b698 [FlexBuffers][Java] Fix wrong access to a string using Reference::asString(). (#5532)
The real position of a string is  calculated by using the indirect() method,
which should be based on parentWidth and not byteWidth, as it was implemented.

We are also fixing the flag BUILDER_FLAG_SHARE_STRINGS on FlexBuffersBuilder
that was set as '1', same value as BUILDER_FLAG_SHARE_KEYS.
2019-09-25 11:59:10 -07:00
Vladimir Glavnyy
cbdf82e2fb Fix Mutate() methods of Array<scalar/struct> (override 5508) (#5526)
* Draft with Array specialization (#5508)

* Array specialization + SFINAE to fold copy-paste (#5508)

* Add implicit specialization of Array<scalar> and Array<struct> (#5508)

- Tag dispatching is used for implicit specialization
- Array<scalar> and Array<struct> have different iterators and accessors
- Array<scalar> and Array<struct> have different Mutate() methods

* Add implicit specialization of Array<scalar> and Array<struct> (#5508)

- Tag dispatching is used for implicit specialization
- Array<scalar> and Array<struct> have different iterators and accessors
- Array<scalar> and Array<struct> have different Mutate() methods
2019-09-23 09:31:51 -07:00
Kulikov Alexey
e365c502ff Java: Added access object for vector of struct and vector of tables. (#5233)
* Java: Added access object for vector of struct and vector of tables.

* Java: Workarounds removed when accessing the union vector.
2019-09-23 09:22:43 -07:00
Wouter van Oortmerssen
917687c7a6 Fixed Reflection Verifier not handling vectors of unions.
Change-Id: Ie94386ff8e10fd2a964bd9155139b50953746a37
2019-09-16 17:48:54 -07:00
Wouter van Oortmerssen
f9277e691d Fixed GenerateText not handling vectors of unions.
Change-Id: Ie82abaf178495c4692e7d10be6b4a13f2fa1bee6
2019-09-16 14:44:14 -07:00
Kevin Fort
2706381eef Add element size parameter to __vector_as_arraysegment [c#] (#5512)
* Add element size parameter to __vector_as_arraysegment

Add element size parameter to __vector_as_arraysegment fixing issue where VectorAsBytes returns incorrect size span for multibyte element types.

* Update codegen

Update codegen and Table to return typed span.

* update test files

update test files
2019-09-10 17:07:46 -07:00
Casper
c0282873fb Rust: Fixed cargo clippy on non-generated code (#5485)
* Cargo clippy lints

* more lints

* more lints

* Restored a doc comment

* Comment on float eps-eq and adjusted casting
2019-09-09 13:02:43 -07:00
Jaemin Park
4b870aca98 [Javascript] Fix syntax error for signed enum (#5503)
* wrap quotes to enum name map to prevent syntax errorn when enum value is negative

* Add a test that covers signed enum case
2019-09-05 10:22:04 -07:00
Nathan Williams
d0e3870c0f [C#] Fix retrieving enumeration vectors as arrays (#5457)
* [C#] Fix retrieving enumeration vectors as arrays

* [C#] Don't generate CreateVectorBlock for enums
2019-09-03 14:10:54 -07:00