Commit Graph

1250 Commits

Author SHA1 Message Date
mustiikhalil
a4b2884e4e Added create function for swift (#5707)
Fixed the create functions and updated the test cases in swift

Removes unneeded code

Updated documentation
2020-01-13 12:34:06 -08:00
mustiikhalil
04d80f255d [Swift] Swift implementation 🎉🎉 (#5603)
* Implemented the swift version of Flatbuffers

Implemented serailzing, reading, and mutating data from object monster

Fixes mis-aligned pointer issue

Fixes issue when shared strings are removed from table

Adds swift enum, structs code gen

Fixed namespace issues + started implementing the table gen

Added Mutate function to the code generator

Generated linux test cases

Fixed an issue with bools, and structs readers in table writer

Swift docker image added

Updated the test cases, and removed a method parameters in swift

Fixed createVector api when called with scalars

Fixed issues with scalar arrays, and fixed the code gen namespaces, added sample_binary.swift

Cleaned up project

Added enum vectors, and their readers

Refactored code

Added swift into the support document

Added documentation in docs, and fixed a small issue with Data() not being returned correctly

Fixes Lowercase issue, and prevents generating lookups for deprecated keys

* Made all the required funcs to have const + removed unneeded code + fix lowercase func

* Removed transform from lowercased and moved it to function

* Fixes an issue with iOS allocation from read

* Refactored cpp code to be more readable

* casts position into int for position

* Fix enums issue, moves scalar writer code to use memcpy

* Removed c_str from struct function

* Fixed script to generate new objects when ran on travis ci: fix

* Handles deallocating space allocated for structs

* Updated the test cases to adhere to the fileprivate lookup, no mutation for unions, and updated the names of the vector functions
2020-01-09 12:12:10 -08:00
Michael Beardsworth
21b7061963 (Optionally) add an additional suffix namespace to generated fbs files. (#5698)
This change allows for the generation of fbs files (from proto) that
don't contain name collisions with the protobuf generated C++ code,
allowing both the proto and fbs message types to be linked into the same binary.
2020-01-06 10:00:59 -08:00
dreifachstein
35daaf83d3 [Java] Replace Table.UTF8_CHARSET with StandardCharsets.UTF_8 (#5696)
StandardCharsets.UTF_8 is already used in FlexBuffersBuilder.
2020-01-06 09:28:19 -08:00
Robert Winslow
3b458f7a17 Rust: Temporarily disable 2 endianness unit tests (#5695)
Rare failures occur on AppVeyor in these functions; the failures appear
spurious.
2020-01-02 13:34:47 -08:00
David P. Sicilia
a5d9d0f7d3 [C++17] Add Traits class for Tables and Factory function within it. (#5678)
* Include flattests_cpp17 in unit tests when C++17 build is enabled.

* [C++17] Generate generic table factory function.

1. For each table, generate a convenient free-standing factory
   function that allows creating the table in a generic way by
   specifying only the type.  This is the first change in a series
   of changes to make Flatbuffers generated C++ code more friendly
   to code bases that make use of C++ template metaprogramming
   techniques to manage the serialization process.  Example:

     Before :(

       // The name of the Flatbuffers type (and namespace) must
       // be hard-coded when writing the factory function.
       auto monster = MyGame::Example::CreateMonster(fbb, ...);

     After  :)

       using type_to_create = MyGame::Example::Monster;
       // No namespace needed on CreateByTagType.
       auto monster = CreateByTagType((type_to_create*)nullptr,
                                      fbb, ...);

   This feature requires building with C++14 or greater, and thus
   it is guarded behind --cpp-std >= c++17 in the flatbuffers C++
   generator.

2. Fix a CMake bug to include C++17 unit tests in test suite.

* [C++17] Replace standalone variadic factory function with type_traits.

Add a `type_traits` to each table class.  This `type_traits` can be
populated with various compile-time info about the table.  Initially,
we have the Create* function and type, but is extensible in the future.

* Remove empty line and fix stale comments.

* Rename type_traits to Traits and move fwd declaration.

* Fix parameter evaluation order issue and use lambda for scope.
2020-01-02 10:12:14 -08:00
Wouter van Oortmerssen
3cd9b6434a Removed code_generators.cpp from library targets
Change-Id: Ia6e032a77983bf1838b8675f51d1c910acc991d8
2020-01-02 08:50:39 -08:00
Max Burke
355dfd48d1 [rust] Make enum names public (#5690)
* Bugfix for Rust generation of union fields named with language keywords

Looking at ParseField, it appears that in the case of unions, an extra field with a `UnionTypeFieldSuffix` is added to the type definition, however, if the name of this field is a keyword in the target language, it isn't escaped.

For example, if generating code for rust for a union field named `type`, flatc will generate a (non-keyword escaped) field named `type_type` for this hidden union field, and one (keyword escaped) called `type_` for the actual union contents.

When the union accessors are generated, they refer to this `type_type` field, but they will escape it mistakenly, generating code like this:

```
  #[inline]
  #[allow(non_snake_case)]
  pub fn type__as_int(&self) -> Option<Int<'a>> {
    if self.type__type() == Type::Int {
      self.type_().map(|u| Int::init_from_table(u))
    } else {
      None
    }
  }
```

Which will fail to build because the field is called `self.type_type()`, not `self.type__type()`.

* [Rust] Add crate-relative use statements for FBS includes.

At present if a flatbuffer description includes a reference to a type in
another file, the generated Rust code needs to be hand-modified to add
the appropriate `use` statements.

This assumes that the dependencies are built into the same crate, which
I think is a reasonable assumption?

* Revert "[Rust] Add crate-relative use statements for FBS includes."

This reverts commit d554d79fec.

* Address comments raised in PR

* Update documentation comments per feedback

* Fix typo

* [rust] Make enum variant names public.

* Update generated test files

* Add test for public enum names
2019-12-31 10:28:58 -08:00
lu-wang-g
a2c12900aa Optimize Pack method using numpy (#5662)
Add the support to pack using numpy for scalar vectors when numpy is available.
2019-12-26 21:42:11 -05:00
Austin Schuh
901b89e733 [C++] Add Builder and Table typedefs (#5685)
* Add Builder and Table typedefs

This gives us a way to use templates to go from a builder to a table
and back again without having to pass both types in.

* Fix tests/cpp17/generated_cpp17/monster_test_generated.h
2019-12-26 14:56:46 -08:00
Austin Schuh
b4154405d1 Fix --incompatible_load_cc_rules_from_bzl (#5683)
Incompatible flag --incompatible_load_cc_rules_from_bzl will break
FlatBuffers once Bazel 1.2.1 is released.

Fixes #5671
2019-12-26 10:03:22 -08:00
Robert Winslow
7de668053f CI: New Docker tests for Python with numpy (#5677)
* New Docker tests for Python with numpy

* print numpy status in test suite
2019-12-24 02:14:55 -05:00
Wouter van Oortmerssen
c957550511 Removed test proto output.
Change-Id: Iaf64bec068d03dd1b75670e9a28dde7392ebddb5
2019-12-23 12:33:16 -08:00
Vladimir Glavnyy
44bf719883 Add flatc '--cpp_std' switch (#5656)
* Add flatc '--cpp_std' switch and sandbox for C++17 code generator

- Added 'flac --cpp_std legacy' for compatibility with old compilers (VS2010);
- Added experimental switch 'flac --cpp_std c++17' for future development;
- Added C++17 sandbox test_cpp17.cpp;
- C++ code generator generates enums with explicit underlying type to avoid problems with the forward and backward schema compatibility;
- Adjusted CMakeLists.txt, CI and generate code scripts to support of introduced '--cpp_std';

* Fix --cpp_std values: c++0x, c++11, c++17

* Add 'cpp::CppStandard' enum

* Add testing engine into test_cpp17

* Rebase to upstream/master

* Set default '--cpp-std C++0x'

* Fix code generation (--cpp_std C++11) in CMakeLists.txt

- Fix dependency declaration of grpctest target

* Revert --cpp-std for the tests from explicit C++11 to flatc default value (C++0x)
2019-12-23 12:13:48 -08:00
Wouter van Oortmerssen
3e8f15df90 Fix for FlexBuffers FBT_VECTOR_STRING size bit-width.
For details, test.cpp/FlexBuffersDeprecatedTest(), and also
https://github.com/google/flatbuffers/issues/5627

Change-Id: I6e86e1138a5777e31055cfa2f79276d44732efbc
2019-12-23 11:49:59 -08:00
Michael Beardsworth
13c05f4da3 Improve import handling for proto conversion (#5673)
* Keep include prefix when converting from proto.

This change preserves the include prefix when generating flatbuffers
from proto (with FBS_GEN_INCLUDES) defined.

* Improve handling of imports in proto conversion.

Previously, there was no runtime flag to make proto->fbs conversion keep
the import structure of a collection of files. This change makes proto
conversion respect the --no-gen-includes flag and skip the output of
"generated" symbols.
2019-12-23 08:50:29 -08:00
Matt Brubeck
aa75e5734b Make Rust constants public (#5659)
* Make Rust constants public

Otherwise they cannot be accessed by code that consumes the generated
bindings.

* Re-generate test code

* Add a test for enum constants
2019-12-17 12:18:59 -08:00
cryptocode
2790fee257 Add namespace qualification to union types (#5666) 2019-12-17 09:11:26 -08:00
FujiZ
030fee36ab wrap multiple statements in do {} while(!IsConstTrue(true)) (#5655) 2019-12-13 10:14:14 -08:00
bttk
99d11e279f Split Bazel targets into multiple packages (#5640)
* Split Bazel targets into multiple packages

* Merge /include/BUILD back into /BUILD
2019-12-02 14:14:48 -08:00
Google AutoFuzz Team
4fd8eb214b Remove a static_assert (#5643)
Having a static_assert on MSAN and ASAN prevents
the fuzzers from being used with different engines,
like TSAN, UBSAN, … but also with fuzzers that aren't
using MSAN/ASAN like afl for example.
2019-12-02 14:13:28 -08:00
lu-wang-g
65f8703572 Flatbuffers Python Object API (#5616)
* Flatbuffers Python Object API

Implement the logic to generate the Python object API that can
unpack the data from a buf class into an object class, and pack
the data of an object class to a buf class.

* Fix the build issues

Remove unused parameters and replace auto in the for-loop statement
with std::string to make it compatible with VS2010.

* Fix the build issues.

* Add support for Array type

Added logic to handle Array type in Python Object API. Updated the
generated code accordingly.

* Fix the old style casting from int to char

* Fixed another conversion from int to char

* Fixed the import for typing

Importing typing may cause errors when a machine do not have the
moduel typing installed. This PR fixes the issue by guarding
"import typing" with the "try/except" statement.

* Fix issue of iterating the vector of import list

* Update the generated examples using generate_code.sh

* Fix the import order for typing

The import list was stored in unordered_set, so that each generated
codes may have different import order. Therefore, it failed in the
consistency test where two generated copies need to have exactly the
same apperance.

* Optimize unpack using numpy

Use numpy to unpack vector whenever it is possible to improve unpack
performance.

Also, added codegen command for Python specificly in generate_code.sh,
because --no-includes cannot be turn on for Python.

* Fix the import order

* Update generate_code.bat for windows accordingly

* Replace error message with pass

Avoid printing error message for every Python2 users about typing.
Replace it with pass.
2019-12-02 14:11:28 -08:00
Tiago Silva
3c964e10ab [GO] Fix support for enums with underscores and Unions with imported members (#5600)
* Fix Enum Stringer method when Enum has underscores

Fixes #5481

* Fix go package importing itself when Union has imported members.
2019-11-26 12:25:36 -08:00
Vladimir Glavnyy
c3c32ec942 Fix ambiguity of a type deduction in TEST_EQ macro if arguments have enum class type. (#5630) 2019-11-25 12:56:47 -08:00
Derek Bailey
136d75fa65 Changed null checks in test. Removed verifier pointer usage (#5634) 2019-11-21 10:25:31 -08:00
Vladimir Glavnyy
091fa1fd1b Add testing of C++ with sanitizers (CI-Docker) (#5631)
* Add C++ build testing with clang and gcc

This adds Dockerfiles which test building flatc and the C++ library against clang
and gcc. See discussion at #5119.  It is derived from the Travis CI tooling.

The GRPC tests are failing due to #5099 so those are commented out.

These are run from the .travis.yml file rather than the tests/docker/languages
folder because the builds may each take longer than 30 minutes and were hitting
Travis timeouts.

Parallel builds and build caching attempt to keep the build times low.

* Add GCC 8.3 and Clang 7.0 with sanitizers into CI (based on #5130)

- Add a docker based on Debian Buster.
- Add C++ building scripts for the docker.
- Leak-sanitizer requires SYS_PTRACE.
2019-11-18 12:16:41 -08:00
Malar Kannan
6beb9f49cb Support for python grpc - continuing the work from the pull request #4270 #4705 (#5613)
* Support for python grpc

* add few fixes

* Fixes build

* Fix python generator

* Add tests

* Fix grpc python test

* Fix tests and add incomplete python generator

* Fix python generator

* Add python generator methods

* Fix Appveyor build

* grpc python support v0.1

* Update tests

* update grpctest

* Remove duplicated code and fix a brace

* tests for flatbuffers grpc python

* Updated tests + removed SerializeToString, From String

* remove pickle import

* include missing files in ci - BUILD and generated test result
2019-11-14 16:58:35 -08:00
Derek Bailey
0f2ff7eaa9 Lua cleanup (#5624) 2019-11-14 15:46:46 -08:00
Derek Bailey
dda095023d [C++] Adds basic schema evolution tests (#5611)
* Added basic schema evolution tests

* Add BUILD targets for evolution tests. Added to test/generate_code scripts

* Use vector.front() instead of vector.data()

* Added --scoped-enums option for evolution test
2019-11-14 15:44:18 -08:00
Derek Bailey
42c08cbca6 Ran src/clang-format-all.sh (#5617) 2019-11-11 09:29:37 -08:00
Derek Bailey
105dd528e9 Change monster_extra generation to use flatbuffers::unique_ptr (#5612) 2019-11-07 14:10:41 -08:00
Derek Bailey
f0f0efe7b8 [C++] Refactor to conform to Google C++ style guide (#5608)
* Automatic refractor of C++ headers to Google C++ style guide

* Automatic refractor of C++ source to Google C++ style guide

* Automatic refractor of C++ tests to Google C++ style guide

* Fixed clang-format issues by running clang-format twice to correct itself. Kotlin was missing clang-format on after turning it off, so it was changed,
2019-11-07 12:22:54 -08:00
Derek Bailey
46ae3f80a6 [C++, Java, C#, TypeScript, JavaScript] Skip generation of mutable union types (#5599)
* Skip generation of mutable union types

* Removed C# and Java unit tests that mutated a Union type
2019-11-04 14:54:44 -08:00
Derek Bailey
661bedd837 Add Lua FlatbufferBuilder Clean() method to enable reuseable builders (#5606) 2019-11-04 14:27:59 -08:00
Paulo Pinheiro
8526e12d73 [Kotlin] Fix union vector accessor after change in Java API (#5605)
Kolin uses java library as dependency, which changed the way it access union vector recently
(e365c502ff).

This changes updates kotlin code generation to match Java's changes.
2019-11-04 14:25:07 -08:00
Lawrence Chan
964365ba69 [Go] Add UnPackTo functions (#5598) 2019-11-04 12:12:25 -08:00
iceboy
32254b7acd [Go] Object API support (#5339)
* start

* works for current usages!

* unpack: vector of struct

* optimize byte slice

* support nested struct

* support null table

* support struct

* support union

* update generated code

* grumble

* fix compiler warning

* update generated code

* wrap type in namespace

* bug

* wrap in namespace

* enum byte arrays

* generate struct for unions

* basic testing

* remove branching

* fix assert

* pack vector of fixed structs correctly

* omit null vectors

* Refactor Union Pack and UnPack methods

Remove append usage to increase code efficiency when dealing with large vectors

* generate goldens
2019-10-31 11:13:45 -07:00
Mathias Svensson
521e255ad9 Rust: Add idiomatic iterator for Vector type (#5579)
* Rust: Add idiomatic iterator for Vector type

* Add comments explaining some implementation details
2019-10-28 14:19:47 -04:00
Derek Bailey
480815447a C++ verifier for evolved union fields should return true (#5586) 2019-10-28 09:43:33 -07:00
Derek Bailey
8d5e424c65 Add ByteBuffer copy for vector of bytes in Java (#5587) 2019-10-28 09:30:31 -07:00
Mathias Svensson
b4774d2354 Rust: Fix Copy and Clone impls for a few generic types (#5577)
* Rust: Fix Copy and Clone impls for a few generic types

* Add tests for Copy+Clone

* Wrap Copy+Clone checks in a #[test] function
2019-10-28 00:20:29 -04:00
Bharat Tak
e9d29c21a7 Python: Add forceDefaults opt to python Builder (#5564)
* Add forceDefaults opt to python Builder

* Add test functions for force_default option for python builder

* Simplify

* Add force default test for UOffsetTFlags
2019-10-17 15:25:05 -07:00
Wouter van Oortmerssen
df3e8bf4a7 Fixed warnings generated by recent JSON sorting feature.
Change-Id: I6fd6283b616c7a39bb878b1610e4ddf6e208fa0a
2019-10-17 15:16:40 -07:00
tira-misu
5797540ed0 #5544 Fix of Array of table is not sorted if Create<type>Direct() is used (#5546)
* Fix C/C++ Create<Type>Direct with sorted vectors

If a struct has a key the vector has to be sorted. To sort the vector
you can't use "const".

* Changes due to code review

* Improve code readability
2019-10-17 14:22:21 -07:00
Wouter van Oortmerssen
32f47ad247 Fixed JSON parser not sorting vectors of tables/structs with key.
Change-Id: Iacc0c8513af80a736700e6cbaf513ebdf8e3ac89
2019-10-10 15:25:39 -07:00
Paulo Pinheiro
d4cae0a623 Fix issue #5542 (#5543)
Empty objects that inherit from Sized would try to access internal
ByteBuffer when Sized::size was called. So we add a single byte in
the empty buffer, so when size() is called it would return 0
2019-09-30 14:05:19 -07:00
Robert Winslow
a92039687a Update Rust versions under test from 1.30.1 to 1.37.0 (#5538)
Fixes build errors from using newer crate versions with an old Rust version.
2019-09-26 14:31:11 -07:00
Paulo Pinheiro
3f8ce99c50 [FlexBuffers][Java] Add override Key::toString (#5533) 2019-09-25 14:14:39 -07:00
Paulo Pinheiro
0798b7b698 [FlexBuffers][Java] Fix wrong access to a string using Reference::asString(). (#5532)
The real position of a string is  calculated by using the indirect() method,
which should be based on parentWidth and not byteWidth, as it was implemented.

We are also fixing the flag BUILDER_FLAG_SHARE_STRINGS on FlexBuffersBuilder
that was set as '1', same value as BUILDER_FLAG_SHARE_KEYS.
2019-09-25 11:59:10 -07:00
Vladimir Glavnyy
cbdf82e2fb Fix Mutate() methods of Array<scalar/struct> (override 5508) (#5526)
* Draft with Array specialization (#5508)

* Array specialization + SFINAE to fold copy-paste (#5508)

* Add implicit specialization of Array<scalar> and Array<struct> (#5508)

- Tag dispatching is used for implicit specialization
- Array<scalar> and Array<struct> have different iterators and accessors
- Array<scalar> and Array<struct> have different Mutate() methods

* Add implicit specialization of Array<scalar> and Array<struct> (#5508)

- Tag dispatching is used for implicit specialization
- Array<scalar> and Array<struct> have different iterators and accessors
- Array<scalar> and Array<struct> have different Mutate() methods
2019-09-23 09:31:51 -07:00