Commit Graph

113 Commits

Author SHA1 Message Date
Robert
3c54fd964b Port FlatBuffers to Rust (#4898)
This is a port of FlatBuffers to Rust. It provides code generation and a
runtime library derived from the C++ implementation. It utilizes the
Rust type system to provide safe and fast traversal of FlatBuffers data.

There are 188 tests, including many fuzz tests of roundtrips for various
serialization scenarios. Initial benchmarks indicate that the canonical
example payload can be written in ~700ns, and traversed in ~100ns.

Rustaceans may be interested in the Follow, Push, and SafeSliceAccess
traits. These traits lift traversals, reads, writes, and slice accesses
into the type system, providing abstraction with no runtime penalty.
2018-09-02 18:26:55 -07:00
Christopher Cifra
d0321df8cf C# support for directly reading and writting to memory other than byte[]. For example, ByteBuffer can be initialized with a custom allocator which uses shared memory / memory mapped files. (#4886)
Public access to the backing buffer uses Span<T> instead of ArraySegment<T>.

Writing to the buffer now supports Span<T> in addition to T[].

To maintain backwards compatibility ENABLE_SPAN_T must be defined.
2018-08-23 10:05:31 -07:00
Derek Bailey
d8f49e18d7 Mono Fix for Unsafe Mode (#4887)
* Added preprocessor define for C++ if Template Aliases are supported by the compiler

* Revert "Revert "Performance Increase of Vector of Structures using .NET BlockCopy (#4830)""

This reverts commit 1f5eae5d6a.

* Put<T> method was inside #if UNSAFE_BYTEBUFFER which caused compilation failure when building in unsafe mode

* Revert "Added preprocessor define for C++ if Template Aliases are supported by the compiler"

This reverts commit a75af73521.
2018-08-20 16:31:44 -07:00
Wouter van Oortmerssen
1f5eae5d6a Revert "Performance Increase of Vector of Structures using .NET BlockCopy (#4830)"
This reverts commit 7b50004ec9.

Change-Id: I09d6869c16aa3c7fadc537fc9c76eaa3cf7ee7ea
2018-08-20 12:08:21 -07:00
Andy Martin
c2c3a84aaf Add C#/Java generator behaviour for 'private' attribute (#4882)
* Added 'private' attribute, supported when generating C# and Java

* Added use of 'private' attribute in monster_test
2018-08-16 15:25:33 -07:00
Derek Bailey
7b50004ec9 Performance Increase of Vector of Structures using .NET BlockCopy (#4830)
* Added Get<vector_name>Array() method for accessing vectors of structures in C# using Buffer.Blockcopy().

* Added Get<vector_name>Array() method for accessing vectors of structures in C# using Buffer.Blockcopy().

Added Create<Name>VectorBlock() method to add a typed array using Buffer.BlockCopy() to speed up creation of vector of arrays

New Lua files for namespace test

* fixed c++ style issue
2018-07-26 13:44:40 -07:00
Kamil Rojewski
f11ffedb2b Vector of unions support for java and c# (#4735)
* Eclipse ignore

* TypeScript support

* Prefixing enums

* Test results

* Merged JS and TS generators

* Fixed AppVeyor build problems

* Fixed more AppVeyor build problems

* Fixed more AppVeyor build problems

* Changed TS flag to options struct

* Storing options by value

* Removed unneeded const

* Re-export support for unions

* Uint support

* Casting bools to numbers for mutation

* TS shell tests

* Reverted generates js test file to original version

* Backing up js tests and properly generating test data

* Not importing flatbuffers for TS test generation

* Not overwriting generated js for tests

* AppVeyor test fixes

* Generating the most strict TS code possible

* Not returning null when creating vectors

* Not returning null from struct contructors

* Vector of unions for ts/js

* Sanity check for languages

* Indentation fix + output test files

* Vectors of unions for php

* Fixes to union vector handling + tests

* Fix for strictPropertyInitialization

* Fix for new aligned operator new for gcc >= 7.1

* Not generating imports/ns prefixes with --gen-all

* TypeScript docs

* Missing imports of enums

* Missing TS links

* Enabled vector of unions for java, since it seems to work

* Added jitpack config

* Added obj to vector of unions getter

* Removed unneeded accessor

* Bumped jdk version in pom.xml

* Vector of unions support for c#
2018-05-14 11:12:24 -07:00
Mike Mansell
3a2c535592 Remove @Nullable (Java) for required fields (fixes #4724) (#4726)
@Nullable is only a compiler informational attribute.
Removing for required fields improves compiler warnings
but doesn't impact any running code.
2018-05-10 09:27:23 -07:00
Vladimir Glavnyy
a66f9e769b The asserts replaced by FLATBUFFERS_ASSERT. (#4701)
* The asserts replaced by FLATBUFFERS_ASSERT. Several asserts have converted to static_asserts.

* Regenerate header monster generate_code.sh
2018-04-16 08:57:59 -07:00
Flaviu
79f62ee353 Const correctness in generated code and in code generators. Added missing \reflection\generate_code.bat file. (#4679) 2018-03-23 08:58:07 -07:00
Robert Schmidtke
08cf50c54a Java/C#/Python prefixed size support (#4445)
* initial changes to support size prefixed buffers in Java

* add slice equivalent to CSharp ByteBuffer

* resolve TODO for slicing in CSharp code generation

* add newly generated Java and CSharp test sources

* fix typo in comment

* add FinishSizePrefixed methods to CSharp FlatBufferBuilder as well

* add option to allow writing the prefix as well

* generate size-prefixed monster binary as well

* extend JavaTest to test the size prefixed binary as well

* use constants for size prefix length

* fuse common code for getRootAs and getSizePrefixedRootAs

* pulled file identifier out of if

* add FinishSizePrefixed, GetSizePrefixedRootAs support for Python

* Revert "extend JavaTest to test the size prefixed binary as well"

This reverts commit 68be4420dd.

* Revert "generate size-prefixed monster binary as well"

This reverts commit 2939516fdf.

* fix ByteBuffer.cs Slice() method; add proper CSharp and Java tests

* fix unused parameter

* increment version number

* pulled out generated methods into separate utility class

* pulled out generated methods into separate utility class for Python

* fix indentation

* remove unnecessary comment

* fix newline and copyright

* add ByteBufferUtil to csproj compilation

* hide ByteBuffer's internal data; track offset into parent's array

* test unsafe versions as well; compile and run in debug mode

* clarify help text for size prefix

* move ByteBuffer slicing behavior to subclass

* fix protection levels

* add size prefix support for text generation

* add ByteBufferSlice to csproj compilation

* revert size prefix handling for nested buffers

* use duplicate instead of slice for removing size prefix

* remove slice subclass and use duplicate for removing size prefix

* remove slice specific tests

* remove superfluous command line option
2018-03-12 11:30:46 -07:00
Wouter van Oortmerssen
4bc6de9a88 Fix broken Java/C# codegen 2018-03-05 16:37:04 -08:00
Nik Hemmings
fb94af8899 Protobufs: Added '--oneof-union' option. (#4647)
* Added '--oneof-union' option.

Used with the .proto -> .fbs converter, will translate protobuff oneofs to flatbuffer unions.
Updated proto test to check both methods of converting oneofs.

* Added '--oneof-union' option.

Used with the .proto -> .fbs converter, will translate protobuff oneofs to flatbuffer unions.
Updated proto test to check both methods of converting oneofs.

* FlatBuffers: Moved MakeCamel() into idl_parser.cpp

Removes library dependency on Java/C# generator code.
2018-03-05 08:45:25 -08:00
Mitchel
4ea1be53d4 Java + C#, reuse object in lookup_by_key (#4648)
* Java + C#, reuse object in lookup_by_key

* Java + C#, reuse object in lookup_by_key
2018-03-02 15:22:46 -08:00
Mitchel
55ddb84eb2 In Java, allow reusing ByteBuffer in getters (#4633)
* In Java, allow reusing ByteBuffer in getters

* In Java, allow reusing ByteBuffer in getters

* In Java, allow reusing ByteBuffer in getters
2018-02-23 14:01:05 -08:00
Yuri Finkelstein
a4dbe13486 fixed java codegen bug documented in #4563 (#4565)
* grpc bindings generator for Java and a few minor supporting changes in improvements

* restored formatting before my previous changes for ease of review

* Fixed grpc java code generation bug resulting in duplicate extractor declarations in case the same is used in more than a single RPC method

* fixed previous merge issue

* removed extra space

* restored extra space

* restored extra space

* fixed java codegen bug documented in https://github.com/google/flatbuffers/issues/4563
2017-12-21 13:32:34 -08:00
Wouter van Oortmerssen
89711c9c47 Made all C++ files clang-formatted.
Also added missing generated files.

Change-Id: Ifd22a643a08e3f2edfce92812ed57b87fc0e1875
2017-12-21 11:45:02 -08:00
Angela Sheu
61fe2a4fac Fix valid params limit exceeded for generated table constructor (#4490) 2017-11-13 13:39:49 -08:00
Yonggang Li
01bac38c84 Nullable java (#4455)
* add _Nullable Support for C++ interface

* generate @Nullable for Java generated code
2017-10-16 09:24:11 -07:00
Wouter van Oortmerssen
03e2899985 Made the parser not have any hard-coded keywords.
This prevented any keywords showing up as JSON field names, for example.

Change-Id: Ie9d0cada96778e06016ca02ca96d052410a37038
Tested: on Linux.
2017-09-22 16:30:26 -07:00
Wouter van Oortmerssen
321a1c9dc0 Cleaned up namespace handling.
A Namespace object is now guaranteed unique.
This cleaned up some old workarounds and latent bugs.

Change-Id: Ic3f12d89947871b03b2c449ba51b3186f953adde
Tested: on Linux.
Bug: 21336857
2017-08-24 13:39:45 -07:00
MikkelFJ
f2b3705c2c Add scalar type aliases for int8, uint8, ..., f32, f64 (#4400)
* Add type aliases

* Add generated code

* Rebase master
2017-08-02 08:07:43 -07:00
Christian Helmich
89a68942ac Added support for nested flatbuffers serializing to json (#4391)
* added bool for nested_flatbuffer
setting bool flag nested_flatbuffer according to attributes
setting nested type

* added JSON serialization for nested flatbuffers

* simplified lookup of nested_flatbuffer StructDef

* added nested_flatbuffer parsing workound relying on flexbuffers

* moved nested flatbuffer parsing into its own function for clarity
removed flexbuffers workound to simplify and speed up nested_flatbuffer parsing
added support for 'legacy' nested_flatbuffers, ubyte arrays

* inlined functions for nested parser init/clean since only used once

* whitespace formatting

* changed type of FieldDef.nested_flatbuffer from bool to StructDef*
removed subsequent type lookups

* removed copies of unrequired data when initializing nested parser

* applied changes requested by reviewer

* removed superfluous lookup of nested_flatbuffer field attributes

* renamed camelCased variables to snake_case
2017-07-28 10:27:51 -07:00
Stewart Miles
a892322203 C++98 (stlport) support for core FlatBuffers and FlexBuffers.
* Added internal - limited - implementation of flatbuffers::unique_ptr
  for STLs that don't ship with std::unique_ptr.  In C++11 and beyond
  this is just an alias for std::unique_ptr.
* Aliased used type traits structs is_scalar is_floating_point is_unsigned
  into flatbuffers namespace so they can be replaced in C++98 implementations.
  Right now these point at stlport's TR1 implementations.
* Wrapped vector::data() in vector_data().
* Wrapped vector::emplace_back() in vector_emplace_back().
* Wrapper string::back() in string_back().
* Added variants of FlatBufferBuilder::CreateVector() and
  FlatBufferBuilder::CreateVectorOfStructs() that allow the use of plain
  function pointers.
  Generated code has also been modified to use plain functions to build objects
  rather than std::function() so all generated code will work in C++98
  applications.
* Added flexbuffers::Builder::Vector(), flexbuffers::Builder::TypedVector()
  and flexbuffers::Builder::Map() methods that allow the use of plain function
  pointers.
* Changed Parser to internally use plain function pointers when parsing table
  and vector delimiters.
* Added specializations of NumToString() for 64-bit types that aren't supported
  by stringstream in stlport.
* Overloaded numeric_limits for 64-bit types not supported by stlport.
* Replaced build_apk.sh (which was broken by deprecation of the
  "android" tool in the Android SDK) with build.gradle and the
  appropriate gradle wrapper to build an APK.
* Switched Android build to build against all STL variants.
* Updated travis configuration to build Android test and sample.

Tested:
* Verified all tests continue to work on Linux, OSX and Android.
* Verified Travis build is green.

Change-Id: I9e634363793f85b9f141d21454b10686020a2065
2017-07-20 11:01:00 -07:00
Alex Wasserman
625c989875 Java: Added ByteBufferFactory interface and sizedInputStream method. (#4379)
The ByteBufferFactory interface gives the user an option to specify
the method in which the internal ByteBuffer is allocated. This provides
flexibility in the type of ByteBuffer that can be used.
The sizedInputStream method is an alternative to sizedByteArray that
does not make a copy of the data in memory.
2017-07-13 08:33:32 -07:00
schoetbi
0f5f7faa9f C#: Added <autogenerated> in cs generator file header comment (#4291)
* Added <autogenerated> in cs generator file header comment
#4287

* CS: Added xml-correct file header comment including "<auto-generated>...</auto-generated>"
code_generators.cpp: FlatBuffersGeneratedWarning() function replaced by compile time constant "GeneratedWarning"
 also removed extra newlines at end of GeneratedWarning to be able to generate a xml well formed file comment for cs files
#4291

* code_generators: Changed static string GeneratedWarning back to function FlatBuffersGeneratedWarning()
#4291

* Added modified Unit-Test files
#4291

* idl_gen_general: Add autogenerated only in C# code
#4291
2017-05-16 14:28:34 -07:00
Wouter van Oortmerssen
8468ea1ab4 Fixed LookupByKey for Java & C#
Change-Id: I05c02223675dee241d1ae8cb466e5186444058c8
Tested: on Linux.
2017-05-10 12:55:27 -07:00
Heiko Becker
bbb72f0b73 Tweak fallthrough comments to get recognized by gcc7 (#4298)
GCC gained a new warning, -Wimplicit-fallthrough, which warns about
implicitly falling through a case statement. The regular expressions
used at the default level (-Wimplicit-fallthrough=3) don't match with
a colon at the end. The comment also needs to be followed (after
optional whitespace and other comments) by a 'case' or 'default'
keyword, i.e. it will not be recognized with a '}' between the comment
and the keyword.
2017-05-08 13:35:55 -07:00
Spencer Bench
ccfa317486 C#: Fixed possible conflicts between usings and user-supplied namespace (#4247)
* C#: Fixed possible conflicts between usings and user-supplied namespace

C#: Added the global qualifier to using directives to prevent possible conflicts with the user-supplied namespace. Also prevents unintentional type hiding. Resolves issue #4242.

* Updated C# generated code files
2017-03-29 11:09:25 -07:00
tianyapiaozi
a5cc2092a6 Support binary search for struct in cpp (#4245)
* Support binary search for struct in cpp

CreateVectorOfSortedStruct is provided for convenience.

* fix continuous-integration error

* add generated files

* compile Ability.cs in csharp test

* compile Ability.cs in csharp

* modify according to code review
2017-03-29 09:51:12 -07:00
Raman
0b379211dc Fixed problem with uint keys (#4172)
* Fixed problem with uint keys

* removed excess check

* Resolved problem with big uint, ubyte keys in Java
2017-02-28 14:16:50 -08:00
Bei Li
68bbe983e9 Union Vector 2017-01-24 11:52:36 -08:00
Wouter van Oortmerssen
7b94eab2b1 Add CodeWriter utility class.
Helps simplify code generation code.  Instead of this:
  code += "inline const " + cpp_qualified_name + " *Get";
  code += name;
  code += "(const void *buf) {\n  return flatbuffers::GetRoot<";
  code += cpp_qualified_name + ">(buf);\n}\n\n";

You do this:
  code.SetValue("NAME", struct_def.name);
  code.SetValue("CPP_NAME", cpp_qualified_name);
  code += "inline const {{CPP_NAME}} *Get{{NAME}}(const void *buf) {";
  code += "  return flatbuffers::GetRoot<{{CPP_NAME}}>(buf);";
  code += "}";
  code += "";

Updated the CPP code generator to use the CodeWriter class.  Most of the
changes in the generated code are white-space changes, esp. around new
lines (since the code generator class automatically appends new lines
when appending a string).  Actual code changes include:

* Renamed "rehasher" to "_rehasher" for consistency with other args in
  Pack function.

* Renamed "union_obj" to "obj: in UnPack function.

* Always do "(void)_o;" to prevent unused variable warning in Create
  function (instead of only doing it if there are no fields) in order
  to avoid two-passes.

* Renamed padding variables from __paddingX to paddingX__.
  "Each name that contains a double underscore (_ _) [...] is reserved
   to the implementation for any use."  C++ standards 17.4.3.1.2.

* Add braces around switch cases.

* Calculate index as a separate statement in EnumName function, eg.
    const size_t index = ...;
    return EnumNamesX()[index];
  vs.
    return EnumNamesX()[...];

* Stored end table offset in variable in Finish() functions, eg.
    const auto end = fbb_.EndTable(start_, ...);
    auto o = flatbuffers::Offset<T>(end);
  vs.
    auto o = flatbuffers::Offset<T>(fbb_.EndTable(start, ...));

* Separate reinterpret_cast calls from function calls in Union
  functions, eg.
    auto ptr = reinterpret_cast<const T *>(obj);
    return ptr->UnPack(resolver);
  vs.
    return reinterpret_cast<const T *>(obj)->UnPack(resolver);

* Removed unecessary (void)(padding__X) no-ops from constructors, eg.
    Test(int16_t a, int8_t b) : ... {
      (void)__padding0;  // <-- Removed this line.
    }

In the idl_gen_cpp.cpp file itself, I refactored some code generation into
new functions: GenParam, GenNativeTable, GenVerifyCall, GenBuilders,
GenUnpackFieldStatement, and GenCreateParam.

Change-Id: I727b1bd8719d05b7ce33cbce00eb58fda817b25d
2017-01-13 17:44:42 -08:00
Wouter van Oortmerssen
cc84240098 Misc idl_gen_cpp cleanup
- Update to be const-correct where possible.
- Consistently pass |code| as pointer instead of non-const-ref.
- No newlines (\n) characters in the middle of code strings.
- Use if-else if-else statements instead of nested ternary operators.
- Ensure all lines end at 80 chars.
- Make utility functions static.

From cl/143505731.

Change-Id: If0fab9ee75de5af963367a948dddf53af93f73b4
2017-01-04 10:12:39 -08:00
Baptiste Lepilleur
5b5fcbfc00 Java: fix unsigned default value code generation and add 'L' suffix for long default value (#4051)
* Java: emit "signed" equivalent of unsigned default value in generated code and add "L" suffix to long default value.

* Updated generated code

* Only convert ulong to "signed" equivalent. ubyte and ushort don't need specific handling as "user facing" type is int. uint need 'L' suffix as "user facing" type is long.

* Added missing cast to primitive type of default value which is in "user facing" type in builder.add<type>() calls.

* Do not cast default value to actual type in C#.
2016-10-21 14:27:15 -07:00
Wouter van Oortmerssen
424fc0c3ac Fixed inconsistent use of namespace directory prefixing.
Change-Id: I2cab7a1f68b8e643c1057e0a366572421a94ed41
Tested: on Linux.
2016-10-12 16:05:57 -07:00
Baptiste Lepilleur
199157e8f4 flatc java generator: namespace_test2.fbs is missing namespace prefix for referToA2() return type (#4040)
* Fix flatc generating incorrect java/c# code for namespace_test2.fbs. In code for TableInC, method referToA2() returned type SecondTableInA instead of NamespaceA.SecondTableInA.

* Updated generated code.

* Fixed indendation.
2016-10-07 09:42:29 -07:00
Alberto Fernández
02dfa64a89 Allow access to underlying ByteBuffer 2016-09-12 09:35:08 +02:00
Wouter van Oortmerssen
52ca75506a Switched C# accessors from classes to structs 2016-08-31 17:34:51 -07:00
TGIshib
722b903f89 Remove languages check 2016-08-27 01:21:15 +03:00
TGIshib
2ff6152204 Update idl_gen_general.cpp 2016-08-27 00:42:06 +03:00
Wouter van Oortmerssen
481d332e72 Fixed compile error in LookupByKey fix.
Change-Id: Ice94f09197235cf71b41a22d0767bd7b119284e1
2016-08-26 14:18:04 -07:00
Wouter van Oortmerssen
e1f8037cb5 Fixed Java LookupByKey functionality for Java 1.6
Tested: on Linux.

Change-Id: Iea336f75a3b6e722743563813c3c9ed9db4d02fe
2016-08-26 13:58:16 -07:00
TGIshib
7c69c5dc3d Fix lookupByKey, improve compareStrings 2016-08-26 19:41:32 +03:00
TGIshib
9f16090f90 Improve LookupByKey , update docs 2016-08-22 18:10:52 +03:00
TGIshib
fa74ce6d16 Update idl_gen_general.cpp 2016-08-15 13:30:30 +03:00
TGIshib
8fdced4e11 Update 2016-08-14 14:58:51 +03:00
TGIshib
dc7f5bc0d8 Remake 2016-08-03 13:29:50 +03:00
Wouter van Oortmerssen
ce3e7fbd72 VS2010 fixes 2016-07-20 10:41:26 -07:00
Jonathan Tullett
d9767b8315 Fix for issue #3922
Also, clean up redundant ';' at end of java classes.
2016-07-09 09:16:26 +01:00