Commit Graph

277 Commits

Author SHA1 Message Date
Vladimir Glavnyy
82836a62be [idl_parser] Improve stack overflow protection (#6364)
* [idl_parser] Improve stack overflow protection

Add stack overflow protection for Flexbuffer and nested Flatbuffer parsers.
Replaces the `Recurse()` method by the new ParseDepthGuard RAII class.

* Remove move operator from Parser.

It was wrong decision to add move ctor and assignment into Parser class.
These operators will make it extremely difficult to add constant or reference fields in the future.

* Remove ';' from definition of FLATBUFFERS_DELETE_FUNC

* Format code

* Make this PR compatible with MSVC2010 (it doesn't support inherited ctor)
2021-01-04 12:39:12 -08:00
Vladimir Glavnyy
e7430bbebd [idl_parser] Check the range of explicitly set field's id value (#6363)
* [idl_parser] Check the range of explicitly set field's id value

The explicitly set `id` attribute should be a non-negative value of the `voffset_t` type.

* Format FieldIdentifierTest()
2021-01-04 12:33:31 -08:00
mqy
05192553f4 Fix typos in usage/comments; Make rust generator respect to --filenam… (#6342)
* Fix typos in usage/comments; Make rust generator respect to --filename-suffix

* run clang format
2020-12-17 13:15:27 -08:00
tira-misu
9fca5e4f42 Add flatc option to inhibit all warnings #6005 (#6301)
* Fix C/C++ Create<Type>Direct with sorted vectors

If a struct has a key the vector has to be sorted. To sort the vector
you can't use "const".

* Changes due to code review

* Improve code readability

* Add generate of JSON schema to string to lib

* option indent_step is supported

* Remove unused variables

* Fix break in test

* Fix style to be consistent with rest of the code

* Add option --no-warnings to inhibit all warnings

* Fix order of member initialization

* Add documentation for --no-warnings
2020-12-10 14:03:53 -08:00
Vladimir Glavnyy
8d9eae9ac9 [idl_parser] Unify parsing of NaN values read from .fbs and .json files (#6296)
This commit unifies parsing of NaN values read
from .fbs and .json files by converting them to unsigned NaN.
2020-12-07 11:12:58 -08:00
Vladimir Glavnyy
c27bc2d76f [C++] Add ParseJson(), Parser(Parser&&), update fuzzers (#6284)
- add a new method ParseJson to minimize failures during fuzzing
- add default (conditional) move-constructor for Parser
- add a new monster_fuzzer
- switch fuzzers to C++17 and `test/cpp17` generated code
2020-11-23 10:17:44 -08:00
Wouter van Oortmerssen
bc518a5127 Fixed FlexBufferBuilder asserting on duplicate keys 2020-11-23 09:02:40 -08:00
Austin Schuh
e58c182443 Add --require-explicit-ids to require explicit ids (#6277)
* Add --require-explicit-ids to require explicit ids

We just got bit by a well intentioned developer forgetting that field
order by default is the field index.  3 people missed it in review.

I'm looking at ways to make it harder to mess up.  We are requesting
that developers explicitly id all fields in tables.  Automatic (opt in
for others) enforcement of this will help the effort succeed.  This
patch adds a command line flag which lets the user require ids on all
fields in tables.

* Added docs to Compiler.md as well
2020-11-19 17:16:45 -08:00
Stefan F
d0d51e2a5c flatc should support --binary --schema with optional scalar fields. (#6252)
* Added missing EndTable() call to VerifyObject()

VerifyObject called VerifyTableStart() but not EndTable(). This made Verifier::VerifyComplexity() increase depth_ with each table, not with the depth of tables.

https://groups.google.com/forum/#!topic/flatbuffers/OpxtW5UFAdg

* Added Check to VerifyAlignment

https://stackoverflow.com/questions/59376308/flatbuffers-verifier-returns-false-without-any-assertion-flatbuffers-debug-veri

* Add GetStringView (Convenience function to get string_view from a String returning an empty string_view on null pointer) like GetString, GetCstring

* flatc should warn, when an attribute is attached more than once.

flatc.exe -b duplicate.fbs
warning: duplicate.fbs(5, 36): warning: attribute already found: priority

duplicate.fbs:

namespace MyGame;

attribute "priority";

table Monster (priority:1, priority:2) {
}

root_type Monster;

* flatc should support --binary --schema with optional scalar fields.

This fixes 'error: Optional scalars are not yet supported in at least one the of the specified programming languages.' when calling flatc.exe --binary --schema with a schema containing optional scalars.
2020-11-12 10:06:09 -08:00
Derek Bailey
ea92a668d8 [C#] Optional Scalars (#6217)
* [C#] Optional Scalars

* Moved scalar optional check to avoid null enum conversion
2020-10-30 13:18:59 -07:00
Kamil Rojewski
a0182cdb11 optional scalars for ts/js (#6215)
* optional scalars for ts/js

* removed range based for

* removed range based for
2020-10-28 09:45:23 -07:00
Paulo Pinheiro
f9a18ea635 [Java] Implement optional scalars (#6212)
Java implementation of optional scalars, following issue #6014.
2020-10-26 17:46:50 -07:00
Casper
fe8e3c7e5d Mass Refactoring to use IsString and other BASE_TYPE helpers (#6193)
* Updated SupportsAdvancedUnionFeatures to look out for string

* Mass refactoring to use BASE_TYPE helper functions.

Co-authored-by: Casper Neo <cneo@google.com>
2020-10-20 08:00:31 -07:00
Casper
8f6fa4b715 Updated SupportsAdvancedUnionFeatures to look out for string (#6190)
Co-authored-by: Casper Neo <cneo@google.com>
2020-10-19 13:16:00 -07:00
mustiikhalil
7b9e61fccf [TS] GRPC Implementation (#6141)
* GRPC implementation for Typescript

* Fixes a couple of issues

* Finished implementing the typescript support for grpc

* Updated generated code

* Fixes CI
2020-10-07 09:56:30 -07:00
Stefan F
ab01ae1620 flatc should output a warning, when an attribute is attached more than once (#6146)
* Added missing EndTable() call to VerifyObject()

VerifyObject called VerifyTableStart() but not EndTable(). This made Verifier::VerifyComplexity() increase depth_ with each table, not with the depth of tables.

https://groups.google.com/forum/#!topic/flatbuffers/OpxtW5UFAdg

* Added Check to VerifyAlignment

https://stackoverflow.com/questions/59376308/flatbuffers-verifier-returns-false-without-any-assertion-flatbuffers-debug-veri

* Add GetStringView (Convenience function to get string_view from a String returning an empty string_view on null pointer) like GetString, GetCstring

* flatc should warn, when an attribute is attached more than once.

flatc.exe -b duplicate.fbs
warning: duplicate.fbs(5, 36): warning: attribute already found: priority

duplicate.fbs:

namespace MyGame;

attribute "priority";

table Monster (priority:1, priority:2) {
}

root_type Monster;
2020-09-28 09:28:47 -07:00
Vladimir Glavnyy
fb4e1c34f9 Add CharToLower and CharToUpper into util.s (#6126)
This commit adds replacement of `::tolower` and `::toupper`.
Added CharToLower and CharToUpper routines reduce the number of cast operators
that required for correct usage of standard C/C++ `::tolower/toupper` routines.
2020-09-21 09:31:27 -07:00
Vladimir Glavnyy
8c67b5b129 Add support of Optional<T> scalars to C++ code generator (#6092) 2020-09-18 12:51:17 -07:00
Paulo Pinheiro
6228b66d3d [Kotlin] Support for optional scalars. (#6115)
More information on #6014
2020-09-17 23:31:51 -07:00
Casper
c75ae24293 Optional-ness in reflection (#6097)
* Optional scalars in reflection

* fixed name collision

* Remove code duplicated by merge

Co-authored-by: Casper Neo <cneo@google.com>
2020-09-11 14:20:32 -07:00
Casper
338944d3d9 Rename Nullable scalars to Optional scalars (#6112)
Co-authored-by: Casper Neo <cneo@google.com>
2020-09-10 13:04:36 -07:00
Wouter van Oortmerssen
77f966f89f [Lobster] optional scalars support 2020-08-18 14:00:02 -07:00
mustiikhalil
d76e93f277 adds code gen for optional scalars in swift (#6038)
Adds swift test code

Replaces if statments

Adds swift to supported languages for optionals

Moved std::string to auto

Adds nullable scalars support in object api
2020-07-25 17:59:34 +03:00
Casper
043b52bd4a Optional Scalars support for Rust (#6034)
* First draft of rust optionals

* Code cleanup around ftBool and ftVectorOfBool

* Tests for Rust optional scalars

* test bools too

Co-authored-by: Casper Neo <cneo@google.com>
2020-07-23 16:30:27 -07:00
Casper
9ecd2e16c2 Flatc parser support for nullable scalars (#6026)
* Parser support for nullable scalars

* Use older C++ features

* use default element

* Add a test for json, flexbuffers, and null

* test comments and names

Co-authored-by: Casper Neo <cneo@google.com>
2020-07-16 13:43:47 -07:00
Derek Bailey
60b6066feb Add warning to schema parser if field name is not snake_case. (#6005)
* Added code gen for evolution tests back in.

* General generate code and clang format

* Added code gen for evolution tests back in.

* General generate code and clang format

* reran generate_code.sh

* Added code gen for evolution tests back in.

* General generate code and clang format

* Added code gen for evolution tests back in.

* General generate code and clang format

* Enforce snake_case for schema field names

* Switched to basic for loop, Fixed warning message
2020-06-26 11:16:02 -07:00
Derek Bailey
165a6e3d1e Re-added Evolution Schema Code Generation Command (#5999)
* Fixed refractoring issue in reflection/generate_code.sh. Also, mv deletes the original file, so I don't need to clean it up manually in that case.

* Fixed Dart Tests by removing code-gen for included files.

* Added code gen for evolution tests back in.

* General generate code and clang format

* Added evolution schema generation to .bat file

* Added code gen for evolution tests back in.

* General generate code and clang format

* Added evolution schema generation to .bat file

* reran generate_code.sh

* Removed wildcard from generate_code.bat that doesn't work
2020-06-25 15:35:23 -07:00
Alex Ames
b124b76258 Removed requirement that enums be declared in ascending order. (#5887) 2020-06-22 17:02:15 -07:00
Javier Serrano
0ec7600c67 Do not remove the last digit from float values (#5974)
Trailing zeros are already removed inside the function FloatToString,
that is called immediately before the lines deleted by this commit.
2020-06-22 16:57:45 -07:00
Liu Liu
cc44a44427 [idl_parser] Mark typefield as deprecated (#5958)
* Mark typefield as deprecated

* Add UnionDeprecation Test.

* Update to use evolution schema instead.

* Use --scoped-enums to compile.
2020-06-15 23:18:12 -07:00
Mika Raento
a28357d7ac Propagate boolean default values from proto to fbs (#5964) 2020-06-11 10:15:19 -07:00
Vladimir Glavnyy
4e45f7c9e8 Fix error in SimpleQSort (#5955) 2020-06-09 23:35:07 -07:00
Wouter van Oortmerssen
94a78e3853 Fixed: Access violation and ASAN/UNSAN failures with sorted tables
https://github.com/google/flatbuffers/issues/5945
2020-06-08 09:50:40 -07:00
Derek Bailey
7179a5a8ba General Codebase clean up (#5939)
* Fixed refractoring issue in reflection/generate_code.sh. Also, mv deletes the original file, so I don't need to clean it up manually in that case.

* Ensuring test/generate_code.sh was ran

* Fixed Dart Tests by removing code-gen for included files.

* General cleanup of codebase.
2020-06-01 08:40:55 -07:00
Derek Bailey
0fa087657e [Dart] Getting tests/DartTest.sh to work on master. (#5915)
* Fixed refractoring issue in reflection/generate_code.sh. Also, mv deletes the original file, so I don't need to clean it up manually in that case.

* Fixed Dart Tests by removing code-gen for included files.
2020-05-15 18:13:29 -07:00
Vladimir Glavnyy
424a473e1f Schema parser: prohibit declaration of an array of pointers inside structs (#5907)
* Fix issue #5906, Prohibit declaration of an array of pointers inside structs

- idl_parser.cpp: Prohibit declaration of an array of pointers inside structs
- idl_gen_cpp.cpp: Extract GenStructConstructor() method from GenStruct() to simplify future modification
- idl_gen_cpp.cpp: Add assert for checking of Array fields in structs on code-generation stage

* Fix the error 'unused local variable' in release build

* Fix: format the PR code according to coding rules

* Add test-case and fix review notes
2020-05-14 11:39:58 -07:00
Mika Raento
6e9f5d9810 Respect shared attribute in Parser (#5885)
Respects the 'shared' field attribute when reading data with Parser.
2020-05-04 10:33:11 -07:00
Vladimir Glavnyy
9b034eee12 Fix interpretation of 'nan(number)' by the idl_parser (#5810)
* Parser reject "nan(n)" string as it does with nan(n)

* Adjust scalar fuzzer to ignore '$schema' substrings

- Scalar fuzzer ignores '$schema' substrings at the input
- Added 'scalar_debug' target to simplify research of fuzzed cases

* Improve formatting of './tests/fuzzer/CMakeLists.txt'
2020-03-16 11:59:34 -07:00
Vladimir Glavnyy
d10c163142 Replace 'assert' by 'FLATBUFFERS_ASSERT' inside idl_parser.cpp (#5799) 2020-03-12 11:18:10 -07:00
Myrice
9954e09ab7 [C++] Generate code for vector force_align attribute. (#5796)
* [Flatbuffer] Generate code for force_align with CreateXDirect and Pack functions.

* Fixed Visual Studio 10.0 compile error for std::to_string.

* Fixed Visual Studio 10.0 compile error for std::to_string.
2020-03-12 08:32:29 -07:00
Derek Bailey
6ff1898413 Added --filename-suffix and --filename-ext to flatc (#5778)
* Fixed refractoring issue in reflection/generate_code.sh. Also, mv deletes the original file, so I don't need to clean it up manually in that case.

* Added --filename-suffix and --filename-ext to flatc

* Fixed typo and added example generation of suffix and extension for C++

* Removed extra ;

* Removed clang-format block from a region that didn't need it. Fixed an auto format of another clang-format block

* Added docs, fixed pointer alignment, removed suffix test file
2020-03-02 10:15:23 -08:00
mustiikhalil
04d80f255d [Swift] Swift implementation 🎉🎉 (#5603)
* Implemented the swift version of Flatbuffers

Implemented serailzing, reading, and mutating data from object monster

Fixes mis-aligned pointer issue

Fixes issue when shared strings are removed from table

Adds swift enum, structs code gen

Fixed namespace issues + started implementing the table gen

Added Mutate function to the code generator

Generated linux test cases

Fixed an issue with bools, and structs readers in table writer

Swift docker image added

Updated the test cases, and removed a method parameters in swift

Fixed createVector api when called with scalars

Fixed issues with scalar arrays, and fixed the code gen namespaces, added sample_binary.swift

Cleaned up project

Added enum vectors, and their readers

Refactored code

Added swift into the support document

Added documentation in docs, and fixed a small issue with Data() not being returned correctly

Fixes Lowercase issue, and prevents generating lookups for deprecated keys

* Made all the required funcs to have const + removed unneeded code + fix lowercase func

* Removed transform from lowercased and moved it to function

* Fixes an issue with iOS allocation from read

* Refactored cpp code to be more readable

* casts position into int for position

* Fix enums issue, moves scalar writer code to use memcpy

* Removed c_str from struct function

* Fixed script to generate new objects when ran on travis ci: fix

* Handles deallocating space allocated for structs

* Updated the test cases to adhere to the fileprivate lookup, no mutation for unions, and updated the names of the vector functions
2020-01-09 12:12:10 -08:00
Wouter van Oortmerssen
55686100aa Changed direct calls to strtod to use StringToNumber
StringToNumber will correctly use locale-insensitive functions
when available.

Change-Id: I6bde11039a541634186f8f791012af2eb0d86b8d
2020-01-09 11:48:09 -08:00
Vladimir Glavnyy
44bf719883 Add flatc '--cpp_std' switch (#5656)
* Add flatc '--cpp_std' switch and sandbox for C++17 code generator

- Added 'flac --cpp_std legacy' for compatibility with old compilers (VS2010);
- Added experimental switch 'flac --cpp_std c++17' for future development;
- Added C++17 sandbox test_cpp17.cpp;
- C++ code generator generates enums with explicit underlying type to avoid problems with the forward and backward schema compatibility;
- Adjusted CMakeLists.txt, CI and generate code scripts to support of introduced '--cpp_std';

* Fix --cpp_std values: c++0x, c++11, c++17

* Add 'cpp::CppStandard' enum

* Add testing engine into test_cpp17

* Rebase to upstream/master

* Set default '--cpp-std C++0x'

* Fix code generation (--cpp_std C++11) in CMakeLists.txt

- Fix dependency declaration of grpctest target

* Revert --cpp-std for the tests from explicit C++11 to flatc default value (C++0x)
2019-12-23 12:13:48 -08:00
Vladimir Glavnyy
075e8d676b Simplify declarations of x-macro FLATBUFFERS_TD (#5638) 2019-11-25 12:54:59 -08:00
Derek Bailey
f0f0efe7b8 [C++] Refactor to conform to Google C++ style guide (#5608)
* Automatic refractor of C++ headers to Google C++ style guide

* Automatic refractor of C++ source to Google C++ style guide

* Automatic refractor of C++ tests to Google C++ style guide

* Fixed clang-format issues by running clang-format twice to correct itself. Kotlin was missing clang-format on after turning it off, so it was changed,
2019-11-07 12:22:54 -08:00
Wouter van Oortmerssen
df3e8bf4a7 Fixed warnings generated by recent JSON sorting feature.
Change-Id: I6fd6283b616c7a39bb878b1610e4ddf6e208fa0a
2019-10-17 15:16:40 -07:00
Wouter van Oortmerssen
32f47ad247 Fixed JSON parser not sorting vectors of tables/structs with key.
Change-Id: Iacc0c8513af80a736700e6cbaf513ebdf8e3ac89
2019-10-10 15:25:39 -07:00
Jack Zhou
69d3fec488 Fix namespaced struct/field name collision detection (#5540) (#5545)
Changes the use of `LookupStruct` to `LookupCreateStruct` in
`ParseField` to also detect when collisions happen in namespaces.
2019-09-30 11:50:03 -07:00
Paulo Pinheiro
cfb4ecf6f0 [flac] Add FlexBuffers option for generating data (#5519)
Alongside --binary and --json, an additional
switch (--flexbuffers) can be passed to convert
data using FlexBuffers
2019-09-26 15:06:11 -07:00