Commit Graph

284 Commits

Author SHA1 Message Date
Casper
e581013e3d Refactor FieldDef to model presense as an enum rather than 2 bools. (#6420)
* Define presence.

* Migrate to IsRequired and IsOptional methods

* moved stuff around

* Removed optional and required bools from FieldDef

* change assert to return error

* Fix tests.cpp

* MakeFieldPresence helper

* fmt

* old c++ compatibility stuff

Co-authored-by: Casper Neo <cneo@google.com>
2021-01-25 09:29:43 -08:00
mustiikhalil
786f69b248 Formats cpp code (#6349) 2021-01-22 10:46:53 -08:00
Björn Harrtell
760c657551 [TS/JS] New gen TS code gen (#6302)
* TS/ES6 modules spike iteration 1

* Initial modularized dasherized output

* Remove obsoleted parts and namespace wrapping

* Use _flatbuffers_ prefix

* First part of imports logic

* Second part of imports logic

* Fix TS/JS code removal mixup

* Alias imported symbols if same name from different namespaces and some fixes

* Use star import for bare imports

* Fix messed up string concat

* var to const and remove not needed semi

* Remove some cases of ns prefixing

* Add missing space

* Cleanups

* Completed initial import tracking logic

* Compilable output

* Adjust TypeScriptTest and dependents to work

* Use local flatbuffers package for tests

* Refactor away use of any

* Remove obsolete imported_fileset and reexport_map

* Still need any and fix JavaScriptTest.sh

* Fix test runs out of the box

* Temp add generated files

* TypeScriptTest replaces JavaScriptTest and cleanups

* Also remove reference to JavaScriptTest in TestAll.sh

* Remove old generated ts/js files

* Remove use of --js in generate_code scripts

* idl_gen_js_ts to idl_gen_ts and removal of js gen

* Remove obsoleted options

* Fix obsolete ts test detection

* Tweak ts compilation be as strict as possible

* Remove jsdoc type annotation generation

* Generated test ts files

* Fix search and replace messup

* Regenerated ts test output

* Use CharToLower

* Use normal for loop

* Rework namespacedir

* Revert "Rework namespacedir"

This reverts commit 6f4eb0104ceeb86011bb076ebca901138c48e068.

* Revert "Use normal for loop"

This reverts commit 676b2135bfaa1853dfbb06c92b5c16a0d81bb13a.

* Revert "Use CharToLower"

This reverts commit 2d08648d0d72d0af201fad80d54cdc76412b35e9.

* Again do rework but correct

* Avoid runtime cast

* Fix test runs

* Also add npm install to get tsc

* Bump node test versions

* for range to std for loop

* Clang format

* Missed one clang format

* Move accessor to later

* Attempt to make windows version of TypeScriptTest

* Want to see the output

* Try to get newer node at appveyor

* Style changes
2021-01-19 12:51:13 -08:00
Vladimir Glavnyy
75c859e98f [idl_parser] Improve symbols lookup thru parent namespaces (#6407)
Added the new method LookupTableByName for searching symbols in parent namespaces.
This method speedup (x50 or greater) symbol resolution (enum or struct) in parent namespaces.
The speedup was measured without `table.empty()` guard condition.

This method should suppress fuzzer timeout issue without artificial limits for nested namespaces (https://oss-fuzz.com/testcase-detail/6244168439169024).

Additionally, this commit speedup (x5) the GetFullyQualifiedName method by removing unnecessary temporary std::string object.
2021-01-19 12:49:24 -08:00
Mark Spatz
91b0958c43 Search for includes in the directory containg the current file (#6371)
* [codegen] Search for includes in the directory containg the current file

This matches the behavior of a C preprocessor, see:
https://gcc.gnu.org/onlinedocs/cpp/Search-Path.html

* Skip FileExists() when file is guaranteed to not exist

* end comment with period to match others
2021-01-19 12:26:34 -08:00
Casper
8573108bbe Unset FieldDef.optional if its key (#6402)
* Unset FieldDef.optional if its key

* removed StringKey table, just removed an extra "required, key"

* removed extra newline

Co-authored-by: Casper Neo <cneo@google.com>
2021-01-12 14:06:43 -05:00
Vladimir Glavnyy
83ce29cc22 [C++, JSON] Fix nullptr access when reading a key with a default value. (#6375)
This commit fixes handling of default and NULL `key` fields in `Parser::ParseVector` (#5928).

The JSON generator updated. It outputs `key` fields even if the `--force-defaults` option is inactive.

Additional test cases for `key` added.
2021-01-07 11:24:59 -08:00
Vladimir Glavnyy
82836a62be [idl_parser] Improve stack overflow protection (#6364)
* [idl_parser] Improve stack overflow protection

Add stack overflow protection for Flexbuffer and nested Flatbuffer parsers.
Replaces the `Recurse()` method by the new ParseDepthGuard RAII class.

* Remove move operator from Parser.

It was wrong decision to add move ctor and assignment into Parser class.
These operators will make it extremely difficult to add constant or reference fields in the future.

* Remove ';' from definition of FLATBUFFERS_DELETE_FUNC

* Format code

* Make this PR compatible with MSVC2010 (it doesn't support inherited ctor)
2021-01-04 12:39:12 -08:00
Vladimir Glavnyy
e7430bbebd [idl_parser] Check the range of explicitly set field's id value (#6363)
* [idl_parser] Check the range of explicitly set field's id value

The explicitly set `id` attribute should be a non-negative value of the `voffset_t` type.

* Format FieldIdentifierTest()
2021-01-04 12:33:31 -08:00
mqy
05192553f4 Fix typos in usage/comments; Make rust generator respect to --filenam… (#6342)
* Fix typos in usage/comments; Make rust generator respect to --filename-suffix

* run clang format
2020-12-17 13:15:27 -08:00
tira-misu
9fca5e4f42 Add flatc option to inhibit all warnings #6005 (#6301)
* Fix C/C++ Create<Type>Direct with sorted vectors

If a struct has a key the vector has to be sorted. To sort the vector
you can't use "const".

* Changes due to code review

* Improve code readability

* Add generate of JSON schema to string to lib

* option indent_step is supported

* Remove unused variables

* Fix break in test

* Fix style to be consistent with rest of the code

* Add option --no-warnings to inhibit all warnings

* Fix order of member initialization

* Add documentation for --no-warnings
2020-12-10 14:03:53 -08:00
Vladimir Glavnyy
8d9eae9ac9 [idl_parser] Unify parsing of NaN values read from .fbs and .json files (#6296)
This commit unifies parsing of NaN values read
from .fbs and .json files by converting them to unsigned NaN.
2020-12-07 11:12:58 -08:00
Vladimir Glavnyy
c27bc2d76f [C++] Add ParseJson(), Parser(Parser&&), update fuzzers (#6284)
- add a new method ParseJson to minimize failures during fuzzing
- add default (conditional) move-constructor for Parser
- add a new monster_fuzzer
- switch fuzzers to C++17 and `test/cpp17` generated code
2020-11-23 10:17:44 -08:00
Wouter van Oortmerssen
bc518a5127 Fixed FlexBufferBuilder asserting on duplicate keys 2020-11-23 09:02:40 -08:00
Austin Schuh
e58c182443 Add --require-explicit-ids to require explicit ids (#6277)
* Add --require-explicit-ids to require explicit ids

We just got bit by a well intentioned developer forgetting that field
order by default is the field index.  3 people missed it in review.

I'm looking at ways to make it harder to mess up.  We are requesting
that developers explicitly id all fields in tables.  Automatic (opt in
for others) enforcement of this will help the effort succeed.  This
patch adds a command line flag which lets the user require ids on all
fields in tables.

* Added docs to Compiler.md as well
2020-11-19 17:16:45 -08:00
Stefan F
d0d51e2a5c flatc should support --binary --schema with optional scalar fields. (#6252)
* Added missing EndTable() call to VerifyObject()

VerifyObject called VerifyTableStart() but not EndTable(). This made Verifier::VerifyComplexity() increase depth_ with each table, not with the depth of tables.

https://groups.google.com/forum/#!topic/flatbuffers/OpxtW5UFAdg

* Added Check to VerifyAlignment

https://stackoverflow.com/questions/59376308/flatbuffers-verifier-returns-false-without-any-assertion-flatbuffers-debug-veri

* Add GetStringView (Convenience function to get string_view from a String returning an empty string_view on null pointer) like GetString, GetCstring

* flatc should warn, when an attribute is attached more than once.

flatc.exe -b duplicate.fbs
warning: duplicate.fbs(5, 36): warning: attribute already found: priority

duplicate.fbs:

namespace MyGame;

attribute "priority";

table Monster (priority:1, priority:2) {
}

root_type Monster;

* flatc should support --binary --schema with optional scalar fields.

This fixes 'error: Optional scalars are not yet supported in at least one the of the specified programming languages.' when calling flatc.exe --binary --schema with a schema containing optional scalars.
2020-11-12 10:06:09 -08:00
Derek Bailey
ea92a668d8 [C#] Optional Scalars (#6217)
* [C#] Optional Scalars

* Moved scalar optional check to avoid null enum conversion
2020-10-30 13:18:59 -07:00
Kamil Rojewski
a0182cdb11 optional scalars for ts/js (#6215)
* optional scalars for ts/js

* removed range based for

* removed range based for
2020-10-28 09:45:23 -07:00
Paulo Pinheiro
f9a18ea635 [Java] Implement optional scalars (#6212)
Java implementation of optional scalars, following issue #6014.
2020-10-26 17:46:50 -07:00
Casper
fe8e3c7e5d Mass Refactoring to use IsString and other BASE_TYPE helpers (#6193)
* Updated SupportsAdvancedUnionFeatures to look out for string

* Mass refactoring to use BASE_TYPE helper functions.

Co-authored-by: Casper Neo <cneo@google.com>
2020-10-20 08:00:31 -07:00
Casper
8f6fa4b715 Updated SupportsAdvancedUnionFeatures to look out for string (#6190)
Co-authored-by: Casper Neo <cneo@google.com>
2020-10-19 13:16:00 -07:00
mustiikhalil
7b9e61fccf [TS] GRPC Implementation (#6141)
* GRPC implementation for Typescript

* Fixes a couple of issues

* Finished implementing the typescript support for grpc

* Updated generated code

* Fixes CI
2020-10-07 09:56:30 -07:00
Stefan F
ab01ae1620 flatc should output a warning, when an attribute is attached more than once (#6146)
* Added missing EndTable() call to VerifyObject()

VerifyObject called VerifyTableStart() but not EndTable(). This made Verifier::VerifyComplexity() increase depth_ with each table, not with the depth of tables.

https://groups.google.com/forum/#!topic/flatbuffers/OpxtW5UFAdg

* Added Check to VerifyAlignment

https://stackoverflow.com/questions/59376308/flatbuffers-verifier-returns-false-without-any-assertion-flatbuffers-debug-veri

* Add GetStringView (Convenience function to get string_view from a String returning an empty string_view on null pointer) like GetString, GetCstring

* flatc should warn, when an attribute is attached more than once.

flatc.exe -b duplicate.fbs
warning: duplicate.fbs(5, 36): warning: attribute already found: priority

duplicate.fbs:

namespace MyGame;

attribute "priority";

table Monster (priority:1, priority:2) {
}

root_type Monster;
2020-09-28 09:28:47 -07:00
Vladimir Glavnyy
fb4e1c34f9 Add CharToLower and CharToUpper into util.s (#6126)
This commit adds replacement of `::tolower` and `::toupper`.
Added CharToLower and CharToUpper routines reduce the number of cast operators
that required for correct usage of standard C/C++ `::tolower/toupper` routines.
2020-09-21 09:31:27 -07:00
Vladimir Glavnyy
8c67b5b129 Add support of Optional<T> scalars to C++ code generator (#6092) 2020-09-18 12:51:17 -07:00
Paulo Pinheiro
6228b66d3d [Kotlin] Support for optional scalars. (#6115)
More information on #6014
2020-09-17 23:31:51 -07:00
Casper
c75ae24293 Optional-ness in reflection (#6097)
* Optional scalars in reflection

* fixed name collision

* Remove code duplicated by merge

Co-authored-by: Casper Neo <cneo@google.com>
2020-09-11 14:20:32 -07:00
Casper
338944d3d9 Rename Nullable scalars to Optional scalars (#6112)
Co-authored-by: Casper Neo <cneo@google.com>
2020-09-10 13:04:36 -07:00
Wouter van Oortmerssen
77f966f89f [Lobster] optional scalars support 2020-08-18 14:00:02 -07:00
mustiikhalil
d76e93f277 adds code gen for optional scalars in swift (#6038)
Adds swift test code

Replaces if statments

Adds swift to supported languages for optionals

Moved std::string to auto

Adds nullable scalars support in object api
2020-07-25 17:59:34 +03:00
Casper
043b52bd4a Optional Scalars support for Rust (#6034)
* First draft of rust optionals

* Code cleanup around ftBool and ftVectorOfBool

* Tests for Rust optional scalars

* test bools too

Co-authored-by: Casper Neo <cneo@google.com>
2020-07-23 16:30:27 -07:00
Casper
9ecd2e16c2 Flatc parser support for nullable scalars (#6026)
* Parser support for nullable scalars

* Use older C++ features

* use default element

* Add a test for json, flexbuffers, and null

* test comments and names

Co-authored-by: Casper Neo <cneo@google.com>
2020-07-16 13:43:47 -07:00
Derek Bailey
60b6066feb Add warning to schema parser if field name is not snake_case. (#6005)
* Added code gen for evolution tests back in.

* General generate code and clang format

* Added code gen for evolution tests back in.

* General generate code and clang format

* reran generate_code.sh

* Added code gen for evolution tests back in.

* General generate code and clang format

* Added code gen for evolution tests back in.

* General generate code and clang format

* Enforce snake_case for schema field names

* Switched to basic for loop, Fixed warning message
2020-06-26 11:16:02 -07:00
Derek Bailey
165a6e3d1e Re-added Evolution Schema Code Generation Command (#5999)
* Fixed refractoring issue in reflection/generate_code.sh. Also, mv deletes the original file, so I don't need to clean it up manually in that case.

* Fixed Dart Tests by removing code-gen for included files.

* Added code gen for evolution tests back in.

* General generate code and clang format

* Added evolution schema generation to .bat file

* Added code gen for evolution tests back in.

* General generate code and clang format

* Added evolution schema generation to .bat file

* reran generate_code.sh

* Removed wildcard from generate_code.bat that doesn't work
2020-06-25 15:35:23 -07:00
Alex Ames
b124b76258 Removed requirement that enums be declared in ascending order. (#5887) 2020-06-22 17:02:15 -07:00
Javier Serrano
0ec7600c67 Do not remove the last digit from float values (#5974)
Trailing zeros are already removed inside the function FloatToString,
that is called immediately before the lines deleted by this commit.
2020-06-22 16:57:45 -07:00
Liu Liu
cc44a44427 [idl_parser] Mark typefield as deprecated (#5958)
* Mark typefield as deprecated

* Add UnionDeprecation Test.

* Update to use evolution schema instead.

* Use --scoped-enums to compile.
2020-06-15 23:18:12 -07:00
Mika Raento
a28357d7ac Propagate boolean default values from proto to fbs (#5964) 2020-06-11 10:15:19 -07:00
Vladimir Glavnyy
4e45f7c9e8 Fix error in SimpleQSort (#5955) 2020-06-09 23:35:07 -07:00
Wouter van Oortmerssen
94a78e3853 Fixed: Access violation and ASAN/UNSAN failures with sorted tables
https://github.com/google/flatbuffers/issues/5945
2020-06-08 09:50:40 -07:00
Derek Bailey
7179a5a8ba General Codebase clean up (#5939)
* Fixed refractoring issue in reflection/generate_code.sh. Also, mv deletes the original file, so I don't need to clean it up manually in that case.

* Ensuring test/generate_code.sh was ran

* Fixed Dart Tests by removing code-gen for included files.

* General cleanup of codebase.
2020-06-01 08:40:55 -07:00
Derek Bailey
0fa087657e [Dart] Getting tests/DartTest.sh to work on master. (#5915)
* Fixed refractoring issue in reflection/generate_code.sh. Also, mv deletes the original file, so I don't need to clean it up manually in that case.

* Fixed Dart Tests by removing code-gen for included files.
2020-05-15 18:13:29 -07:00
Vladimir Glavnyy
424a473e1f Schema parser: prohibit declaration of an array of pointers inside structs (#5907)
* Fix issue #5906, Prohibit declaration of an array of pointers inside structs

- idl_parser.cpp: Prohibit declaration of an array of pointers inside structs
- idl_gen_cpp.cpp: Extract GenStructConstructor() method from GenStruct() to simplify future modification
- idl_gen_cpp.cpp: Add assert for checking of Array fields in structs on code-generation stage

* Fix the error 'unused local variable' in release build

* Fix: format the PR code according to coding rules

* Add test-case and fix review notes
2020-05-14 11:39:58 -07:00
Mika Raento
6e9f5d9810 Respect shared attribute in Parser (#5885)
Respects the 'shared' field attribute when reading data with Parser.
2020-05-04 10:33:11 -07:00
Vladimir Glavnyy
9b034eee12 Fix interpretation of 'nan(number)' by the idl_parser (#5810)
* Parser reject "nan(n)" string as it does with nan(n)

* Adjust scalar fuzzer to ignore '$schema' substrings

- Scalar fuzzer ignores '$schema' substrings at the input
- Added 'scalar_debug' target to simplify research of fuzzed cases

* Improve formatting of './tests/fuzzer/CMakeLists.txt'
2020-03-16 11:59:34 -07:00
Vladimir Glavnyy
d10c163142 Replace 'assert' by 'FLATBUFFERS_ASSERT' inside idl_parser.cpp (#5799) 2020-03-12 11:18:10 -07:00
Myrice
9954e09ab7 [C++] Generate code for vector force_align attribute. (#5796)
* [Flatbuffer] Generate code for force_align with CreateXDirect and Pack functions.

* Fixed Visual Studio 10.0 compile error for std::to_string.

* Fixed Visual Studio 10.0 compile error for std::to_string.
2020-03-12 08:32:29 -07:00
Derek Bailey
6ff1898413 Added --filename-suffix and --filename-ext to flatc (#5778)
* Fixed refractoring issue in reflection/generate_code.sh. Also, mv deletes the original file, so I don't need to clean it up manually in that case.

* Added --filename-suffix and --filename-ext to flatc

* Fixed typo and added example generation of suffix and extension for C++

* Removed extra ;

* Removed clang-format block from a region that didn't need it. Fixed an auto format of another clang-format block

* Added docs, fixed pointer alignment, removed suffix test file
2020-03-02 10:15:23 -08:00
mustiikhalil
04d80f255d [Swift] Swift implementation 🎉🎉 (#5603)
* Implemented the swift version of Flatbuffers

Implemented serailzing, reading, and mutating data from object monster

Fixes mis-aligned pointer issue

Fixes issue when shared strings are removed from table

Adds swift enum, structs code gen

Fixed namespace issues + started implementing the table gen

Added Mutate function to the code generator

Generated linux test cases

Fixed an issue with bools, and structs readers in table writer

Swift docker image added

Updated the test cases, and removed a method parameters in swift

Fixed createVector api when called with scalars

Fixed issues with scalar arrays, and fixed the code gen namespaces, added sample_binary.swift

Cleaned up project

Added enum vectors, and their readers

Refactored code

Added swift into the support document

Added documentation in docs, and fixed a small issue with Data() not being returned correctly

Fixes Lowercase issue, and prevents generating lookups for deprecated keys

* Made all the required funcs to have const + removed unneeded code + fix lowercase func

* Removed transform from lowercased and moved it to function

* Fixes an issue with iOS allocation from read

* Refactored cpp code to be more readable

* casts position into int for position

* Fix enums issue, moves scalar writer code to use memcpy

* Removed c_str from struct function

* Fixed script to generate new objects when ran on travis ci: fix

* Handles deallocating space allocated for structs

* Updated the test cases to adhere to the fileprivate lookup, no mutation for unions, and updated the names of the vector functions
2020-01-09 12:12:10 -08:00
Wouter van Oortmerssen
55686100aa Changed direct calls to strtod to use StringToNumber
StringToNumber will correctly use locale-insensitive functions
when available.

Change-Id: I6bde11039a541634186f8f791012af2eb0d86b8d
2020-01-09 11:48:09 -08:00