Commit Graph

2147 Commits

Author SHA1 Message Date
Mark Spatz
91b0958c43 Search for includes in the directory containg the current file (#6371)
* [codegen] Search for includes in the directory containg the current file

This matches the behavior of a C preprocessor, see:
https://gcc.gnu.org/onlinedocs/cpp/Search-Path.html

* Skip FileExists() when file is guaranteed to not exist

* end comment with period to match others
2021-01-19 12:26:34 -08:00
Casper
8008dde117 Upgrade Rust dependencies (#6406)
Co-authored-by: Casper Neo <cneo@google.com>
2021-01-19 13:39:29 -05:00
Björn Harrtell
c81cf82492 [TS/JS] New gen TS code gen prequel to preserve history (#6404)
* Rename idl_gen_js_ts to idl_gen_ts to preserve history

* Change build related files
2021-01-14 11:34:44 -08:00
Casper
8573108bbe Unset FieldDef.optional if its key (#6402)
* Unset FieldDef.optional if its key

* removed StringKey table, just removed an extra "required, key"

* removed extra newline

Co-authored-by: Casper Neo <cneo@google.com>
2021-01-12 14:06:43 -05:00
Vladimir Glavnyy
7abe612b59 [fuzzer] Fix the binary schema loading in the monster_fuzzer (#6396)
This commit should fix issue 29374 in oss-fuzz.
2021-01-12 10:20:47 -08:00
Casper
408cf58024 Fix Rust UB problems (#6393)
* Fix miri problems by assuming alignment is 1 in rust

* Removed is_aligned fn from rust verifier.

* Add back is_aligned, but make it w.r.t. buffer[0]

* touch unused variable

* touch unused variable

* +nightly

* Move Rust miri testing into its own docker

* fix bash

* missing one endian conversion

* fix endianness2

* format stuff

Co-authored-by: Casper Neo <cneo@google.com>
2021-01-11 15:24:52 -05:00
brian-brt
39e115fdb4 Define Vector::value_type for STL compatibility (#6394)
Specifically, this lets the googlemock container matchers work with a
flatbuffers::Vector.

https://github.com/google/googletest/blob/master/googlemock/docs/cheat_sheet.md#container-matchers
2021-01-08 11:35:55 -08:00
Vladimir Glavnyy
85719669cb [fuzzer] Debug the monster_tets.bfbs on clusterfuzz server (#6392)
For some reason, this fuzzer failed to load the binary schema file
when run on the `/clusterfuzz` server.
Issue: https://oss-fuzz.com/testcase-detail/6215075358703616
This issue doesn't reproduce locally with the latest oss-fuzz docker image.
2021-01-08 11:09:41 -08:00
Dan Field
809fe49c7a Fix up scripts, fix generated enum value for strong mode, regenerate files (#6389)
* Fix up scripts, fix generated enum value for strong mode, regenerate files

* missing files

* GH action

* remove action
2021-01-07 14:51:52 -08:00
Aaron Son
41253e574e [go] tests/GoTest.sh: Fix flags.Parse location to work on new go SDKs. (#6388)
* tests/GoTest.sh: Fix flags.Parse location to work on new go SDKs.

Calling flags.Parse() within init() races with other packages which register
flags in their init(), and in particular with the testing package itself. It is
more reliable to call flags.Parse() from a TestMain implementation.

See https://github.com/golang/go/issues/31859,
https://github.com/golang/go/issues/33869.

* .github: Enable build-go action in build.yaml workflow.
2021-01-07 11:54:00 -08:00
Ivan Dlugos
08d2ce844b fix Dart Builder._writeString() - always write trailing zero byte (#6390) 2021-01-07 11:40:47 -08:00
Ivan Dlugos
a15a8d930d fix Dart Builder.reset() - clear vTables (#6386) 2021-01-07 11:28:33 -08:00
Vladimir Glavnyy
83ce29cc22 [C++, JSON] Fix nullptr access when reading a key with a default value. (#6375)
This commit fixes handling of default and NULL `key` fields in `Parser::ParseVector` (#5928).

The JSON generator updated. It outputs `key` fields even if the `--force-defaults` option is inactive.

Additional test cases for `key` added.
2021-01-07 11:24:59 -08:00
Richard A Hofer
4363c1d2cb Fix generated EndVector. (#6385) 2021-01-05 14:45:23 -08:00
Stefan de Konink
1bf1ec0270 Implements type promotion for Java enum (#6382)
* Implements type promotion for Java enum as suggested in #3890, #5111, #6369

* After generate_code.sh
2021-01-05 11:11:29 -08:00
Derek Bailey
0800976533 Delete label_notify.yml
Removing this workflow as it isn't used.
2021-01-05 10:06:33 -08:00
Wouter van Oortmerssen
795408115a Disabled PHP CI (#6381)
It errors with "Fatal error: Uncaught exception 'InvalidArgumentException' with message 'bad number  for type byte.. in /home/runner/work/flatbuffers/flatbuffers/php/ByteBuffer.php:490" which I can't reproduce locally, and trying to fix it on CI runs into PHP's insane handling of numbers vs strings.
2021-01-04 14:50:56 -08:00
Kamil Rojewski
46545e6273 fixed warnings (#6355)
* semver-compatible deprecations

* removed unneeded lifetimes (+ clippy warnings)

* silenced too many args warning
2021-01-04 14:11:59 -08:00
Bruno ZIKI Kongawi
0168178a17 Fix multiple fbs code generation failure (#6365) 2021-01-04 13:29:32 -08:00
Vladimir Glavnyy
82836a62be [idl_parser] Improve stack overflow protection (#6364)
* [idl_parser] Improve stack overflow protection

Add stack overflow protection for Flexbuffer and nested Flatbuffer parsers.
Replaces the `Recurse()` method by the new ParseDepthGuard RAII class.

* Remove move operator from Parser.

It was wrong decision to add move ctor and assignment into Parser class.
These operators will make it extremely difficult to add constant or reference fields in the future.

* Remove ';' from definition of FLATBUFFERS_DELETE_FUNC

* Format code

* Make this PR compatible with MSVC2010 (it doesn't support inherited ctor)
2021-01-04 12:39:12 -08:00
Vladimir Glavnyy
e7430bbebd [idl_parser] Check the range of explicitly set field's id value (#6363)
* [idl_parser] Check the range of explicitly set field's id value

The explicitly set `id` attribute should be a non-negative value of the `voffset_t` type.

* Format FieldIdentifierTest()
2021-01-04 12:33:31 -08:00
Richard A Hofer
24dd85fd28 Generate code to encode and decode nested flatbuffers in Python. (#6354)
* Generate code to encode and decode nested flatbuffers in Python.

* Delete accidental trailing whitespace.

* Fully delete trailing whitespace.
2021-01-04 12:18:19 -08:00
mustiikhalil
57f68e2896 [Rust] Shared String (#6367)
* Adds shared strings and tests for shared strings

* Adds resets on string_map

* Moved shared strings to use vector instead of hashmap

* Addresses all the issues

* Resolves some comments
2021-01-04 06:18:35 -08:00
mustiikhalil
44cf2bde19 Updates license date to 2021 (#6378) 2021-01-02 11:46:46 +03:00
Kamil Rojewski
be37d4da14 include_prefix support for rust (#6330)
* include_prefix support for rust

* include_prefix support for java

* formatting fixes

* Revert "include_prefix support for java"

* style fix
2020-12-30 12:02:36 -08:00
mustiikhalil
4e79d129cb [Swift] Rebuild the way swift handles structs from scratch (#6326)
* Rebuild the way swift handles structs from scratch

* Updates docs, and sample binary

* Replaces InMemory to Mutable

* Migrates docs from inmemory

* use inline for some functions

* Renamed Mutable objects

* Updates documentation
2020-12-17 14:55:32 -08:00
mqy
05192553f4 Fix typos in usage/comments; Make rust generator respect to --filenam… (#6342)
* Fix typos in usage/comments; Make rust generator respect to --filename-suffix

* run clang format
2020-12-17 13:15:27 -08:00
Kjetil Østerås
f2511d7d49 Renaming infinity variables in test.cpp (#6340)
The infinityf symbol is causing a conflict when building for cygwin. In
the cygwin math.h header there is also a symbol called infinityf. So
this patch is needed to be able to build the flatbuffer tests in a
cygwin environment.
2020-12-17 13:08:23 -08:00
Richard A Hofer
f8b203c9c4 Add vectorNumElements attribute to Builder for simpler vector creation. (#6328)
* Add vectorNumElements attribute to Builder for simpler vector creation.

This adds a default to EndVector which should simplify its use.

* Update tutorial to reflect new default arg in Python EndVector.

* Remove optional argument to Python EndVector.

* Add generated files.

* Unset Builder.vectorNumElems when not in use.
2020-12-14 12:00:28 -08:00
mustiikhalil
8ab7c7e2c8 [CI] Adds formatter to CI (#6272)
* Adds formatters CI

Adds Error message & setup formatting like cpp

Adds Swift

Adds typescript

Adds python

tests yarn

Adds format.md

* Removes unneeded scripts + moves install script to install phase

* Adds format.md content

* Adds cpp to the formatter.md and fixes ci

* Adds cpp to formatter ci
2020-12-11 10:46:10 -08:00
Wouter van Oortmerssen
7e00390289 Fix Max CI build path (#6333)
apparently the default xcodebuild path of ./build clashes with the BUILD file present (case insensitive file system?)
2020-12-11 10:31:20 -08:00
Stefan F
65c415911a Generate nullable properties in C# object-based API for optional scalars. (without -gen-mutable) (#6273)
* Added missing EndTable() call to VerifyObject()

VerifyObject called VerifyTableStart() but not EndTable(). This made Verifier::VerifyComplexity() increase depth_ with each table, not with the depth of tables.

https://groups.google.com/forum/#!topic/flatbuffers/OpxtW5UFAdg

* Added Check to VerifyAlignment

https://stackoverflow.com/questions/59376308/flatbuffers-verifier-returns-false-without-any-assertion-flatbuffers-debug-veri

* Add GetStringView (Convenience function to get string_view from a String returning an empty string_view on null pointer) like GetString, GetCstring

* flatc should warn, when an attribute is attached more than once.

flatc.exe -b duplicate.fbs
warning: duplicate.fbs(5, 36): warning: attribute already found: priority

duplicate.fbs:

namespace MyGame;

attribute "priority";

table Monster (priority:1, priority:2) {
}

root_type Monster;

* flatc should support --binary --schema with optional scalar fields.

This fixes 'error: Optional scalars are not yet supported in at least one the of the specified programming languages.' when calling flatc.exe --binary --schema with a schema containing optional scalars.

* Generate nullable properties in C# object-based API for optional scalars.
tests\generate_code.bat extended to test this.

Ran tests\generate_code.bat
Ran tests\Flatbuffers.Test\NetTest.bat

* %TEST_BASE_FLAGS% replaced with --gen-object-api in generate_code.bat, because only this is part of this PR. Added this same flag to generate_code.sh

* generate_code.bat and generate_code.sh changed to only test c# with object based api.
2020-12-10 16:50:30 -08:00
David P. Sicilia
a9e91116d2 [Python] Commit some orphan python genfile diffs. (#6325) 2020-12-10 16:43:40 -08:00
Derek Bailey
80a745d9b0 Fixed missing ending quotes in labeller (#6327) 2020-12-10 14:19:46 -08:00
tira-misu
9fca5e4f42 Add flatc option to inhibit all warnings #6005 (#6301)
* Fix C/C++ Create<Type>Direct with sorted vectors

If a struct has a key the vector has to be sorted. To sort the vector
you can't use "const".

* Changes due to code review

* Improve code readability

* Add generate of JSON schema to string to lib

* option indent_step is supported

* Remove unused variables

* Fix break in test

* Fix style to be consistent with rest of the code

* Add option --no-warnings to inhibit all warnings

* Fix order of member initialization

* Add documentation for --no-warnings
2020-12-10 14:03:53 -08:00
Vladimir Glavnyy
92a806b4e8 [fuzzer] Rename fuzzing dictionaries for oss-fuzz (#6318)
This commit makes the names of fuzzing dictionaries the same as the target binary names.
Also it explicitly limits size of test inputs to prevent failures in `regex` and fuzzing time-outs.
2020-12-10 14:02:01 -08:00
Casper
9c9baf6d58 bumprust (#6322)
Co-authored-by: Casper Neo <cneo@google.com>
2020-12-09 11:34:19 -08:00
Richard A Hofer
aafc5dc950 Set default initialSize for Builder to 0 (#6310)
* Set default initialSize for Builder to 0

* Change default size of builder to 1024.

This matches what the C++ and Java versions do.
2020-12-07 16:20:26 -08:00
Casper
442949bc11 Rust Flatbuffers Verifier (#6269)
* Updated comments and fixed a fundemental type error.

* bump rust flatbuffers semver

* Initial commit with verifier, need to clean up

* Verifier tested. Needs clean up and refactoring.

* Display for InvalidFlatbuffer and better errors for strings

* SimpleToVerify, some refactoring

* Combined VerifierType TableAccessorFuncBody into FollowType

* scrub todos

* Update Rust get_root functions.

There are 6 variants, with verifier options, default verifier options
and no verification "fast".

* Rename root fns

* inline

* Update to use thiserror

* fix for bad compiler

* improve error formatting

* Replace multiply with saturating_multiply

* saturating adds too

* Add docs disclaiming experimental verification system

Co-authored-by: Casper Neo <cneo@google.com>
2020-12-07 18:37:51 -05:00
David P. Sicilia
9064072e8c Version message should be a "STATUS" to avoid going to stderr. (#6316) 2020-12-07 11:50:29 -08:00
Austin Schuh
fd4c1b5ff7 Replace std::string and const char* CreateSharedString with string_view (#6315)
It is useful to be able to call CreateSharedString with a string_view.
A string_view can be implicitly converted from a std::string or a const
char*.  This means if string_view is available, we can use it instead of
both other functions and get all 3.
2020-12-07 11:49:24 -08:00
Vladimir Glavnyy
bc7eb8adeb [fuzzer] Fix mistakes in the parser and scalar fuzzers. (#6314)
The flatbuffers::Parser::Parse() isn't an idempotent method for schema parsing.
This commit removes a wrong for-loop that tried to check the same schema twice.
2020-12-07 11:47:33 -08:00
Richard A Hofer
fc960f3670 Add default to offset param of Python generated GetRootAs (#6312) 2020-12-07 11:34:07 -08:00
Vladimir Glavnyy
f437f0f7ed [fuzzer] Fix loading of schema in monster_fuzzer (#6308)
This is fix for (https://oss-fuzz.com/testcase-detail/6251772204810240)
2020-12-07 11:21:23 -08:00
Vladimir Glavnyy
7f33cf682a [C++] Switch flatc to --cpp-std c++11 C++ code generator (#6306)
Prior to this commit the default C++ code generator was `c++0x`.
A code generated with `c++0x` code-gen might have a vulnerability (undefined behavior) connected evolution of enums in a schema. This UB could break the backward compatibility if previously generated code casts an unknown enumerator to enum type that knows nothing about future enumerators added to the schema.

The main differences between `c++0x` and `c++11`:

- generated enums use explicitly declared underlying type;
- generated object-API tables don't declare default ctor() explicitly, instead of it default data member initializers are generated.

Please use `flatc --cpp-std c++0x` option for backward compatibility with old compilers.
2020-12-07 11:19:36 -08:00
Vladimir Glavnyy
8d9eae9ac9 [idl_parser] Unify parsing of NaN values read from .fbs and .json files (#6296)
This commit unifies parsing of NaN values read
from .fbs and .json files by converting them to unsigned NaN.
2020-12-07 11:12:58 -08:00
Wouter van Oortmerssen
2046bffa40 Moved various language tests from AppVeyor to GitHub Actions (#6300) 2020-12-03 11:26:27 -08:00
Casper
338393f854 Documentation updates for Optional Scalars (#6014) (#6270)
* Documentation updates for Optional Scalars

* Updated Support

* Reword stuff

* s/NULL/null

Co-authored-by: Casper Neo <cneo@google.com>
2020-11-24 14:05:39 -05:00
Vladimir Glavnyy
c27bc2d76f [C++] Add ParseJson(), Parser(Parser&&), update fuzzers (#6284)
- add a new method ParseJson to minimize failures during fuzzing
- add default (conditional) move-constructor for Parser
- add a new monster_fuzzer
- switch fuzzers to C++17 and `test/cpp17` generated code
2020-11-23 10:17:44 -08:00
Wouter van Oortmerssen
bc518a5127 Fixed FlexBufferBuilder asserting on duplicate keys 2020-11-23 09:02:40 -08:00