Fix for: https://bugs.chromium.org/p/chromium/issues/detail?id=834710
Before, the verifier would create pointers to objects, and then
verify they are inside the buffer. But since even constructing pointers
that are outside a valid allocation is Undefinied Behavior in C++, this
can trigger UBSAN (with -fsanitize=pointer-overflow).
Now instead the bounds checking is first performed using offsets
before pointers are even created.
Change-Id: If4d376e90df9847e543247e70a062671914dae1b
Tested: on Linux.
* starting Lua port of python implmention. Syncing commit
* Bulk of Lua module port from Python done. Not tested, only static analysis. Need to work on binary strings. Started work on flatc lua code generation
* Fixed all the basic errors to produced a binary output from the builder, don't know if it is generated correctly, but it contains data, so that must be good
* fixed binary set command that was extending the array improperly
* continued improvement
* Moved lua submodules down a directory so their names don't clash with potential other modules. Added compat module to provide Lua versioning logic
* Successful sample port from Python
* working on testing Lua code with formal tests
* continued to work on tests and fixes to code to make tests pass
* Added reading buffer test
* Changed binaryarray implmentation to use a temporary table for storing data, and then serialize it to a string when requested. This double the rate of building flatbuffers compared to the string approach.
* Didn't need encode module as it just added another layer of indirection that isn't need
* profiled reading buffers, optimizations to increase read performance of monster data to ~7 monster / millisecond
* Writing profiler improvments. Get about
~2 monsters/millisecond building rate
* removed Numpy generation from Lua (came from the Python port)
* math.pow is deprecated in Lua 5.3, so changed to ^ notation. Also added .bat script for starting Lua tests
* adding results of generate_code.bat
* simple edits for code review in PR.
* There was a buffer overflow in inserting the keywords into the unorder set for both the Lua and Python code gens. Changed insertion to use iterators.
* fixed spacing issue
* basic documenation/tutorial updates. Updated sample_binary.lua to reflect the tutorial better
* removed windows-specific build step in Lua tests
Adds helper function to get empty string when String is nullptr.
This is to get over the fact that flat buffer builders will record null when data
is not present.
* Add --force-defaults option to flatc
To emit default values for fields which are not present or which are set
to the default value.
* flatc option --force-defaults should have a default value (false) and take action on the builder_ within the Parser constructor
* Add help text from flatc --force-defaults to Compiler.md doc
* Clarified docs for flatc --force-defaults, and imply that this behaviour is not normally needed.
* Updated docs and flatc help text for --force-defaults option
Current comment is a bit ambiguous. Default values can be read either if field is not written (like in table), or if they are written explicitly by client but not serialized due to optimization. Impression from current comment is that all the default values which are coming during read are from binaries when we turn-on Force-Defaults. However, that will be a wrong interpretation.
Force_Defaults = true ensures to turn OFF later optimization. In case a field is not written, during read we will get default values but they will still not be serialized.
This function cannot work with any offset types (since offsets
must always point forward) so this avoid possible mistakes.
Change-Id: I1b3dfbefc8d40da630345b9b04f9aff4a990e8e5
* Add suppport for ES6 style exports
Adds support for ECMAScript 6 module exports during Javascript
generation. This is useful as many development projects are
switching to this new standard and away from custom module
solutions. By integrating support into flatbuffers, users
do not need to perform additional post-processing of generated
files in order to use flatbuffers output directly in their
codebases.
Reference to ECMAScript 6 modules:
https://www.ecma-international.org/ecma-262/6.0/#sec-exports
Changes:
* Added `--es6-js-export` option to cli parser tool
* Added conditional code to generate a ES6 style export
line, replacing the normal NodeJS/RequireJS line.
* Fixed missing export statements
Added exports for definition and struct names that were not inside namespaces
* Updated Compiler.md with new generator option
Added entry to Compiler.md in docs for the `--es6-js-export` flag, including a brief description of the effects and usefulness.
This is to not need static variables, which could trip up users
with destruction order problems.
This potentially makes these operations slightly slower, but I
think they're infrequent enough that this should not be noticable.
Also there is one breaking API change, for a method that is not
used by any code in FlatBuffers and is assumed to affect very
few if any users. A namechange and comment ensures that those
affected, if any, will not run into trouble silently.
Change-Id: I16c1352d1dfc9092c816ddb7e353ed7f5f417444
Tested: on Linux.
Fix for the issue #4744: Ambiguous side-effect execution on vector_downward::make_space() method.
C++ does not impose evaluation order on the two expressions on the right side of the assignment, so compiler can freely decide. As ensure_space() method can change the value of "cur_" variable, the result of the subtraction may be different depending on the evaluation order, which is ambiguous in C++.
In order to make this code deterministic and correct, cur_ must be evaluated after ensure_space() is called.
* Add define/ifdef blocks for FLATBUFFERS_PREFER_PRINTF to avoid using std::*streams for idl_parser
* Use string::size() as limit in snprintf
* Refactored FLATBUFFERS_PREFER_PRINTF guarded feature into NumToStringImplWrapper around sprintf
* Remove '.0' where not needed from IntToDigitCount
* Remove leading dot from name in GetFullyQualifiedName when FLATBUFFERS_PREFER_PRINTF is enabled
* Return string directly from conversion functions where possible when FLATBUFFERS_PREFER_PRINTF is enabled
* Use string instead of stringstream for GetFullyQualifiedName
* Revert removing leading dot from GetFullyQualifiedName, it does need to be there for parity with the stringstream implementation
* Dot is single char in Namespace::GetFullyQualifiedName
* Remove trailing (duplicate) null-byte from NumToStringImplWrapper when using FLATBUFFERS_PREFER_PRINTF.
* Update preprocessor indenting (and use clang-format off/on) for FLATBUFFERS_PREFER_PRINTF
* Reduce whitespace, unneeded braces in FLATBUFFERS_PREFER_PRINTF string width functions
* Remove unneeded use of iostream from idl_parser.cpp, std::string is used instead
* Tell snprintf to use the trailing null byte expected at the end of std::string buffer
* Add view() method on flatbuffers::String, to return a string_view type
if support for std::string_view (or alternately
std::experimental::string_view) is found
* Move detection/definition of FLATBUFFERS_STRING_VIEW to base.h, use the
macro (if it is defined) as the argument type for an overload of CreateString
* Rename String::view() to String::string_view() and use the existing c_str() method for the data pointer
* Add and explain minimum C++ standard version checks for FLATBUFFERS_STRING_VIEW implementations
* Updated preprocessor indenting for FLATBUFFERS_STRING_VIEW
* Convert FLATBUFFERS_STRING_VIEW macro to typedef in flatbuffers:: namespace, and boolean feature toggle macro FLATBUFFERS_HAS_STRING_VIEW
* Prepend flatbuffers:: namespace to disambiguate flatbuffers::string_view typedef from String::string_view()
* clang-format as-she-is-spoke for FLATBUFFERS_HAS_STRING_VIEW
* Added support for the non-escaped print of utf-8 string.
* EscapeString: the first invalid symbol resets print_natural_utf8 flag to false.
* Move the test to ParseAndGenerateTextTest. Fixes.
* Removed dependence between `natural_utf8` and `allow_non_utf8` flags.
In file included from include/flatbuffers/flexbuffers.h:24,
from src/idl_gen_text.cpp:20:
include/flatbuffers/util.h: In function 'int flatbuffers::FromUTF8(const char**)':
include/flatbuffers/util.h:324:45: error: type qualifiers ignored on cast result type [-Werror=ignored-qualifiers]
if ((static_cast<const unsigned char>(**in) << len) & 0x80) return -1; // Bit after leading 1's must be 0.
^
cc1plus: all warnings being treated as errors
make[2]: *** [CMakeFiles/flatbuffers_shared.dir/build.make:92: CMakeFiles/flatbuffers_shared.dir/src/idl_gen_text.cpp.o] Error 1
Const does not make sense here, and compiler actually throws warning
(error with -Werror) when you would try to compile it.
In file included from include/flatbuffers/flexbuffers.h:24,
from include/flatbuffers/idl.h:26,
from include/flatbuffers/code_generators.h:22,
from src/code_generators.cpp:17:
include/flatbuffers/util.h: In function ‘int flatbuffers::FromUTF8(const char**)’:
include/flatbuffers/util.h:325:44: error: type qualifiers ignored on cast result type [-Werror=ignored-qualifiers]
if ((static_cast<const unsigned char>(tmp) << len) & 0x80) return -1; // Bit after leading 1's must be 0.
^
cc1plus: all warnings being treated as errors
This warning caught by gcc8:
$ g++ --version
g++ (GCC) 8.0.1 20180222 (Red Hat 8.0.1-0.16)
An assert in flexbuffers was bit-shifting a 64-bit number by
64 bits, which throws up warnings in some automated tools.
The same assert also checks to see if the number of bytes
being shifted is 8. Swapped the order, so that the bitshift
only occurs if the number of bits being shifted is not 64.
Should be the same behavior, but plays nicer with diagnostic
tools.
* Added '--oneof-union' option.
Used with the .proto -> .fbs converter, will translate protobuff oneofs to flatbuffer unions.
Updated proto test to check both methods of converting oneofs.
* Added '--oneof-union' option.
Used with the .proto -> .fbs converter, will translate protobuff oneofs to flatbuffer unions.
Updated proto test to check both methods of converting oneofs.
* FlatBuffers: Moved MakeCamel() into idl_parser.cpp
Removes library dependency on Java/C# generator code.
* added intended use-cases to monster_test.fbs
* added check for `cpp_ptr_type` on hashed fields
added default value 'naked' to `cpp_ptr_type` on hashed fields
* added C++ generation of cpp_type vectors
removed ctor call for vector fields
added condition !vector for cpp_type check
added Pack() and UnPack() code generation for vector of hashes
added generation of correct resolve/rehash for cpp_type elements
* added attribute 'cpp_ptr_type_get' to hold accessor for pointer types possible where '.get()' does not work
use case: cpp_ptr_type:"std::weak_ptr", cpp_ptr_type_get:".lock().get()"
* run flatc to re-generate headers
* added bool param is_ctor to GetDefaultScalarValue() to differentiate between usage places
* modified monster_test.fbs to remove usage of shared_ptr/weak_ptr
reason: STLport does not support std::shared_ptr and std::weak_ptr
* run flatc again to re-generate headers
* fixed symbol unique_ptr not in namespace std when building with STLport
It was missing some helpers when we choose to use
size prefixed FlatBuffers.
* Add general helper : GetPrefixedSize
* Add generated helpers :
* GetSizePrefixedXXX
* VerfifySizePrefixedXXXBuffer
* FinishSizePrefixedXXXBuffer
- Fixed ForceVectorAlignment (and possibly other call-sites) not
setting minalign_.
- Fixed flipped alignment parameters in CopyTable (reflection).
- Made aligment for FlatBufferBuilder internal buffer configurable
(useful when reading a constructed buffer directly).
- Ensured Alignment rounding is always up.
Change-Id: I33ca4887d92a09cb11a369c14a109f4b07ae707a
Tested: on Linux.
Some implementations (e.g. C++98) won't support 64-bit enum values,
but there is no reason to silently truncate them.
Change-Id: I8629563523a96e887068f9c0efcd53741f60e0d6
Tested: on Linux.
Date: Mon Jan 15 11:38:20 2018 -0200
Compilation failure with grpc.h
If cmake run with flag FLATBUFFERS_BUILD_GRPCTEST=ON
compilation fails.
Fix :
-Fix argument list for overriden function in grpc.
-Fix member function name called by FlatBufferBuilder from
buf() to scratch_data()
