Unit tests
Update flatbuffers + gRPC build instructions
Update CMakeLists.txt with cmake variables for grpc and protobuf install paths
Update tests for travis build
The verifier must be resilient against any corrupt data, so
now using size_t thru-out to ensure any 64-bit offsets can
be represented.
Also added verification of alignment.
Change-Id: I87a22aa6b045c2d83b69b47a47153f2e15ad7e06
Tested: on Linux, also with libfuzzer.
Fix for: https://bugs.chromium.org/p/chromium/issues/detail?id=834710
Before, the verifier would create pointers to objects, and then
verify they are inside the buffer. But since even constructing pointers
that are outside a valid allocation is Undefinied Behavior in C++, this
can trigger UBSAN (with -fsanitize=pointer-overflow).
Now instead the bounds checking is first performed using offsets
before pointers are even created.
Change-Id: If4d376e90df9847e543247e70a062671914dae1b
Tested: on Linux.
Adds helper function to get empty string when String is nullptr.
This is to get over the fact that flat buffer builders will record null when data
is not present.
Current comment is a bit ambiguous. Default values can be read either if field is not written (like in table), or if they are written explicitly by client but not serialized due to optimization. Impression from current comment is that all the default values which are coming during read are from binaries when we turn-on Force-Defaults. However, that will be a wrong interpretation.
Force_Defaults = true ensures to turn OFF later optimization. In case a field is not written, during read we will get default values but they will still not be serialized.
This function cannot work with any offset types (since offsets
must always point forward) so this avoid possible mistakes.
Change-Id: I1b3dfbefc8d40da630345b9b04f9aff4a990e8e5
This is to not need static variables, which could trip up users
with destruction order problems.
This potentially makes these operations slightly slower, but I
think they're infrequent enough that this should not be noticable.
Also there is one breaking API change, for a method that is not
used by any code in FlatBuffers and is assumed to affect very
few if any users. A namechange and comment ensures that those
affected, if any, will not run into trouble silently.
Change-Id: I16c1352d1dfc9092c816ddb7e353ed7f5f417444
Tested: on Linux.
Fix for the issue #4744: Ambiguous side-effect execution on vector_downward::make_space() method.
C++ does not impose evaluation order on the two expressions on the right side of the assignment, so compiler can freely decide. As ensure_space() method can change the value of "cur_" variable, the result of the subtraction may be different depending on the evaluation order, which is ambiguous in C++.
In order to make this code deterministic and correct, cur_ must be evaluated after ensure_space() is called.
* Add view() method on flatbuffers::String, to return a string_view type
if support for std::string_view (or alternately
std::experimental::string_view) is found
* Move detection/definition of FLATBUFFERS_STRING_VIEW to base.h, use the
macro (if it is defined) as the argument type for an overload of CreateString
* Rename String::view() to String::string_view() and use the existing c_str() method for the data pointer
* Add and explain minimum C++ standard version checks for FLATBUFFERS_STRING_VIEW implementations
* Updated preprocessor indenting for FLATBUFFERS_STRING_VIEW
* Convert FLATBUFFERS_STRING_VIEW macro to typedef in flatbuffers:: namespace, and boolean feature toggle macro FLATBUFFERS_HAS_STRING_VIEW
* Prepend flatbuffers:: namespace to disambiguate flatbuffers::string_view typedef from String::string_view()
* clang-format as-she-is-spoke for FLATBUFFERS_HAS_STRING_VIEW
It was missing some helpers when we choose to use
size prefixed FlatBuffers.
* Add general helper : GetPrefixedSize
* Add generated helpers :
* GetSizePrefixedXXX
* VerfifySizePrefixedXXXBuffer
* FinishSizePrefixedXXXBuffer
- Fixed ForceVectorAlignment (and possibly other call-sites) not
setting minalign_.
- Fixed flipped alignment parameters in CopyTable (reflection).
- Made aligment for FlatBufferBuilder internal buffer configurable
(useful when reading a constructed buffer directly).
- Ensured Alignment rounding is always up.
Change-Id: I33ca4887d92a09cb11a369c14a109f4b07ae707a
Tested: on Linux.
Date: Mon Jan 15 11:38:20 2018 -0200
Compilation failure with grpc.h
If cmake run with flag FLATBUFFERS_BUILD_GRPCTEST=ON
compilation fails.
Fix :
-Fix argument list for overriden function in grpc.
-Fix member function name called by FlatBufferBuilder from
buf() to scratch_data()
Previously, FlatBufferBuilder used 3 resizable buffers:
- serialization (vector_downward)
- field offsets (std::vector)
- vtable offsets (std::vector)
Since the serialization buffer grows downwards, the bottom part of
it can be used as a "scratchpad" storage for the other two. Since
field offsets are only accumulated during table construction, and
vtable offsets only after table construction, the two can trivially
share the same storage.
Not only does this reduce the amount of allocation, it also removes
the bulk of std::vector usage from FlatBufferBuilder which was
the #1 cause of slow-down in debug mode, see e.g.:
https://stackoverflow.com/questions/36085285/any-way-to-improve-flatbuffer-performance-in-debug-c-msvc
Change-Id: I0224cf2f2a863d2d7ef762bc9163b52fdc149522
Tested: on Linux.
Without this change, the compiler tries to select the following overload
when CreateString is passed a `char *`:
template<typename T>
Offset<String> CreateString(const T &str) {
return CreateString(str.c_str(), str.length());
}
which is not valid since char pointers don't have methods.
(Fixes#4579)
Signed-off-by: Andrew Gunnerson <chenxiaolong@cxl.epac.to>
* flatbuffers: Move EndianSwap template to flatbuffers/base.h
Clang complains
call to function 'EndianSwap' that is neither visible in the template definition nor found by argument-dependent lookup
return EndianSwap(t);
This seems to be due to limitation of two-phase lookup of dependent names in template definitions
Its not being found using associated namespaces therefore
it has to be made visible at the template definition site as well
Signed-off-by: Khem Raj <raj.khem@gmail.com>
* use __builtin_bswap16 when building with clang
clang pretends to be gcc 4.2.0 and therefore the code does
not use __builtin_bswap16 but tries to synthesize it
Signed-off-by: Khem Raj <raj.khem@gmail.com>
* Fix DetachedBuffer move assignment lifetime
DetachedBuffer move assignment now destroys it's own content,
rather than waiting for the destructor of other.
See more under #4435
Add missing function return type
* Rename empty_assign to reset
Reset after destroy
reset() now operates on itself
This is something the format supports, but none of the builders
were doing. Can save 10-20% on FlatBuffer binary size!
Also fixed the Go tests.
Change-Id: I616c56ce9bbcfcaee23aa24f0532fcb60b6a8c75
Tested: on Linux.
* Added internal - limited - implementation of flatbuffers::unique_ptr
for STLs that don't ship with std::unique_ptr. In C++11 and beyond
this is just an alias for std::unique_ptr.
* Aliased used type traits structs is_scalar is_floating_point is_unsigned
into flatbuffers namespace so they can be replaced in C++98 implementations.
Right now these point at stlport's TR1 implementations.
* Wrapped vector::data() in vector_data().
* Wrapped vector::emplace_back() in vector_emplace_back().
* Wrapper string::back() in string_back().
* Added variants of FlatBufferBuilder::CreateVector() and
FlatBufferBuilder::CreateVectorOfStructs() that allow the use of plain
function pointers.
Generated code has also been modified to use plain functions to build objects
rather than std::function() so all generated code will work in C++98
applications.
* Added flexbuffers::Builder::Vector(), flexbuffers::Builder::TypedVector()
and flexbuffers::Builder::Map() methods that allow the use of plain function
pointers.
* Changed Parser to internally use plain function pointers when parsing table
and vector delimiters.
* Added specializations of NumToString() for 64-bit types that aren't supported
by stringstream in stlport.
* Overloaded numeric_limits for 64-bit types not supported by stlport.
* Replaced build_apk.sh (which was broken by deprecation of the
"android" tool in the Android SDK) with build.gradle and the
appropriate gradle wrapper to build an APK.
* Switched Android build to build against all STL variants.
* Updated travis configuration to build Android test and sample.
Tested:
* Verified all tests continue to work on Linux, OSX and Android.
* Verified Travis build is green.
Change-Id: I9e634363793f85b9f141d21454b10686020a2065
flatbuffers.h(591): error C2220: warning treated as error - no 'object' file generated
flatbuffers.h(591): warning C4267: 'return': conversion from 'size_t' to 'flatbuffers::uoffset_t', possible loss of data
* [C++] Remove std::iterator usage (#4340)
Inheriting from std::iterator has never been required, and it's
deprecated in C++17. It can be replaced by directly providing typedefs.
Include <iterator> for std::random_access_iterator_tag.
Note that structs default to public access control.
* [C++] Change whitespace style in typedefs.
* Rework flatbuffers + gRPC integration
- Introduce `flatbuffers::grpc::Message<T>`, a `grpc_slice`-backed
message buffer that handles refcounting and allows flatbuffers to
transfer ownership to gRPC efficiently. This replaces
`flatbuffers::BufferRef<T>`, which required a copy call and was also
unsafe w.r.t. buffer lifetime.
- Introduce `flatbuffers::grpc::MessageBuilder`, a gRPC-specific builder
that forces a `grpc_slice`-backed allocator and also adds some helpful
`Message<T>`-related methods.
- Update serializers accordingly (now zero-copy between flatbuffers and
gRPC).
* gRPC: verify messages by default, but allow user to override
* gRPC: fix some formatting issues
* Disable verification by default, but add helper method
* Make FlatBufferBuilder fields protected + remove vec accessor
* Use bool add_ref parameter to toggle refcount incr
* Remove unnecessary inline specifiers
* Fix formatting
* Use auto
* Remove empty lines
* Use grpc_slice helper macros
* Simplify reset code
* Disable Message copy ctor and assignment by default
* Remove unused member
* Enable gRPC verification by default
* Use auto
* Bake in message verification (remove template specialization)
* Add RoundUp func
* Consolidate gRPC message copy flag
* Make vector_downward allocations fully lazy
* Test message verification failure code/message
* Add grpctest verification test comments
* Simplify reallocate implementation
* Make initial_size a size_t
* Use ternary op for growth_policy
* Use truthiness rather than dont explicit nullptr check
* Indent preprocessor directives
* Remove grpc message copy/assignment
* Fix a few bugs
* Add gRPC example
* Add basic gRPC docs
* Use doxygen EXAMPLE_PATH + @include
* Reference example fbs in grpc docs
* Move gRPC examples into grpc/samples
* Fix pointer/reference formatting
* Use std::function rather than templated callback func
* Create fresh message builder for each request
* Use Clear() in Reset() impl
* Use FLATBUFFERS_CONSTEXPR
* Remove std::move around a raw pointer
Calling std::move on a raw pointer has no advantage to just copying its value. Moreover, it is confusing, because it indicates that the argument is movable in some non-trivial way (e.g., is it actually meant to be a smart pointer?). More context in https://crbug.com/729393.
* Remove the move constructor altogether
FlexBuffers are actually completely unrelated to FlatBuffers and can be used separately.
However, since they utilize several utility functions from flatbuffer.h, they require this header file, which creates unreasonable dependency.
By moving those utility functions to separate base header, both libraries can use same code but still be independent
* Improve Allocator handling
- Templatize Allocator on vector_downward, and make it own the allocator
instance so it can manage lifetimes.
- Templatize + rename FlatBufferBuilderT accordingly, and add a typedef
to FlatBufferBuilder so old code continues to work.
- Fix some issues with the release deleter
- More details in github issue #4311
* Fix constexpr for older MSVC
* Reimplement allocator improvements via inheritance
Instead of templates, use an abstract base class and some unique_ptrs to
implement the new and improved allocator.
* Fix misplaced newline
* Add missing override keyword
* Add macro for func delete to support older compilers
* Explicitly move BufferDeleter (maybe fixes VS 10?)
* Revert previous attempt at VS10 fix
* Try yet another workaround for MS 10
* Use FLATBUFFERS_NOEXCEPT macro
* Add FLATBUFFERS_OVERRIDE macro
* Fix some issues with MSVC 16 hack
* Remove dep on unique_ptr
* Update DEPRECATED macro with a message
* Sigh, no delegating ctors before c++11
* Fix simple_allocator stub
* Relax FlatBufferBuilder ctor
* Add vector_downward reset + assert in clear
* Rename DetachedBuffer to FlatBuffer + remove unnecessary deprecations
* Add flatbuffers::FlatBufferBuilder::Release()
* Address remaining allocator-improvements comments
