VerifyKey() returns true on the first non-zero byte instead of
checking for a null terminator. This causes VerifyBuffer() to accept
FlexBuffers with non-null-terminated keys. Subsequent access to those
keys via strlen()/strcmp() reads out of bounds.
The condition if (*p++) should be if (!*p++) — return true
when a null terminator is found, not when any non-zero byte is found.
Confirmed with AddressSanitizer: heap-buffer-overflow in strlen()
after VerifyBuffer() returns true on a corrupted buffer.
A blob is an array of bytes and has no intrinsic alignment (i.e. the
alignment is 1). The alignment of the existing flexbuffers blob is
solely affected by the width of the integer needed to store the blob's
size: that integer's width becomes the alignment of the blob.
The proposed AlignedBlob function here piggybacks on this effect and
simply uses a user-defined alignment for the width of the integer that
stores the blob's size; this automatically imparts that same alignment
on the blob itself. (The width is bounded below by the actual width
needed to store the blob's size.)
The ability to control the alignment of a blob is important for use
cases in which the blob itself stores structured data that we want to
access without further copies (e.g. other flatbuffer messages).
* Add --ts-undefined-for-optionals command line option
# Details
- Fixes#7656
- Added a new `--ts-undefined-for-optionals` command line option for `flatc`.
- If enabled, generated TypeScript code uses `undefined` for optional fields rather than `null`.
* Also add TS generated test files
* Run `sh scripts/clang-format-git.sh`
* also add tests/ts/lalala-options.ts to the repo
* move new tests to tests/ts/optional_values dir
* add tests/ts/optional_values/optional_values_generated.cjs to the repo
* reuse existing optional_scalars.fbs and add new test
* add comma
* sh scripts/clang-format-git.sh
* remove comma
* sh scripts/clang-format-git.sh
* trying things
* sh scripts/clang-format-git.sh
* done
* address feedback
* sh scripts/clang-format-git.sh
* run `sh scripts/clang-format-git.sh`
* remove uneeded `eslint-disable @typescript-eslint/no-namespace` line
---------
Co-authored-by: José Luis Millán <jmillan@aliax.net>
* chore(idl): Check for case insensitive keywords
Most languages are not affected by this change. In PHP, some names such
as Bool cannot be used because it is a reserved keyword for to the bool
data type. The field `keywords_casing` in the configs enables checking
all characters in lowercase against every keyword. This should be safe
as flatbuffers does not allow non-ASCII characters in its grammar.
* chore: Fix formatting to follow google's coding style for enums
* chore: Extract convert case to lower when CaseInsensitive
---------
Co-authored-by: Justin Davis <jtdavis777@gmail.com>
Co-authored-by: Derek Bailey <derekbailey@google.com>
* flatc builds and seems to work, some of the extra targets are having linker errors
* fix build system
* pipeline failures
* un-rename files
* refactor to use unique_ptr
* typo
* rm make_unique, add comments
* fix cmake
---------
Co-authored-by: Wouter van Oortmerssen <aardappel@gmail.com>
Android libraries include <android/api-level.h> which defines the __ANDROID_API__ symbol even when targeting non-Android platforms and not using Android's libc. This updates the FLATBUFFERS_LOCALE_INDEPENDENT ifdef to check for __ANDROID__ before checking the Android API level.
Removes an extra check from the __Fuchsia__ branch. Fuchsia's libc does not support locales or the locale independent entry points.
Updates the Android API check to check for an API level >= 26 instead of 21. This matches the Android header file's availability macros and Bionic documentation:
https://android.googlesource.com/platform/bionic/+/HEAD/docs/status.md
* Fixes to make SizeVerifier work.
In particular change all the places in the Flatbuffers library
and generated code that were using `Verifier` to instead use
`VerifierTemplate<TrackBufferSize>` and wrap them all inside
`template <bool TrackBufferSize = false>`.
Also add unit tests for SizeVerifier.
* Format using `sh scripts/clang-format-git.sh`
* Use `B` rather than `TrackBufferSize` for the name of the template parameter.
* Update generated files.
* Prevent `make_span` from working with vectors and arrays of pointers
* support `make_structs_span` for little-endian
* fix build: add the required parentheses
---------
Co-authored-by: Wouter van Oortmerssen <aardappel@gmail.com>
new_p is a local addr but is owned now by slice_
thus the life time does not end at the end of the function
Co-authored-by: Wouter van Oortmerssen <aardappel@gmail.com>
This prevents the include of the type defined in the pyi,
otherwise this leads to error message like this:
error: Name XYZ already defined (possibly by an import) [no-redef]
For Vector or Array of structures the dereference operator of an
iterator returns the pointer to the structure. However, IndirectHelper,
which is used in the implementation of this operator, is instantiated
in the way that the IndirectHelper::Read returns structure by value.
This is because, Vector and Array instantiate IndirectHelper with
const T*, but VectorIterator instantiates IndirectHelper with T. There
are three IndirectHelper template definition: first for T, second for
Offset<T> and the last one for const T*. Those have different
IndirectHelper:Read implementations and (more importantly) return type.
This is the reason of mismatch in VectorIterator::operator* between
return type declaration and what was exactly returned.
That is, for Array<T,...> where T is scalar the VectorIterator is
instantiated as VectorIterator<T, T>, dereference operator returns T
and its implementation uses IndirectHelper<T> which Read function
returns T.
When T is not scalar, then VectorIterator is instantiated as
VectorIterator<T, const T *>, dereference operator returns const T * and
its implementation uses IndirectHelper<T> which Read function returns T.
The fix is done as follows:
* implement type trait is_specialization_of_Offset and
is_specialization_of_Offset64,
* change partial specialization of IndirectHelper with const T * that
it is instantiated by T and enabled only if T is not scalar and not
specialization of Offset or Offset64,
* remove type differentiation (due to scalar) from Array..
The above makes the IndirectHelper able to correctly instantiate itself
basing only on T. Thus, the instantiation in VectorIterator correctly
instantiate IndirectHelper::Read function, especially the return type.
This allows this type to meet the requirements of e.g.
std::ranges::range, which is necessary for it to work with the
std::span range constructor, or the "non-legacy" constructor for
Chromium's base::span.
Bug: none
Co-authored-by: Derek Bailey <derekbailey@google.com>
In case when flatbuffers are being used along with other project that defines "max" preprocessor macro, the ::max() in FLATBUFFERS_MAX_BUFFER_SIZE and FLATBUFFERS_MAX_64_BUFFER_SIZE is incorrectly being expanded to the macro. Adding parentheses enforces function-like interpretation.
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Move `namer.h` and `idl_namer.h` to `include/codegen` so they can be reused from `grpc` dirqectory.
* [gRPC] Update the Python generator to produce typed handlers and Python stubs if requested.
* [gRPC] Document the newly added compiler flags.
* [Python] Generate `.pyi` stub files when `--python-typing` is on.
To support this change, the following modifications were made:
- added a new option to disable `numpy` helpers generation;
- added a new flag to control the target Python version:
`--python-version` can be one of the following:
- `0.x.x` – compatible with any Python version;
- `2.x.x` – compatible with Python 2;
- `3.x.x` – compatible with Python 3.
- added codegen utilities for Python;
- added a note that the generated .py file is empty.
* [Python] Update Bazel build rules.
* [Python] Update Bazel build rules.
* [Python] Run buildifier on BUILD.bazel files.
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
This will allow the code to be compiled with `-Wfloat-equal`
as this would result in the folowing warning/error:
vendor/flatbuffers/include/flatbuffers/base.h:465:69:
error: comparing floating point with == or != is unsafe [-Werror,-Wfloat-equal]
template<typename T> inline bool IsTheSameAs(T e, T def) { return e == def; }
But the way it is used in flatbuffers it is ok to compare floating
points with ==.
Co-authored-by: Derek Bailey <derekbailey@google.com>
The new options are:
- `--grpc-filename-suffix` controls the suffix of the generated files;
- `--grpc-use-system-headers` controls the type of C++ includes generated;
- `--grpc-search-path` controls the directory that contains gRPC runtime;
- `--grpc-additional-header` allows to provide additional dependencies for the generated code.
Fix "One Definition Rule" violation when using flatbuffers::Verifier with
FLATBUFFERS_TRACK_VERIFIER_BUFFER_SIZE defined in some compilation units
and not defined in other compilation units.
The fix is to make Verifier a template class, with a boolean template
parameter replacing the "#ifdef" conditionals; to rename it as
VerifierTemplate; and then to use "#ifdef" only for a "using" declaration
that defines the original name Verifier an an alias for the instantiated
template. In this way, even if FLATBUFFERS_TRACK_VERIFIER_BUFFER_SIZE is
defined in some compilation units and not in others, as long as clients
only reference flatbuffers::Verifier in .cc files, not header files, there
will be no ODR violation, since the only part whose definition varies is the
"using" declaration, which does not have external linkage.
There is still some possibility of clients creating ODR violations
if the client header files (rather than .cc files) reference
flatbuffers::Verifier. To avoid that, this change also deprecates
FLATBUFFERS_TRACK_VERIFIER_BUFFER_SIZE, and instead introduces
flatbuffers::SizeVerifier as a public name for the template instance with
the boolean parameter set to true, so that clients don't need to define
the macro at all.
* Reproduce the error in a unit test
Reproduces #8200
* Overload KeyCompareWithValue to work for string-like objects
This fixes#8200.
* Extra tests
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
