Commit Graph

1267 Commits

Author SHA1 Message Date
Derek Bailey
6ff9e90e7e FlatBuffers Version v24.3.7 2024-03-07 15:16:33 -08:00
Derek Bailey
7cd216c51e FlatBuffers Version v24.3.6 2024-03-07 06:52:51 +00:00
Björn Harrtell
129ef422e8 Target .NET Standard 2.1, .NET 6, .NET 8 only (#8184)
* Target .NET Standard 2.1, .NET 6, .NET 8 only

* Remove mono usage

* Fix bat name ref

* Up deps

* Up deps

* Reinstate build-windows

* Fix name

---------

Co-authored-by: Derek Bailey <derekbailey@google.com>
2023-12-19 14:43:55 -08:00
razvanalex
c0d16995a4 [TS/JS] Create byte vectors (#8185)
* Add createByteVector and use set in createString

* Add test for CreateByteVector

---------

Co-authored-by: Derek Bailey <derekbailey@google.com>
2023-12-19 06:42:21 +00:00
razvanalex
d47cd10d77 Add Clear() for python Builder (#8186)
Co-authored-by: Derek Bailey <derekbailey@google.com>
2023-12-17 22:18:11 -08:00
Björn Harrtell
6dfc59dfcc [TS/JS] Upgrade dependencies (#7996) 2023-12-13 16:57:46 -08:00
Aaron Barany
5ba80c24e0 Fix python type annotation output when not enabled (#7983)
Fixes #7971

Co-authored-by: Derek Bailey <derekbailey@google.com>
2023-11-29 09:58:21 -08:00
mustiikhalil
94ff188a3e [Swift] Migrating benchmarks to a newer lib. (#8168)
* Adds Nativestructs pointer push into ByteBuffer

Updates benchmarks & cleanup

Adds native struct vector tests

* Address PR comments

* Add more benchmarks

* Some benchmark cleanup

* Return back to 1M structs

* Tweak Structs benchmark

* Moves swift Benchmarks folder from /tests to /benchmarks

---------

Co-authored-by: Joakim Hassila <jocke@ordo.one>
2023-11-22 16:08:55 -08:00
Taylor Holliday
b08abbbbf6 [Swift] Push contiguous bytes (#8157)
* Add version of push which takes ContiguousBytes

* Ensure overloads aren't ambiguous

* Add version of createVector

* Add version of push which takes ContiguousBytes

* Ensure overloads aren't ambiguous

* Add version of createVector

* Add similar conditional to other use of ContiguousBytes

* Attempt CI fix

* Use memcpy instead of copyMemory

memcpy is faster in tests

* Add testContiguousBytes

* Add benchmarks

* Add version of createVector

* Add benchmarks

* Update push to copy memory

Since we don't care about endianness, we can simply memcpy the array of scalars

* Remove function and benchmarks

Since we don't care about endianness, a FixedWidthInteger version of createVector isn't needed

* Improve naming

* Add doc comment
2023-11-20 17:47:11 +01:00
mustiikhalil
0dc5a75dc0 Update testReadFromOtherLanguages test to also run on macos (#8044) 2023-11-19 07:18:49 +00:00
mustiikhalil
526c92546f Fixes forward offset verifiable objects within arrays (#8135)
Fixes failing tests & removes XCTestsManifests

Co-authored-by: Derek Bailey <derekbailey@google.com>
2023-11-18 21:14:55 +00:00
Derek Bailey
4354945727 Fix CI builds (#8161)
* Update build.yml

Upgrade to gcc 13 and clang 15

* switch to __is_trivially_copyable

* fix cmake issue and warning about sign comparison

* Use libc++ for C++23 on clang for now

* Use libc++ for C++23 on clang for now

* exclude clang+15 for C++13 builds
2023-11-18 00:19:03 -08:00
adsnaider
205285c35c [Rust] Add the Allocator trait for the builder API (#8106)
* Add an Allocator trait for FlatBufferBuilder

* Update rust generated code
2023-10-07 15:28:52 -04:00
Curt Hagenlocher
f4e23bf91e Fix verification for C# unions (#7970)
* Fix verification for unions

* Run scripts\generate_code.py

---------

Co-authored-by: Michael Le <michael.le647@gmail.com>
2023-09-30 18:54:48 -07:00
Björn Harrtell
c4211538bd TS: Add missing generate files (#8075)
Co-authored-by: Michael Le <michael.le647@gmail.com>
2023-09-30 10:51:32 -07:00
Artem Shilkin
bcb9ef1876 moved function to namespace (#8068) 2023-09-29 07:50:06 -07:00
Elior Schneider
c5441dc199 corrected a typo (#8063) 2023-09-29 07:40:02 -07:00
Pavlo Bashmakov
15f16f149e Fix misalignment of small structs in a Vector (C++) (#7883)
* Rare fix: kick fix test

* Rare fix: real fix

* Rare fix: separate test

* Rare fix: remove comments

* Rare fix: updates

* Rare fix: less

* Rare fix: size_t switch to uoffset_t

* Rare fix: swap exp/val

* Rare fix: add annotated before/after

* Rare fix: remove unnecessary changes

---------

Co-authored-by: Derek Bailey <derekbailey@google.com>
2023-09-28 22:53:50 -07:00
mustiikhalil
6f71b76e6f Allow reading unaligned buffers starting from swift 5.7, while keeping the creating aligned since its created by the library (#8061)
Addresses a warning on xcode 15 regarding copying a pointer without safeguards

Address PR comments regarding initializing buffers with flag

Adds a test case for copying unaligned buffers

Formatting code
2023-09-27 01:50:03 -04:00
Wouter van Oortmerssen
d3e8cb60a1 Lobster namespace change 2023-09-17 10:21:58 -07:00
jviel-beta
f625ff3330 [TS] Allows object API to set 0 for a null-default scalar. (#7864)
* Fixes bug where null default allows 0 as a value.

* Undoes one bit, adds null type allowance to addField<> default.

* Undoes IDE auto-format of imports.

* Adds generated changes after scripts/generate_code.py

* Removes unused symbol.

* Revert "Removes unused symbol."

This reverts commit 9cece17325.

---------

Co-authored-by: Derek Bailey <derekbailey@google.com>
2023-09-12 19:58:55 +02:00
Philipp Schrader
362dd663f8 Fix BUILD.bazel style violations (#8081)
`buildifier` was complaining as follows:

    #### :bazel: buildifier: found 2 lint issues in your WORKSPACE, BUILD and *.bzl files
    <pre><code>tests/ts/bazel_repository_test_dir/BUILD:3:1: <a href="https://github.com/bazelbuild/buildtools/blob/master/WARNINGS.md#out-of-order-load">out-of-order-load</a>: Load statement is out of its lexicographical order.
    ts/BUILD.bazel:2:1: <a href="https://github.com/bazelbuild/buildtools/blob/master/WARNINGS.md#out-of-order-load">out-of-order-load</a>: Load statement is out of its lexicographical order.</pre></code>

This can be fixed locally like so:

    $ buildifier -lint fix $(git ls-files | grep -e '/BUILD.bazel$' -e '/BUILD$' -e '\<WORKSPACE$')

I also took this opportunity to fix one of the filenames.

I accidentally introduced these errors in #8078.
2023-08-28 09:20:10 +02:00
OptoCloud
48da238920 Make eslint less pedantic (#8012)
* Disable eslint spam

* Generate TS example files
2023-07-10 15:48:16 +00:00
Wouter van Oortmerssen
23922e7eba FlexBuffers: JSON output supports indentation 2023-06-14 18:12:55 -07:00
Derek Bailey
0cc525b722 fix android typo 2023-05-31 19:31:47 +00:00
Derek Bailey
28861d1d7d various fixes (#7986) 2023-05-31 11:52:05 -07:00
Derek Bailey
96294e9f84 Add ForceVectorAlignment64 and test (#7977) 2023-05-26 11:49:06 -07:00
Paulo Pinheiro
b7856f8e27 Add Kotlin multiplatform support (#7969)
* [Kotlin] Introduction to Kotlin Multiplaform

The first implementation of the Kotlin code generation was made years
ago at the time Kotlin Multiplaform was not stable and Kotlin is mostly
used on JVM-based targets. For this reason the generated code uses java
based runtime.

That design decision comes with many drawbacks, leaving the code
generated more java-like and making it impossible to use more advanced
features of the Kotlin language.

In this change we are adding two parts: A pure, multi-plaform, Kotlin
runtime and a new code generator to accompany it.

* [Kotlin] Remove scalar sign cast from code generation

Now that we have a new runtime the accepts unsigned types, we don't
need to code generate casting back and from signed scalars. This
MR removes this from both code generations and adds the necessary
API to the runtime.

* [Kotlin] Use offset on public API to represent buffer position

Currently, kotlin was following Java's approach of representing objects,
vectors, tables as "Int" (the position of it in the buffer). This change
replaces naked Int with Offset<T>, offering a type-safe API. So,
instead of

fun Table.createTable(b: FlatBufferBuilder, subTable: Int)

We will have

fun Table.createTable(b: FlatBufferBuilder, subTable: Offset<SubTable>)

Making impossible to accidentally switch parameters.

The performance should be similar to use Int as we are using value
class for Offset and ArrayOffset, which most of the time translate to
Int in the bytecode.

* [Kotlin] Add builder for tables

Add builder constructor to make create of table more ergonomic.
For example the movie sample for the test set could be written as:

Movie.createMovie(fbb,
    mainCharacterType = Character_.MuLan,
    mainCharacter = att) {
    charactersType = charsType
    this.characters = characters
}

instead of:

Movie.startMovie(fbb)
Movie.addMainCharacterType(fbb, Character_.MuLan)
Movie.addMainCharacter(fbb, att as Offset<Any>)
Movie.addCharactersType(fbb, charsType)
Movie.addCharacters(fbb, charsVec)
Movie.endMovie(fbb)

* [Kotlin] Move enum types to value class

Moving to flatbuffer enums to value class adds type safety for parameters
with minimum to no performance impact.

* [Kotlin] Simplify Union parameters to avoid naked casting

Just a small change on the APIs that receive union as parameters,
creating a typealias UnionOffset to avoid using Offset<Any>. To "convert"
an table offset to an union, one just call Offset.toUnion().

* [Kotlin] Apply clang-format on kotlin code generators

* [Kotlin] Update kotlin generator to follow official naming conventions

Updating directory, package and enum naming to follow Kotlin official
convention.

https://kotlinlang.org/docs/coding-conventions.html#naming-rules

* [Kotlin] Add fixes to improve performance

1 - Add benchmark comparing serialization between Java & Kotlin
2 - ReadWriteBuffer does not auto-grow (thus avoid check size in every op)
3 - Add specialized add functions on FlatBufferBuilder to avoid boxing
offsets.
4 - Remove a few Kotlin syntax sugar that generated performance penalties.

* [Kotlin] Remove builder from Kotlin KMP and add some optimizations
to avoid boxing of Offset classes

---------

Co-authored-by: Derek Bailey <derekbailey@google.com>
2023-05-26 11:00:33 -07:00
Derek Bailey
0100f6a577 FlatBuffers Version 23.5.26 (#7976) 2023-05-26 10:33:09 -07:00
Derek Bailey
b67f1ad6d0 don't emit flatbuffers include in bfbs generated output (#7968) 2023-05-17 21:13:16 -07:00
Chih-Hsuan Yen
a352bdbc34 Fix python tests (#7960)
* Don't generate types unless --python-typing specified

Fixes https://github.com/google/flatbuffers/issues/7944

* Fix incorrect import statements

Fixes https://github.com/google/flatbuffers/issues/7951

* Fix $PYTHONPATH in PythonTest.sh

Regressed from https://github.com/google/flatbuffers/pull/7529

* PythonTest: fail if something goes wrong

GitHub Actions runs `bash PythonTest.sh`, and thus failures were not
visible.

* Build flatc for Python tests

* Regenerate codes

---------

Co-authored-by: Derek Bailey <derekbailey@google.com>
2023-05-17 13:10:51 -07:00
sssooonnnggg
cb14043f82 fix(rust): fixed rust namer isses, resolve #7865 and ##7782 (#7964)
Co-authored-by: Derek Bailey <derekbailey@google.com>
2023-05-17 19:54:49 +02:00
sssooonnnggg
ea7cfcd591 chore: add rust windows test to CI, fix RustTest.bat (#7963)
Co-authored-by: Derek Bailey <derekbailey@google.com>
2023-05-16 16:06:41 -07:00
sssooonnnggg
b128b802d9 feat: Support union underlying type for TS/JS (#7961) 2023-05-15 13:18:49 -07:00
sssooonnnggg
1d3afb90c5 feat(C++): Support underlying_type for union (#7954)
* feat(C++): support underlying type for union

* chore: add conform checks for underlying type changes
2023-05-14 21:22:38 -07:00
Derek Bailey
05743591e1 remove and auto-generate arrays_test_generated.h 2023-05-12 16:44:28 -07:00
Derek Bailey
2bc2529245 remove and auto-generate alignment_test_generated.h 2023-05-12 16:26:21 -07:00
Wouter van Oortmerssen
950a71ab89 Rename GenerateText
to make it a compile-time breaking change, to alert any users to the new meaning of the return value
2023-05-11 18:14:27 -07:00
Derek Bailey
86486a1735 run scripts/clang-format-all.sh 2023-05-11 12:23:49 -07:00
Derek Bailey
c2f764c22b add GetSizePrefixedBufferLength() 2023-05-11 12:22:05 -07:00
Derek Bailey
e97ff95970 Change SizedPrefixed verifier to be <= provided size 2023-05-11 12:22:05 -07:00
Derek Bailey
33212657ae Change SizedPrefixed verifier to be <= provided size (#7957)
* Change SizedPrefixed verifier to be <= provided size

* add GetSizePrefixedBufferLength()
2023-05-11 12:21:50 -07:00
Derek Bailey
66e9d9823a fixed other occurenace of big buffer 2023-05-11 09:39:54 -07:00
Derek Bailey
9fc153a8f8 remove large buffer creation in test to speed it up 2023-05-11 09:34:50 -07:00
Philipp Schrader
c2bf810638 Fix //tests/ts:bazel_repository_test (#7952)
The test was not actually invoking the bazel that was downloaded with
the `http_file` rule. I failed to add `executable = True` to the
`http_file` call. This caused the test to ignore that bazel binary and
went to the next one on the system.

This patch fixes the issue by adding the missing attribute. Also, this
patch changes the check in the test to make sure that the downloaded
file is indeed executable.
2023-05-10 21:35:58 -07:00
Derek Bailey
10b79d87c1 removed extern code generation declarations preferring direct includes (#7948)
* removed extern code generation definitions, preferring direct includes

* add static to functions

* remove idl_gen_lua
2023-05-09 21:50:28 -07:00
Derek Bailey
76f5e9816a fix out-of-source builds with cmake (#7946) 2023-05-09 20:33:54 -07:00
Björn Harrtell
368428cb94 Upgrade package.json dependencies (#7933)
Co-authored-by: Derek Bailey <derekbailey@google.com>
2023-05-09 20:33:17 -07:00
Derek Bailey
72b56fd081 FlatBuffers Version 23.5.9 (#7945) 2023-05-09 09:33:30 -07:00
Derek Bailey
63b7b25289 FlatBuffers 64 for C++ (#7935)
* First working hack of adding 64-bit. Don't judge :)

* Made vector_downward work on 64 bit types

* vector_downward uses size_t, added offset64 to reflection

* cleaned up adding offset64 in parser

* Add C++ testing skeleton for 64-bit

* working test for CreateVector64

* working >2 GiB buffers

* support for large strings

* simplified CreateString<> to just provide the offset type

* generalize CreateVector template

* update test_64.afb due to upstream format change

* Added Vector64 type, which is just an alias for vector ATM

* Switch to Offset64 for Vector64

* Update for reflection bfbs output change

* Starting to add support for vector64 type in C++

* made a generic CreateVector that can handle different offsets and vector types

* Support for 32-vector with 64-addressing

* Vector64 basic builder + tests working

* basic support for json vector64 support

* renamed fields in test_64bit.fbs to better reflect their use

* working C++ vector64 builder

* Apply --annotate-sparse-vector to 64-bit tests

* Enable Vector64 for --annotate-sparse-vectors

* Merged from upstream

* Add `near_string` field for testing 32-bit offsets alongside

* keep track of where the 32-bit and 64-bit regions are for flatbufferbuilder

* move template<> outside class body for GCC

* update run.sh to build and run tests

* basic assertion for adding 64-bit offset at the wrong time

* started to separate `FlatBufferBuilder` into two classes, 1 64-bit aware, the other not

* add test for nested flatbuffer vector64, fix bug in alignment of big vectors

* fixed CreateDirect method by iterating by Offset64 first

* internal refactoring of flatbufferbuilder

* block not supported languages in the parser from using 64-bit

* evolution tests for adding a vector64 field

* conformity tests for adding/removing offset64 attributes

* ensure test is for a big buffer

* add parser error tests for `offset64` and `vector64` attributes

* add missing static that GCC only complains about

* remove stdint-uintn.h header that gets automatically added

* move 64-bit CalculateOffset internal

* fixed return size of EndVector

* various fixes on windows

* add SizeT to vector_downward

* minimze range of size changes in vector and builder

* reworked how tracking if 64-offsets are added

* Add ReturnT to EndVector

* small cleanups

* remove need for second Array definition

* combine IndirectHelpers into one definition

* started support for vector of struct

* Support for 32/64-vectors of structs + Offset64

* small cleanups

* add verification for vector64

* add sized prefix for 64-bit buffers

* add fuzzer for 64-bit

* add example of adding many vectors using a wrapper table

* run the new -bfbs-gen-embed logic on the 64-bit tests

* remove run.sh and fix cmakelist issue

* fixed bazel rules

* fixed some PR comments

* add 64-bit tests to cmakelist
2023-05-09 09:16:30 -07:00