* Add --ts-undefined-for-optionals command line option
# Details
- Fixes#7656
- Added a new `--ts-undefined-for-optionals` command line option for `flatc`.
- If enabled, generated TypeScript code uses `undefined` for optional fields rather than `null`.
* Also add TS generated test files
* Run `sh scripts/clang-format-git.sh`
* also add tests/ts/lalala-options.ts to the repo
* move new tests to tests/ts/optional_values dir
* add tests/ts/optional_values/optional_values_generated.cjs to the repo
* reuse existing optional_scalars.fbs and add new test
* add comma
* sh scripts/clang-format-git.sh
* remove comma
* sh scripts/clang-format-git.sh
* trying things
* sh scripts/clang-format-git.sh
* done
* address feedback
* sh scripts/clang-format-git.sh
* run `sh scripts/clang-format-git.sh`
* remove uneeded `eslint-disable @typescript-eslint/no-namespace` line
---------
Co-authored-by: José Luis Millán <jmillan@aliax.net>
* flatbuffers Rust reflection: replace num with num-traits
num crate is a wrapper over num-traits and a few other crates, that
reexports the APIs from all of them. We only need num-traits.
Signed-off-by: Marcin Radomski <dextero@google.com>
* Rust reflection: drop dependency on stdint crate
We only use it to get intmax_t for deriving alignment, which is an alias
for `core::ffi::c_long` [1]. We can use that directly instead.
[1] https://docs.rs/stdint/1.0.0/stdint/type.intmax_t.html
Signed-off-by: Marcin Radomski <dextero@google.com>
* Rust reflection: drop dependency on escape_string crate
It's used to format a string used for debugging only, so we might as
well use the builtin Debug representation of a string.
Signed-off-by: Marcin Radomski <dextero@google.com>
* Rust codegen: add derives on generated bitflags
Otherwise it limits the use of structs generated for reflection.fbs
in Rust reflection API.
Signed-off-by: Marcin Radomski <dextero@google.com>
* Rust flatbuffers: update bitflags dependency to 2.8
Signed-off-by: Marcin Radomski <dextero@google.com>
* Rust codegen: use bitflags v2 API for converting from bits
from_bits_unchecked was replaced with safe from_bits_retain.
Signed-off-by: Marcin Radomski <dextero@google.com>
* Regenerate Rust code after idl change
Signed-off-by: Marcin Radomski <dextero@google.com>
* Regenerate reflection_generated.rs
With flatc --rust ../../../reflection/reflection.fbs
Signed-off-by: Marcin Radomski <dextero@google.com>
* ts/BUILD.bazel: add missing import
Found by Buildifire presubmit:
Function "sh_binary" is not global anymore and needs to be loaded from
"@rules_shell//shell:sh_binary.bzl".
Signed-off-by: Marcin Radomski <dextero@google.com>
* Update expected value in generated_code_debug_prints_correctly test
In bitflags v2, the debug string representation of enum values is
different than it was in v1:
Blue -> Color(Blue)
(empty) -> LongEnum(0x0)
This change adjusts the expected test value.
Signed-off-by: Marcin Radomski <dextero@google.com>
* Fix tests build on Swift 5.8
grpc-swift 1.4.1 depends on swift-nio-ssl 2.14.0+ [1]. swift-nio-ssl 2.29.1
published on 2025-01-30, introduced some code [2] that uses a "switch
expression syntax" supported since Swift 5.9 [3]. Attempts to compile it with
Swift 5.8 cause build errors.
swift-nio-ssl project doesn't seem to support Swift 5.8. A commit from
2024-10-29 removes a "deprecated reference to a Swift 5.8 pipeline" [4].
swift-nio-ssl 2.29.0 is the last version that can be compiled with Swift
5.8. This commit pins it to that exact version.
[1] 66e27d7e84/Package.swift (L33)
[2] 3cb4d5ad12 (diff-bc1db1321ff689c2819245dcce1a3080554f0fc13f81b8d326c97e7d42717c8fR54)
[3] https://github.com/swiftlang/swift-evolution/blob/main/proposals/0380-if-switch-expressions.md
[4] 8a6b89d9a4
---------
Signed-off-by: Marcin Radomski <dextero@google.com>
Co-authored-by: Marcin Radomski <dextero@google.com>
* Add missing file to filegroup for bazel integration tests
Fixup after a9257b6963.
* Align versions in bazel_respository_test_dir with root
* Update XCode version to 15.2
This is the oldest available version.
* Add WORKSPACE.bzlmod
* Add support for Bazel 7
* Add support for Bazel 8 in CI
* Restrict visibility of exported file
* Align npm_translate_lock attrs
* Remove defs_bzl_filename attr
* Align root_package with pnpm-lock.yaml location
Use a symlink to avoid copying the file.
When flatbuffers is being used from a project that has no use for
JavaScript, users encounter an error similar to the following:
ERROR: Skipping '@com_github_google_flatbuffers//:flatbuffers': error loading package '@com_github_google_flatbuffers//': Unable to find package for @npm//:defs.bzl: The repository '@npm' could not be resolved: Repository '@npm' is not defined.
WARNING: Target pattern parsing failed.
ERROR: error loading package '@com_github_google_flatbuffers//': Unable to find package for @npm//:defs.bzl: The repository '@npm' could not be resolved: Repository '@npm' is not defined.
INFO: Elapsed time: 0.023s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
currently loading: @com_github_google_flatbuffers//
That's not ideal. Users that only care about C++ for example
shouldn't be forced to deal with rules_js and friends.
This patch attempts to fix that by moving the rules_js-specific things
into the `ts` and `tests/ts` directories. This should allow
non-JavaScript projects to ignore rules_js and friends completely.
Here I basically followed the `rules_foo` example from rules_js:
https://github.com/aspect-build/rules_js/tree/main/e2e/rules_foo
The idea is that flatbuffers has its own npm dependencies regardless
of what other projects may have. This means we should not force the
user to import flatbuffers's npm dependencies. The new
`ts/repositories.bzl` file is used by dependents to import
flatbuffers's dependencies. They can still import their own
dependencies. This cleanup allowed me to move all
JavaScript-specific stuff from the top-level directory into
subdirectories.
There should be no changes in this patch in terms of functionality.
It's just a refactor of the rules_js call sites. Users will have to
add a call to the function in `ts/repositories.bzl` in their own
`WORKSPACE` file. They can use
`tests/ts/bazel_repository_test/WORKSPACE` as an example.
Co-authored-by: Derek Bailey <derekbailey@google.com>
`buildifier` was complaining as follows:
#### :bazel: buildifier: found 2 lint issues in your WORKSPACE, BUILD and *.bzl files
<pre><code>tests/ts/bazel_repository_test_dir/BUILD:3:1: <a href="https://github.com/bazelbuild/buildtools/blob/master/WARNINGS.md#out-of-order-load">out-of-order-load</a>: Load statement is out of its lexicographical order.
ts/BUILD.bazel:2:1: <a href="https://github.com/bazelbuild/buildtools/blob/master/WARNINGS.md#out-of-order-load">out-of-order-load</a>: Load statement is out of its lexicographical order.</pre></code>
This can be fixed locally like so:
$ buildifier -lint fix $(git ls-files | grep -e '/BUILD.bazel$' -e '/BUILD$' -e '\<WORKSPACE$')
I also took this opportunity to fix one of the filenames.
I accidentally introduced these errors in #8078.
* Migrate from rules_nodejs to rules_js/rules_ts (take 2)
This is the second version of patch #7923. The first version got
reverted because bazel query was failing:
$ bazel --nosystem_rc --nohome_rc query tests(set('//...')) except tests(attr("tags", "manual", set('//...')))
ERROR: Traceback (most recent call last):
File "/workdir/tests/ts/bazel_repository_test_dir/BUILD", line 6, column 22, in <toplevel>
npm_link_all_packages(name = "node_modules")
File "/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/ec321eb2cc2d0f8f91b676b6d4c66c29/external/npm/defs.bzl", line 188, column 13, in npm_link_all_packages
fail(msg)
Error in fail: The npm_link_all_packages() macro loaded from @npm//:defs.bzl and called in bazel package 'tests/ts/bazel_repository_test_dir' may only be called in bazel packages that correspond to the pnpm root package '' and pnpm workspace projects ''
This was happening because the `.bazelrc` file only added
`--deleted_packages` to the `build` command. We also need it for the
`query` command. This second version of the patch fixes that.
Original commit message:
This patch migrates the current use of rules_nodejs to the new rules_js.
rules_js is the intended replacement of rules_nodejs as per this note:
https://github.com/aspect-build/rules_js#relationship-to-rules_nodejs
> rules_js is an alternative to the build_bazel_rules_nodejs Bazel module
> and accompanying npm packages hosted in
> https://github.com/bazelbuild/rules_nodejs, which is now
> unmaintained. All users are recommended to use rules_js instead.
There are a few notable changes in this patch:
1. The `flatbuffer_ts_library` macro no longer accepts a `package_name`
attribute. This is because rules_js appears to manage the import
naming of dependencies via top-level `npm_link_package` targets.
Users will have to migrate.
2. I added a few more arguments to `flatbuffer_library_public()`. These
helped with exposing esbuild to `ts/compile_flat_file.sh`.
3. I pinned the version of `typescript` in `package.json` so that
rules_ts can download the exact same version. rules_ts doesn't know
what to do if the version isn't exact.
4. Since rules_js uses the pnpm locking mechanism, we now have a
`pnpm-lock.yaml` file instead of a yarn lock file.
4. I added bazel targets for a few of the existing tests in `tests/ts`.
They can be run with `bazel test //test/ts:all`. Since there is no
flexbuffers bazel target, I did not add a bazel target for the
corresponding test.
5. I added a separate workspace in `tests/ts/bazel_repository_test_dir/`
to validate that the flatbuffers code can be imported as an external
repository. You can run the test with
`bazel test //test/ts:bazel_repository_test`. For this to work, I
needed to expose a non-trivial chunk of the flatbuffers code to the
test. I achieved this through some recursive `distribution`
filegroups. This is inspired by rules_python's workspace tests.
I did not do anything special to validate that the `gen_reflections`
parameter works the same. This patch doesn't change anything about
the TypeScript generation.
As a side note: I am not an expert with rules_js. This patch is my
attempt based on my limited understanding of the rule set.
Fixes#7817
* Fix the query
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Start using pnpm
* Add @npm
* get more stuff set up
* Get the analysis phase passing.
* Get esbuild working?
* Get it compiling?
$ bazel build //tests/ts/...
* Try to get the test working
* test is passing
* Get the other tests working
* clarify comment
* clean up a bit
* Try to add another test
* Add another test
* clean up more
* remove unused reference
* Add e2e test
* Get more of the test working
* add lock file
* Get test working on its own
* Get e2e test passing
* fix infinite recursion
* Add comments
* clean up some more
* clean up more again
* Source typescript version from package.json
* run buildifier
* lint
* Fix unset `extra_env`
* Incorporate feedback
* run buildifier
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
* [TS/JS] Entry point per namespace
* Fix handling of outputpath and array_test
* Attempt to fix generate_code
* Fix cwd for ts in generate_code
* Attempt to fixup bazel and some docs
* Add --ts-flat-files to bazel build to get bundle
* Move to DEFAULT_FLATC_TS_ARGS
* Attempt to add esbuild
* Attempt to use npm instead
* Remove futile attempt to add esbuild
* Attempt to as bazel esbuild
* Shuffle
* Upgrade bazel deps
* Revert failed attempts to get bazel working
* Ignore flatc tests for now
* Add esbuild dependency
* `package.json` Include esbuild
* `WORKSPACE` Add fetching esbuild binary
* Update WORKSPACE
* Unfreeze Lockfile
* Update WORKSPACE
* Update BUILD.bazel
* Rework to suggest instead of running external bundler
* Add esbuild generation to test script
* Prelim bundle test
* Run test JavaScriptTest from flatbuffers 1.x
* Deps upgrade
* Clang format fix
* Revert bazel changes
* Fix newline
* Generate with type declarations
* Handle "empty" root namespace
* Adjust tests for typescript_keywords.ts
* Separate test procedure for old node resolution module output
* Fix rel path for root level re-exports
* Bazel support for esbuild-based flatc
Unfortunately, we lose typing information because the new esbuild method
of generating single files does not generate type information.
The method used here is a bit hack-ish because it relies on parsing the
console output of flatc to figure out what to do.
* Try to fix bazel build for when node isn't present on host
* Auto formatting fixes
* Fix missing generated code
Co-authored-by: Derek Bailey <derekbailey@google.com>
Co-authored-by: James Kuszmaul <jabukuszmaul+collab@gmail.com>
* [TS]: Fix vtable creation for consecutive required fileds (#7739)
* handle feedback
* comment the schema
* comment change in builder.ts
* [TS]: builder, Fix requiredField()
Verifty that the field is present in the vtable.
* restore monsterdata binary file
Co-authored-by: Derek Bailey <derekbailey@google.com>
The headline here is adding a flatbuffer_ts_library rule for generating
typescript code in bazel. This entails some non-trivial other changes,
but ideally none are user-visible.
In particular:
* Added a --ts-flat-file flag that generates a single *_generated.ts
file instead of separate files for each typescript type. This makes
bazel much happier.
* Import the bazel rules_nodejs stuff needed to support building
typescript in bazel
* Move flatbuffers.ts to index.ts because I wasn't sure how to make
bazel comprehend the "main" attribute of the package.json. Happy
to take another stab at figuring that out if really needed.
* Fix another couple keyword escaping spots in typescript...
* [ts] Builder incorrectly serializing empty strings
The builder was returning an offset of zero for empty strings. This is
leading to flatbuffers which fail verification in other languages, such
as Rust.
* tests expect 0 offset for null or undefined strings
* TS/ES6 modules spike iteration 1
* Initial modularized dasherized output
* Remove obsoleted parts and namespace wrapping
* Use _flatbuffers_ prefix
* First part of imports logic
* Second part of imports logic
* Fix TS/JS code removal mixup
* Alias imported symbols if same name from different namespaces and some fixes
* Use star import for bare imports
* Fix messed up string concat
* var to const and remove not needed semi
* Remove some cases of ns prefixing
* Add missing space
* Cleanups
* Completed initial import tracking logic
* Compilable output
* Adjust TypeScriptTest and dependents to work
* Use local flatbuffers package for tests
* Refactor away use of any
* Remove obsolete imported_fileset and reexport_map
* Still need any and fix JavaScriptTest.sh
* Fix test runs out of the box
* Temp add generated files
* TypeScriptTest replaces JavaScriptTest and cleanups
* Also remove reference to JavaScriptTest in TestAll.sh
* Remove old generated ts/js files
* Remove use of --js in generate_code scripts
* idl_gen_js_ts to idl_gen_ts and removal of js gen
* Remove obsoleted options
* Fix obsolete ts test detection
* Tweak ts compilation be as strict as possible
* Remove jsdoc type annotation generation
* Generated test ts files
* Fix search and replace messup
* Regenerated ts test output
* Use CharToLower
* Use normal for loop
* Rework namespacedir
* Revert "Rework namespacedir"
This reverts commit 6f4eb0104ceeb86011bb076ebca901138c48e068.
* Revert "Use normal for loop"
This reverts commit 676b2135bfaa1853dfbb06c92b5c16a0d81bb13a.
* Revert "Use CharToLower"
This reverts commit 2d08648d0d72d0af201fad80d54cdc76412b35e9.
* Again do rework but correct
* Avoid runtime cast
* Fix test runs
* Also add npm install to get tsc
* Bump node test versions
* for range to std for loop
* Clang format
* Missed one clang format
* Move accessor to later
* Attempt to make windows version of TypeScriptTest
* Want to see the output
* Try to get newer node at appveyor
* Style changes
