Commit Graph

196 Commits

Author SHA1 Message Date
Max Burke
a397dd7e8c Optionally generate Python type annotations (#7858)
* optionally generate type prefixes and suffixes for python code

* fix codegen error when qualified name is empty

* WIP: Python typing

* more progress towards python typing

* Further iterate on Python generated code typing

* clang-format

* Regenerate code

* add documentation for Python type annotations option

* generate code with Python type annotations

* handle forward references

* clang-format
2023-04-28 09:38:29 -07:00
Max Burke
56ecc1f548 Optionally generate type prefixes and suffixes for python code (#7857)
* optionally generate type prefixes and suffixes for python code

* fix codegen error when qualified name is empty

* generated code updated
2023-04-25 21:38:16 -07:00
Khanh Nguyen
0916f1c87e Add a FileWriter interface (#7821)
* Add a FileWriter interface

* Change interface

* Provide 2 impl for File interface: FileManager & FileNameManager

* Update

* update

* Update

* Add file_writer file

* Update

* Format files

* Update based on review

* Update

* Format bzl file

* Add LoadFile function

* Format

---------

Co-authored-by: Derek Bailey <derekbailey@google.com>
2023-04-06 01:49:29 +00:00
Ben Beasley
d4d355d883 Fix help output for --java-checkerframework (#7854) 2023-03-09 06:53:37 +00:00
Chuck Atkins
b90cc35a10 Add a --java-package-prefix option to flatc (#7848)
Co-authored-by: Derek Bailey <derekbailey@google.com>
2023-03-02 21:52:03 -08:00
Khanh Nguyen
4a34cd70dc Add Code Generator for idl_gen_fbs to parse .proto files (#7832)
* Add code generator for proto files

* Update

* Add --proto to script

* Remove cmt

* Move proto parsing logic into else block to share same set up logic for code_generator

* Remove IsValidCodeGenerator
2023-03-02 10:01:44 -08:00
Derek Bailey
535ead8d8c [Annotated Buffers] Improve efficiency (#7820)
* AnnotatedBinaryTextGen switch to ofstream instead of building giant string

* Add --annotate-sparse-vectors to reduce AFB size
2023-02-06 21:42:44 -08:00
Saman
f838017860 Parsing from proto should keep field ID. (fixes #7645) (#7655)
* Parsing from proto should keep field ID. (fixes #7645)

* Fix failed tests

* Fix windows warning

* Improve attribute generation in proto to fbs

* Check if id is used twice. fix Some clang-format problems

* Test if fake id can solve the test problem

* Validate proto file in proto -> fbs generation.

* Fix error messages

* Ignore id in union

* Add keep proto id for legacy and check gap flag have been added. Reserved id will be checked.

* Add needed flags

* unit tests

* fix fromat problem. fix comments and error messages.

* clear

* More unit tests

* Fix windows build

* Fix include problems

* Fake commit to invoke rebuild

* Fix buzel build

* Fix some issues

* Fix comments, fix return value and sort for android NDK

* Fix return type

* Break down big function

* Place todo

---------

Co-authored-by: Derek Bailey <derekbailey@google.com>
2023-02-01 11:17:35 -08:00
Khanh Nguyen
08ebd202e2 Final refactor for bfsb_generator* and text generator (#7806)
* Refactor BfbsGenerator to use CodeGenerator interface

* Update

* Refactor bfbs generator

* Refactor bfbs generator for lua and nim. Remove old code that use Generator
interface.

* Update import

* Update CMakeLists

* Update BUILD file

* Update BUILD file for src

* Remove from Android CMakeLists and add error message

* Update

* Add generate root file function to Code Generator interface

* Update

* Update

* Minor format fix
2023-01-31 09:35:34 -08:00
Khanh Nguyen
a105c26eca Refactor usage message (#7803)
* Update usage string formation

* Rework help message to use code generator interface

* update

* refactor
2023-01-28 13:17:36 -06:00
Björn Harrtell
ef76b5ece4 [TS/JS] Entry point per namespace and reworked 1.x compatible single file build (#7510)
* [TS/JS] Entry point per namespace

* Fix handling of outputpath and array_test

* Attempt to fix generate_code

* Fix cwd for ts in generate_code

* Attempt to fixup bazel and some docs

* Add --ts-flat-files to bazel build to get bundle

* Move to DEFAULT_FLATC_TS_ARGS

* Attempt to add esbuild

* Attempt to use npm instead

* Remove futile attempt to add esbuild

* Attempt to as bazel esbuild

* Shuffle

* Upgrade bazel deps

* Revert failed attempts to get bazel working

* Ignore flatc tests for now

* Add esbuild dependency

* `package.json` Include esbuild

* `WORKSPACE` Add fetching esbuild binary

* Update WORKSPACE

* Unfreeze Lockfile

* Update WORKSPACE

* Update BUILD.bazel

* Rework to suggest instead of running external bundler

* Add esbuild generation to test script

* Prelim bundle test

* Run test JavaScriptTest from flatbuffers 1.x

* Deps upgrade

* Clang format fix

* Revert bazel changes

* Fix newline

* Generate with type declarations

* Handle "empty" root namespace

* Adjust tests for typescript_keywords.ts

* Separate test procedure for old node resolution module output

* Fix rel path for root level re-exports

* Bazel support for esbuild-based flatc

Unfortunately, we lose typing information because the new esbuild method
of generating single files does not generate type information.

The method used here is a bit hack-ish because it relies on parsing the
console output of flatc to figure out what to do.

* Try to fix bazel build for when node isn't present on host

* Auto formatting fixes

* Fix missing generated code

Co-authored-by: Derek Bailey <derekbailey@google.com>
Co-authored-by: James Kuszmaul <jabukuszmaul+collab@gmail.com>
2023-01-21 12:22:22 -08:00
Derek Bailey
3b8644d32c Defined CodeGenerator Interface and implement C++ (#7771) 2023-01-08 15:01:33 -08:00
Derek Bailey
641fbe4658 Refactor FlatC to receive FlatCOptions (#7770)
* Refactor FlatC to receive `FlatCOptions`

* switch to c++11 unique_ptr
2023-01-08 13:29:00 -08:00
Chris
c2668fc0e2 Add ts-no-import-ext flag (#7748)
Co-authored-by: Derek Bailey <derekbailey@google.com>
2023-01-07 13:42:28 -08:00
jalitriver
920f3827a0 [C++] Add Command-Line Flag to Suppress MIN and MAX Enums (#7705)
Add the --no-minmax-values flag to prevent flatc from generating C++
enums with MIN and MAX enumerated values that otherwise would be set
to the inclusive lower and upper bound respectively of the enum.

This command-line flag is needed to avoid collisions when an enum that
is being ported to FlatBuffers already has a MIN or MAX enumerated
value.

It is also needed to work around a long-standing problem with
magic_enum that causes magic_enum to not see enumerated values that
are not unique.  For example, if FlatBuffers sets MIN = FOO and MAX =
BAR, MIN and FOO share the same underlying value so they are not
unique.  The same is true of MAX and BAR.  This prevents magic_enum
from converting FOO and BAR to and from strings as well as causing
magic_enum to return a count of enumerated values that is two fewer
than it should be.

Co-authored-by: Paul Serice <paul@serice.net>
2023-01-07 10:33:11 -08:00
Michael Le
e000458bb1 Add --go-module-name flag to support generating Go module compatible code (#7651)
* Add --go-module-name flag to support generating code for go modules

* Rename echo example folder

* Grammar

* Update readme for go-echo example

* Update readme for go-echo example

* Re-enable go modules after test is done
2022-11-22 14:28:01 -08:00
RishabhDeep Singh
879622fc57 Fixes #7345 to add the option to minify enums (#7566)
* Added cpp minified enums

* Update generated files

* remove initializer and fix comma

* Fix .gitignore

* Fix comma

* Add tests for cpp minify enums
2022-11-11 04:47:28 +00:00
Ben Beasley
dbc58ab77c Fix help output for --gen-includes (#7611)
Fixes the --help output documenting the deprecated --gen-includes
option, in which the option name contained a typo (--gen-inclues).
2022-11-08 18:16:17 +00:00
Ben Beasley
2facfeec7e Fix missing spaces in flatc help text (#7612)
* Fix error in --json-nested-bytes help text

Correct “bytesin” to “bytes in”

* Fix missing space in --no-leak-private-annotation help text
2022-11-08 17:59:48 +00:00
Even Rouault
a22434e2a1 Add missing #include <algorithm> for std::min/std::max uses, and #include <limits> for std::numeric_limits<> (#7624) 2022-11-08 09:36:35 -08:00
Paul Harris
f7c511957f Audit and fixups for GCC and Clang (#7212)
Added (for compiler versions that support it):
-Wmissing-declarations
-Wzero-as-null-pointer-constant

Then, fixes to problems identified by the extra warnings
Tested only on GCC 9.4.0

Adjusted the CPP code generator to output nullptr where appropriate,
to satisfy -Wzero-as-null-pointer-constant

Added a lot of 'static' declarations in front of functions,
to satisfy -Wmissing-declarations,
and wrap static function defs in anonymous namespaces.

There are advantages to both anonymous namespaces and static,
it seems that marking a function as static will not publish the name in
the symbol table at all, thus giving the linker less work to do.
2022-08-16 10:48:41 -07:00
Derek Bailey
a66de58af9 Partial support for --ts-flat-files and --gen-all (#7446)
* Partial support for --ts-flat-files and --gen-all

* Add generated code changes

* remove some debugging code left over

* missed grpc files
2022-08-15 16:11:45 -07:00
SF-Zhou
32328075d8 Fix error msg format when generate GRPC failed (#7350) 2022-08-05 22:12:15 -07:00
Derek Bailey
6e2791640e keep-prefix keeps relative pathing (#7394) 2022-07-26 15:11:54 -07:00
mustiikhalil
11a1988705 Started implementation for private flags in rust (#7269)
Adds flag to the cpp to fix bad annotations in all the languages

Addresses comments to the PR, and fixes logic for leaking annotations
2022-06-05 23:04:05 +02:00
mustiikhalil
832c618f5f Adds implementation flag for swift (#7202)
* Adds implementation flag for swift

Forces internal flag when using @_implementationOnly in swift

Fixes access type for verifier functions & encoder functions

Updates generated code

* Addresses PR comments & adds a code gen dir within the swift tests

* Adds test case for no-include

* Fixes code gen script

Removes prefix
2022-04-06 13:31:38 -07:00
Derek Bailey
e2be0c0b06 Handle root offset and root table vtable invalidation (#7177)
* Handle invalid root offset

* Handle vtable offset invalidation

* Added script generator. Add more cases through vtable ref table size

* review responses

* vtable offset validation

* Moved padding insertion to the end. Tests invalid field lenghts

* table offsets validated. Added type after field

* validate string length

* add todo

* invalid vector length

* invalid structs

* general cleanup

* reworded invalid offsets

* example for vector of structs

* invalid vector of tables

* invalid vector of strings

* invalid vector of scalars

* vector of unions

* validate union type value

* invalid vector union type values
2022-03-23 21:51:32 -07:00
Stefan F
824763b316 Typo in flatc options (warning-as-errors instead of warnings-as-errors) (#7180) 2022-03-21 18:08:14 -04:00
Derek Bailey
d3aeee32bb Annotated Flatbuffer Binary (#7174)
* Annotated Flatbuffer Binary

* Various fixes

* Handles old schema

* handle multiple missing fields

* minor edits

* bazel fix, spelling fix, ascii fix
2022-03-18 14:08:05 -07:00
James Kuszmaul
e5f331db99 [TS] Add single-file ts codegen & bazel rule for typescript (#7161)
The headline here is adding a flatbuffer_ts_library rule for generating
typescript code in bazel. This entails some non-trivial other changes,
but ideally none are user-visible.

In particular:
* Added a --ts-flat-file flag that generates a single *_generated.ts
  file instead of separate files for each typescript type. This makes
  bazel much happier.
* Import the bazel rules_nodejs stuff needed to support building
  typescript in bazel
* Move flatbuffers.ts to index.ts because I wasn't sure how to make
  bazel comprehend the "main" attribute of the package.json. Happy
  to take another stab at figuring that out if really needed.
* Fix another couple keyword escaping spots in typescript...
2022-03-10 10:08:13 -08:00
Derek Bailey
914344ea9b some minor help edits 2022-02-22 15:52:51 -08:00
Derek Bailey
70e2f49bff fixed string-json -> strict-json typo 2022-02-21 20:26:40 -08:00
Casper
615616cb55 Change Rust generated file defaults (#7101)
* Change Rust generated file defaults

After #6731, flatc changed its default behavior
for generating rust code to fix some importing issues.
This was a breaking change which invlidated the patch release,
`flatc 2.0.5` (#7081). This PR reverses the default so we can
release a patch update. However, does break Rust users who work at
HEAD.

* Bump flatc patch version (2.0.6)

Co-authored-by: Casper Neo <cneo@google.com>
2022-02-15 11:48:42 -05:00
Derek Bailey
c85fb690f4 Add Parsing Completed function for Rust (#7084) 2022-02-08 07:40:55 -08:00
Derek Bailey
bc366a7f9e Provide a short help text and default in error case (#6992)
* Provide a short help text and default in error case

* clean up short options a bit
2022-01-31 21:24:48 -08:00
Max Burke
1d294a31b8 Implement Serialize on generated rust types (#7022)
* fix for rust build

* Rust: Implement Serialize on generated types

For debugging convenience it is really handy to be able to dump out
types as another format (ie: json). For example, if we are logging a
type to a structured logging system, or even printing it out in a
structured way to the console.

Right now we handle this by shelling out to `flatc` which is not ideal;
by implementing Serialize on the generated types we can use any of the
Serializer-implementing packages for our structured debug output.

* clang-format

* Make the flatbuffers Rust crate only have an optional dependency on the `serde` packages.

* fix warning

* fix rust test build

* Oh yeah this needs to be initialized

* fix toml syntax

* code review feedback

* rebuild test data
2022-01-30 19:29:18 -05:00
Jon Simantov
a2d38fbb98 Add --warnings-as-errors to flatc compiler. (#7034)
* Add --warnings-as-errors to flatc compiler.

With this option set, flatc will return an error on parsing if
any warnings occurred.

* Add unit test for opts.warnings_as_errors.

* Change explicit option setting to default.
2022-01-25 14:01:16 -08:00
Wouter van Oortmerssen
b8aaccee82 Disable parsing of nested_flatbuffers as bytes by default
Parsing as bytes produces buffers that are unsafe to access unless passed thru a verifier,
whereas users could reasonably assume that any JSON parsed without errors is safe to access.
Users that still have legacy JSON files with such bytes in it will get a helpful error point them
to the option to turn on to have it work again.
2021-12-15 10:41:29 -08:00
Wouter van Oortmerssen
f8148b8dad Made FlexBuffers reuse tracker track types 2021-12-14 11:20:23 -08:00
Derek Bailey
3cbc120a08 Refractor Flatc Options (#6987)
* Moved error/warnings to bottom of std error

* Refactor flatc options

* pass program name to flatc

* extra -- in one option

* merge of upstream
2021-12-13 13:56:47 -08:00
Derek Bailey
b78002ff3e Validate C# json/object-api options (#6985) 2021-12-13 11:52:01 -08:00
Wouter van Oortmerssen
e367ca32ad Verifier for FlexBuffers (#6977)
* Verifier for FlexBuffers

* Verifier improvements & fuzzer
2021-12-10 14:59:08 -08:00
lu-wang-g
5fc87f4c4b Enable --gen-onefile in Python (#6953)
* Enable --gen-onefile in Python

Made it possible to generate all python code in one file. Modified
py_test.py so that it can switch between the multi-file code and
the one-file code.

Updated PythonTest.sh and py_test.py so that the multi-file code
and the one-file code can be tested based on the same test code.

* Sync with google/flatbuffers

* Add --gen-onefile to generate_code.py
2021-12-09 15:47:09 -08:00
Derek Bailey
061d61f3f8 Lua Generator using IR. (#6940)
* initial hack to get new Lua generator into flatc

* Starting to output enum defs for Lua

* Continue to work on table generation for Lua

* Finished basic getter access for Lua

* Added ability to get object by index

* Finished struct builder

* aliased reflection to r

* finish table builder generation

* register requiring files

* better generated header info

* Tying up loose ends

* Updated reflection to handle struct padding

* Addd type sizes to reflection

* Fixed some vector indirect issues

* Lua tests passed

* Misc cleanup

* ci fixes 1

* ci fixes 2

* renaming

* up size of type sizes

* manually ran clang-format-11 -i src/idl_parser.cpp

* fixed some windows casting

* remove stupid auto import

* more static_casting

* remove std

* update other build environments

* remove scoped enums

* replaced std::to_string with NumToString

* more win fixes

* more win fixes

* replaced old lua with new

* removed auto import

* review responses

* more style fixes

* refactor bfbs_gen_len to use code +=

* added consts

* fix lambda capture for windows

* remove unused return type
2021-12-02 21:29:19 -08:00
Alec Theriault
c0394bb09c Support --conform for vector of unions (#6883)
It seems like `--conform` already works for vectors of unions - there is
just a spurious check that prevents it from running. Fixes #6882

Also, if schemas do not conform, `flatc` no longer prints out the usage
(since the error is not due to bad usage). Fixes #6496
2021-11-01 11:26:36 -07:00
mustiikhalil
4c7a9c10d3 Adds JSON encoding to swift (#6874)
Updates generated code & removes unneeded protocol

Updates cpp to only generate code when flag is passed

Updates code gen script
2021-10-07 23:22:22 +02:00
Todd Hansen
6f7a57eaa0 [C#] Using 'global::' as qualifying_start_ within BaseGenerator (#6767)
* Update idl_gen_csharp.cpp

Change csharp generator to use "global::" for it's qualifying_start_ to disambiguate namespaces

* regenerate testing files

regenerate testing files

* Missed TableInC.cs

updated with global prefix

* Remove "global::" from qualifying_start_ for csharp generator

* C# global alias

* Tests and docs for --cs-global-alias

Add tests for --cs-global-alias to demonstrate use case for why it's needed.

Add documentation to Compiler.md

* Add also to help text

Add also to help text
2021-09-15 16:57:29 -05:00
Casper
0a3b017f09 Delete IDLOptions::lang (#6841)
Co-authored-by: Casper Neo <cneo@google.com>
2021-09-13 18:18:00 -04:00
Herr Frei
338331b55b Changes to support binary schema file loading and parsing (flatc) (#6823)
* Changes to support binary schema file loading and parsing (flatc)
- parser.reset() is also called if binary schema file is given
- code flow changed to not try to load a binary schema as textual schema

* Removed unneeded braces
2021-09-08 13:19:00 -07:00
Casper
12e341e4f8 Rework how Rust generated files are laid out (#6731)
* Refactored Rust Generated code into a module directory.

Each symbol will be generated into one file and then
imported into a module. This breaks the "out_dir"
pattern where some users would generate code in their
target/ directory. Also, these objects are best used
in their own module. It will be hard for users to share
their own module structure with flatbuffers namespaces.

There may be solutions to these drawbacks but that should
be discussed. I don't want to overengineer here.

* shadow error

* try fix .bat file

* fix .bat 2

* Restore accidentally deleted files

* Fixed some DONOTSUBMITs and made Rust outdir pattern use symlinks.

* fixed binary files

* git clang format

* make generated onefiles not public and fix .bat

* reduced diff with master in generate_code.sh

* fix shadowed variable

* add object api flags to .bat

* space

* Removed extern crate and extra &

* use statement

* more clippy lints

* format

* Undo extern crate -> use change, it actually matters to our tests

Co-authored-by: Casper Neo <cneo@google.com>
2021-07-22 19:02:28 -04:00