This commit significantly improves the developer experience for the Python Object-Based API by overhauling the generated `__init__` method for `T`-suffixed classes.
Previously, `T` objects had to be instantiated with an empty constructor, and their fields had to be populated manually one by one. This was verbose and not idiomatic Python.
This change modifies the Python code generator (`GenInitialize`) to produce `__init__` methods that are:
1. **Keyword-Argument-Friendly**: The constructor now accepts all table/struct fields as keyword arguments, allowing for concise, single-line object creation.
2. **Fully Typed**: The signature of the `__init__` method is now annotated with Python type hints. This provides immediate benefits for static analysis tools (like Mypy) and IDEs, enabling better autocompletion and type checking.
3. **Correctly Optional**: The generator now correctly wraps types in `Optional[...]` if their default value is `None`. This applies to strings, vectors, and other nullable fields, ensuring strict type safety.
The new approach remains **fully backward-compatible**, as all arguments have default values. Existing code that uses the empty constructor will continue to work without modification.
#### Example of a Generated `__init__`
**Before:**
```python
class KeyValueT(object):
def __init__(self):
self.key = None # type: str
self.value = None # type: str
```
**After:**
```python
class KeyValueT(object):
def __init__(self, key: Optional[str] = None, value: Optional[str] = None):
self.key = key
self.value = value
```
#### Example of User Code
**Before:**
```python
# Old, verbose way
kv = KeyValueT()
kv.key = "instrument"
kv.value = "EUR/USD"
```
**After:**
```python
# New, Pythonic way
kv = KeyValueT(key="instrument", value="EUR/USD")
```
* optionally generate type prefixes and suffixes for python code
* fix codegen error when qualified name is empty
* WIP: Python typing
* more progress towards python typing
* Further iterate on Python generated code typing
* clang-format
* Regenerate code
* add documentation for Python type annotations option
* generate code with Python type annotations
* handle forward references
* clang-format
* Fix C/C++ Create<Type>Direct with sorted vectors
If a struct has a key the vector has to be sorted. To sort the vector
you can't use "const".
* Changes due to code review
* Improve code readability
* Add generate of JSON schema to string to lib
* option indent_step is supported
* Remove unused variables
* Fix break in test
* Fix style to be consistent with rest of the code
* [TS] Fix reserved words as arguments (#6955)
* [TS] Fix generation of reserved words in object api (#7106)
* [TS] Fix generation of object api
* [TS] Fix MakeCamel -> ConvertCase
* [C#] Fix collision of field name and type name
* [TS] Add test for struct of struct of struct
* Update generated files
* Add missing files
* [TS] Fix query of null/undefined fields in object api
* Add .Net verfier
* Add some fuzz tests for .Net
* Remove additional files
* Fix .net test
* Changes due to PR
* Fix generated files
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
* [Kotlin] Control the generation of reflection with --reflect-names.
Tested:
```
$ cmake -G "Unix Makefiles" && make && ./tests/flatc/main.py
...
KotlinTests.EnumValAttributes
[PASSED]
KotlinTests.EnumValAttributes_ReflectNames
[PASSED]
KotlinTests: 2 of 2 passsed
...
35 of 35 tests passed
```
* [Kotlin] Fix SampleBinary by converting Byte to UByte for ubyte fields.
* [Kotlin] Annotate all generated classes with kotlin.ExperimentalUnsignedTypes.
* feat: Fixed the issue with nested unions relying on InitFromBuf.
Problem: Issue #7569
Nested Unions were broken with the introduction of parsing buffers with an initial encoding offset.
Fix:
Revert the InitFromBuf method to the previous version and introduction of InitFromPackedBuf that allows
users to read types from packed buffers applying the offset automatically.
Test:
Added in TestNestedUnionTables to test the encoding and decoding ability using a nested table with a
union field.
* fix: Uncommented generate code command
* Bfbs Nim Generator
* Remove commented out tests
* add missing line to idl.h
* Commit python reflection changes
* Commit python reflection changes and move tests
* Remove default string addition
* Move tests to python file
* Fix element size check when element is table
* remove whitespace changes
* add element_type docs and commit further to namer and remove kkeep
* Bfbs Nim Generator
* Remove commented out tests
* add missing line to idl.h
* Commit python reflection changes
* Commit python reflection changes and move tests
* Remove default string addition
* Move tests to python file
* Fix element size check when element is table
* remove whitespace changes
* add element_type docs and commit further to namer and remove kkeep
* remove unused variables
* added tests to ci
* added tests to ci
* fixes
* Added reflection type Field, Variable to namer
* Moved reflection namer impl to bfbsnamer
* Remove whitespace at end of line
* Added nim to generated code
* Revert whitespace removal
Co-authored-by: Derek Bailey <derekbailey@google.com>
* feat: Added support for fixed sized arrays to python
Problem:
We encountered that using fixed arrays from C++ to python that python would
not read those arrays correctly due to no size information being encoded in the byte
array itself.
Fix:
Encode the sizes within the generated python file during code generation.
Specfically we add GetArrayAsNumpy to the python version of table, which takes as input
the length of the vector. When generating the python message files we include this length
from the VectorType().fixed_length.
* fix: added digit support for camel case to snake case conversion
Problem:
When including a number in the message name we would encounter cases where SnakeCase would
not add the appropirate breaks. e.g. Int32Stamped -> int_32stamped rather than int_32_stamped.
Fix:
To fix this we can add the condition that we check if the current character is not lower and
not a digit, that we check if the previous character was a lower or digit. If it was a lower
or digit then we add the break.
* fix: Array support for structures
Problem:
The python generated code for handling non-struct and struct vectors
and arrays was inconsistent. The calls to populate the obj api was
creating incorrect code.
Solution:
To fix this the VectorOfStruct and VectorOfNonStruct was rewritten
to handle array cases and bring the two methods in line which each
other.
Testing:
PythonTesting.sh now correctly runs and generates the code for
array_test.fbs.
Minor modifications were done on the test to use the new index
accessor for struct arrays and the script correctly sources the
location of the python code.
* chore: clang format changes
* Added code generated by scripts/generate_code. Modified GetArrayOfNonStruct slightly
to allow for function overloading allowing the user to get a single element of an array
or the whole array.
* Added new_line parameter to OffsetPrefix to allow optional new lines to be added.
This allows us to use the GenIndents method that automatically adds new lines instead.
* Reupload of generated code from the scripts/generate_code.py
* Removed new line in GetVectorAsNumpy.
* Updated Array lengths to use Length methods where possible. Added fallthrough for GenTypePointer. Added digit check to CamelToSnake method. Added and modified tests for ToSnakeCase and CamelToSnake.
* Added range check on the getter methods for vector and array types. Renamed == as is for python
* Change Rust generated file defaults
After #6731, flatc changed its default behavior
for generating rust code to fix some importing issues.
This was a breaking change which invlidated the patch release,
`flatc 2.0.5` (#7081). This PR reverses the default so we can
release a patch update. However, does break Rust users who work at
HEAD.
* Bump flatc patch version (2.0.6)
Co-authored-by: Casper Neo <cneo@google.com>
* initial hack to get new Lua generator into flatc
* Starting to output enum defs for Lua
* Continue to work on table generation for Lua
* Finished basic getter access for Lua
* Added ability to get object by index
* Finished struct builder
* aliased reflection to r
* finish table builder generation
* register requiring files
* better generated header info
* Tying up loose ends
* Updated reflection to handle struct padding
* Addd type sizes to reflection
* Fixed some vector indirect issues
* Lua tests passed
* Misc cleanup
* ci fixes 1
* ci fixes 2
* renaming
* up size of type sizes
* manually ran clang-format-11 -i src/idl_parser.cpp
* fixed some windows casting
* remove stupid auto import
* more static_casting
* remove std
* update other build environments
* remove scoped enums
* replaced std::to_string with NumToString
* more win fixes
* more win fixes
* replaced old lua with new
* removed auto import
* review responses
* more style fixes
* refactor bfbs_gen_len to use code +=
* added consts
* fix lambda capture for windows
* remove unused return type
* Keep methods with struct name and switch them to default
This PR can help fix the following two issues:
1): A set of simplified API (without struct name) was added in
https://github.com/google/flatbuffers/pull/6336. It causes name
conflict when merging all generated python file into a single one
(the primary usage senario in Google).
2): Flatbuffers 2.0 generates absolute import path, which may cause
name space conflicts. See more details in
https://github.com/google/flatbuffers/issues/5840.
The solution for both is to generate the merged Python code, similar
C++. The merged code will not contain the simplied API, but only the
method with struct name. For issue (1), it will mimic the exactly
usage pattern for Google internal. For issue (2), users can generate
the merged flatbuffer code, without worrying about the imports.
The above idea will be implemented in the following steps:
Step 1 (this PR): revert changes in https://github.com/google/flatbuffers/pull/6336
that set the simplified API as default. Remove statements that the
original API will be deprecated, and reset the original API as default.
Step 2 (the following PR): create a flag to generate the merged code.
The Simplified API will be removed from the merged code, otherwise it
will cause name conflict.
* Update the generated sample code
* Update the generated example code
* Reverst the changes of GetRootAs
* Update examples from grpc/example/generate.sh
* Remove a lot of redundancy from the Python generated code.
Update tutorial to reflect new Python generated code.
* Add aliases for newly deprecated Python generated methods.
This should help with backwards compatibility.
* Fix incorrect names in deprecated comments.
