* Fix generate_code script path
* [Python] Make StartVector public
Make StartVector vector public since it is already being used in
generated code
* [Python] Improve vector creation for Python API
Makes Python API for vectors cleaner like Rust and Swift
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
* [Python] Sync PythonTest.sh flags with generate_code.py
* [Python] Update generated code to latest flatc version for tests
* [Python] Fix test support for numpy newer than 2.0.0
* [Python] Remove unused variable
* [Python] Fix __eq__ for numpy arrays
* [Python] Run clang-format over the entire file
This commit significantly improves the developer experience for the Python Object-Based API by overhauling the generated `__init__` method for `T`-suffixed classes.
Previously, `T` objects had to be instantiated with an empty constructor, and their fields had to be populated manually one by one. This was verbose and not idiomatic Python.
This change modifies the Python code generator (`GenInitialize`) to produce `__init__` methods that are:
1. **Keyword-Argument-Friendly**: The constructor now accepts all table/struct fields as keyword arguments, allowing for concise, single-line object creation.
2. **Fully Typed**: The signature of the `__init__` method is now annotated with Python type hints. This provides immediate benefits for static analysis tools (like Mypy) and IDEs, enabling better autocompletion and type checking.
3. **Correctly Optional**: The generator now correctly wraps types in `Optional[...]` if their default value is `None`. This applies to strings, vectors, and other nullable fields, ensuring strict type safety.
The new approach remains **fully backward-compatible**, as all arguments have default values. Existing code that uses the empty constructor will continue to work without modification.
#### Example of a Generated `__init__`
**Before:**
```python
class KeyValueT(object):
def __init__(self):
self.key = None # type: str
self.value = None # type: str
```
**After:**
```python
class KeyValueT(object):
def __init__(self, key: Optional[str] = None, value: Optional[str] = None):
self.key = key
self.value = value
```
#### Example of User Code
**Before:**
```python
# Old, verbose way
kv = KeyValueT()
kv.key = "instrument"
kv.value = "EUR/USD"
```
**After:**
```python
# New, Pythonic way
kv = KeyValueT(key="instrument", value="EUR/USD")
```
* [Python] Use correct type for str with None
Otherwise mypy will correctly flag code like this
def __init__(self):
self.fooBar = None # type: Optional[str]
error: Incompatible types in assignment (expression has type "None", variable has type "str")
* [Python] Make list type optional as they can contain None
* optionally generate type prefixes and suffixes for python code
* fix codegen error when qualified name is empty
* WIP: Python typing
* more progress towards python typing
* Further iterate on Python generated code typing
* clang-format
* Regenerate code
* add documentation for Python type annotations option
* generate code with Python type annotations
* handle forward references
* clang-format
* feat: Fixed the issue with nested unions relying on InitFromBuf.
Problem: Issue #7569
Nested Unions were broken with the introduction of parsing buffers with an initial encoding offset.
Fix:
Revert the InitFromBuf method to the previous version and introduction of InitFromPackedBuf that allows
users to read types from packed buffers applying the offset automatically.
Test:
Added in TestNestedUnionTables to test the encoding and decoding ability using a nested table with a
union field.
* fix: Uncommented generate code command
* feat: Added support for fixed sized arrays to python
Problem:
We encountered that using fixed arrays from C++ to python that python would
not read those arrays correctly due to no size information being encoded in the byte
array itself.
Fix:
Encode the sizes within the generated python file during code generation.
Specfically we add GetArrayAsNumpy to the python version of table, which takes as input
the length of the vector. When generating the python message files we include this length
from the VectorType().fixed_length.
* fix: added digit support for camel case to snake case conversion
Problem:
When including a number in the message name we would encounter cases where SnakeCase would
not add the appropirate breaks. e.g. Int32Stamped -> int_32stamped rather than int_32_stamped.
Fix:
To fix this we can add the condition that we check if the current character is not lower and
not a digit, that we check if the previous character was a lower or digit. If it was a lower
or digit then we add the break.
* fix: Array support for structures
Problem:
The python generated code for handling non-struct and struct vectors
and arrays was inconsistent. The calls to populate the obj api was
creating incorrect code.
Solution:
To fix this the VectorOfStruct and VectorOfNonStruct was rewritten
to handle array cases and bring the two methods in line which each
other.
Testing:
PythonTesting.sh now correctly runs and generates the code for
array_test.fbs.
Minor modifications were done on the test to use the new index
accessor for struct arrays and the script correctly sources the
location of the python code.
* chore: clang format changes
* Added code generated by scripts/generate_code. Modified GetArrayOfNonStruct slightly
to allow for function overloading allowing the user to get a single element of an array
or the whole array.
* Added new_line parameter to OffsetPrefix to allow optional new lines to be added.
This allows us to use the GenIndents method that automatically adds new lines instead.
* Reupload of generated code from the scripts/generate_code.py
* Removed new line in GetVectorAsNumpy.
* Updated Array lengths to use Length methods where possible. Added fallthrough for GenTypePointer. Added digit check to CamelToSnake method. Added and modified tests for ToSnakeCase and CamelToSnake.
* Added range check on the getter methods for vector and array types. Renamed == as is for python
* Keep methods with struct name and switch them to default
This PR can help fix the following two issues:
1): A set of simplified API (without struct name) was added in
https://github.com/google/flatbuffers/pull/6336. It causes name
conflict when merging all generated python file into a single one
(the primary usage senario in Google).
2): Flatbuffers 2.0 generates absolute import path, which may cause
name space conflicts. See more details in
https://github.com/google/flatbuffers/issues/5840.
The solution for both is to generate the merged Python code, similar
C++. The merged code will not contain the simplied API, but only the
method with struct name. For issue (1), it will mimic the exactly
usage pattern for Google internal. For issue (2), users can generate
the merged flatbuffer code, without worrying about the imports.
The above idea will be implemented in the following steps:
Step 1 (this PR): revert changes in https://github.com/google/flatbuffers/pull/6336
that set the simplified API as default. Remove statements that the
original API will be deprecated, and reset the original API as default.
Step 2 (the following PR): create a flag to generate the merged code.
The Simplified API will be removed from the merged code, otherwise it
will cause name conflict.
* Update the generated sample code
* Update the generated example code
* Reverst the changes of GetRootAs
* Update examples from grpc/example/generate.sh
* Remove a lot of redundancy from the Python generated code.
Update tutorial to reflect new Python generated code.
* Add aliases for newly deprecated Python generated methods.
This should help with backwards compatibility.
* Fix incorrect names in deprecated comments.
* Add vectorNumElements attribute to Builder for simpler vector creation.
This adds a default to EndVector which should simplify its use.
* Update tutorial to reflect new default arg in Python EndVector.
* Remove optional argument to Python EndVector.
* Add generated files.
* Unset Builder.vectorNumElems when not in use.
* Flatbuffers Python Object API
Implement the logic to generate the Python object API that can
unpack the data from a buf class into an object class, and pack
the data of an object class to a buf class.
* Fix the build issues
Remove unused parameters and replace auto in the for-loop statement
with std::string to make it compatible with VS2010.
* Fix the build issues.
* Add support for Array type
Added logic to handle Array type in Python Object API. Updated the
generated code accordingly.
* Fix the old style casting from int to char
* Fixed another conversion from int to char
* Fixed the import for typing
Importing typing may cause errors when a machine do not have the
moduel typing installed. This PR fixes the issue by guarding
"import typing" with the "try/except" statement.
* Fix issue of iterating the vector of import list
* Update the generated examples using generate_code.sh
* Fix the import order for typing
The import list was stored in unordered_set, so that each generated
codes may have different import order. Therefore, it failed in the
consistency test where two generated copies need to have exactly the
same apperance.
* Optimize unpack using numpy
Use numpy to unpack vector whenever it is possible to improve unpack
performance.
Also, added codegen command for Python specificly in generate_code.sh,
because --no-includes cannot be turn on for Python.
* Fix the import order
* Update generate_code.bat for windows accordingly
* Replace error message with pass
Avoid printing error message for every Python2 users about typing.
Replace it with pass.
* Python: Added support for file_identifiers
* Added tests. Fixed file_identifier code.
* Python: Fixed excessive padding of file_identifier. Repaired tests.
* Python: Made code compatible with python2.7
* Python: Typo fix in @endcond
* whitespace normaalization
* Stylistic change from if(not X is None) to if(X is not None). Added comment to type string.
* Python: Added support for automatic code generation of file_identifiers. Added tests for said code generation.
* converted sprintf to snprintf
* Bugfix, added snprint deffinition for MSVC
* changed snprint deffinition for MSVC to sprint_s
* changed scanf to IntToStringHex. Renamed HasFileIdentifier to GenHasFileIdentifier.
* Added updated genereated code to commit
* Python bugix: flatc no longer produces HasFileIdentfier for shcemas with no file identifier
* Added tests to verify `MonsterBufferHasIdentifier` returns false on no Identifier
* Python: added tests for GetBufferIdentifier and BufferHasIdentifier
Python: removed unessasary parenethesis in if statements
Minor format changes.
* Python : correceted instances of keyword arguments being called as positional arguments
* fixed typos and grammer in comments
* Minor style fixes
* Indentation fix
* Equals style changes
* Python: Fixed Alignment Issues. Changed test code to test against atual output
* Ran make(forgot to run make last commit)
* Python: Style changes
* Style changes
* indentation and style
* readded CONTRIBUTING.md
* Formatting tweak
Mostly to make CI run again
* More formatting fixes
* More formatting fixes
* More formatting fixes
* More formatting fixes
* Formatting fix
* More formatting fixes
* Formatting
* ran generate_code.sh
* Extend the test of MonsterExtra
- Extend C++ test of MonsterExtra
- Add conversion of fbs/json NaNs to unsigned quiet-NaN
- Update documentation (cross-platform interoperability)
* Fix declaration of infinity constants int the test
