Commit Graph

32 Commits

Author SHA1 Message Date
lu-wang-g
5fc87f4c4b Enable --gen-onefile in Python (#6953)
* Enable --gen-onefile in Python

Made it possible to generate all python code in one file. Modified
py_test.py so that it can switch between the multi-file code and
the one-file code.

Updated PythonTest.sh and py_test.py so that the multi-file code
and the one-file code can be tested based on the same test code.

* Sync with google/flatbuffers

* Add --gen-onefile to generate_code.py
2021-12-09 15:47:09 -08:00
Richard A Hofer
13d9e35858 Better python generated code naming (#6336)
* Remove a lot of redundancy from the Python generated code.

Update tutorial to reflect new Python generated code.

* Add aliases for newly deprecated Python generated methods.

This should help with backwards compatibility.

* Fix incorrect names in deprecated comments.
2021-01-28 12:35:37 -08:00
Richard A Hofer
24dd85fd28 Generate code to encode and decode nested flatbuffers in Python. (#6354)
* Generate code to encode and decode nested flatbuffers in Python.

* Delete accidental trailing whitespace.

* Fully delete trailing whitespace.
2021-01-04 12:18:19 -08:00
Richard A Hofer
f8b203c9c4 Add vectorNumElements attribute to Builder for simpler vector creation. (#6328)
* Add vectorNumElements attribute to Builder for simpler vector creation.

This adds a default to EndVector which should simplify its use.

* Update tutorial to reflect new default arg in Python EndVector.

* Remove optional argument to Python EndVector.

* Add generated files.

* Unset Builder.vectorNumElems when not in use.
2020-12-14 12:00:28 -08:00
Joseph Pyott
17557f9131 [Python] Fixed issue #5499 (#5764) 2020-02-11 15:02:21 -08:00
Robert Winslow
7de668053f CI: New Docker tests for Python with numpy (#5677)
* New Docker tests for Python with numpy

* print numpy status in test suite
2019-12-24 02:14:55 -05:00
lu-wang-g
65f8703572 Flatbuffers Python Object API (#5616)
* Flatbuffers Python Object API

Implement the logic to generate the Python object API that can
unpack the data from a buf class into an object class, and pack
the data of an object class to a buf class.

* Fix the build issues

Remove unused parameters and replace auto in the for-loop statement
with std::string to make it compatible with VS2010.

* Fix the build issues.

* Add support for Array type

Added logic to handle Array type in Python Object API. Updated the
generated code accordingly.

* Fix the old style casting from int to char

* Fixed another conversion from int to char

* Fixed the import for typing

Importing typing may cause errors when a machine do not have the
moduel typing installed. This PR fixes the issue by guarding
"import typing" with the "try/except" statement.

* Fix issue of iterating the vector of import list

* Update the generated examples using generate_code.sh

* Fix the import order for typing

The import list was stored in unordered_set, so that each generated
codes may have different import order. Therefore, it failed in the
consistency test where two generated copies need to have exactly the
same apperance.

* Optimize unpack using numpy

Use numpy to unpack vector whenever it is possible to improve unpack
performance.

Also, added codegen command for Python specificly in generate_code.sh,
because --no-includes cannot be turn on for Python.

* Fix the import order

* Update generate_code.bat for windows accordingly

* Replace error message with pass

Avoid printing error message for every Python2 users about typing.
Replace it with pass.
2019-12-02 14:11:28 -08:00
Bharat Tak
e9d29c21a7 Python: Add forceDefaults opt to python Builder (#5564)
* Add forceDefaults opt to python Builder

* Add test functions for force_default option for python builder

* Simplify

* Add force default test for UOffsetTFlags
2019-10-17 15:25:05 -07:00
svenk177
4525c91be3 Fix incorrect padding in arrays of structs (Issue #5484) (#5491) 2019-08-23 10:46:47 -07:00
Joseph Pyott
9fa8245e81 Python: Added support for file_identifiers (#5123)
* Python: Added support for file_identifiers

* Added tests. Fixed file_identifier code.

* Python: Fixed excessive padding of file_identifier. Repaired tests.

* Python: Made code compatible with python2.7

* Python: Typo fix in @endcond

* whitespace normaalization

* Stylistic change from if(not X is None) to if(X is not None). Added comment to type string.

* Python: Added support for automatic code generation of file_identifiers. Added tests for said code generation.

* converted sprintf to snprintf

* Bugfix, added snprint deffinition for MSVC

* changed snprint deffinition for MSVC to sprint_s

* changed scanf to IntToStringHex. Renamed HasFileIdentifier to GenHasFileIdentifier.

* Added updated genereated code to commit

* Python bugix: flatc no longer produces HasFileIdentfier for shcemas with no file identifier

* Added tests to verify `MonsterBufferHasIdentifier` returns false on no Identifier

* Python: added tests for GetBufferIdentifier and BufferHasIdentifier
Python: removed unessasary parenethesis in if statements
Minor format changes.

* Python : correceted instances of keyword arguments being called as positional arguments

* fixed typos and grammer in comments

* Minor style fixes

* Indentation fix

* Equals style changes

* Python: Fixed Alignment Issues. Changed test code to test against atual output

* Ran make(forgot to run make last commit)

* Python: Style changes

* Style changes

* indentation and style

* readded CONTRIBUTING.md

* Formatting tweak

Mostly to make CI run again

* More formatting fixes

* More formatting fixes

* More formatting fixes

* More formatting fixes

* Formatting fix

* More formatting fixes

* Formatting

* ran generate_code.sh
2019-07-26 11:06:25 -07:00
Vladimir Glavnyy
e304f8c115 Extend the test of MonsterExtra (#5428)
* Extend the test of MonsterExtra

- Extend C++ test of MonsterExtra
- Add conversion of fbs/json NaNs to unsigned quiet-NaN
- Update documentation (cross-platform interoperability)

* Fix declaration of infinity constants int the test
2019-07-08 11:22:56 -07:00
svenk177
e635141d5b Add support for fixed-size arrays (#5313) 2019-06-18 00:15:13 +02:00
Malthe Borch
e47ca7ab40 Use a hash table to index existing vtables (#5314)
* Use a hash table to index existing vtables

This allows for quick deduplication even in situations where there
might be thousands of vtables due to 'combinatoric explosion'.

This fixes issue #5301.

* Refactor 0-offset trimming

* Improve deduplication benchmark

The routine now generates a set of realistic logical layouts and
uses a timer function that randomly picks a layout for each iteration.

The benchmark runs in batches of # of logical layouts = 1, 10, 100, 1000.

(Note that due to alignment, the actual number of vtables is usually slightly
higher.)
2019-05-06 15:00:02 -07:00
Wouter van Oortmerssen
600f3fbcd4 Reduced force_align in tests to 8, to work with --object-api.
More detail:
https://github.com/google/flatbuffers/projects/6#card-17401359

See also the .md changes in this commit.

Change-Id: Idfa68b2fd3bdb19979833737d3a3cf83ec1d6775
2019-02-07 14:51:04 -08:00
Vladimir Glavnyy
4fa4d36706 Add default NaN/Inf to C#, Java, Python (#5107)
* Add default NaN/Inf to C#, Java, Python

* Python NaN/Inf test added

* Remove MSVC2010/13 dependent code
2019-01-14 09:23:42 -08:00
Felix Frank
ad8b1e5dbd [Python] Fast serialization of numpy vectors (#4829)
[Python] Fast serialization of numpy vectors (#4829)
2018-10-15 17:07:08 -07:00
Tin Tvrtković
b10123ff63 Python: return None for missing strings. (#4733)
[BREAKING CHANGE] Python: return None for missing strings.
2018-05-10 10:34:20 -07:00
Robert Schmidtke
08cf50c54a Java/C#/Python prefixed size support (#4445)
* initial changes to support size prefixed buffers in Java

* add slice equivalent to CSharp ByteBuffer

* resolve TODO for slicing in CSharp code generation

* add newly generated Java and CSharp test sources

* fix typo in comment

* add FinishSizePrefixed methods to CSharp FlatBufferBuilder as well

* add option to allow writing the prefix as well

* generate size-prefixed monster binary as well

* extend JavaTest to test the size prefixed binary as well

* use constants for size prefix length

* fuse common code for getRootAs and getSizePrefixedRootAs

* pulled file identifier out of if

* add FinishSizePrefixed, GetSizePrefixedRootAs support for Python

* Revert "extend JavaTest to test the size prefixed binary as well"

This reverts commit 68be4420dd.

* Revert "generate size-prefixed monster binary as well"

This reverts commit 2939516fdf.

* fix ByteBuffer.cs Slice() method; add proper CSharp and Java tests

* fix unused parameter

* increment version number

* pulled out generated methods into separate utility class

* pulled out generated methods into separate utility class for Python

* fix indentation

* remove unnecessary comment

* fix newline and copyright

* add ByteBufferUtil to csproj compilation

* hide ByteBuffer's internal data; track offset into parent's array

* test unsafe versions as well; compile and run in debug mode

* clarify help text for size prefix

* move ByteBuffer slicing behavior to subclass

* fix protection levels

* add size prefix support for text generation

* add ByteBufferSlice to csproj compilation

* revert size prefix handling for nested buffers

* use duplicate instead of slice for removing size prefix

* remove slice subclass and use duplicate for removing size prefix

* remove slice specific tests

* remove superfluous command line option
2018-03-12 11:30:46 -07:00
Robert
86fb05d320 Python: CreateByteVector function in builder. (#4453) 2017-10-06 08:34:07 -07:00
Dariusz Ostolski
5e4739184f Fix issue #4389, if field is ommitted the return type is the same for… (#4444)
* Fix issue #4389, if field is ommitted the return type is the same for python generator

* Fix issue #4389, bytes is returned when field is omitted

* Fix issue #4389, added generated python code after generator modification
2017-10-05 14:07:42 -07:00
Wouter van Oortmerssen
ac1015e3c4 Trimmed vtables of trailing zeroes.
This is something the format supports, but none of the builders
were doing. Can save 10-20% on FlatBuffer binary size!

Also fixed the Go tests.

Change-Id: I616c56ce9bbcfcaee23aa24f0532fcb60b6a8c75
Tested: on Linux.
2017-08-24 09:35:54 -07:00
Kevin Rose
3282a84e30 [Python] (scalar) vector reading speedup via numpy (#4390)
* Add numpy accessor to python flatbuffers scalar vectors

* Update python tests to test numpy vector accessor

* Update appveyor CI to run Python tests, save generated code as artifact

* Update example generated python code

* Add numpy info to python usage docs

* Update test schema and python tests w/ multi-byte vector

* did not mean to push profiling code

* adding float64 numpy tests
2017-08-01 08:34:00 -07:00
Robert
a56c6e5195 Merge pull request #363 from danring/fix-341
Emit GetRootAs methods for all types in Go and Python
2016-07-22 15:12:35 -07:00
rw
fd43d3709e Fix test using compat.binary_types (from March). 2016-06-18 01:12:56 -07:00
Dan Ring
4832238708 Add tests for GetRootAs* in Go and Python 2016-06-14 18:41:33 -04:00
Wouter van Oortmerssen
0afac8f75a Merge branch 'master' of https://github.com/google/flatbuffers 2016-01-20 13:30:09 -08:00
Wouter van Oortmerssen
519dcdd0b2 Fixed Python unit test breakage.
Test was testing for a field to be None, even though it is populated
in one particular instance.

Change-Id: Id19848d01498d4c0a684efdb2bf49ff12c5b6758
Tested: on Linux.
2016-01-20 13:04:03 -08:00
Faizan Rashid
af23683311 [BUG FIX] [MINOR] Fix encoding with unicode characters.
When passing a unicode string to builder.CreateString, the
default encoding assumed all characters can be encoded
using ascii. Added a fix so a user can specify the
encoding and how to handle errors when creating strings.
2015-12-31 09:45:09 +05:00
rw
3232727ace Python: Improve Builder user interface.
+ Add state to the Builder object to track if we are inside a table,
  and if we are finished building the buffer.
+ Use this data to check that a buffer is being built correctly.
+ Raise an exception if a buffer is not being built correctly.
+ Test that the exceptions happen as expected.

Based on d236dea.
2015-11-11 19:43:53 -08:00
tguo-aa
d9db48257b Add self.assertNotNested() in CreateString
And also add a test case.
If you try to nest CreateString you will get a clear exception.
2015-09-29 08:34:10 +08:00
rw
2746aabcf1 fix string catenation 2015-05-13 10:50:47 -07:00
rw
48dfc69ee6 Port FlatBuffers to Python.
Implement code generation and self-contained runtime library for Python.

The test suite verifies:
  - Correctness of generated Python code by comparing output to that of
    the other language ports.
  - The exact bytes in the Builder buffer during many scenarios.
  - Vtable deduplication correctness.
  - Edge cases for table construction, via a fuzzer derived from the Go
    implementation.
  - All code is simultaneously valid in Python 2.6, 2.7, and 3.4.

The test suite includes benchmarks for:
  - Building 'gold' data.
  - Parsing 'gold' data.
  - Deduplicating vtables.

All tests pass on this author's system for the following Python
implementations:
  - CPython 2.6.7
  - CPython 2.7.8
  - CPython 3.4.2
  - PyPy 2.5.0 (CPython 2.7.8 compatible)
2015-05-12 15:40:29 -07:00