Commit Graph

3398 Commits

Author SHA1 Message Date
Wouter van Oortmerssen a98bff3ea1 Add simple libfuzzer based test.
This tests the verifier and the parser.

Tested: on Linux.
Bug: 27230459
Change-Id: I3417faba9385dc941199c57a08d8a50c2faec1aa
2016-05-25 17:25:40 -07:00
Wouter van Oortmerssen 06eb6e94e3 Merge pull request #3880 from Lakedaemon/IsEverythingGenerated
shared method that exits early if everything is generated
2016-05-25 10:38:56 -07:00
Lakedaemon 4a8801da34 shared method that exits early if everything is generated 2016-05-25 15:15:56 +02:00
James Swift b1e8be27a9 MSVC doesn't allow alignment on function params. This prevents the copy and thus avoids this issue https://msdn.microsoft.com/en-us/library/373ak2y1.aspx 2016-05-24 17:16:03 +02:00
Wouter van Oortmerssen b922a3c952 Merge "Uncomment the corpus attribute since b/28762769 is fixed, add more items to the corpus (taken from the ClusterFuzz-generated corpus)." into ub-games-master 2016-05-23 23:58:50 +00:00
Wouter van Oortmerssen b52826e7cc Merge "Add a fuzzer for flatbuffers, derived from the one used in Chromium: https://code.google.com/p/chromium/codesearch#chromium/src/testing/libfuzzer/fuzzers/flatbuffers_verifier_fuzzer.cc" into ub-games-master 2016-05-23 23:58:27 +00:00
Wouter van Oortmerssen 0ef72db7bf Merge pull request #3865 from bhamiltoncx/surrogate-pairs
Handle \u-escaped surrogate pairs correctly in IDL parser
2016-05-23 16:51:07 -07:00
Stewart Miles 505d0f2288 Merge "Removed reference to deprecated NDK toolchain version." into ub-games-master 2016-05-23 23:49:56 +00:00
Ben Gertzfield 6704b19db6 Handle \u-escaped surrogate pairs correctly in IDL parser 2016-05-23 16:38:55 -07:00
Wouter van Oortmerssen 64b91da9cf Uncomment the corpus attribute since b/28762769 is fixed,
add more items to the corpus (taken from the ClusterFuzz-generated corpus).

(mirrored from cl 122690857)

Change-Id: I98c7672169a9a27fe18bedae8501e6fb85a0681d
2016-05-23 16:37:57 -07:00
Wouter van Oortmerssen fe9787e58d Add a fuzzer for flatbuffers, derived from the one used in Chromium:
https://code.google.com/p/chromium/codesearch#chromium/src/testing/libfuzzer/fuzzers/flatbuffers_verifier_fuzzer.cc

(mirrored from cl 122676317)

Change-Id: I0074af435cf37f5e6368b040563c3733d9e02345
2016-05-23 16:35:20 -07:00
Stewart Miles d06e571b31 Removed reference to deprecated NDK toolchain version.
Change-Id: I8d39a66ca5642427619c60f349fa78917aec2455
2016-05-23 16:27:35 -07:00
Wouter van Oortmerssen 208c15f29d Merge branch 'master' of https://github.com/google/flatbuffers 2016-05-23 14:56:47 -07:00
Wouter van Oortmerssen a82dbb6e01 Fixed Google Groups link in documentation.
Change-Id: I64307ccd0a6f23a7c3860d602813e637415e7c91
2016-05-23 14:08:21 -07:00
Wouter van Oortmerssen a42a99029f Made .sh files all have same comment formatting.
Change-Id: Ie7cc268a8823aec1d06c8b7cbef54068d832de3a
2016-05-23 14:08:21 -07:00
Wouter van Oortmerssen 0c7b700895 Merge pull request #3878 from pjulien/no_alloc_on_cb
Remove Remaining FBB#createString Allocation
2016-05-23 13:55:21 -07:00
pjulien 1aa3ee2170 Clarify documentation based on review comments 2016-05-23 16:47:41 -04:00
Wouter van Oortmerssen 8c0d56d55a Removed BOM marker from some files.
Also removed some trailing whitespace.

Change-Id: I40bf4f776ca4d467b7ee6f2c9a7500c13700f9d5
2016-05-23 12:24:44 -07:00
Wouter van Oortmerssen c41a0453c1 Merge branch 'master' of https://github.com/google/flatbuffers 2016-05-23 10:59:47 -07:00
Wouter van Oortmerssen 1a161a8333 Added missing licenses to some sh/cs/py files.
Bug: https://github.com/google/flatbuffers/issues/3872
Change-Id: I5d551168e9bc925e867e5e4ddf5d809418fd44eb
Tested: on Linux.
2016-05-23 09:51:41 -07:00
pjulien 9d8ae1b0c3 If the user passes in a {{CharBuffer}} himser, reward him by
eliminating the last remaining allocation in string creation
2016-05-21 09:19:43 -04:00
Wouter van Oortmerssen 2de558057d Merge pull request #3867 from dreifachstein/master
Use the build output flatc to compile test schemas by default.
2016-05-20 13:19:52 -07:00
Wouter van Oortmerssen 0756caa42e Merge pull request #3866 from bhamiltoncx/fix-undefined-bit-shift
Fix undefined behavior when CheckBitsFit left-shifts 64-bit value by 64 bits
2016-05-20 13:10:52 -07:00
Wouter van Oortmerssen 99c1ddc792 Merge branch 'master' of https://github.com/google/flatbuffers 2016-05-18 18:01:17 -07:00
Wouter van Oortmerssen 69eef0a0ec Fixes verifier not catching odd vtable sizes.
Bug: 28762769
Change-Id: Iec370482f67c4585e97dd5175c1a4258ddba47fe
Tested: on Linux.
2016-05-18 18:00:25 -07:00
Wouter van Oortmerssen 42bd6447c0 Merge pull request #3877 from ozkuran/patch-1
Fixed Library Location in CppUsage.md
2016-05-18 10:33:55 -07:00
Mahmut Ali ÖZKURAN 747a8c628e Fixed Library Location in CppUsage.md
Fixed Library Location 
"https://github.com/google/flatbuffers/tree/master/include/flatbuffers"
in CppUsage.md
2016-05-18 12:02:32 +03:00
Martin Ankerl 023fec627e Added helpers to access objects while creating the flatbuffer.
GetObject and GetMutableObject are similar to GetRoot and GetMutableRoot,
and can be useful when wanting to access data that has just been created.
Unfortunately there is a danger in using these methods, as it is possible
that the buffer reallocates which will invalidate the pointers.
2016-05-03 17:20:14 +02:00
Xiaolei Yu 3a8bc309e2 Use the build output flatc to compile test schemas by default. 2016-05-01 11:15:53 +08:00
Ben Gertzfield 3400727fff Fix undefined behavior in CheckBitsFit bit-shift on size_t 2016-04-29 15:15:09 -07:00
Robert 77742a3fba Merge pull request #3806 from mediagraph/fix-create-string
Fix CreateString with already-encoded string or bytearray in Python 2.7.
2016-04-27 20:31:55 -07:00
Wouter van Oortmerssen 7cce55d8f0 Merge pull request #3854 from pjulien/3853
Fix for #3853
2016-04-27 18:12:15 -07:00
Wouter van Oortmerssen 933c195e51 Merge pull request #3843 from Lakedaemon/refactoring
Transition 1 (nice diffs) Go, C++ and General generators with class
2016-04-27 13:59:16 -07:00
Wouter van Oortmerssen d0898fd0c8 Merge pull request #3863 from pjulien/cseq
Can now use ``CharSequence`` of ``String``.  Works with
2016-04-27 13:50:04 -07:00
Lakedaemon 6765c19d45 memory leak fix + php/js/python transition to class based generator 2016-04-26 20:36:43 +02:00
pjulien 599f5e3915 Can now use `CharSequence of String`. Works with
1. [AsciiString]
(http://netty.io/4.1/api/io/netty/util/AsciiString.html)
2. [CharBuffer]
(https://docs.oracle.com/javase/8/docs/api/java/nio/CharBuffer.html)
3. [StringBuilder]
(https://docs.oracle.com/javase/8/docs/api/java/lang/StringBuilder.html)
2016-04-25 22:07:57 -04:00
Robert ba7204a7b7 Merge pull request #3859 from rw/go-zero-alloc-strings
Go: Fix heap allocation when reading a string.
2016-04-23 18:38:03 -07:00
rw e8ac0f293e Fix heap allocation when reading a string. 2016-04-23 12:32:43 -07:00
Wouter van Oortmerssen a0d1161feb Merge pull request #3850 from pjulien/3849
Fix for #3849
2016-04-22 11:42:14 -07:00
Wouter van Oortmerssen 828b2680c0 Merge branch 'master' of https://github.com/google/flatbuffers 2016-04-22 11:35:10 -07:00
Wouter van Oortmerssen c17cf022ea Fixed a verifier check that didn't return from its parent.
Change-Id: I1aef5fdba72de0c3bb8276c9f50fffa494f26e29
Tested: on Linux.
2016-04-22 11:31:35 -07:00
Wouter van Oortmerssen f6330ab8f1 Fixed SkipByteOrderMark advancing cursor_ too early.
Change-Id: Ie60f01d935ca6b4aa6ce0eab7598602ac0758342
Tested: on Linux.
2016-04-22 11:27:35 -07:00
pjulien 6908826f95 Fix for #3849
This avoids the following allocations over repeated use:

- A ``CharsetEncoder``
- A byte array to contain the contents of the string
- A wrapper, via ``ByteBuffer#wrap``, for the previously mentioned byte
  array

This also removes a copy of the temporary byte array.  This extra copy
is needed because ``String#getBytes`` returns a byte array and its
length must match the exact size of the contents.

This implementation requires that the builder retain a ``ByteBuffer``
and ``CharEncoder``.  This is considered slower only for users that
will only allocate a single string in their buffers or none at all.

The improvement is especially potent if the builder is constantly reused
by its caller.  This also eliminates the penalty for users that do not
use strings at all since the cost of allocating these fields is
now amortized.

The only allocation left in this code is a ``CharBuffer`` wrapper.  It's
possible to eliminate this allocation using an additional copy but need
further profiling to see which is the bigger problem.
2016-04-21 20:20:14 -04:00
Wouter van Oortmerssen fd542c71e3 Fixed float suffix not being added on implicit float defaults.
Change-Id: I8dcbb5e93836356511d411ef86452babd3dd3c0b
Tested: on Linux.
2016-04-20 12:06:32 -07:00
pjulien 9fb87f813b size the allocation to the required bytes 2016-04-18 21:43:28 -04:00
pjulien b0146b3d9a fix file permission 2016-04-18 20:19:56 -04:00
pjulien 286587d151 Fix for #3853
Removes the following allocations:
- ``CharsetDecoder`` is reused between calls
- ``CharBuffer#wrap`` removed in favor of heap
  based char buffer that is reused
- Temporary ``char[]``, an intermediate copy inside ``StringCoding``
- Another ``char[]``, this is needed because ``StringCoding`` uses
  a ``CharBuffer`` internally but returns a ``char[]``.  Extra
  characters need to be trimmed so this means yet another allocation
- Yet another ``char[]`` directly from ``__string`` for non-heap
  based buffers

Removes the following copies
- No copy is performed to trim the allocation since a ``CharBuffer``
  is used directly
- For non-heap based byte buffers, removes the copy that was
  previously done in the __string function

This does need to get the TLS entry which implies at least some
contention on the thread object table and a fence.
2016-04-18 20:04:00 -04:00
Wouter van Oortmerssen 6f751d5d26 Merge pull request #3848 from pjulien/3847
Fix for 3847
2016-04-18 15:54:34 -07:00
Wouter van Oortmerssen 4d7890c2c9 Fixed whole-number float default values missing a .0 suffix.
A previous commit that added "f" for C++/Java/C# would break
on gcc of constants like 3f, which are now output as 3.0f

Tested: on Linux

Change-Id: If9cabbe3c6d6948a5050b8b123bda9c06e181f52
2016-04-18 11:50:46 -07:00
pjulien fdfaf23361 3847 2016-04-16 20:09:28 -04:00