BigfootDev/flatbuffers-bigfoot
1
0
Fork 0
forked from BigfootDev/flatbuffers
Code Pull Requests Activity

Documentation updates for Optional Scalars (#6014) (#6270)

Browse Source 
* Documentation updates for Optional Scalars

* Updated Support

* Reword stuff

* s/NULL/null

Co-authored-by: Casper Neo <cneo@google.com>
This commit is contained in:
Casper
2020-11-24 11:05:39 -08:00
committed by GitHub
parent c27bc2d76f
commit 338393f854
3 changed files with 100 additions and 78 deletions
Download Patch File Download Diff File Expand all files Collapse all files

102
docs/source/Schemas.md
View File

@@ -39,16 +39,16 @@ first:
### Tables
Tables are the main way of defining objects in FlatBuffers, and consist
of a name (here `Monster`) and a list of fields. Each field has a name,
a type, and optionally a default value (if omitted, it defaults to `0` /
`NULL`).
Tables are the main way of defining objects in FlatBuffers, and consist of a
name (here `Monster`) and a list of fields. Each field has a name, a type, and
optionally a default value. If the default value is not specified in the schema,
it will be `0` for scalar types, or `null` for other types. Some languages
support setting a scalar's default to `null`. This makes the scalar optional.
Each field is optional: It does not have to appear in the wire
representation, and you can choose to omit fields for each individual
object. As a result, you have the flexibility to add fields without fear of
bloating your data. This design is also FlatBuffer's mechanism for forward
and backwards compatibility. Note that:
Fields do not have to appear in the wire representation, and you can choose
to omit fields when constructing an object. You have the flexibility to add
fields without fear of bloating your data. This design is also FlatBuffer's
mechanism for forward and backwards compatibility. Note that:
- You can add new fields in the schema ONLY at the end of a table
definition. Older data will still
@@ -135,24 +135,35 @@ Both representations are binary equivalent.
Arrays are currently only supported in a `struct`.
### (Default) Values
### Default, Optional and Required Values
Values are a sequence of digits. Values may be optionally followed by a decimal
point (`.`) and more digits, for float constants, or optionally prefixed by
a `-`. Floats may also be in scientific notation; optionally ending with an `e`
or `E`, followed by a `+` or `-` and more digits.
There are three, mutually exclusive, reactions to the non-presence of a table's
field in the binary data:
1. Default valued fields will return the default value (as defined in the schema).
2. Optional valued fields will return some form of `null` depending on the
local language. (In a sense, `null` is the default value).
3. Required fields will cause an error. Flatbuffer verifiers would
consider the whole buffer invalid. See the `required` tag below.
When writing a schema, values are a sequence of digits. Values may be optionally
followed by a decimal point (`.`) and more digits, for float constants, or
optionally prefixed by a `-`. Floats may also be in scientific notation;
optionally ending with an `e` or `E`, followed by a `+` or `-` and more digits.
Values can also be the keyword `null`.
Only scalar values can have defaults, non-scalar (string/vector/table) fields
default to `NULL` when not present.
default to `null` when not present.
You generally do not want to change default values after they're initially
defined. Fields that have the default value are not actually stored in the
serialized data (see also Gotchas below) but are generated in code,
so when you change the default, you'd
now get a different value than from code generated from an older version of
the schema. There are situations, however, where this may be
desirable, especially if you can ensure a simultaneous rebuild of
all code.
serialized data (see also Gotchas below). Values explicitly written by code
generated by the old schema old version, if they happen to be the default, will
be read as a different value by code generated with the new schema. This is
slightly less bad when converting an optional scalar into a default valued
scalar since non-presence would not be overloaded with a previous default value.
There are situations, however, where this may be desirable, especially if you
can ensure a simultaneous rebuild of all code.
### Enums
@@ -325,18 +336,16 @@ Current understood attributes:
deprecate a field that was previous required, old code may fail to validate
new data (when using the optional verifier).
- `required` (on a non-scalar table field): this field must always be set.
By default, all fields are optional, i.e. may be left out. This is
By default, fields do not need to be present in the binary. This is
desirable, as it helps with forwards/backwards compatibility, and
flexibility of data structures. It is also a burden on the reading code,
since for non-scalar fields it requires you to check against NULL and
take appropriate action. By specifying this field, you force code that
constructs FlatBuffers to ensure this field is initialized, so the reading
code may access it directly, without checking for NULL. If the constructing
code does not initialize this field, they will get an assert, and also
the verifier will fail on buffers that have missing required fields. Note
that if you add this attribute to an existing field, this will only be
valid if existing data always contains this field / existing code always
writes this field.
flexibility of data structures. By specifying this attribute, you make non-
presence in an error for both reader and writer. The reading code may access
the field directly, without checking for null. If the constructing code does
not initialize this field, they will get an assert, and also the verifier
will fail on buffers that have missing required fields. Both adding and
removing this attribute may be forwards/backwards incompatible as readers
will be unable read old or new data, respectively, unless the data happens to
always have the field set.
- `force_align: size` (on a struct): force the alignment of this struct
to be something higher than what it is naturally aligned to. Causes
these structs to be aligned to that amount inside a buffer, IF that
@@ -486,7 +495,7 @@ of related data structures is a union. Unions do have a cost however,
so an alternative to a union is to have a single table that has
all the fields of all the data structures you are trying to
represent, if they are relatively similar / share many fields.
Again, this is efficient because optional fields are cheap.
Again, this is efficient because non-present fields are cheap.
FlatBuffers supports the full range of integer sizes, so try to pick
the smallest size needed, rather than defaulting to int/long.
@@ -611,22 +620,25 @@ Most serialization formats (e.g. JSON or Protocol Buffers) make it very
explicit in the format whether a field is present in an object or not,
allowing you to use this as "extra" information.
In FlatBuffers, this also holds for everything except scalar values.
FlatBuffers will not write fields that are equal to their default value,
sometimes resulting in significant space savings. However, this also means we
cannot disambiguate the meaning of non-presence as "written default value" or
"not written at all". This only applies to scalar fields since only they support
default values. Unless otherwise specified, their default is 0.
FlatBuffers by default will not write fields that are equal to the default
value (for scalars), sometimes resulting in a significant space savings.
However, this also means testing whether a field is "present" is somewhat
meaningless, since it does not tell you if the field was actually written by
calling `add_field` style calls, unless you're only interested in this
information for non-default values.
If you care about the presence of scalars, most languages support "optional
scalars." You can set `null` as the default value in the schema. `null` is a
value that's outside of all types, so we will always write if `add_field` is
called. The generated field accessor should use the local language's canonical
optional type.
Some `FlatBufferBuilder` implementations have an option called `force_defaults`
that circumvents this behavior, and writes fields even if they are equal to
the default. You can then use `IsFieldPresent` to query this.
that circumvents this "not writing defaults" behavior you can then use
`IsFieldPresent` to query presence.
Another option that works in all languages is to wrap a scalar field in a
struct. This way it will return null if it is not present. The cool thing
is that structs don't take up any more space than the scalar they represent.
struct. This way it will return null if it is not present. This will be slightly
less ergonomic but structs don't take up any more space than the scalar they
represent.
[Interface Definition Language]: https://en.wikipedia.org/wiki/Interface_description_language

59
docs/source/Support.md
View File

@@ -18,31 +18,40 @@ In general:
NOTE: this table is a start, it needs to be extended.
Feature | C++ | Java | C# | Go | Python | JS | TS | C | PHP | Dart | Lobster | Rust | Swift
------------------------------ | ------ | ------ | ------ | ------ | ------ | --------- | --------- | ------ | --- | ------- | ------- | ------- | ------
Codegen for all basic features | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | WiP | Yes | Yes | Yes | Yes
JSON parsing | Yes | No | No | No | No | No | No | Yes | No | No | Yes | No | No
Simple mutation | Yes | Yes | Yes | Yes | No | No | No | No | No | No | No | No | Yes
Reflection | Yes | No | No | No | No | No | No | Basic | No | No | No | No | No
Buffer verifier | Yes | No | No | No | No | No | No | Yes | No | No | No | No | No
Testing: basic | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | ? | Yes | Yes | Yes | Yes
Testing: fuzz | Yes | No | No | Yes | Yes | No | No | No | ? | No | No | Yes | No
Performance: | Superb | Great | Great | Great | Ok | ? | ? | Superb | ? | ? | Great | Superb | Great
Platform: Windows | VS2010 | Yes | Yes | ? | ? | ? | Yes | VS2010 | ? | Yes | Yes | Yes | No
Platform: Linux | GCC282 | Yes | ? | Yes | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes | Yes
Platform: OS X | Xcode4 | ? | ? | ? | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes | Yes
Platform: Android | NDK10d | Yes | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ? | No
Platform: iOS | ? | ? | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ? | Yes
Engine: Unity | ? | ? | Yes | ? | ? | ? | ? | ? | ? | ? | No | ? | No
Primary authors (github) | aard* | aard* | ev*/js*| rw | rw | evanw/ev* | kr* | mik* | ch* | dnfield | aard* | rw | mi*/mz*
Feature | C++ | Java | C# | Go | Python | JS | TS | C | PHP | Dart | Lobster | Rust | Swift
------------------------------ | ------ | ----- | -------- | ----- | ------ | ----- | --- | ------ | --- | ------- | ------- | ------ | ------
Codegen for all basic features | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | WiP | Yes | Yes | Yes | Yes
JSON parsing | Yes | No | No | No | No | No | No | Yes | No | No | Yes | No | No
Simple mutation | Yes | Yes | Yes | Yes | No | No | No | No | No | No | No | No | Yes
Reflection | Yes | No | No | No | No | No | No | Basic | No | No | No | No | No
Buffer verifier | Yes | No | No | No | No | No | No | Yes | No | No | No | No | No
Native Object API | Yes | No | Yes | Yes | Yes | Yes | Yes | No | No | No | No | No | No
Optional Scalars | Yes | Yes | Yes | No | No | Yes | Yes | Yes | No | No | Yes | Yes | Yes
Flexbuffers | Yes | Yes | ? | ? | ? | ? | ? | ? | ? | ? | ? | Yes | ?
Testing: basic | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | ? | Yes | Yes | Yes | Yes
Testing: fuzz | Yes | No | No | Yes | Yes | No | No | No | ? | No | No | Yes | No
Performance: | Superb | Great | Great | Great | Ok | ? | ? | Superb | ? | ? | Great | Superb | Great
Platform: Windows | VS2010 | Yes | Yes | ? | ? | ? | Yes | VS2010 | ? | Yes | Yes | Yes | No
Platform: Linux | GCC282 | Yes | ? | Yes | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes | Yes
Platform: OS X | Xcode4 | ? | ? | ? | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes | Yes
Platform: Android | NDK10d | Yes | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ? | No
Platform: iOS | ? | ? | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ? | Yes
Engine: Unity | ? | ? | Yes | ? | ? | ? | ? | ? | ? | ? | No | ? | No
Primary authors (github) | aard | aard | ev/js/df | rw | rw | ew/ev | kr | mik | ch | df | aard | rw/cn | mi/mz
* aard = aardappel (previously: gwvo)
* ev = evolutional
* js = jonsimantov
* mik = mikkelfj
* ch = chobie
* kr = krojew
* mi = mustiikhalil
* mz = mzaks
Above | Github username
----- | -----------------------------
aard | aardappel (previously: gwvo)
ch | chobie
cn | caspern
df | dnfield
ev | evolutional
ew | evanw
js | jonsimantov
kr | krojew
mi | mustiikhalil
mik | mikkelfj
mz | mzaks
rw | rw
<br>

17
docs/source/Tutorial.md
View File

@@ -227,13 +227,14 @@ less memory and have faster lookup.
The `Monster` table is the main object in our FlatBuffer. This will be used as
the template to store our `orc` monster. We specify some default values for
fields, such as `mana:short = 150`. All unspecified fields will default to `0`
or `NULL`. Another thing to note is the line
`friendly:bool = false (deprecated);`. Since you cannot delete fields from a
`table` (to support backwards compatability), you can set fields as
`deprecated`, which will prevent the generation of accessors for this field in
the generated code. Be careful when using `deprecated`, however, as it may break
legacy code that used this accessor.
fields, such as `mana:short = 150`. If unspecified, scalar fields (like `int`,
`uint`, or `float`) will be given a default of `0` while strings and tables will
be given a default of `null`. Another thing to note is the line `friendly:bool =
false (deprecated);`. Since you cannot delete fields from a `table` (to support
backwards compatability), you can set fields as `deprecated`, which will prevent
the generation of accessors for this field in the generated code. Be careful
when using `deprecated`, however, as it may break legacy code that used this
accessor.
The `Weapon` table is a sub-table used within our FlatBuffer. It is
used twice: once within the `Monster` table and once within the `Equipment`
@@ -2702,7 +2703,7 @@ To access sub-objects, in the case of our `pos`, which is a `Vec3`:
`x`, `y`, and `z` will contain `1.0`, `2.0`, and `3.0`, respectively.
*Note: Had we not set `pos` during serialization, it would be a `NULL`-value.*
*Note: Had we not set `pos` during serialization, it would be a `null`-value.*
Similarly, we can access elements of the inventory `vector` by indexing it. You
can also iterate over the length of the array/vector representing the