BigfootDev/flatbuffers
1
0
Fork 1
mirror of https://github.com/google/flatbuffers.git synced 2026-06-25 16:38:40 +00:00
Code Issues Packages Projects Releases Wiki Activity

Documentation updates for Optional Scalars (#6014) (#6270)

Browse Source 
* Documentation updates for Optional Scalars

* Updated Support

* Reword stuff

* s/NULL/null

Co-authored-by: Casper Neo <cneo@google.com>
This commit is contained in:
Casper
2020-11-24 11:05:39 -08:00
committed by GitHub
parent c27bc2d76f
commit 338393f854
3 changed files with 100 additions and 78 deletions
Download Patch File Download Diff File Expand all files Collapse all files

102
docs/source/Schemas.md
View File

@@ -39,16 +39,16 @@ first:
### Tables ### Tables
Tables are the main way of defining objects in FlatBuffers, and consist Tables are the main way of defining objects in FlatBuffers, and consist of a
of a name (here `Monster`) and a list of fields. Each field has a name, name (here `Monster`) and a list of fields. Each field has a name, a type, and
a type, and optionally a default value (if omitted, it defaults to `0` / optionally a default value. If the default value is not specified in the schema,
`NULL`). it will be `0` for scalar types, or `null` for other types. Some languages
support setting a scalar's default to `null`. This makes the scalar optional.
Each field is optional: It does not have to appear in the wire Fields do not have to appear in the wire representation, and you can choose
representation, and you can choose to omit fields for each individual to omit fields when constructing an object. You have the flexibility to add
object. As a result, you have the flexibility to add fields without fear of fields without fear of bloating your data. This design is also FlatBuffer's
bloating your data. This design is also FlatBuffer's mechanism for forward mechanism for forward and backwards compatibility. Note that:
and backwards compatibility. Note that:
- You can add new fields in the schema ONLY at the end of a table - You can add new fields in the schema ONLY at the end of a table
definition. Older data will still definition. Older data will still
@@ -135,24 +135,35 @@ Both representations are binary equivalent.
Arrays are currently only supported in a `struct`. Arrays are currently only supported in a `struct`.
### (Default) Values ### Default, Optional and Required Values
Values are a sequence of digits. Values may be optionally followed by a decimal There are three, mutually exclusive, reactions to the non-presence of a table's
point (`.`) and more digits, for float constants, or optionally prefixed by field in the binary data:
a `-`. Floats may also be in scientific notation; optionally ending with an `e`
or `E`, followed by a `+` or `-` and more digits. 1. Default valued fields will return the default value (as defined in the schema).
2. Optional valued fields will return some form of `null` depending on the
local language. (In a sense, `null` is the default value).
3. Required fields will cause an error. Flatbuffer verifiers would
consider the whole buffer invalid. See the `required` tag below.
When writing a schema, values are a sequence of digits. Values may be optionally
followed by a decimal point (`.`) and more digits, for float constants, or
optionally prefixed by a `-`. Floats may also be in scientific notation;
optionally ending with an `e` or `E`, followed by a `+` or `-` and more digits.
Values can also be the keyword `null`.
Only scalar values can have defaults, non-scalar (string/vector/table) fields Only scalar values can have defaults, non-scalar (string/vector/table) fields
default to `NULL` when not present. default to `null` when not present.
You generally do not want to change default values after they're initially You generally do not want to change default values after they're initially
defined. Fields that have the default value are not actually stored in the defined. Fields that have the default value are not actually stored in the
serialized data (see also Gotchas below) but are generated in code, serialized data (see also Gotchas below). Values explicitly written by code
so when you change the default, you'd generated by the old schema old version, if they happen to be the default, will
now get a different value than from code generated from an older version of be read as a different value by code generated with the new schema. This is
the schema. There are situations, however, where this may be slightly less bad when converting an optional scalar into a default valued
desirable, especially if you can ensure a simultaneous rebuild of scalar since non-presence would not be overloaded with a previous default value.
all code. There are situations, however, where this may be desirable, especially if you
can ensure a simultaneous rebuild of all code.
### Enums ### Enums
@@ -325,18 +336,16 @@ Current understood attributes:
deprecate a field that was previous required, old code may fail to validate deprecate a field that was previous required, old code may fail to validate
new data (when using the optional verifier). new data (when using the optional verifier).
- `required` (on a non-scalar table field): this field must always be set. - `required` (on a non-scalar table field): this field must always be set.
By default, all fields are optional, i.e. may be left out. This is By default, fields do not need to be present in the binary. This is
desirable, as it helps with forwards/backwards compatibility, and desirable, as it helps with forwards/backwards compatibility, and
flexibility of data structures. It is also a burden on the reading code, flexibility of data structures. By specifying this attribute, you make non-
since for non-scalar fields it requires you to check against NULL and presence in an error for both reader and writer. The reading code may access
take appropriate action. By specifying this field, you force code that the field directly, without checking for null. If the constructing code does
constructs FlatBuffers to ensure this field is initialized, so the reading not initialize this field, they will get an assert, and also the verifier
code may access it directly, without checking for NULL. If the constructing will fail on buffers that have missing required fields. Both adding and
code does not initialize this field, they will get an assert, and also removing this attribute may be forwards/backwards incompatible as readers
the verifier will fail on buffers that have missing required fields. Note will be unable read old or new data, respectively, unless the data happens to
that if you add this attribute to an existing field, this will only be always have the field set.
valid if existing data always contains this field / existing code always
writes this field.
- `force_align: size` (on a struct): force the alignment of this struct - `force_align: size` (on a struct): force the alignment of this struct
to be something higher than what it is naturally aligned to. Causes to be something higher than what it is naturally aligned to. Causes
these structs to be aligned to that amount inside a buffer, IF that these structs to be aligned to that amount inside a buffer, IF that
@@ -486,7 +495,7 @@ of related data structures is a union. Unions do have a cost however,
so an alternative to a union is to have a single table that has so an alternative to a union is to have a single table that has
all the fields of all the data structures you are trying to all the fields of all the data structures you are trying to
represent, if they are relatively similar / share many fields. represent, if they are relatively similar / share many fields.
Again, this is efficient because optional fields are cheap. Again, this is efficient because non-present fields are cheap.
FlatBuffers supports the full range of integer sizes, so try to pick FlatBuffers supports the full range of integer sizes, so try to pick
the smallest size needed, rather than defaulting to int/long. the smallest size needed, rather than defaulting to int/long.
@@ -611,22 +620,25 @@ Most serialization formats (e.g. JSON or Protocol Buffers) make it very
explicit in the format whether a field is present in an object or not, explicit in the format whether a field is present in an object or not,
allowing you to use this as "extra" information. allowing you to use this as "extra" information.
In FlatBuffers, this also holds for everything except scalar values. FlatBuffers will not write fields that are equal to their default value,
sometimes resulting in significant space savings. However, this also means we
cannot disambiguate the meaning of non-presence as "written default value" or
"not written at all". This only applies to scalar fields since only they support
default values. Unless otherwise specified, their default is 0.
FlatBuffers by default will not write fields that are equal to the default If you care about the presence of scalars, most languages support "optional
value (for scalars), sometimes resulting in a significant space savings. scalars." You can set `null` as the default value in the schema. `null` is a
value that's outside of all types, so we will always write if `add_field` is
However, this also means testing whether a field is "present" is somewhat called. The generated field accessor should use the local language's canonical
meaningless, since it does not tell you if the field was actually written by optional type.
calling `add_field` style calls, unless you're only interested in this
information for non-default values.
Some `FlatBufferBuilder` implementations have an option called `force_defaults` Some `FlatBufferBuilder` implementations have an option called `force_defaults`
that circumvents this behavior, and writes fields even if they are equal to that circumvents this "not writing defaults" behavior you can then use
the default. You can then use `IsFieldPresent` to query this. `IsFieldPresent` to query presence.
Another option that works in all languages is to wrap a scalar field in a Another option that works in all languages is to wrap a scalar field in a
struct. This way it will return null if it is not present. The cool thing struct. This way it will return null if it is not present. This will be slightly
is that structs don't take up any more space than the scalar they represent. less ergonomic but structs don't take up any more space than the scalar they
represent.
[Interface Definition Language]: https://en.wikipedia.org/wiki/Interface_description_language [Interface Definition Language]: https://en.wikipedia.org/wiki/Interface_description_language

59
docs/source/Support.md
View File

@@ -18,31 +18,40 @@ In general:
NOTE: this table is a start, it needs to be extended. NOTE: this table is a start, it needs to be extended.
Feature | C++ | Java | C# | Go | Python | JS | TS | C | PHP | Dart | Lobster | Rust | Swift Feature | C++ | Java | C# | Go | Python | JS | TS | C | PHP | Dart | Lobster | Rust | Swift
------------------------------ | ------ | ------ | ------ | ------ | ------ | --------- | --------- | ------ | --- | ------- | ------- | ------- | ------ ------------------------------ | ------ | ----- | -------- | ----- | ------ | ----- | --- | ------ | --- | ------- | ------- | ------ | ------
Codegen for all basic features | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | WiP | Yes | Yes | Yes | Yes Codegen for all basic features | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | WiP | Yes | Yes | Yes | Yes
JSON parsing | Yes | No | No | No | No | No | No | Yes | No | No | Yes | No | No JSON parsing | Yes | No | No | No | No | No | No | Yes | No | No | Yes | No | No
Simple mutation | Yes | Yes | Yes | Yes | No | No | No | No | No | No | No | No | Yes Simple mutation | Yes | Yes | Yes | Yes | No | No | No | No | No | No | No | No | Yes
Reflection | Yes | No | No | No | No | No | No | Basic | No | No | No | No | No Reflection | Yes | No | No | No | No | No | No | Basic | No | No | No | No | No
Buffer verifier | Yes | No | No | No | No | No | No | Yes | No | No | No | No | No Buffer verifier | Yes | No | No | No | No | No | No | Yes | No | No | No | No | No
Testing: basic | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | ? | Yes | Yes | Yes | Yes Native Object API | Yes | No | Yes | Yes | Yes | Yes | Yes | No | No | No | No | No | No
Testing: fuzz | Yes | No | No | Yes | Yes | No | No | No | ? | No | No | Yes | No Optional Scalars | Yes | Yes | Yes | No | No | Yes | Yes | Yes | No | No | Yes | Yes | Yes
Performance: | Superb | Great | Great | Great | Ok | ? | ? | Superb | ? | ? | Great | Superb | Great Flexbuffers | Yes | Yes | ? | ? | ? | ? | ? | ? | ? | ? | ? | Yes | ?
Platform: Windows | VS2010 | Yes | Yes | ? | ? | ? | Yes | VS2010 | ? | Yes | Yes | Yes | No Testing: basic | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | ? | Yes | Yes | Yes | Yes
Platform: Linux | GCC282 | Yes | ? | Yes | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes | Yes Testing: fuzz | Yes | No | No | Yes | Yes | No | No | No | ? | No | No | Yes | No
Platform: OS X | Xcode4 | ? | ? | ? | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes | Yes Performance: | Superb | Great | Great | Great | Ok | ? | ? | Superb | ? | ? | Great | Superb | Great
Platform: Android | NDK10d | Yes | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ? | No Platform: Windows | VS2010 | Yes | Yes | ? | ? | ? | Yes | VS2010 | ? | Yes | Yes | Yes | No
Platform: iOS | ? | ? | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ? | Yes Platform: Linux | GCC282 | Yes | ? | Yes | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes | Yes
Engine: Unity | ? | ? | Yes | ? | ? | ? | ? | ? | ? | ? | No | ? | No Platform: OS X | Xcode4 | ? | ? | ? | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes | Yes
Primary authors (github) | aard* | aard* | ev*/js*| rw | rw | evanw/ev* | kr* | mik* | ch* | dnfield | aard* | rw | mi*/mz* Platform: Android | NDK10d | Yes | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ? | No
Platform: iOS | ? | ? | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ? | Yes
Engine: Unity | ? | ? | Yes | ? | ? | ? | ? | ? | ? | ? | No | ? | No
Primary authors (github) | aard | aard | ev/js/df | rw | rw | ew/ev | kr | mik | ch | df | aard | rw/cn | mi/mz
* aard = aardappel (previously: gwvo) Above | Github username
* ev = evolutional ----- | -----------------------------
* js = jonsimantov aard | aardappel (previously: gwvo)
* mik = mikkelfj ch | chobie
* ch = chobie cn | caspern
* kr = krojew df | dnfield
* mi = mustiikhalil ev | evolutional
* mz = mzaks ew | evanw
js | jonsimantov
kr | krojew
mi | mustiikhalil
mik | mikkelfj
mz | mzaks
rw | rw
<br> <br>

17
docs/source/Tutorial.md
View File

@@ -227,13 +227,14 @@ less memory and have faster lookup.
The `Monster` table is the main object in our FlatBuffer. This will be used as The `Monster` table is the main object in our FlatBuffer. This will be used as
the template to store our `orc` monster. We specify some default values for the template to store our `orc` monster. We specify some default values for
fields, such as `mana:short = 150`. All unspecified fields will default to `0` fields, such as `mana:short = 150`. If unspecified, scalar fields (like `int`,
or `NULL`. Another thing to note is the line `uint`, or `float`) will be given a default of `0` while strings and tables will
`friendly:bool = false (deprecated);`. Since you cannot delete fields from a be given a default of `null`. Another thing to note is the line `friendly:bool =
`table` (to support backwards compatability), you can set fields as false (deprecated);`. Since you cannot delete fields from a `table` (to support
`deprecated`, which will prevent the generation of accessors for this field in backwards compatability), you can set fields as `deprecated`, which will prevent
the generated code. Be careful when using `deprecated`, however, as it may break the generation of accessors for this field in the generated code. Be careful
legacy code that used this accessor. when using `deprecated`, however, as it may break legacy code that used this
accessor.
The `Weapon` table is a sub-table used within our FlatBuffer. It is The `Weapon` table is a sub-table used within our FlatBuffer. It is
used twice: once within the `Monster` table and once within the `Equipment` used twice: once within the `Monster` table and once within the `Equipment`
@@ -2702,7 +2703,7 @@ To access sub-objects, in the case of our `pos`, which is a `Vec3`:
`x`, `y`, and `z` will contain `1.0`, `2.0`, and `3.0`, respectively. `x`, `y`, and `z` will contain `1.0`, `2.0`, and `3.0`, respectively.
*Note: Had we not set `pos` during serialization, it would be a `NULL`-value.* *Note: Had we not set `pos` during serialization, it would be a `null`-value.*
Similarly, we can access elements of the inventory `vector` by indexing it. You Similarly, we can access elements of the inventory `vector` by indexing it. You
can also iterate over the length of the array/vector representing the can also iterate over the length of the array/vector representing the