BigfootDev/flatbuffers
1
0
Fork 1
mirror of https://github.com/google/flatbuffers.git synced 2026-06-06 13:37:25 +00:00
Code Issues Packages Projects Releases Wiki Activity

FlatBuffers implementation for the Lobster programming language

Browse Source 
Language, see: http://strlen.com/lobster/ and https://github.com/aardappel/lobster
This commit is contained in:
aardappel
2018-07-23 19:03:11 -07:00
parent ca5aaf62d3
commit 4898809eca
25 changed files with 2017 additions and 41 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
docs/source/Compiler.md
View File

@@ -31,12 +31,18 @@ For any schema input files, one or more generators can be specified:
- `--js`, `-s`: Generate JavaScript code.
- `--ts`: Generate TypeScript code.
- `--php`: Generate PHP code.
- `--grpc`: Generate RPC stub code for GRPC.
- `--dart`: Generate Dart code.
- `--lua`: Generate Lua code.
- `--lobster`: Generate Lobster code.
For any data input files:
- `--binary`, `-b` : If data is contained in this file, generate a

4
docs/source/FlatBuffers.md
View File

@@ -4,7 +4,7 @@ FlatBuffers {#flatbuffers_index}
# Overview {#flatbuffers_overview}
[FlatBuffers](@ref flatbuffers_overview) is an efficient cross platform
serialization library for C++, C#, C, Go, Java, JavaScript, Lua, TypeScript, PHP, and Python.
serialization library for C++, C#, C, Go, Java, JavaScript, Lobster, Lua, TypeScript, PHP, and Python.
It was originally created at Google for game development and other
performance-critical applications.
@@ -142,6 +142,8 @@ sections provide a more in-depth usage guide.
own programs.
- How to [use FlatBuffers in C with `flatcc`](@ref flatbuffers_guide_use_c) in your
own programs.
- How to [use the generated Lobster code](@ref flatbuffers_guide_use_lobster) in your
own programs.
- [Support matrix](@ref flatbuffers_support) for platforms/languages/features.
- Some [benchmarks](@ref flatbuffers_benchmarks) showing the advantage of
using FlatBuffers.

85
docs/source/LobsterUsage.md Normal file
View File

@@ -0,0 +1,85 @@
Use in Lobster {#flatbuffers_guide_use_lobster}
==============
## Before you get started
Before diving into the FlatBuffers usage in Lobster, it should be noted that the
[Tutorial](@ref flatbuffers_guide_tutorial) page has a complete guide to general
FlatBuffers usage in all of the supported languages (including Lobster). This
page is designed to cover the nuances of FlatBuffers usage, specific to
Lobster.
You should also have read the [Building](@ref flatbuffers_guide_building)
documentation to build `flatc` and should be familiar with
[Using the schema compiler](@ref flatbuffers_guide_using_schema_compiler) and
[Writing a schema](@ref flatbuffers_guide_writing_schema).
## FlatBuffers Lobster library code location
The code for the FlatBuffers Lobster library can be found at
`flatbuffers/lobster`. You can browse the library code on the
[FlatBuffers GitHub page](https://github.com/google/flatbuffers/tree/master/
lobster).
## Testing the FlatBuffers Lobster library
The code to test the Lobster library can be found at `flatbuffers/tests`.
The test code itself is located in [lobstertest.lobster](https://github.com/google/
flatbuffers/blob/master/tests/lobstertest.lobster).
To run the tests, run `lobster lobstertest.lobster`. To obtain Lobster itself,
go to the [Lobster homepage](http://strlen.com/lobster) or
[github](https://github.com/aardappel/lobster) to learn how to build it for your
platform.
## Using the FlatBuffers Lobster library
*Note: See [Tutorial](@ref flatbuffers_guide_tutorial) for a more in-depth
example of how to use FlatBuffers in Lobster.*
There is support for both reading and writing FlatBuffers in Lobster.
To use FlatBuffers in your own code, first generate Lobster classes from your
schema with the `--lobster` option to `flatc`. Then you can include both
FlatBuffers and the generated code to read or write a FlatBuffer.
For example, here is how you would read a FlatBuffer binary file in Lobster:
First, import the library and the generated code. Then read a FlatBuffer binary
file into a string, which you pass to the `GetRootAsMonster` function:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.lobster}
include "monster_generated.lobster"
let fb = read_file("monsterdata_test.mon")
assert fb
let monster = MyGame_Example_GetRootAsMonster(fb)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now you can access values like this:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.lobster}
let hp = monster.hp
let pos = monster.pos
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
As you can see, even though `hp` and `pos` are functions that access FlatBuffer
data in-place in the string buffer, they appear as field accesses.
## Speed
Using FlatBuffers in Lobster should be relatively fast, as the implementation
makes use of native support for writing binary values, and access of vtables.
Both generated code and the runtime library are therefore small and fast.
Actual speed will depend on wether you use Lobster as bytecode VM or compiled to
C++.
## Text Parsing
Lobster has full support for parsing JSON into FlatBuffers, or generating
JSON from FlatBuffers. See `samples/sample_test.lobster` for an example.
This uses the C++ parser and generator underneath, so should be both fast and
conformant.
<br>

36
docs/source/Support.md
View File

@@ -18,25 +18,25 @@ In general:
NOTE: this table is a start, it needs to be extended.
Feature | C++ | Java | C# | Go | Python | JS | TS | C | PHP | Ruby | Dart
------------------------------ | ------ | ------ | ------ | ------ | ------ | --------- | --------- | ------ | --- | ---- | ----
Codegen for all basic features | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | WiP | WiP | Yes
JSON parsing | Yes | No | No | No | No | No | No | Yes | No | No | No
Simple mutation | Yes | Yes | Yes | Yes | No | No | No | No | No | No | No
Reflection | Yes | No | No | No | No | No | No | Basic | No | No | No
Buffer verifier | Yes | No | No | No | No | No | No | Yes | No | No | No
Testing: basic | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | ? | ? | Yes
Testing: fuzz | Yes | No | No | Yes | Yes | No | No | No | ? | ? | No
Performance: | Superb | Great | Great | Great | Ok | ? | ? | Superb | ? | ? | ?
Platform: Windows | VS2010 | Yes | Yes | ? | ? | ? | Yes | VS2010 | ? | ? | Yes
Platform: Linux | GCC282 | Yes | ? | Yes | Yes | ? | Yes | Yes | ? | ? | Yes
Platform: OS X | Xcode4 | ? | ? | ? | Yes | ? | Yes | Yes | ? | ? | Yes
Platform: Android | NDK10d | Yes | ? | ? | ? | ? | ? | ? | ? | ? | Flutter
Platform: iOS | ? | ? | ? | ? | ? | ? | ? | ? | ? | ? | Flutter
Engine: Unity | ? | ? | Yes | ? | ? | ? | ? | ? | ? | ? | ?
Primary authors (github) | gwvo | gwvo | ev*/js*| rw | rw | evanw/ev* | kr | mik* | ch* | rw | dnfield
Feature | C++ | Java | C# | Go | Python | JS | TS | C | PHP | Dart | Lobster
------------------------------ | ------ | ------ | ------ | ------ | ------ | --------- | --------- | ------ | --- | ------- | -------
Codegen for all basic features | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | WiP | Yes | Yes
JSON parsing | Yes | No | No | No | No | No | No | Yes | No | No | Yes
Simple mutation | Yes | Yes | Yes | Yes | No | No | No | No | No | No | No
Reflection | Yes | No | No | No | No | No | No | Basic | No | No | No
Buffer verifier | Yes | No | No | No | No | No | No | Yes | No | No | No
Testing: basic | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | ? | Yes | Yes
Testing: fuzz | Yes | No | No | Yes | Yes | No | No | No | ? | No | No
Performance: | Superb | Great | Great | Great | Ok | ? | ? | Superb | ? | ? | Great
Platform: Windows | VS2010 | Yes | Yes | ? | ? | ? | Yes | VS2010 | ? | Yes | Yes
Platform: Linux | GCC282 | Yes | ? | Yes | Yes | ? | Yes | Yes | ? | Yes | Yes
Platform: OS X | Xcode4 | ? | ? | ? | Yes | ? | Yes | Yes | ? | Yes | Yes
Platform: Android | NDK10d | Yes | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes
Platform: iOS | ? | ? | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes
Engine: Unity | ? | ? | Yes | ? | ? | ? | ? | ? | ? | ? | No
Primary authors (github) | aard* | aard* | ev*/js*| rw | rw | evanw/ev* | kr* | mik* | ch* | dnfield | aard*
* aard = aardappel (previously: gwvo)
* ev = evolutional
* js = jonsimantov
* mik = mikkelfj

165
docs/source/Tutorial.md
View File

@@ -32,6 +32,7 @@ Please select your desired language for our quest:
<input type="radio" name="language" value="c">C</input>
<input type="radio" name="language" value="dart">Dart</input>
<input type="radio" name="language" value="lua">Lua</input>
<input type="radio" name="language" value="lobster">Lobster</input>
</form>
\endhtmlonly
@@ -138,7 +139,10 @@ For your chosen language, please cross-reference with:
[example.dart](https://github.com/google/flatbuffers/blob/master/dart/example/example.dart)
</div>
<div class="language-lua">
[sample_binary.lua](https://github.com/google/flatbuffers/blob/master/dart/example/sample_binary.lua)
[sample_binary.lua](https://github.com/google/flatbuffers/blob/master/samples/sample_binary.lua)
</div>
<div class="language-lobster">
[sample_binary.lobster](https://github.com/google/flatbuffers/blob/master/samples/sample_binary.lobster)
</div>
@@ -333,6 +337,12 @@ Please be aware of the difference between `flatc` and `flatcc` tools.
./../flatc --lua monster.fbs
~~~
</div>
<div class="language-lobster">
~~~{.sh}
cd flatbuffers/sample
./../flatc --lobster monster.fbs
~~~
</div>
For a more complete guide to using the `flatc` compiler, please read the
[Using the schema compiler](@ref flatbuffers_guide_using_schema_compiler)
@@ -463,6 +473,12 @@ The first step is to import/include the library, generated files, etc.
local weapon = require("MyGame.Sample.Weapon")
~~~
</div>
<div class="language-lobster">
~~~{.lobster}
include from "../lobster/" // Where to find flatbuffers.lobster
include "monster_generated.lobster"
~~~
</div>
Now we are ready to start building some buffers. In order to start, we need
to create an instance of the `FlatBufferBuilder`, which will contain the buffer
@@ -548,6 +564,12 @@ which will grow automatically if needed:
local builder = flatbuffers.Builder(1024)
~~~
</div>
<div class="language-lobster">
~~~{.lobster}
-- get access to the builder
let builder = flatbuffers_builder {}
~~~
</div>
After creating the `builder`, we can start serializing our data. Before we make
our `orc` Monster, lets create some `Weapon`s: a `Sword` and an `Axe`.
@@ -753,6 +775,19 @@ our `orc` Monster, lets create some `Weapon`s: a `Sword` and an `Axe`.
local axe = weapon.End(builder)
~~~
</div>
<div class="language-lobster">
~~~{.lobster}
let weapon_names = [ "Sword", "Axe" ]
let weapon_damages = [ 3, 5 ]
weapon_offsets := map(weapon_names) name, i:
let ns = builder.CreateString(name)
builder.MyGame_Sample_WeaponStart()
builder.MyGame_Sample_WeaponAddName(ns)
builder.MyGame_Sample_WeaponAddDamage(weapon_damages[i])
builder.MyGame_Sample_WeaponEnd()
~~~
</div>
Now let's create our monster, the `orc`. For this `orc`, lets make him
`red` with rage, positioned at `(1.0, 2.0, 3.0)`, and give him
@@ -915,6 +950,15 @@ traversal. This is generally easy to do on any tree structures.
local inv = builder:EndVector(10)
~~~
</div>
<div class="language-lobster">
~~~{.lobster}
// Name of the monster.
let name = builder.CreateString("Orc")
// Inventory.
let inv = builder.MyGame_Sample_MonsterCreateInventoryVector(map(10): _)
~~~
</div>
We serialized two built-in data types (`string` and `vector`) and captured
their return values. These values are offsets into the serialized data,
@@ -1037,6 +1081,11 @@ offsets.
local weapons = builder:EndVector(2)
~~~
</div>
<div class="language-lobster">
~~~{.lobster}
let weapons = builder.MyGame_Sample_MonsterCreateWeaponsVector(weapon_offsets)
~~~
</div>
<div class="language-cpp">
<br>
@@ -1146,6 +1195,14 @@ for the `path` field above:
local path = builder:EndVector(2)
~~~
</div>
<div class="language-lobster">
~~~{.lobster}
builder.MyGame_Sample_MonsterStartPathVector(2)
builder.MyGame_Sample_CreateVec3(1.0, 2.0, 3.0)
builder.MyGame_Sample_CreateVec3(4.0, 5.0, 6.0)
let path = builder.EndVector(2)
~~~
</div>
We have now serialized the non-scalar components of the orc, so we
can serialize the monster itself:
@@ -1366,6 +1423,21 @@ can serialize the monster itself:
local orc = monster.End(builder)
~~~
</div>
<div class="language-lobster">
~~~{.lobster}
builder.MyGame_Sample_MonsterStart()
builder.MyGame_Sample_MonsterAddPos(builder.MyGame_Sample_CreateVec3(1.0, 2.0, 3.0))
builder.MyGame_Sample_MonsterAddHp(300)
builder.MyGame_Sample_MonsterAddName(name)
builder.MyGame_Sample_MonsterAddInventory(inv)
builder.MyGame_Sample_MonsterAddColor(MyGame_Sample_Color_Red)
builder.MyGame_Sample_MonsterAddWeapons(weapons)
builder.MyGame_Sample_MonsterAddEquippedType(MyGame_Sample_Equipment_Weapon)
builder.MyGame_Sample_MonsterAddEquipped(weapon_offsets[1])
builder.MyGame_Sample_MonsterAddPath(path)
let orc = builder.MyGame_Sample_MonsterEnd()
~~~
</div>
Note how we create `Vec3` struct in-line in the table. Unlike tables, structs
are simple combinations of scalars that are always stored inline, just like
@@ -1514,6 +1586,12 @@ Here is a repetition these lines, to help highlight them more clearly:
monster.AddEquipped(builder, axe) -- Union data
~~~
</div>
<div class="language-lobster">
~~~{.lobster}
builder.MyGame_Sample_MonsterAddEquippedType(MyGame_Sample_Equipment_Weapon)
builder.MyGame_Sample_MonsterAddEquipped(axe)
~~~
</div>
After you have created your buffer, you will have the offset to the root of the
data in the `orc` variable, so you can finish the buffer by calling the
@@ -1591,7 +1669,12 @@ appropriate `finish` method.
builder:Finish(orc)
~~~
</div>
<div class="language-lobster">
~~~{.lobster}
// Call `Finish()` to instruct the builder that this monster is complete.
builder.Finish(orc)
~~~
</div>
The buffer is now ready to be stored somewhere, sent over the network, be
compressed, or whatever you'd like to do with it. You can access the buffer
@@ -1695,6 +1778,12 @@ like so:
local bufAsString = builder:Output()
~~~
</div>
<div class="language-lobster">
~~~{.lobster}
// This must be called after `Finish()`.
let buf = builder.SizedCopy() // Of type `string`.
~~~
</div>
Now you can write the bytes to a file, send them over the network..
@@ -1706,7 +1795,7 @@ which will lead to hard to find problems when you read the buffer.
Now that we have successfully created an `Orc` FlatBuffer, the monster data can
be saved, sent over a network, etc. Let's now adventure into the inverse, and
deserialize a FlatBuffer.
access a FlatBuffer.
This section requires the same import/include, namespace, etc. requirements as
before:
@@ -1822,6 +1911,12 @@ import './monster_my_game.sample_generated.dart' as myGame;
local weapon = require("MyGame.Sample.Weapon")
~~~
</div>
<div class="language-lobster">
~~~{.lobster}
include from "../lobster/" // Where to find flatbuffers.lobster
include "monster_generated.lobster"
~~~
</div>
Then, assuming you have a buffer of bytes received from disk,
network, etc., you can create start accessing the buffer like so:
@@ -1941,6 +2036,14 @@ myGame.Monster monster = new myGame.Monster(data);
local mon = monster.GetRootAsMonster(buf, 0)
~~~
</div>
<div class="language-lobster">
~~~{.lobster}
buf = /* the data you just read, in a string */
// Get an accessor to the root object inside the buffer.
let monster = MyGame_Sample_GetRootAsMonster(buf)
~~~
</div>
If you look in the generated files from the schema compiler, you will see it generated
accessors for all non-`deprecated` fields. For example:
@@ -2026,6 +2129,13 @@ accessors for all non-`deprecated` fields. For example:
local name = mon:Name()
~~~
</div>
<div class="language-lobster">
~~~{.lobster}
let hp = monster.hp
let mana = monster.mana
let name = monster.name
~~~
</div>
These should hold `300`, `150`, and `"Orc"` respectively.
@@ -2127,7 +2237,14 @@ To access sub-objects, in the case of our `pos`, which is a `Vec3`:
local z = pos:Z()
~~~
</div>
<div class="language-lobster">
~~~{.lobster}
let pos = monster.pos
let x = pos.x
let y = pos.y
let z = pos.z
~~~
</div>
`x`, `y`, and `z` will contain `1.0`, `2.0`, and `3.0`, respectively.
@@ -2206,6 +2323,12 @@ FlatBuffers `vector`.
local thirdItem = mon:Inventory(3) -- Lua is 1-based
~~~
</div>
<div class="language-lobster">
~~~{.lobster}
let inv_len = monster.inventory_length
let third_item = monster.inventory(2)
~~~
</div>
For `vector`s of `table`s, you can access the elements like any other vector,
except your need to handle the result as a FlatBuffer `table`:
@@ -2294,6 +2417,13 @@ except your need to handle the result as a FlatBuffer `table`:
local secondWeaponDamage = mon:Weapon(2):Damage()
~~~
</div>
<div class="language-lobster">
~~~{.lobster}
let weapons_length = monster.weapons_length
let second_weapon_name = monster.weapons(1).name
let second_weapon_damage = monster.weapons(1).damage
~~~
</div>
Last, we can access our `Equipped` FlatBuffer `union`. Just like when we created
the `union`, we need to get both parts of the `union`: the type and the data.
@@ -2442,6 +2572,19 @@ We can access the type to dynamically cast the data as needed (since the
end
~~~
</div>
<div class="language-lobster">
~~~{.lobster}
union_type = monster.equipped_type
if union_type == MyGame_Sample_Equipment_Weapon:
// `monster.equipped_as_Weapon` returns a FlatBuffer handle much like normal table fields,
// but this is only valid to call if we already know it is the correct type.
let union_weapon = monster.equipped_as_Weapon
let weapon_name = union_weapon.name // "Axe"
let weapon_damage = union_weapon.damage // 5
~~~
</div>
## Mutating FlatBuffers
@@ -2527,6 +2670,11 @@ mutators like so:
<API for mutating FlatBuffers is not yet available in Lua.>
~~~
</div>
<div class="language-lobster">
~~~{.lobster}
<API for mutating FlatBuffers is not yet available in Lobster.>
~~~
</div>
We use the somewhat verbose term `mutate` instead of `set` to indicate that this
is a special use case, not to be confused with the default way of constructing
@@ -2601,6 +2749,11 @@ printers that you can compile and use at runtime. The `flatc` compiler (not
flatbuffer conversion from a given schema. There are no current plans
for `flatcc` to support this.*
</div>
<div class="language-lobster">
*Note: If you're working in Lobster, you can also parse JSON at runtime. See the
[Use in Lobster](@ref flatbuffers_guide_use_lobster) section of the Programmer's
Guide for more information.*
</div>
## Advanced Features for Each Language
@@ -2642,6 +2795,8 @@ For your chosen language, see:
<div class="language-lua">
[Use in Lua](@ref flatbuffers_guide_use_lua)
</div>
<div class="language-lobster">
[Use in Lobster](@ref flatbuffers_guide_use_lobster)
</div>
<br>

1
docs/source/doxyfile
View File

@@ -759,6 +759,7 @@ INPUT = "FlatBuffers.md" \
"PHPUsage.md" \
"PythonUsage.md" \
"LuaUsage.md" \
"LobsterUsage.md" \
"Support.md" \
"Benchmarks.md" \
"WhitePaper.md" \

2
docs/source/doxygen_layout.xml
View File

@@ -43,6 +43,8 @@
title="Use in Dart"/>
<tab type="user" url="@ref flatbuffers_guide_use_lua"
title="Use in Lua"/>
<tab type="user" url="@ref flatbuffers_guide_use_lobster"
title="Use in Lobster"/>
<tab type="user" url="@ref flexbuffers"
title="Schema-less version"/>
<tab type="usergroup" url="" title="gRPC">