BigfootDev/flatbuffers
1
0
Fork 1
mirror of https://github.com/google/flatbuffers.git synced 2026-06-01 19:58:15 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add Kotlin multiplatform support (#7969)

Browse Source 
* [Kotlin] Introduction to Kotlin Multiplaform

The first implementation of the Kotlin code generation was made years
ago at the time Kotlin Multiplaform was not stable and Kotlin is mostly
used on JVM-based targets. For this reason the generated code uses java
based runtime.

That design decision comes with many drawbacks, leaving the code
generated more java-like and making it impossible to use more advanced
features of the Kotlin language.

In this change we are adding two parts: A pure, multi-plaform, Kotlin
runtime and a new code generator to accompany it.

* [Kotlin] Remove scalar sign cast from code generation

Now that we have a new runtime the accepts unsigned types, we don't
need to code generate casting back and from signed scalars. This
MR removes this from both code generations and adds the necessary
API to the runtime.

* [Kotlin] Use offset on public API to represent buffer position

Currently, kotlin was following Java's approach of representing objects,
vectors, tables as "Int" (the position of it in the buffer). This change
replaces naked Int with Offset<T>, offering a type-safe API. So,
instead of

fun Table.createTable(b: FlatBufferBuilder, subTable: Int)

We will have

fun Table.createTable(b: FlatBufferBuilder, subTable: Offset<SubTable>)

Making impossible to accidentally switch parameters.

The performance should be similar to use Int as we are using value
class for Offset and ArrayOffset, which most of the time translate to
Int in the bytecode.

* [Kotlin] Add builder for tables

Add builder constructor to make create of table more ergonomic.
For example the movie sample for the test set could be written as:

Movie.createMovie(fbb,
    mainCharacterType = Character_.MuLan,
    mainCharacter = att) {
    charactersType = charsType
    this.characters = characters
}

instead of:

Movie.startMovie(fbb)
Movie.addMainCharacterType(fbb, Character_.MuLan)
Movie.addMainCharacter(fbb, att as Offset<Any>)
Movie.addCharactersType(fbb, charsType)
Movie.addCharacters(fbb, charsVec)
Movie.endMovie(fbb)

* [Kotlin] Move enum types to value class

Moving to flatbuffer enums to value class adds type safety for parameters
with minimum to no performance impact.

* [Kotlin] Simplify Union parameters to avoid naked casting

Just a small change on the APIs that receive union as parameters,
creating a typealias UnionOffset to avoid using Offset<Any>. To "convert"
an table offset to an union, one just call Offset.toUnion().

* [Kotlin] Apply clang-format on kotlin code generators

* [Kotlin] Update kotlin generator to follow official naming conventions

Updating directory, package and enum naming to follow Kotlin official
convention.

https://kotlinlang.org/docs/coding-conventions.html#naming-rules

* [Kotlin] Add fixes to improve performance

1 - Add benchmark comparing serialization between Java & Kotlin
2 - ReadWriteBuffer does not auto-grow (thus avoid check size in every op)
3 - Add specialized add functions on FlatBufferBuilder to avoid boxing
offsets.
4 - Remove a few Kotlin syntax sugar that generated performance penalties.

* [Kotlin] Remove builder from Kotlin KMP and add some optimizations
to avoid boxing of Offset classes

---------

Co-authored-by: Derek Bailey <derekbailey@google.com>
This commit is contained in:
Paulo Pinheiro
2023-05-26 20:00:33 +02:00
committed by GitHub
parent 0100f6a577
commit b7856f8e27
43 changed files with 4597 additions and 359 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
.github/labeler.yml vendored
View File

@@ -48,6 +48,7 @@ java:
kotlin:
- '**/*.kt'
- src/idl_gen_kotlin.cpp
- src/idl_gen_kotlin_kmp.cpp
lua:
- '**/*.lua'

12
.github/workflows/build.yml vendored
View File

@@ -422,9 +422,14 @@ jobs:
with:
distribution: 'temurin'
java-version: '11'
- name: Build flatc
run: |
cmake -DFLATBUFFERS_BUILD_TESTS=OFF -DFLATBUFFERS_BUILD_FLATLIB=OFF -DFLATBUFFERS_BUILD_FLATHASH=OFF .
make -j
echo "${PWD}" >> $GITHUB_PATH
- name: Build
working-directory: kotlin
run: ./gradlew clean iosX64Test macosX64Test
run: ./gradlew clean iosSimulatorArm64Test macosX64Test macosArm64Test
build-kotlin-linux:
name: Build Kotlin Linux
@@ -437,6 +442,11 @@ jobs:
distribution: 'temurin'
java-version: '11'
- uses: gradle/wrapper-validation-action@v1.0.5
- name: Build flatc
run: |
cmake -DFLATBUFFERS_BUILD_TESTS=OFF -DFLATBUFFERS_BUILD_FLATLIB=OFF -DFLATBUFFERS_BUILD_FLATHASH=OFF .
make -j
echo "${PWD}" >> $GITHUB_PATH
- name: Build
working-directory: kotlin
# we are using docker's version of gradle

2
.gitignore vendored
View File

@@ -151,3 +151,5 @@ flatbuffers.pc
# https://cmake.org/cmake/help/latest/module/FetchContent.html#variable:FETCHCONTENT_BASE_DIR
cmake-build-debug/
_deps/
**/.gradle/**
kotlin/**/generated

1
CMakeLists.txt
View File

@@ -160,6 +160,7 @@ set(FlatBuffers_Compiler_SRCS
src/idl_gen_csharp.cpp
src/idl_gen_dart.cpp
src/idl_gen_kotlin.cpp
src/idl_gen_kotlin_kmp.cpp
src/idl_gen_go.cpp
src/idl_gen_java.cpp
src/idl_gen_ts.cpp

2
android/.project
View File

@@ -10,7 +10,7 @@
</natures>
<filteredResources>
<filter>
<id>1677235311958</id>
<id>1672434305228</id>
<name></name>
<type>30</type>
<matcher>

6
include/flatbuffers/idl.h
View File

@@ -342,7 +342,10 @@ struct FieldDef : public Definition {
bool Deserialize(Parser &parser, const reflection::Field *field);
bool IsScalarOptional() const {
return IsScalar(value.type.base_type) && IsOptional();
return IsScalar() && IsOptional();
}
bool IsScalar() const {
return ::flatbuffers::IsScalar(value.type.base_type);
}
bool IsOptional() const { return presence == kOptional; }
bool IsRequired() const { return presence == kRequired; }
@@ -725,6 +728,7 @@ struct IDLOptions {
kSwift = 1 << 16,
kNim = 1 << 17,
kProto = 1 << 18,
kKotlinKmp = 1 << 19,
kMAX
};

89
kotlin/benchmark/build.gradle.kts
View File

@@ -1,5 +1,3 @@
import org.jetbrains.kotlin.ir.backend.js.compile
plugins {
kotlin("multiplatform")
id("org.jetbrains.kotlinx.benchmark")
@@ -27,7 +25,7 @@ benchmark {
iterationTime = 300
iterationTimeUnit = "ms"
// uncomment for benchmarking JSON op only
include(".*JsonBenchmark.*")
include(".*FlatbufferBenchmark.*")
}
}
targets {
@@ -36,24 +34,34 @@ benchmark {
}
kotlin {
jvm()
jvm {
compilations {
val main by getting { }
// custom benchmark compilation
val benchmarks by compilations.creating {
defaultSourceSet {
dependencies {
// Compile against the main compilation's compile classpath and outputs:
implementation(main.compileDependencyFiles + main.output.classesDirs)
}
}
}
}
}
sourceSets {
all {
languageSettings.enableLanguageFeature("InlineClasses")
}
val jvmMain by getting {
dependencies {
implementation(kotlin("stdlib-common"))
implementation(project(":flatbuffers-kotlin"))
implementation(libs.kotlinx.benchmark.runtime)
implementation("com.google.flatbuffers:flatbuffers-java:2.0.3")
implementation("com.google.flatbuffers:flatbuffers-java:23.5.9")
// json serializers
implementation(libs.moshi.kotlin)
implementation(libs.gson)
}
kotlin.srcDir("src/jvmMain/generated/kotlin/")
kotlin.srcDir("src/jvmMain/generated/java/")
}
}
}
@@ -67,3 +75,64 @@ tasks.register<de.undercouch.gradle.tasks.download.Download>("downloadMultipleFi
dest(File("${project.projectDir.absolutePath}/src/jvmMain/resources"))
overwrite(false)
}
abstract class GenerateFBTestClasses : DefaultTask() {
@get:InputFiles
abstract val inputFiles: ConfigurableFileCollection
@get:Input
abstract val includeFolder: Property<String>
@get:Input
abstract val outputFolder: Property<String>
@get:Input
abstract val variants: ListProperty<String>
@Inject
protected open fun getExecActionFactory(): org.gradle.process.internal.ExecActionFactory? {
throw UnsupportedOperationException()
}
init {
includeFolder.set("")
}
@TaskAction
fun compile() {
val execAction = getExecActionFactory()!!.newExecAction()
val sources = inputFiles.asPath.split(":")
val langs = variants.get().map { "--$it" }
val args = mutableListOf("flatc","-o", outputFolder.get(), *langs.toTypedArray())
if (includeFolder.get().isNotEmpty()) {
args.add("-I")
args.add(includeFolder.get())
}
args.addAll(sources)
println(args)
execAction.commandLine = args
print(execAction.execute())
}
}
// Use the default greeting
tasks.register<GenerateFBTestClasses>("generateFBTestClassesKt") {
inputFiles.setFrom("$projectDir/monster_test_kotlin.fbs")
includeFolder.set("$rootDir/../tests/include_test")
outputFolder.set("${projectDir}/src/jvmMain/generated/kotlin/")
variants.addAll("kotlin-kmp")
}
tasks.register<GenerateFBTestClasses>("generateFBTestClassesJava") {
inputFiles.setFrom("$projectDir/monster_test_java.fbs")
includeFolder.set("$rootDir/../tests/include_test")
outputFolder.set("${projectDir}/src/jvmMain/generated/java/")
variants.addAll("kotlin")
}
project.tasks.forEach {
if (it.name.contains("compileKotlin")) {
it.dependsOn("generateFBTestClassesKt")
it.dependsOn("generateFBTestClassesJava")
}
}

37
kotlin/benchmark/monster_test_java.fbs Normal file
View File

@@ -0,0 +1,37 @@
// Example IDL file for our monster's schema.
namespace jmonster;
enum JColor:byte { Red = 0, Green, Blue = 2 }
union JEquipment { JWeapon } // Optionally add more tables.
struct JVec3 {
x:float;
y:float;
z:float;
}
table JMonster {
pos:JVec3;
mana:short = 150;
hp:short = 100;
name:string;
friendly:bool = false (deprecated);
inventory:[ubyte];
color:JColor = Blue;
weapons:[JWeapon];
equipped:JEquipment;
path:[JVec3];
}
table JWeapon {
name:string;
damage:short;
}
table JAllMonsters {
monsters: [JMonster];
}
root_type JAllMonsters;

37
kotlin/benchmark/monster_test_kotlin.fbs Normal file
View File

@@ -0,0 +1,37 @@
// Example IDL file for our monster's schema.
namespace monster;
enum Color:byte { Red = 0, Green, Blue = 2 }
union Equipment { Weapon } // Optionally add more tables.
struct Vec3 {
x:float;
y:float;
z:float;
}
table Monster {
pos:Vec3;
mana:short = 150;
hp:short = 100;
name:string;
friendly:bool = false (deprecated);
inventory:[ubyte];
color:Color = Blue;
weapons:[Weapon];
equipped:Equipment;
path:[Vec3];
}
table Weapon {
name:string;
damage:short;
}
table AllMonsters {
monsters: [Monster];
}
root_type AllMonsters;

1
kotlin/benchmark/src/jvmMain/java
View File

@@ -1 +0,0 @@
../../../../java/src/main/java

68
kotlin/benchmark/src/jvmMain/kotlin/com/google/flatbuffers/kotlin/benchmark/FlatbufferBenchmark.kt Normal file
View File

@@ -0,0 +1,68 @@
@file:OptIn(ExperimentalUnsignedTypes::class)
package com.google.flatbuffers.kotlin.benchmark
import com.google.flatbuffers.kotlin.FlatBufferBuilder
import jmonster.JAllMonsters
import jmonster.JMonster
import jmonster.JVec3
import monster.AllMonsters.Companion.createAllMonsters
import monster.AllMonsters.Companion.createMonstersVector
import monster.Monster
import monster.Monster.Companion.createInventoryVector
import monster.MonsterOffsetArray
import monster.Vec3
import org.openjdk.jmh.annotations.*
import java.util.concurrent.TimeUnit
@State(Scope.Benchmark)
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
@Measurement(iterations = 20, time = 1, timeUnit = TimeUnit.NANOSECONDS)
open class FlatbufferBenchmark {
val repetition = 1000000
val fbKotlin = FlatBufferBuilder(1024 * repetition)
val fbJava = com.google.flatbuffers.FlatBufferBuilder(1024 * repetition)
@OptIn(ExperimentalUnsignedTypes::class)
@Benchmark
fun monstersKotlin() {
fbKotlin.clear()
val monsterName = fbKotlin.createString("MonsterName");
val items = ubyteArrayOf(0u, 1u, 2u, 3u, 4u)
val inv = createInventoryVector(fbKotlin, items)
val monsterOffsets: MonsterOffsetArray = MonsterOffsetArray(repetition) {
Monster.startMonster(fbKotlin)
Monster.addName(fbKotlin, monsterName)
Monster.addPos(fbKotlin, Vec3.createVec3(fbKotlin, 1.0f, 2.0f, 3.0f))
Monster.addHp(fbKotlin, 80)
Monster.addMana(fbKotlin, 150)
Monster.addInventory(fbKotlin, inv)
Monster.endMonster(fbKotlin)
}
val monsters = createMonstersVector(fbKotlin, monsterOffsets)
val allMonsters = createAllMonsters(fbKotlin, monsters)
fbKotlin.finish(allMonsters)
}
@Benchmark
fun monstersjava() {
fbJava.clear()
val monsterName = fbJava.createString("MonsterName");
val inv = JMonster.createInventoryVector(fbJava, byteArrayOf(0, 1, 2, 3, 4).asUByteArray())
val monsters = JAllMonsters.createMonstersVector(fbJava, IntArray(repetition) {
JMonster.startJMonster(fbJava)
JMonster.addName(fbJava, monsterName)
JMonster.addPos(fbJava, JVec3.createJVec3(fbJava, 1.0f, 2.0f, 3.0f))
JMonster.addHp(fbJava, 80)
JMonster.addMana(fbJava, 150)
JMonster.addInventory(fbJava, inv)
JMonster.endJMonster(fbJava)
})
val allMonsters = JAllMonsters.createJAllMonsters(fbJava, monsters)
fbJava.finish(allMonsters)
}
}

2
kotlin/benchmark/src/jvmMain/kotlin/com/google/flatbuffers/kotlin/benchmark/FlexBuffersBenchmark.kt
View File

@@ -13,6 +13,8 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
@file:OptIn(ExperimentalUnsignedTypes::class)
package com.google.flatbuffers.kotlin.benchmark
import com.google.flatbuffers.ArrayReadWriteBuf
import com.google.flatbuffers.FlexBuffers

7
kotlin/benchmark/src/jvmMain/kotlin/com/google/flatbuffers/kotlin/benchmark/JsonBenchmark.kt
View File

@@ -53,9 +53,10 @@ open class JsonBenchmark {
val fbParser = JSONParser()
final val twitterData = this.javaClass.classLoader.getResourceAsStream("twitter.json")!!.readBytes()
final val canadaData = this.javaClass.classLoader.getResourceAsStream("canada.json")!!.readBytes()
final val citmData = this.javaClass.classLoader.getResourceAsStream("citm_catalog.json")!!.readBytes()
final val classLoader = this.javaClass.classLoader
final val twitterData = classLoader.getResourceAsStream("twitter.json")!!.readBytes()
final val canadaData = classLoader.getResourceAsStream("canada.json")!!.readBytes()
final val citmData = classLoader.getResourceAsStream("citm_catalog.json")!!.readBytes()
val fbCitmRef = JSONParser().parse(ArrayReadBuffer(citmData))
val moshiCitmRef = moshi.adapter(Map::class.java).fromJson(citmData.decodeToString())

14
kotlin/benchmark/src/jvmMain/kotlin/com/google/flatbuffers/kotlin/benchmark/UTF8Benchmark.kt
View File

@@ -35,14 +35,14 @@ import kotlin.random.Random
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@Measurement(iterations = 100, time = 1, timeUnit = TimeUnit.MICROSECONDS)
class UTF8Benchmark {
open class UTF8Benchmark {
private final val sampleSize = 5000
private final val stringSize = 25
final var sampleSmallUtf8 = (0..sampleSize).map { populateUTF8(stringSize) }.toList()
final var sampleSmallUtf8Decoded = sampleSmallUtf8.map { it.encodeToByteArray() }.toList()
final var sampleSmallAscii = (0..sampleSize).map { populateAscii(stringSize) }.toList()
final var sampleSmallAsciiDecoded = sampleSmallAscii.map { it.encodeToByteArray() }.toList()
private val sampleSize = 5000
private val stringSize = 25
private var sampleSmallUtf8 = (0..sampleSize).map { populateUTF8(stringSize) }.toList()
private var sampleSmallUtf8Decoded = sampleSmallUtf8.map { it.encodeToByteArray() }.toList()
private var sampleSmallAscii = (0..sampleSize).map { populateAscii(stringSize) }.toList()
private var sampleSmallAsciiDecoded = sampleSmallAscii.map { it.encodeToByteArray() }.toList()
@Setup
fun setUp() {

25
kotlin/build.gradle.kts
View File

@@ -1,5 +1,7 @@
group = "com.google.flatbuffers"
version = "2.0.0-SNAPSHOT"
import org.gradle.internal.impldep.org.testng.ITestResult.STARTED
import org.jetbrains.kotlin.gradle.dsl.KotlinCommonOptions
import org.jetbrains.kotlin.gradle.tasks.KotlinCompile
import java.nio.charset.StandardCharsets
buildscript {
repositories {
@@ -21,3 +23,22 @@ allprojects {
mavenCentral()
}
}
tasks.withType<org.jetbrains.kotlin.gradle.dsl.KotlinCompile<KotlinCommonOptions>>().configureEach {
kotlinOptions {
freeCompilerArgs += "-progressive" // https://kotlinlang.org/docs/whatsnew13.html#progressive-mode
}
}
tasks.withType<org.jetbrains.kotlin.gradle.tasks.KotlinJvmCompile>().configureEach {
kotlinOptions {
jvmTarget = JavaVersion.VERSION_1_8.toString()
freeCompilerArgs += "-Xjvm-default=all"
}
}
tasks.withType<JavaCompile> {
options.encoding = StandardCharsets.UTF_8.toString()
sourceCompatibility = JavaVersion.VERSION_1_8.toString()
targetCompatibility = JavaVersion.VERSION_1_8.toString()
}

124
kotlin/flatbuffers-kotlin/build.gradle.kts
View File

@@ -1,29 +1,37 @@
import org.gradle.internal.impldep.org.fusesource.jansi.AnsiRenderer.test
import org.jetbrains.kotlin.gradle.plugin.mpp.apple.XCFramework
import org.jetbrains.kotlin.cli.common.toBooleanLenient
import org.jetbrains.kotlin.gradle.plugin.mpp.KotlinNativeTarget
import org.jetbrains.kotlin.gradle.plugin.mpp.NativeBuildType
import org.jetbrains.kotlin.gradle.plugin.mpp.apple.XCFrameworkConfig
plugins {
kotlin("multiplatform")
}
val libName = "Flatbuffers"
group = "com.google.flatbuffers.kotlin"
version = "2.0.0-SNAPSHOT"
kotlin {
explicitApi()
jvm()
js {
js(IR) {
browser {
testTask {
useKarma {
useChromeHeadless()
}
testTask {
enabled = false
}
}
binaries.executable()
}
macosX64()
iosArm32()
macosArm64()
iosArm64()
iosX64()
iosSimulatorArm64()
sourceSets {
val commonMain by getting {
dependencies {
implementation(kotlin("stdlib-common"))
@@ -34,47 +42,33 @@ kotlin {
dependencies {
implementation(kotlin("test"))
}
kotlin.srcDir("src/commonTest/generated/kotlin/")
}
val jvmTest by getting {
dependencies {
implementation(kotlin("test-junit"))
implementation("com.google.flatbuffers:flatbuffers-java:2.0.3")
}
}
val jvmMain by getting {
kotlin.srcDir("java")
}
val jsMain by getting {
dependsOn(commonMain)
}
val jsTest by getting {
dependsOn(commonTest)
dependencies {
implementation(kotlin("test-js"))
}
}
val macosX64Main by getting
val macosArm64Main by getting
val iosArm64Main by getting
val iosSimulatorArm64Main by getting
val nativeMain by creating {
dependsOn(commonMain)
}
val nativeTest by creating {
// this sourceSet will hold common cold for all iOS targets
dependsOn(commonMain)
}
val macosX64Main by getting {
dependsOn(nativeMain)
}
val iosArm32Main by getting {
dependsOn(nativeMain)
}
val iosArm64Main by getting {
dependsOn(nativeMain)
}
val iosX64Main by getting {
dependsOn(nativeMain)
macosArm64Main.dependsOn(this)
macosX64Main.dependsOn(this)
iosArm64Main.dependsOn(this)
iosSimulatorArm64Main.dependsOn(this)
}
all {
languageSettings.enableLanguageFeature("InlineClasses")
languageSettings.optIn("kotlin.ExperimentalUnsignedTypes")
}
}
@@ -83,4 +77,66 @@ kotlin {
// Fixes JS issue: https://youtrack.jetbrains.com/issue/KT-49109
rootProject.plugins.withType<org.jetbrains.kotlin.gradle.targets.js.nodejs.NodeJsRootPlugin> {
rootProject.the<org.jetbrains.kotlin.gradle.targets.js.nodejs.NodeJsRootExtension>().nodeVersion = "16.0.0"
}
// Use the default greeting
tasks.register<GenerateFBTestClasses>("generateFBTestClassesKt") {
inputFiles.setFrom("$rootDir/../tests/monster_test.fbs",
"$rootDir/../tests/dictionary_lookup.fbs",
// @todo Seems like nesting code generation is broken for all generators.
// disabling test for now.
// "$rootDir/../tests/namespace_test/namespace_test1.fbs",
// "$rootDir/../tests/namespace_test/namespace_test2.fbs",
"$rootDir/../tests/union_vector/union_vector.fbs",
"$rootDir/../tests/optional_scalars.fbs")
includeFolder.set("$rootDir/../tests/include_test")
outputFolder.set("${projectDir}/src/commonTest/generated/kotlin/")
variant.set("kotlin-kmp")
}
project.tasks.forEach {
if (it.name.contains("compileKotlin"))
it.dependsOn("generateFBTestClassesKt")
}
fun String.intProperty() = findProperty(this).toString().toInt()
abstract class GenerateFBTestClasses : DefaultTask() {
@get:InputFiles
abstract val inputFiles: ConfigurableFileCollection
@get:Input
abstract val includeFolder: Property<String>
@get:Input
abstract val outputFolder: Property<String>
@get:Input
abstract val variant: Property<String>
@Inject
protected open fun getExecActionFactory(): org.gradle.process.internal.ExecActionFactory? {
throw UnsupportedOperationException()
}
init {
includeFolder.set("")
}
@TaskAction
fun compile() {
val execAction = getExecActionFactory()!!.newExecAction()
val sources = inputFiles.asPath.split(":")
val args = mutableListOf("flatc","-o", outputFolder.get(), "--${variant.get()}")
if (includeFolder.get().isNotEmpty()) {
args.add("-I")
args.add(includeFolder.get())
}
args.addAll(sources)
println(args)
execAction.commandLine = args
print(execAction.execute())
}
}

253
kotlin/flatbuffers-kotlin/src/commonMain/kotlin/com/google/flatbuffers/kotlin/Buffers.kt
View File

@@ -110,16 +110,23 @@ public interface ReadBuffer {
public fun getDouble(index: Int): Double
/**
* Read an UTF-8 string from the buffer.
* Read a UTF-8 string from the buffer.
* @param start initial element of the string
* @param size size of the string in bytes.
* @return a `String`
*/
public fun getString(start: Int, size: Int): String
public fun getString(start: Int = 0, size: Int = limit): String
/**
* Read a ByteArray from the buffer.
* @param start position from the [ReadBuffer] to be read
* @param length maximum number of bytes to be written in the buffer
*/
public fun getBytes(array: ByteArray, start: Int, length: Int = array.size)
/**
* Expose [ReadBuffer] as an array of bytes.
* This method is meant to be as efficient as possible, so for a array-backed [ReadBuffer], it should
* This method is meant to be as efficient as possible, so for an array-backed [ReadBuffer], it should
* return its own internal data. In case access to internal data is not possible,
* a copy of the data into an array of bytes might occur.
* @return [ReadBuffer] as an array of bytes
@@ -151,6 +158,29 @@ public interface ReadWriteBuffer : ReadBuffer {
*/
public fun clear()
/**
* Request capacity of the buffer relative to [writePosition]. In case buffer is already larger
* than the requested, this method will just return true. Otherwise,
* It might try to resize the buffer. In case of being unable to allocate
* enough memory, an exception will be thrown.
* @param additional capacity in bytes to be added on top of [writePosition]
* @param copyAtEnd copy current data at the end of new underlying buffer
* @return new capacity in bytes
*/
public fun requestAdditionalCapacity(additional: Int, copyAtEnd: Boolean = false): Int =
requestCapacity(writePosition + additional, copyAtEnd)
/**
* Request capacity of the buffer in absolute values. In case buffer is already larger
* than the requested the method is a no-op. Otherwise,
* It might try to resize the buffer. In case of being unable to allocate
* enough memory, an exception will be thrown.
* @param capacity new capacity
* @param copyAtEnd copy current data at the end of new underlying buffer
* @return new capacity in bytes
*/
public fun requestCapacity(capacity: Int, copyAtEnd: Boolean = false): Int
/**
* Put a [Boolean] into the buffer at [writePosition] . Booleans as stored as single byte.
* Write position will be incremented.
@@ -164,7 +194,15 @@ public interface ReadWriteBuffer : ReadBuffer {
* @param start initial position on value to be copied
* @param length amount of bytes to be copied
*/
public fun put(value: ByteArray, start: Int, length: Int)
public fun put(value: ByteArray, start: Int = 0, length: Int = value.size)
/**
* Put an array of bytes into the buffer at [writePosition]. Write position will be incremented.
* @param value [ReadBuffer] the data to be copied
* @param start initial position on value to be copied
* @param length amount of bytes to be copied
*/
public fun put(value: ReadBuffer, start: Int = 0, length: Int = value.limit - start)
/**
* Write a [Byte] into the buffer at [writePosition]. Write position will be incremented.
@@ -182,7 +220,7 @@ public interface ReadWriteBuffer : ReadBuffer {
public fun put(value: Short)
/**
* Writea [UShort] into in the buffer at [writePosition]. Write position will be incremented.
* Write a [UShort] into in the buffer at [writePosition]. Write position will be incremented.
*/
public fun put(value: UShort)
@@ -224,7 +262,7 @@ public interface ReadWriteBuffer : ReadBuffer {
* Write a [String] encoded as UTF-8 into the buffer at [writePosition]. Write position will be incremented.
* @return size in bytes of the encoded string
*/
public fun put(value: String, encodedLength: Int = -1): Int
public fun put(value: CharSequence, encodedLength: Int = -1): Int
/**
* Write an array of bytes into the buffer.
@@ -233,7 +271,16 @@ public interface ReadWriteBuffer : ReadBuffer {
* @param srcStart initial position on [src] that will be copied.
* @param srcLength amount of bytes to be copied
*/
public operator fun set(dstIndex: Int, src: ByteArray, srcStart: Int, srcLength: Int)
public fun set(dstIndex: Int, src: ByteArray, srcStart: Int = 0, srcLength: Int = src.size)
/**
* Write an array of bytes into the buffer.
* @param dstIndex initial position where [src] will be copied into.
* @param src the data to be copied.
* @param srcStart initial position on [src] that will be copied.
* @param srcLength amount of bytes to be copied
*/
public operator fun set(dstIndex: Int, src: ReadBuffer, srcStart: Int = 0, srcLength: Int)
/**
* Write [Boolean] into a given position [index] on the buffer. Booleans as stored as single byte.
@@ -301,63 +348,95 @@ public interface ReadWriteBuffer : ReadBuffer {
*/
public fun set(index: Int, value: Double)
public fun fill(value: Byte, start: Int, end: Int)
/**
* Current position of the buffer to be written. It will be automatically updated on [put] operations.
*/
public var writePosition: Int
/**
* Defines the size of the message in the buffer. It also determines last position that buffer
* can be read or write. Last byte to be accessed is in position `limit() -1`.
* @return indicate last position
* Creates a new [ReadWriteBuffer] point to a region of the current buffer, starting at [offset] with size [size].
* @param offset starting position of the [ReadWriteBuffer]
* @param size in bytes of the [ReadWriteBuffer]
* @return [ReadWriteBuffer] slice.
*/
override val limit: Int
public fun writeSlice(offset: Int, size: Int): ReadWriteBuffer
/**
* Request capacity of the buffer. In case buffer is already larger
* than the requested, this method will just return true. Otherwise
* It might try to resize the buffer. In case of being unable to allocate
* enough memory, an exception will be thrown.
* Special operation where we increase the backed buffer size to [capacity]
* and shift all already written data to the end of the buffer.
*
* This function is mostly used when creating a Flatbuffer message, as
* data is written from the end of the buffer towards index 0.
* @param capacity required in bytes
* @return new capacity in bytes
*/
public fun requestCapacity(capacity: Int)
public fun moveWrittenDataToEnd(capacity: Int): Int
/**
* Maximum size in bytes that the backed buffer supports.
*/
public val capacity: Int
/**
* Defines last relative position of the backed buffer that can be written.
* Any addition to the buffer that goes beyond will throw an exception
* instead of regrow the buffer (default behavior).
*/
public val writeLimit: Int
}
public open class ArrayReadBuffer(protected var buffer: ByteArray, override val limit: Int = buffer.size) : ReadBuffer {
public open class ArrayReadBuffer(protected var buffer: ByteArray,
// offsets writePosition against backed buffer e.g. offset = 1, writePosition = 1
// will write first byte at position 2 of the backed buffer
internal val offset: Int = 0,
override val limit: Int = buffer.size - offset) : ReadBuffer {
override fun findFirst(value: Byte, start: Int, end: Int): Int {
val e = min(end, limit)
val s = max(0, start)
val s = max(0, this.offset + start)
for (i in s until e) if (buffer[i] == value) return i
return -1
}
override fun getBoolean(index: Int): Boolean = buffer[index] != 0.toByte()
override fun getBoolean(index: Int): Boolean = buffer[offset + index] != 0.toByte()
override operator fun get(index: Int): Byte = buffer[index]
override operator fun get(index: Int): Byte = buffer[offset + index]
override fun getUByte(index: Int): UByte = buffer.getUByte(index)
override fun getUByte(index: Int): UByte = buffer.getUByte(offset + index)
override fun getShort(index: Int): Short = buffer.getShort(index)
override fun getShort(index: Int): Short = buffer.getShort(offset + index)
override fun getUShort(index: Int): UShort = buffer.getUShort(index)
override fun getUShort(index: Int): UShort = buffer.getUShort(offset + index)
override fun getInt(index: Int): Int = buffer.getInt(index)
override fun getInt(index: Int): Int = buffer.getInt(offset + index)
override fun getUInt(index: Int): UInt = buffer.getUInt(index)
override fun getUInt(index: Int): UInt = buffer.getUInt(offset + index)
override fun getLong(index: Int): Long = buffer.getLong(index)
override fun getLong(index: Int): Long = buffer.getLong(offset + index)
override fun getULong(index: Int): ULong = buffer.getULong(index)
override fun getULong(index: Int): ULong = buffer.getULong(offset + index)
override fun getFloat(index: Int): Float = buffer.getFloat(index)
override fun getFloat(index: Int): Float = buffer.getFloat(offset + index)
override fun getDouble(index: Int): Double = buffer.getDouble(index)
override fun getDouble(index: Int): Double = buffer.getDouble(offset + index)
override fun getString(start: Int, size: Int): String = buffer.decodeToString(start, start + size)
override fun getString(start: Int, size: Int): String = buffer.decodeToString(this.offset + start,
this.offset + start + size)
override fun getBytes(array: ByteArray, start: Int, length: Int) {
val end = min(this.offset + start + length, buffer.size)
var j = 0
for (i in this.offset + start until end) {
array[j++] = buffer[i]
}
}
override fun data(): ByteArray = buffer
override fun slice(start: Int, size: Int): ReadBuffer = ArrayReadBuffer(buffer, limit)
override fun slice(start: Int, size: Int): ReadBuffer = ArrayReadBuffer(buffer, this.offset + start, size)
}
/**
* Implements `[ReadWriteBuffer]` using [ByteArray] as backing buffer. Using array of bytes are
@@ -365,14 +444,20 @@ public open class ArrayReadBuffer(protected var buffer: ByteArray, override val
*
* This class is not thread-safe, meaning that
* it must operate on a single thread. Operating from
* multiple thread leads into a undefined behavior
* multiple thread leads into an undefined behavior
*
* All operations assumes Little Endian byte order.
* All operations assume Little Endian byte order.
*/
public class ArrayReadWriteBuffer(
buffer: ByteArray,
override var writePosition: Int = 0
) : ArrayReadBuffer(buffer, writePosition), ReadWriteBuffer {
offset: Int = 0,
// Defines last position of the backed buffer that can be written.
// Any addition to the buffer that goes beyond will throw an exception
// instead of regrow the buffer (default behavior).
public override val writeLimit: Int = -1,
override var writePosition: Int = offset
) : ArrayReadBuffer(buffer, offset, writePosition), ReadWriteBuffer {
public constructor(initialCapacity: Int = 10) : this(ByteArray(initialCapacity))
@@ -390,6 +475,11 @@ public class ArrayReadWriteBuffer(
writePosition += length
}
override fun put(value: ReadBuffer, start: Int, length: Int) {
set(writePosition, value, start, length)
writePosition += length
}
override fun put(value: Byte) {
set(writePosition, value)
writePosition++
@@ -440,50 +530,87 @@ public class ArrayReadWriteBuffer(
writePosition += 8
}
override fun put(value: String, encodedLength: Int): Int {
override fun put(value: CharSequence, encodedLength: Int): Int {
val length = if (encodedLength != -1) encodedLength else Utf8.encodedLength(value)
withCapacity(writePosition + length) {
writePosition = setString(writePosition, value)
}
writePosition = buffer.setCharSequence(writePosition, value)
return length
}
override fun set(index: Int, value: Boolean) {
set(index, if (value) 1.toByte() else 0.toByte())
buffer[index] = if (value) 1.toByte() else 0.toByte()
}
override operator fun set(dstIndex: Int, src: ByteArray, srcStart: Int, srcLength: Int) {
withCapacity(dstIndex + (srcLength + srcStart)) {
src.copyInto(buffer, dstIndex, srcStart, srcStart + srcLength)
}
override fun set(dstIndex: Int, src: ByteArray, srcStart: Int, srcLength: Int) {
src.copyInto(buffer, dstIndex, srcStart, srcStart + srcLength)
}
override operator fun set(index: Int, value: Byte): Unit = withCapacity(index + 1) { set(index, value) }
override operator fun set(index: Int, value: UByte): Unit = withCapacity(index + 1) { setUByte(index, value) }
override operator fun set(index: Int, value: Short): Unit = withCapacity(index + 2) { setShort(index, value) }
override operator fun set(index: Int, value: UShort): Unit = withCapacity(index + 2) { setUShort(index, value) }
override operator fun set(index: Int, value: Int): Unit = withCapacity(index + 4) { setInt(index, value) }
override operator fun set(index: Int, value: UInt): Unit = withCapacity(index + 4) { setUInt(index, value) }
override operator fun set(index: Int, value: Long): Unit = withCapacity(index + 8) { setLong(index, value) }
override operator fun set(index: Int, value: ULong): Unit = withCapacity(index + 8) { setULong(index, value) }
override operator fun set(index: Int, value: Float): Unit = withCapacity(index + 4) { setFloat(index, value) }
override operator fun set(index: Int, value: Double): Unit = withCapacity(index + 8) { setDouble(index, value) }
override operator fun set(dstIndex: Int, src: ReadBuffer, srcStart: Int, srcLength: Int) {
when(src) {
is ArrayReadBuffer -> {
src.data().copyInto(buffer, dstIndex, src.offset + srcStart, src.offset + srcStart + srcLength)
}
else -> {
for (i in 0 until srcLength) {
buffer[dstIndex + i] = src[srcStart + i]
}
}
}
}
override fun requestCapacity(capacity: Int) {
override operator fun set(index: Int, value: Byte) { buffer[index] = value }
override operator fun set(index: Int, value: UByte) { buffer.setUByte(index, value) }
override operator fun set(index: Int, value: Short) { buffer.setShort(index, value) }
override operator fun set(index: Int, value: UShort) { buffer.setUShort(index, value) }
override operator fun set(index: Int, value: Int) { buffer.setInt(index, value) }
override operator fun set(index: Int, value: UInt) { buffer.setUInt(index, value) }
override operator fun set(index: Int, value: Long) { buffer.setLong(index, value) }
override operator fun set(index: Int, value: ULong) { buffer.setULong(index, value) }
override operator fun set(index: Int, value: Float) { buffer.setFloat(index, value) }
override operator fun set(index: Int, value: Double) { buffer.setDouble(index, value) }
override fun fill(value: Byte, start: Int, end: Int) { buffer.fill(value, start, end) }
/**
* Request capacity of the buffer. In case buffer is already larger
* than the requested, it is a no-op. Otherwise,
* It might try to resize the buffer. In case of being unable to allocate
* enough memory, an exception will be thrown.
* @param capacity new capacity
* @param copyAtEnd copy current data at the end of new underlying buffer
*/
override fun requestCapacity(capacity: Int, copyAtEnd: Boolean): Int {
if (capacity < 0) error("Capacity may not be negative (likely a previous int overflow)")
if (buffer.size >= capacity) return
if (buffer.size >= capacity) return buffer.size
if (writeLimit > 0 && writeLimit + offset >= buffer.size) error("Buffer in writeLimit mode. In writeLimit mode" +
" the buffer does not grow automatically and any write beyond writeLimit will throw exception. " +
"(writeLimit: $writeLimit, newCapacity: $capacity")
// implemented in the same growing fashion as ArrayList
val oldCapacity = buffer.size
var newCapacity = oldCapacity + (oldCapacity shr 1)
if (newCapacity < capacity) { // Note: this also catches newCapacity int overflow
newCapacity = capacity
if (oldCapacity == Int.MAX_VALUE - 8) { // Ensure we don't grow beyond what fits in an int.
error("FlatBuffers: cannot grow buffer beyond 2 gigabytes.")
}
buffer = buffer.copyOf(newCapacity)
//(old_buf_size & 0xC0000000) != 0 ? MAX_BUFFER_SIZE : old_buf_size << 1;
var newCapacity = 8
while (newCapacity < capacity) { // Note: this also catches newCapacity int overflow
newCapacity = if (newCapacity and -0x40000000 != 0) Int.MAX_VALUE - 8 else newCapacity shl 1
}
val newBuffer = ByteArray(newCapacity)
buffer.copyInto(newBuffer, if (copyAtEnd) newBuffer.size - buffer.size else 0)
buffer = newBuffer
return newCapacity
}
private inline fun withCapacity(size: Int, crossinline action: ByteArray.() -> Unit) {
requestCapacity(size)
buffer.action()
override fun writeSlice(offset: Int, size: Int): ReadWriteBuffer {
return ArrayReadWriteBuffer(this.buffer, offset=offset, writeLimit=size)
}
override fun moveWrittenDataToEnd(capacity: Int): Int = requestCapacity(capacity, true)
override val capacity: Int
get() = buffer.size
}
public val emptyBuffer: ReadWriteBuffer = ArrayReadWriteBuffer(ByteArray(1))

36
kotlin/flatbuffers-kotlin/src/commonMain/kotlin/com/google/flatbuffers/kotlin/ByteArray.kt
View File

@@ -14,13 +14,14 @@
* limitations under the License.
*/
@file:Suppress("NOTHING_TO_INLINE")
package com.google.flatbuffers.kotlin
import kotlin.experimental.and
internal fun ByteArray.getString(index: Int, size: Int): String = Utf8.decodeUtf8Array(this, index, size)
internal fun ByteArray.setString(index: Int, value: String): Int =
internal fun ByteArray.setCharSequence(index: Int, value: CharSequence): Int =
Utf8.encodeUtf8Array(value, this, index, this.size - index)
// List of functions that needs to be implemented on all platforms.
@@ -35,7 +36,7 @@ internal expect inline fun ByteArray.getFloat(index: Int): Float
internal expect inline fun ByteArray.getDouble(index: Int): Double
internal expect inline fun ByteArray.setUByte(index: Int, value: UByte)
internal expect inline fun ByteArray.setShort(index: Int, value: Short)
public expect inline fun ByteArray.setShort(index: Int, value: Short)
internal expect inline fun ByteArray.setUShort(index: Int, value: UShort)
internal expect inline fun ByteArray.setInt(index: Int, value: Int)
internal expect inline fun ByteArray.setUInt(index: Int, value: UInt)
@@ -102,43 +103,20 @@ public object ByteArrayOps {
public inline fun setUInt(ary: ByteArray, index: Int, value: UInt): Unit = setInt(ary, index, value.toInt())
public inline fun setLong(ary: ByteArray, index: Int, value: Long) {
var idx = index
var i = value.toInt()
ary[idx++] = (i and 0xff).toByte()
ary[idx++] = (i shr 8 and 0xff).toByte()
ary[idx++] = (i shr 16 and 0xff).toByte()
ary[idx++] = (i shr 24 and 0xff).toByte()
setInt(ary, index, i)
i = (value shr 32).toInt()
ary[idx++] = (i and 0xff).toByte()
ary[idx++] = (i shr 8 and 0xff).toByte()
ary[idx++] = (i shr 16 and 0xff).toByte()
ary[idx] = (i shr 24 and 0xff).toByte()
setInt(ary, index + 4, i)
}
public inline fun setULong(ary: ByteArray, index: Int, value: ULong): Unit = setLong(ary, index, value.toLong())
public inline fun setFloat(ary: ByteArray, index: Int, value: Float) {
var idx = index
val iValue: Int = value.toRawBits()
ary[idx++] = (iValue and 0xff).toByte()
ary[idx++] = (iValue shr 8 and 0xff).toByte()
ary[idx++] = (iValue shr 16 and 0xff).toByte()
ary[idx] = (iValue shr 24 and 0xff).toByte()
setInt(ary, index, value.toRawBits())
}
public inline fun setDouble(ary: ByteArray, index: Int, value: Double) {
var idx = index
val lValue: Long = value.toRawBits()
var i = lValue.toInt()
ary[idx++] = (i and 0xff).toByte()
ary[idx++] = (i shr 8 and 0xff).toByte()
ary[idx++] = (i shr 16 and 0xff).toByte()
ary[idx++] = (i shr 24 and 0xff).toByte()
i = (lValue shr 32).toInt()
ary[idx++] = (i and 0xff).toByte()
ary[idx++] = (i shr 8 and 0xff).toByte()
ary[idx++] = (i shr 16 and 0xff).toByte()
ary[idx] = (i shr 24 and 0xff).toByte()
setLong(ary, index, value.toRawBits())
}
public inline fun getFloat(ary: ByteArray, index: Int): Float = Float.fromBits(getInt(ary, index))

1105
kotlin/flatbuffers-kotlin/src/commonMain/kotlin/com/google/flatbuffers/kotlin/FlatBufferBuilder.kt Normal file
View File

File diff suppressed because it is too large Load Diff

367
kotlin/flatbuffers-kotlin/src/commonMain/kotlin/com/google/flatbuffers/kotlin/Flatbuffers.kt Normal file
View File

@@ -0,0 +1,367 @@
/*
* Copyright 2021 Google Inc. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.google.flatbuffers.kotlin
import kotlin.jvm.JvmInline
import kotlin.math.min
// For now a typealias to guarantee type safety.
public typealias UnionOffset = Offset<Any>
public typealias UnionOffsetArray = OffsetArray<Any>
public typealias StringOffsetArray = OffsetArray<String>
public inline fun UnionOffsetArray(size: Int, crossinline call: (Int) -> Offset<Any>): UnionOffsetArray =
UnionOffsetArray(IntArray(size) { call(it).value })
public inline fun StringOffsetArray(size: Int, crossinline call: (Int) -> Offset<String>): StringOffsetArray =
StringOffsetArray(IntArray(size) { call(it).value })
/**
* Represents a "pointer" to a pointer types (table, string, struct) within the buffer
*/
@JvmInline
public value class Offset<T>(public val value: Int) {
public fun toUnion(): UnionOffset = UnionOffset(value)
}
/**
* Represents an array of offsets. Used to avoid boxing
* offset types.
*/
@JvmInline
public value class OffsetArray<T>(public val value: IntArray) {
public inline val size: Int
get() = value.size
public inline operator fun get(index: Int): Offset<T> = Offset(value[index])
}
public inline fun <T> OffsetArray(size: Int, crossinline call: (Int) -> Offset<T>): OffsetArray<T> {
return OffsetArray(IntArray(size) { call(it).value })
}
/**
* Represents a "pointer" to a vector type with elements T
*/
@JvmInline
public value class VectorOffset<T>(public val value: Int)
public fun <T> Int.toOffset(): Offset<T> = Offset(this)
public operator fun <T> Offset<T>.minus(other: Int): Offset<T> = Offset(this.value - other)
public operator fun <T> Int.minus(other: Offset<T>): Int {
return this - other.value
}
/**
* All tables in the generated code derive from this class, and add their own accessors.
*/
public open class Table {
/** Used to hold the position of the `bb` buffer. */
public var bufferPos: Int = 0
/** The underlying ReadWriteBuffer to hold the data of the Table. */
public var bb: ReadWriteBuffer = emptyBuffer
/** Used to hold the vtable position. */
public var vtableStart: Int = 0
/** Used to hold the vtable size. */
public var vtableSize: Int = 0
protected inline fun <reified T> Int.invalid(default: T, valid: (Int) -> T) : T =
if (this != 0) valid(this) else default
protected inline fun <reified T> lookupField(i: Int, default: T, found: (Int) -> T) : T =
offset(i).invalid(default) { found(it) }
/**
* Look up a field in the vtable.
*
* @param vtableOffset An `int` offset to the vtable in the Table's ReadWriteBuffer.
* @return Returns an offset into the object, or `0` if the field is not present.
*/
public fun offset(vtableOffset: Int): Int =
if (vtableOffset < vtableSize) bb.getShort(vtableStart + vtableOffset).toInt() else 0
/**
* Retrieve a relative offset.
*
* @param offset An `int` index into the Table's ReadWriteBuffer containing the relative offset.
* @return Returns the relative offset stored at `offset`.
*/
public fun indirect(offset: Int): Int = offset + bb.getInt(offset)
/**
* Create a Java `String` from UTF-8 data stored inside the FlatBuffer.
*
* This allocates a new string and converts to wide chars upon each access,
* which is not very efficient. Instead, each FlatBuffer string also comes with an
* accessor based on __vector_as_ReadWriteBuffer below, which is much more efficient,
* assuming your Java program can handle UTF-8 data directly.
*
* @param offset An `int` index into the Table's ReadWriteBuffer.
* @return Returns a `String` from the data stored inside the FlatBuffer at `offset`.
*/
public fun string(offset: Int): String = string(offset, bb)
/**
* Get the length of a vector.
*
* @param offset An `int` index into the Table's ReadWriteBuffer.
* @return Returns the length of the vector whose offset is stored at `offset`.
*/
public fun vectorLength(offset: Int): Int {
var newOffset = offset
newOffset += bufferPos
newOffset += bb.getInt(newOffset)
return bb.getInt(newOffset)
}
/**
* Get the start data of a vector.
*
* @param offset An `int` index into the Table's ReadWriteBuffer.
* @return Returns the start of the vector data whose offset is stored at `offset`.
*/
public fun vector(offset: Int): Int {
var newOffset = offset
newOffset += bufferPos
return newOffset + bb.getInt(newOffset) + Int.SIZE_BYTES // data starts after the length
}
/**
* Initialize vector as a ReadWriteBuffer.
*
* This is more efficient than using duplicate, since it doesn't copy the data
* nor allocates a new [ReadBuffer], creating no garbage to be collected.
*
* @param buffer The [ReadBuffer] for the array
* @param vectorOffset The position of the vector in the byte buffer
* @param elemSize The size of each element in the array
* @return The [ReadBuffer] for the array
*/
public fun vectorAsBuffer(buffer: ReadWriteBuffer, vectorOffset: Int, elemSize: Int): ReadBuffer {
val o = offset(vectorOffset)
if (o == 0) return emptyBuffer
val vectorStart = vector(o)
return buffer.slice(vectorStart, vectorLength(o) * elemSize)
}
/**
* Initialize any Table-derived type to point to the union at the given `offset`.
*
* @param t A `Table`-derived type that should point to the union at `offset`.
* @param offset An `int` index into the Table's ReadWriteBuffer.
* @return Returns the Table that points to the union at `offset`.
*/
public fun union(t: Table, offset: Int): Table = union(t, offset, bb)
/**
* Sort tables by the key.
*
* @param offsets An 'int' indexes of the tables into the bb.
* @param bb A `ReadWriteBuffer` to get the tables.
*/
public fun <T> sortTables(offsets: Array<Offset<T>>, bb: ReadWriteBuffer) {
val off = offsets.sortedWith { o1, o2 -> keysCompare(o1, o2, bb) }
for (i in offsets.indices) offsets[i] = off[i]
}
/**
* Compare two tables by the key.
*
* @param o1 An 'Integer' index of the first key into the bb.
* @param o2 An 'Integer' index of the second key into the bb.
* @param buffer A `ReadWriteBuffer` to get the keys.
*/
public open fun keysCompare(o1: Offset<*>, o2: Offset<*>, buffer: ReadWriteBuffer): Int = 0
/**
* Re-init the internal state with an external buffer `ReadWriteBuffer` and an offset within.
*
* This method exists primarily to allow recycling Table instances without risking memory leaks
* due to `ReadWriteBuffer` references.
*/
public inline fun <reified T: Table> reset(i: Int, reuseBuffer: ReadWriteBuffer): T {
bb = reuseBuffer
if (bb != emptyBuffer) {
bufferPos = i
vtableStart = bufferPos - bb.getInt(bufferPos)
vtableSize = bb.getShort(vtableStart).toInt()
} else {
bufferPos = 0
vtableStart = 0
vtableSize = 0
}
return this as T
}
/**
* Resets the internal state with a null `ReadWriteBuffer` and a zero position.
*
* This method exists primarily to allow recycling Table instances without risking memory leaks
* due to `ReadWriteBuffer` references. The instance will be unusable until it is assigned
* again to a `ReadWriteBuffer`.
*/
public inline fun <reified T: Table> reset(): T = reset(0, emptyBuffer)
public companion object {
public fun offset(vtableOffset: Int, offset: Offset<*>, bb: ReadWriteBuffer): Int {
val vtable: Int = bb.capacity - offset.value
return bb.getShort(vtable + vtableOffset - bb.getInt(vtable)) + vtable
}
/**
* Retrieve a relative offset.
*
* @param offset An `int` index into a ReadWriteBuffer containing the relative offset.
* @param bb from which the relative offset will be retrieved.
* @return Returns the relative offset stored at `offset`.
*/
public fun indirect(offset: Int, bb: ReadWriteBuffer): Int {
return offset + bb.getInt(offset)
}
/**
* Create a Java `String` from UTF-8 data stored inside the FlatBuffer.
*
* This allocates a new string and converts to wide chars upon each access,
* which is not very efficient. Instead, each FlatBuffer string also comes with an
* accessor based on __vector_as_ReadWriteBuffer below, which is much more efficient,
* assuming your Java program can handle UTF-8 data directly.
*
* @param offset An `int` index into the Table's ReadWriteBuffer.
* @param bb Table ReadWriteBuffer used to read a string at given offset.
* @return Returns a `String` from the data stored inside the FlatBuffer at `offset`.
*/
public fun string(offset: Int, bb: ReadWriteBuffer): String {
var newOffset = offset
newOffset += bb.getInt(newOffset)
val length: Int = bb.getInt(newOffset)
return bb.getString(newOffset + Int.SIZE_BYTES, length)
}
/**
* Initialize any Table-derived type to point to the union at the given `offset`.
*
* @param t A `Table`-derived type that should point to the union at `offset`.
* @param offset An `int` index into the Table's ReadWriteBuffer.
* @param bb Table ReadWriteBuffer used to initialize the object Table-derived type.
* @return Returns the Table that points to the union at `offset`.
*/
public fun union(t: Table, offset: Int, bb: ReadWriteBuffer): Table =
t.reset(indirect(offset, bb), bb)
/**
* Check if a [ReadWriteBuffer] contains a file identifier.
*
* @param bb A `ReadWriteBuffer` to check if it contains the identifier
* `ident`.
* @param ident A `String` identifier of the FlatBuffer file.
* @return True if the buffer contains the file identifier
*/
public fun hasIdentifier(bb: ReadWriteBuffer?, ident: String): Boolean {
val identifierLength = 4
if (ident.length != identifierLength)
throw AssertionError("FlatBuffers: file identifier must be length $identifierLength")
for (i in 0 until identifierLength) {
if (ident[i].code.toByte() != bb!![bb.limit + Int.SIZE_BYTES + i]) return false
}
return true
}
/**
* Compare two strings in the buffer.
*
* @param offsetA An 'int' index of the first string into the bb.
* @param offsetB An 'int' index of the second string into the bb.
* @param bb A `ReadWriteBuffer` to get the strings.
*/
public fun compareStrings(offsetA: Int, offsetB: Int, bb: ReadWriteBuffer): Int {
var offset1 = offsetA
var offset2 = offsetB
offset1 += bb.getInt(offset1)
offset2 += bb.getInt(offset2)
val len1: Int = bb.getInt(offset1)
val len2: Int = bb.getInt(offset2)
val startPos1: Int = offset1 + Int.SIZE_BYTES
val startPos2: Int = offset2 + Int.SIZE_BYTES
val len: Int = min(len1, len2)
for (i in 0 until len) {
if (bb[i + startPos1] != bb[i + startPos2]) {
return bb[i + startPos1] - bb[i + startPos2]
}
}
return len1 - len2
}
/**
* Compare string from the buffer with the 'String' object.
*
* @param offset An 'int' index of the first string into the bb.
* @param key Second string as a byte array.
* @param bb A `ReadWriteBuffer` to get the first string.
*/
public fun compareStrings(offset: Int, key: ByteArray, bb: ReadWriteBuffer): Int {
var offset1 = offset
offset1 += bb.getInt(offset1)
val len1: Int = bb.getInt(offset1)
val len2 = key.size
val startPos: Int = offset1 + Int.SIZE_BYTES
val len: Int = min(len1, len2)
for (i in 0 until len) {
if (bb[i + startPos] != key[i]) return bb[i + startPos] - key[i]
}
return len1 - len2
}
}
}
/**
* All structs in the generated code derive from this class, and add their own accessors.
*/
public open class Struct {
/** Used to hold the position of the `bb` buffer. */
protected var bufferPos: Int = 0
/** The underlying ByteBuffer to hold the data of the Struct. */
protected var bb: ReadWriteBuffer = emptyBuffer
/**
* Re-init the internal state with an external buffer `ByteBuffer` and an offset within.
*
* This method exists primarily to allow recycling Table instances without risking memory leaks
* due to `ByteBuffer` references.
*/
protected inline fun <reified T: Struct> reset(i: Int, reuseBuffer: ReadWriteBuffer): T {
bb = reuseBuffer
bufferPos = if (bb != emptyBuffer) i else 0
return this as T
}
/**
* Resets internal state with a null `ByteBuffer` and a zero position.
*
* This method exists primarily to allow recycling Struct instances without risking memory leaks
* due to `ByteBuffer` references. The instance will be unusable until it is assigned
* again to a `ByteBuffer`.
*/
private inline fun <reified T: Struct> reset(): T = reset(0, emptyBuffer)
}
public inline val <T> T.value: T get() = this
public const val VERSION_2_0_8: Int = 1

30
kotlin/flatbuffers-kotlin/src/commonMain/kotlin/com/google/flatbuffers/kotlin/FlexBuffers.kt
View File

@@ -43,7 +43,7 @@ public class Reference internal constructor(
internal val end: Int,
internal val parentWidth: ByteWidth,
internal val byteWidth: ByteWidth,
internal val type: FlexBufferType
public val type: FlexBufferType
) {
internal constructor(bb: ReadBuffer, end: Int, parentWidth: ByteWidth, packedType: Int) :
@@ -294,7 +294,8 @@ public class Reference internal constructor(
T_KEY -> buffer.getKeyString(buffer.indirect(end, parentWidth))
T_MAP -> "{ ${toMap().entries.joinToString(", ") { "${it.key}: ${it.value}"}} }"
T_VECTOR, T_VECTOR_BOOL, T_VECTOR_FLOAT, T_VECTOR_INT,
T_VECTOR_UINT, T_VECTOR_KEY, T_VECTOR_STRING_DEPRECATED -> "[ ${toVector().joinToString(", ") { it.toString() }} ]"
T_VECTOR_UINT, T_VECTOR_KEY, T_VECTOR_STRING_DEPRECATED ->
"[ ${toVector().joinToString(", ") { it.toString() }} ]"
T_INT -> toLong().toString()
T_UINT -> toULong().toString()
T_FLOAT -> toDouble().toString()
@@ -629,10 +630,18 @@ public open class TypedVector(
return block(childPos, byteWidth)
}
internal fun getBoolean(index: Int): Boolean = resolveAt(index) { pos: Int, _: ByteWidth -> buffer.getBoolean(pos) }
internal fun getInt(index: Int): Long = resolveAt(index) { pos: Int, width: ByteWidth -> buffer.readLong(pos, width) }
internal fun getUInt(index: Int): ULong = resolveAt(index) { pos: Int, width: ByteWidth -> buffer.readULong(pos, width) }
internal fun getFloat(index: Int): Double = resolveAt(index) { pos: Int, width: ByteWidth -> buffer.readFloat(pos, width) }
internal fun getBoolean(index: Int): Boolean = resolveAt(index) {
pos: Int, _: ByteWidth -> buffer.getBoolean(pos)
}
internal fun getInt(index: Int): Long = resolveAt(index) {
pos: Int, width: ByteWidth -> buffer.readLong(pos, width)
}
internal fun getUInt(index: Int): ULong = resolveAt(index) {
pos: Int, width: ByteWidth -> buffer.readULong(pos, width)
}
internal fun getFloat(index: Int): Double = resolveAt(index) {
pos: Int, width: ByteWidth -> buffer.readFloat(pos, width)
}
}
/**
@@ -706,7 +715,8 @@ public data class Key(
/**
* A Map class that provide support to access Key-Value data from Flexbuffers.
*/
public class Map internal constructor(buffer: ReadBuffer, end: Int, byteWidth: ByteWidth) :
public class Map
internal constructor(buffer: ReadBuffer, end: Int, byteWidth: ByteWidth):
Sized(buffer, end, byteWidth),
kotlin.collections.Map<Key, Reference> {
@@ -869,14 +879,14 @@ public class Map internal constructor(buffer: ReadBuffer, end: Int, byteWidth: B
while (otherPos < otherLimit) {
val c2 = other[otherPos]
// not a single byte codepoint
if (c2.toInt() >= 0x80) {
if (c2.code >= 0x80) {
break
}
val b: Byte = buffer[bufferPos]
when {
b == ZeroByte -> return -c2.toInt()
b == ZeroByte -> return -c2.code
b < 0 -> break
b != c2.toByte() -> return b - c2.toByte()
b != c2.code.toByte() -> return b - c2.code.toByte()
}
++bufferPos
++otherPos

89
kotlin/flatbuffers-kotlin/src/commonMain/kotlin/com/google/flatbuffers/kotlin/FlexBuffersBuilder.kt
View File

@@ -17,6 +17,7 @@
package com.google.flatbuffers.kotlin
@ExperimentalUnsignedTypes
public class FlexBuffersBuilder(
public val buffer: ReadWriteBuffer,
private val shareFlag: Int = SHARE_KEYS
@@ -49,13 +50,14 @@ public class FlexBuffersBuilder(
* @return [ReadBuffer] containing the FlexBuffer message
*/
public fun finish(): ReadBuffer {
// If you hit this assert, you likely have objects that were never included
// If you hit this, you likely have objects that were never included
// in a parent. You need to have exactly one root to finish a buffer.
// Check your Start/End calls are matched, and all objects are inside
// some other object.
if (stack.size != 1) error("There is must be only on object as root. Current ${stack.size}.")
// Write root value.
val byteWidth = align(stack[0].elemWidth(buffer.writePosition, 0))
buffer.requestAdditionalCapacity(byteWidth.value + 2)
writeAny(stack[0], byteWidth)
// Write root type.
buffer.put(stack[0].storedPackedType())
@@ -198,7 +200,9 @@ public class FlexBuffersBuilder(
public operator fun set(key: String? = null, value: String): Int {
val iKey = putKey(key)
val holder = if (shareFlag and SHARE_STRINGS != 0) {
stringValuePool.getOrPut(value) { writeString(iKey, value).also { stringValuePool[value] = it } }.copy(key = iKey)
stringValuePool.getOrPut(value) {
writeString(iKey, value).also { stringValuePool[value] = it }
}.copy(key = iKey)
} else {
writeString(iKey, value)
}
@@ -491,13 +495,17 @@ public class FlexBuffersBuilder(
return if ((shareFlag and SHARE_KEYS) != 0) {
stringKeyPool.getOrPut(key) {
val pos: Int = buffer.writePosition
buffer.put(key)
val encodedKeySize = Utf8.encodedLength(key)
buffer.requestAdditionalCapacity(encodedKeySize + 1)
buffer.put(key, encodedKeySize)
buffer.put(ZeroByte)
pos
}
} else {
val pos: Int = buffer.writePosition
buffer.put(key)
val encodedKeySize = Utf8.encodedLength(key)
buffer.requestAdditionalCapacity(encodedKeySize + 1)
buffer.put(key, encodedKeySize)
buffer.put(ZeroByte)
pos
}
@@ -510,26 +518,31 @@ public class FlexBuffersBuilder(
}
private fun writeString(key: Int, s: String): Value {
val size = Utf8.encodedLength(s)
val bitWidth = size.toULong().widthInUBits()
val encodedSize = Utf8.encodedLength(s)
val bitWidth = encodedSize.toULong().widthInUBits()
val byteWidth = align(bitWidth)
writeInt(size, byteWidth)
writeInt(encodedSize, byteWidth)
buffer.requestAdditionalCapacity(encodedSize + 1)
val sloc: Int = buffer.writePosition
if (size > 0)
buffer.put(s, size)
if (encodedSize > 0)
buffer.put(s, encodedSize)
buffer.put(ZeroByte)
return Value(T_STRING, key, bitWidth, sloc.toULong())
}
private fun writeDouble(toWrite: Double, byteWidth: ByteWidth): Unit = when (byteWidth.value) {
4 -> buffer.put(toWrite.toFloat())
8 -> buffer.put(toWrite)
else -> Unit
private fun writeDouble(toWrite: Double, byteWidth: ByteWidth) {
buffer.requestAdditionalCapacity(byteWidth.value)
when (byteWidth.value) {
4 -> buffer.put(toWrite.toFloat())
8 -> buffer.put(toWrite)
else -> Unit
}
}
private fun writeOffset(toWrite: Int, byteWidth: ByteWidth) {
buffer.requestAdditionalCapacity(byteWidth.value)
val relativeOffset = (buffer.writePosition - toWrite)
if (byteWidth.value != 8 && relativeOffset >= 1L shl byteWidth.value * 8) error("invalid offset $relativeOffset, writer pos ${buffer.writePosition}")
writeInt(relativeOffset, byteWidth)
@@ -542,6 +555,7 @@ public class FlexBuffersBuilder(
writeInt(blob.size, byteWidth)
val sloc: Int = buffer.writePosition
buffer.requestAdditionalCapacity(blob.size + trailing.compareTo(false))
buffer.put(blob, 0, blob.size)
if (trailing) {
buffer.put(ZeroByte)
@@ -559,18 +573,12 @@ public class FlexBuffersBuilder(
writeIntegerArray(0, value.size, byteWidth) { value[it].toULong() }
private fun writeFloatArray(value: FloatArray) {
val byteWidth = Float.SIZE_BYTES
// since we know we are writing an array, we can avoid multiple copy/growth of the buffer by requesting
// the right size on the spot
buffer.requestCapacity(buffer.writePosition + (value.size * byteWidth))
buffer.requestAdditionalCapacity(Float.SIZE_BYTES * value.size)
value.forEach { buffer.put(it) }
}
private fun writeFloatArray(value: DoubleArray) {
val byteWidth = Double.SIZE_BYTES
// since we know we are writing an array, we can avoid multiple copy/growth of the buffer by requesting
// the right size on the spot
buffer.requestCapacity(buffer.writePosition + (value.size * byteWidth))
buffer.requestAdditionalCapacity(Double.SIZE_BYTES * value.size)
value.forEach { buffer.put(it) }
}
@@ -580,9 +588,7 @@ public class FlexBuffersBuilder(
byteWidth: ByteWidth,
crossinline valueBlock: (Int) -> ULong
) {
// since we know we are writing an array, we can avoid multiple copy/growth of the buffer by requesting
// the right size on the spot
buffer.requestCapacity(buffer.writePosition + (size * byteWidth))
buffer.requestAdditionalCapacity(size * byteWidth.value)
return when (byteWidth.value) {
1 -> for (i in start until start + size) {
buffer.put(valueBlock(i).toUByte())
@@ -600,20 +606,26 @@ public class FlexBuffersBuilder(
}
}
private fun writeInt(value: Int, byteWidth: ByteWidth) = when (byteWidth.value) {
1 -> buffer.put(value.toUByte())
2 -> buffer.put(value.toUShort())
4 -> buffer.put(value.toUInt())
8 -> buffer.put(value.toULong())
else -> Unit
private fun writeInt(value: Int, byteWidth: ByteWidth) {
buffer.requestAdditionalCapacity(byteWidth.value)
when (byteWidth.value) {
1 -> buffer.put(value.toUByte())
2 -> buffer.put(value.toUShort())
4 -> buffer.put(value.toUInt())
8 -> buffer.put(value.toULong())
else -> Unit
}
}
private fun writeInt(value: ULong, byteWidth: ByteWidth) = when (byteWidth.value) {
1 -> buffer.put(value.toUByte())
2 -> buffer.put(value.toUShort())
4 -> buffer.put(value.toUInt())
8 -> buffer.put(value)
else -> Unit
private fun writeInt(value: ULong, byteWidth: ByteWidth) {
buffer.requestAdditionalCapacity(byteWidth.value)
when(byteWidth.value) {
1 -> buffer.put(value.toUByte())
2 -> buffer.put(value.toUShort())
4 -> buffer.put(value.toUInt())
8 -> buffer.put(value)
else -> Unit
}
}
// Align to prepare for writing a scalar with a certain size.
@@ -621,6 +633,7 @@ public class FlexBuffersBuilder(
private fun align(alignment: BitWidth): ByteWidth {
val byteWidth = 1 shl alignment.value
var padBytes = paddingBytes(buffer.writePosition, byteWidth)
buffer.requestCapacity(buffer.capacity + padBytes)
while (padBytes-- != 0) {
buffer.put(ZeroByte)
}
@@ -659,6 +672,7 @@ public class FlexBuffersBuilder(
private inline fun createVector(key: Int, start: Int, length: Int, keys: Value? = null): Value {
return createAnyVector(key, start, length, T_VECTOR, keys) {
// add types since we are not creating a typed vector.
buffer.requestAdditionalCapacity(stack.size)
for (i in start until stack.size) {
buffer.put(stack[i].storedPackedType(it))
}
@@ -668,6 +682,7 @@ public class FlexBuffersBuilder(
private fun putMap(key: Int, start: Int, length: Int, keys: Value? = null): Value {
return createAnyVector(key, start, length, T_MAP, keys) {
// add types since we are not creating a typed vector.
buffer.requestAdditionalCapacity(stack.size)
for (i in start until stack.size) {
buffer.put(stack[i].storedPackedType(it))
}
@@ -692,7 +707,7 @@ public class FlexBuffersBuilder(
keys: Value? = null,
crossinline typeBlock: (BitWidth) -> Unit = {}
): Value {
// Figure out smallest bit width we can store this vector with.
// Figure out the smallest bit width we can store this vector with.
var bitWidth = W_8.max(length.toULong().widthInUBits())
var prefixElems = 1
if (keys != null) {

20
kotlin/flatbuffers-kotlin/src/commonMain/kotlin/com/google/flatbuffers/kotlin/FlexBuffersInternals.kt
View File

@@ -17,13 +17,18 @@
package com.google.flatbuffers.kotlin
public inline class BitWidth(public val value: Int) {
import kotlin.jvm.JvmInline
@JvmInline
public value class BitWidth(public val value: Int) {
public inline fun max(other: BitWidth): BitWidth = if (this.value >= other.value) this else other
}
public inline class ByteWidth(public val value: Int)
@JvmInline
public value class ByteWidth(public val value: Int)
public inline class FlexBufferType(public val value: Int) {
@JvmInline
public value class FlexBufferType(public val value: Int) {
public operator fun minus(other: FlexBufferType): FlexBufferType = FlexBufferType(this.value - other.value)
public operator fun plus(other: FlexBufferType): FlexBufferType = FlexBufferType(this.value + other.value)
public operator fun compareTo(other: FlexBufferType): Int = this.value - other.value
@@ -108,11 +113,12 @@ internal fun FlexBufferType.isIndirectScalar(): Boolean = when (this) {
internal fun FlexBufferType.isTypedVector(): Boolean =
this >= T_VECTOR_INT && this <= T_VECTOR_STRING_DEPRECATED || this == T_VECTOR_BOOL
internal fun FlexBufferType.isTypedVectorElementType(): Boolean = (this.value in T_INT.value..T_KEY.value) || this == T_BOOL
internal fun FlexBufferType.isTypedVectorElementType(): Boolean =
(this.value in T_INT.value..T_KEY.value) || this == T_BOOL
// returns the typed vector of a given scalar type.
internal fun FlexBufferType.toTypedVector(): FlexBufferType = (this - T_INT) + T_VECTOR_INT
// returns the element type of a given typed vector.
// returns the element type of given typed vector.
internal fun FlexBufferType.toElementTypedVector(): FlexBufferType = this - T_VECTOR_INT + T_INT
// Holds information about the elements inserted on the buffer.
@@ -126,7 +132,8 @@ internal data class Value(
inline fun storedPackedType(parentBitWidth: BitWidth = W_8): Byte = packedType(storedWidth(parentBitWidth), type)
private inline fun packedType(bitWidth: BitWidth, type: FlexBufferType): Byte = (bitWidth.value or (type.value shl 2)).toByte()
private inline fun packedType(bitWidth: BitWidth, type: FlexBufferType): Byte =
(bitWidth.value or (type.value shl 2)).toByte()
private inline fun storedWidth(parentBitWidth: BitWidth): BitWidth =
if (type.isInline()) minBitWidth.max(parentBitWidth) else minBitWidth
@@ -199,7 +206,6 @@ internal fun FlexBufferType.typeToString(): String = when (this) {
}
// Few repeated values used in hot path is cached here
internal val emptyBuffer = ArrayReadWriteBuffer(1)
internal fun emptyBlob() = Blob(emptyBuffer, 1, ByteWidth(1))
internal fun emptyVector() = Vector(emptyBuffer, 1, ByteWidth(1))
internal fun emptyMap() = Map(ArrayReadWriteBuffer(3), 3, ByteWidth(1))

62
kotlin/flatbuffers-kotlin/src/commonMain/kotlin/com/google/flatbuffers/kotlin/Utf8.kt
View File

@@ -14,6 +14,7 @@
* limitations under the License.
*/
@file:Suppress("NOTHING_TO_INLINE")
package com.google.flatbuffers.kotlin
public object Utf8 {
@@ -32,15 +33,15 @@ public object Utf8 {
var i = 0
// This loop optimizes for pure ASCII.
while (i < utf16Length && sequence[i].toInt() < 0x80) {
while (i < utf16Length && sequence[i].code < 0x80) {
i++
}
// This loop optimizes for chars less than 0x800.
while (i < utf16Length) {
val c = sequence[i]
if (c.toInt() < 0x800) {
utf8Length += 0x7f - c.toInt() ushr 31 // branch free!
if (c.code < 0x800) {
utf8Length += 0x7f - c.code ushr 31 // branch free!
} else {
utf8Length += encodedLengthGeneral(sequence, i)
break
@@ -60,8 +61,8 @@ public object Utf8 {
var i = start
while (i < utf16Length) {
val c = sequence[i]
if (c.toInt() < 0x800) {
utf8Length += 0x7f - c.toInt() ushr 31 // branch free!
if (c.code < 0x800) {
utf8Length += 0x7f - c.code ushr 31 // branch free!
} else {
utf8Length += 2
if (c.isSurrogate()) {
@@ -109,7 +110,7 @@ public object Utf8 {
public inline fun isFourByte(b: Byte): Boolean = b < 0xF8.toByte()
public fun handleOneByte(byte1: Byte, resultArr: CharArray, resultPos: Int) {
resultArr[resultPos] = byte1.toChar()
resultArr[resultPos] = byte1.toInt().toChar()
}
public fun handleTwoBytes(
@@ -209,21 +210,21 @@ public object Utf8 {
return 0
}
val c = input[start]
return if (c.toInt() < 0x80) {
return if (c.code < 0x80) {
// One byte (0xxx xxxx)
out[0] = c.toByte()
out[0] = c.code.toByte()
1
} else if (c.toInt() < 0x800) {
} else if (c.code < 0x800) {
// Two bytes (110x xxxx 10xx xxxx)
out[0] = (0xC0 or (c.toInt() ushr 6)).toByte()
out[1] = (0x80 or (0x3F and c.toInt())).toByte()
out[0] = (0xC0 or (c.code ushr 6)).toByte()
out[1] = (0x80 or (0x3F and c.code)).toByte()
2
} else if (c < Char.MIN_SURROGATE || Char.MAX_SURROGATE < c) {
// Three bytes (1110 xxxx 10xx xxxx 10xx xxxx)
// Maximum single-char code point is 0xFFFF, 16 bits.
out[0] = (0xE0 or (c.toInt() ushr 12)).toByte()
out[1] = (0x80 or (0x3F and (c.toInt() ushr 6))).toByte()
out[2] = (0x80 or (0x3F and c.toInt())).toByte()
out[0] = (0xE0 or (c.code ushr 12)).toByte()
out[1] = (0x80 or (0x3F and (c.code ushr 6))).toByte()
out[2] = (0x80 or (0x3F and c.code)).toByte()
3
} else {
// Four bytes (1111 xxxx 10xx xxxx 10xx xxxx 10xx xxxx)
@@ -330,7 +331,10 @@ public object Utf8 {
return resultArr.concatToString(0, resultPos)
}
public fun encodeUtf8Array(input: CharSequence, out: ByteArray, offset: Int = 0, length: Int = out.size - offset): Int {
public fun encodeUtf8Array(input: CharSequence,
out: ByteArray,
offset: Int = 0,
length: Int = out.size - offset): Int {
val utf16Length = input.length
var j = offset
var i = 0
@@ -341,8 +345,8 @@ public object Utf8 {
if (utf16Length == 0)
return 0
var cc: Char = input[i]
while (i < utf16Length && i + j < limit && input[i].also { cc = it }.toInt() < 0x80) {
out[j + i] = cc.toByte()
while (i < utf16Length && i + j < limit && input[i].also { cc = it }.code < 0x80) {
out[j + i] = cc.code.toByte()
i++
}
if (i == utf16Length) {
@@ -352,16 +356,16 @@ public object Utf8 {
var c: Char
while (i < utf16Length) {
c = input[i]
if (c.toInt() < 0x80 && j < limit) {
out[j++] = c.toByte()
} else if (c.toInt() < 0x800 && j <= limit - 2) { // 11 bits, two UTF-8 bytes
out[j++] = (0xF shl 6 or (c.toInt() ushr 6)).toByte()
out[j++] = (0x80 or (0x3F and c.toInt())).toByte()
if (c.code < 0x80 && j < limit) {
out[j++] = c.code.toByte()
} else if (c.code < 0x800 && j <= limit - 2) { // 11 bits, two UTF-8 bytes
out[j++] = (0xF shl 6 or (c.code ushr 6)).toByte()
out[j++] = (0x80 or (0x3F and c.code)).toByte()
} else if ((c < Char.MIN_SURROGATE || Char.MAX_SURROGATE < c) && j <= limit - 3) {
// Maximum single-char code point is 0xFFFF, 16 bits, three UTF-8 bytes
out[j++] = (0xF shl 5 or (c.toInt() ushr 12)).toByte()
out[j++] = (0x80 or (0x3F and (c.toInt() ushr 6))).toByte()
out[j++] = (0x80 or (0x3F and c.toInt())).toByte()
out[j++] = (0xF shl 5 or (c.code ushr 12)).toByte()
out[j++] = (0x80 or (0x3F and (c.code ushr 6))).toByte()
out[j++] = (0x80 or (0x3F and c.code)).toByte()
} else if (j <= limit - 4) {
// Minimum code point represented by a surrogate pair is 0x10000, 17 bits,
// four UTF-8 bytes
@@ -384,7 +388,7 @@ public object Utf8 {
) {
errorSurrogate(i, utf16Length)
}
error("Failed writing character ${c.toShort().toString(radix = 16)} at index $j")
error("Failed writing character ${c.code.toShort().toString(radix = 16)} at index $j")
}
i++
}
@@ -400,13 +404,13 @@ public object Utf8 {
return toCodePoint(c1, c2)
}
}
return c1.toInt()
return c1.code
}
private fun isSurrogatePair(high: Char, low: Char) = high.isHighSurrogate() and low.isLowSurrogate()
private fun toCodePoint(high: Char, low: Char): Int = (high.toInt() shl 10) + low.toInt() +
(MIN_SUPPLEMENTARY_CODE_POINT - (Char.MIN_HIGH_SURROGATE.toInt() shl 10) - Char.MIN_LOW_SURROGATE.toInt())
private fun toCodePoint(high: Char, low: Char): Int = (high.code shl 10) + low.code +
(MIN_SUPPLEMENTARY_CODE_POINT - (Char.MIN_HIGH_SURROGATE.code shl 10) - Char.MIN_LOW_SURROGATE.code)
private fun errorSurrogate(i: Int, utf16Length: Int): Unit =
error("Unpaired surrogate at index $i of $utf16Length length")

112
kotlin/flatbuffers-kotlin/src/commonMain/kotlin/com/google/flatbuffers/kotlin/JSON.kt → kotlin/flatbuffers-kotlin/src/commonMain/kotlin/com/google/flatbuffers/kotlin/json.kt
View File

@@ -19,6 +19,7 @@ package com.google.flatbuffers.kotlin
import com.google.flatbuffers.kotlin.FlexBuffersBuilder.Companion.SHARE_KEYS_AND_STRINGS
import kotlin.experimental.and
import kotlin.jvm.JvmInline
import kotlin.math.pow
/**
@@ -72,19 +73,19 @@ public fun Map.toJson(): String = ArrayReadWriteBuffer(1024).let { toJson(it); i
* @param out [ReadWriteBuffer] the JSON will be written.
*/
public fun Map.toJson(out: ReadWriteBuffer) {
out.put('{'.toByte())
out.put('{'.code.toByte())
// key values pairs
for (i in 0 until size) {
val key = keyAt(i)
out.jsonEscape(buffer, key.start, key.sizeInBytes)
out.put(':'.toByte())
out.put(':'.code.toByte())
get(i).toJson(out)
if (i != size - 1) {
out.put(','.toByte())
out.put(','.code.toByte())
}
}
// close bracket
out.put('}'.toByte())
out.put('}'.code.toByte())
}
/**
@@ -97,20 +98,21 @@ public fun Vector.toJson(): String = ArrayReadWriteBuffer(1024).let { toJson(it)
* @param out that the JSON is being concatenated.
*/
public fun Vector.toJson(out: ReadWriteBuffer) {
out.put('['.toByte())
for (i in 0 until size) {
out.put('['.code.toByte())
for (i in indices) {
get(i).toJson(out)
if (i != size - 1) {
out.put(','.toByte())
out.put(','.code.toByte())
}
}
out.put(']'.toByte())
out.put(']'.code.toByte())
}
/**
* JSONParser class is used to parse a JSON as FlexBuffers. Calling [JSONParser.parse] fiils [output]
* and returns a [Reference] ready to be used.
*/
@ExperimentalUnsignedTypes
public class JSONParser(public var output: FlexBuffersBuilder = FlexBuffersBuilder(1024, SHARE_KEYS_AND_STRINGS)) {
private var readPos = 0
private var scopes = ScopeStack()
@@ -150,7 +152,7 @@ public class JSONParser(public var output: FlexBuffersBuilder = FlexBuffersBuild
TOK_NULL -> T_NULL.also { output.putNull(key) }
TOK_BEGIN_QUOTE -> parseString(data, key)
TOK_NUMBER -> parseNumber(data, data.data(), key)
else -> makeError(data, "Unexpected Character while parsing", 'x'.toByte())
else -> makeError(data, "Unexpected Character while parsing", 'x'.code.toByte())
}
}
@@ -590,7 +592,7 @@ public class JSONParser(public var output: FlexBuffersBuilder = FlexBuffersBuild
val end = i + 4
while (i < end) {
val part: Byte = data[i]
result = (result.toInt() shl 4).toChar()
result = (result.code shl 4).toChar()
result += when (part) {
in CHAR_0..CHAR_9 -> part - CHAR_0
in CHAR_a..CHAR_f -> part - CHAR_a + 10
@@ -606,13 +608,13 @@ public class JSONParser(public var output: FlexBuffersBuilder = FlexBuffersBuild
CHAR_r -> '\r'
CHAR_n -> '\n'
CHAR_f -> 12.toChar() // '\f'
CHAR_DOUBLE_QUOTE, CHAR_BACKSLASH, CHAR_FORWARDSLASH -> byte1.toChar()
CHAR_DOUBLE_QUOTE, CHAR_BACKSLASH, CHAR_FORWARDSLASH -> byte1.toInt().toChar()
else -> makeError(data, "Invalid escape sequence.", byte1)
}
}
private fun Byte.print(): String = when (this) {
in 0x21..0x7E -> "'${this.toChar()}'" // visible ascii chars
in 0x21..0x7E -> "'${this.toInt().toChar()}'" // visible ascii chars
CHAR_EOF -> "EOF"
else -> "'0x${this.toString(16)}'"
}
@@ -685,20 +687,21 @@ private inline fun ReadWriteBuffer.jsonEscape(data: ReadBuffer, start: Int, size
// Following escape strategy defined in RFC7159.
private val JSON_ESCAPE_CHARS: Array<ByteArray?> = arrayOfNulls<ByteArray>(128).apply {
this['\n'.toInt()] = "\\n".encodeToByteArray()
this['\t'.toInt()] = "\\t".encodeToByteArray()
this['\r'.toInt()] = "\\r".encodeToByteArray()
this['\b'.toInt()] = "\\b".encodeToByteArray()
this['\n'.code] = "\\n".encodeToByteArray()
this['\t'.code] = "\\t".encodeToByteArray()
this['\r'.code] = "\\r".encodeToByteArray()
this['\b'.code] = "\\b".encodeToByteArray()
this[0x0c] = "\\f".encodeToByteArray()
this['"'.toInt()] = "\\\"".encodeToByteArray()
this['\\'.toInt()] = "\\\\".encodeToByteArray()
this['"'.code] = "\\\"".encodeToByteArray()
this['\\'.code] = "\\\\".encodeToByteArray()
for (i in 0..0x1f) {
this[i] = "\\u${i.toPaddedHex()}".encodeToByteArray()
}
}
// Scope is used to the define current space that the scanner is operating.
private inline class Scope(val id: Int)
@JvmInline
private value class Scope(val id: Int)
private val SCOPE_DOC_EMPTY = Scope(0)
private val SCOPE_DOC_FILLED = Scope(1)
private val SCOPE_OBJ_EMPTY = Scope(2)
@@ -738,7 +741,8 @@ private class ScopeStack(
}
}
private inline class Token(val id: Int) {
@JvmInline
private value class Token(val id: Int) {
fun print(): String = when (this) {
TOK_EOF -> "TOK_EOF"
TOK_NONE -> "TOK_NONE"
@@ -767,41 +771,41 @@ private val TOK_FALSE = Token(7)
private val TOK_NULL = Token(8)
private val TOK_BEGIN_QUOTE = Token(9)
private const val CHAR_NEWLINE = '\n'.toByte()
private const val CHAR_OPEN_OBJECT = '{'.toByte()
private const val CHAR_COLON = ':'.toByte()
private const val CHAR_CLOSE_OBJECT = '}'.toByte()
private const val CHAR_OPEN_ARRAY = '['.toByte()
private const val CHAR_CLOSE_ARRAY = ']'.toByte()
private const val CHAR_DOUBLE_QUOTE = '"'.toByte()
private const val CHAR_BACKSLASH = '\\'.toByte()
private const val CHAR_FORWARDSLASH = '/'.toByte()
private const val CHAR_f = 'f'.toByte()
private const val CHAR_a = 'a'.toByte()
private const val CHAR_r = 'r'.toByte()
private const val CHAR_t = 't'.toByte()
private const val CHAR_n = 'n'.toByte()
private const val CHAR_b = 'b'.toByte()
private const val CHAR_e = 'e'.toByte()
private const val CHAR_E = 'E'.toByte()
private const val CHAR_u = 'u'.toByte()
private const val CHAR_A = 'A'.toByte()
private const val CHAR_F = 'F'.toByte()
private const val CHAR_NEWLINE = '\n'.code.toByte()
private const val CHAR_OPEN_OBJECT = '{'.code.toByte()
private const val CHAR_COLON = ':'.code.toByte()
private const val CHAR_CLOSE_OBJECT = '}'.code.toByte()
private const val CHAR_OPEN_ARRAY = '['.code.toByte()
private const val CHAR_CLOSE_ARRAY = ']'.code.toByte()
private const val CHAR_DOUBLE_QUOTE = '"'.code.toByte()
private const val CHAR_BACKSLASH = '\\'.code.toByte()
private const val CHAR_FORWARDSLASH = '/'.code.toByte()
private const val CHAR_f = 'f'.code.toByte()
private const val CHAR_a = 'a'.code.toByte()
private const val CHAR_r = 'r'.code.toByte()
private const val CHAR_t = 't'.code.toByte()
private const val CHAR_n = 'n'.code.toByte()
private const val CHAR_b = 'b'.code.toByte()
private const val CHAR_e = 'e'.code.toByte()
private const val CHAR_E = 'E'.code.toByte()
private const val CHAR_u = 'u'.code.toByte()
private const val CHAR_A = 'A'.code.toByte()
private const val CHAR_F = 'F'.code.toByte()
private const val CHAR_EOF = (-1).toByte()
private const val CHAR_COMMA = ','.toByte()
private const val CHAR_0 = '0'.toByte()
private const val CHAR_1 = '1'.toByte()
private const val CHAR_2 = '2'.toByte()
private const val CHAR_3 = '3'.toByte()
private const val CHAR_4 = '4'.toByte()
private const val CHAR_5 = '5'.toByte()
private const val CHAR_6 = '6'.toByte()
private const val CHAR_7 = '7'.toByte()
private const val CHAR_8 = '8'.toByte()
private const val CHAR_9 = '9'.toByte()
private const val CHAR_MINUS = '-'.toByte()
private const val CHAR_PLUS = '+'.toByte()
private const val CHAR_DOT = '.'.toByte()
private const val CHAR_COMMA = ','.code.toByte()
private const val CHAR_0 = '0'.code.toByte()
private const val CHAR_1 = '1'.code.toByte()
private const val CHAR_2 = '2'.code.toByte()
private const val CHAR_3 = '3'.code.toByte()
private const val CHAR_4 = '4'.code.toByte()
private const val CHAR_5 = '5'.code.toByte()
private const val CHAR_6 = '6'.code.toByte()
private const val CHAR_7 = '7'.code.toByte()
private const val CHAR_8 = '8'.code.toByte()
private const val CHAR_9 = '9'.code.toByte()
private const val CHAR_MINUS = '-'.code.toByte()
private const val CHAR_PLUS = '+'.code.toByte()
private const val CHAR_DOT = '.'.code.toByte()
// This template utilizes the One Definition Rule to create global arrays in a
// header. As seen in:

55
kotlin/flatbuffers-kotlin/src/commonTest/kotlin/com/google/flatbuffers/kotlin/Asserts.kt Normal file
View File

@@ -0,0 +1,55 @@
package com.google.flatbuffers.kotlin
import kotlin.test.assertTrue
fun <T> assertArrayEquals(expected: Array<out T>, actual: Array<out T>) =
assertTrue(expected contentEquals actual, arrayFailMessage(expected, actual))
fun assertArrayEquals(expected: IntArray, actual: IntArray) =
assertTrue(expected contentEquals actual, arrayFailMessage(expected, actual))
fun assertArrayEquals(expected: ShortArray, actual: ShortArray) =
assertTrue(expected contentEquals actual, arrayFailMessage(expected, actual))
fun assertArrayEquals(expected: LongArray, actual: LongArray) =
assertTrue(expected contentEquals actual, arrayFailMessage(expected, actual))
fun assertArrayEquals(expected: ByteArray, actual: ByteArray) =
assertTrue(expected contentEquals actual, arrayFailMessage(expected, actual))
fun assertArrayEquals(expected: DoubleArray, actual: DoubleArray) =
assertTrue(expected contentEquals actual, arrayFailMessage(expected, actual))
fun assertArrayEquals(expected: FloatArray, actual: FloatArray) =
assertTrue(expected contentEquals actual, arrayFailMessage(expected, actual))
fun <T> arrayFailMessage(expected: Array<out T>, actual: Array<out T>): String =
failMessage(expected.contentToString(), actual.contentToString())
fun arrayFailMessage(expected: IntArray, actual: IntArray): String =
failMessage(expected.contentToString(), actual.contentToString())
fun arrayFailMessage(expected: ShortArray, actual: ShortArray): String =
failMessage(expected.contentToString(), actual.contentToString())
fun arrayFailMessage(expected: LongArray, actual: LongArray): String =
failMessage(expected.contentToString(), actual.contentToString())
fun failMessage(expected: String, actual: String): String =
"Expected: $expected\nActual: $actual"
fun arrayFailMessage(expected: FloatArray, actual: FloatArray): String {
return "Expected: ${expected.contentToString()}\nActual: ${actual.contentToString()}"
}
fun arrayFailMessage(expected: DoubleArray, actual: DoubleArray): String {
return "Expected: ${expected.contentToString()}\nActual: ${actual.contentToString()}"
}
fun arrayFailMessage(expected: BooleanArray, actual: BooleanArray): String {
return "Expected: ${expected.contentToString()}\nActual: ${actual.contentToString()}"
}
fun arrayFailMessage(expected: ByteArray, actual: ByteArray): String {
return "Expected: ${expected.contentToString()}\nActual: ${actual.contentToString()}"
}

78
kotlin/flatbuffers-kotlin/src/commonTest/kotlin/com/google/flatbuffers/kotlin/BuffersTest.kt Normal file
View File

@@ -0,0 +1,78 @@
package com.google.flatbuffers.kotlin
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertFailsWith
class BuffersTest {
@Test
fun readBufferStringTest() {
val text = "Hello world!"
val bytes = text.encodeToByteArray()
val fullRead = ArrayReadBuffer(bytes)
val helloRead = ArrayReadBuffer(bytes, limit = 5)
val worldRead = fullRead.slice(6, 6)
assertEquals(bytes.size, fullRead.limit)
assertEquals(text, fullRead.getString(0, fullRead.limit))
assertEquals("Hello" , helloRead.getString(0, helloRead.limit))
assertEquals("world!" , worldRead.getString())
assertEquals(fullRead.getString(0, 5) , helloRead.getString(0, helloRead.limit))
assertEquals(fullRead.getString(6, 6) , worldRead.getString(0, worldRead.limit))
for (i in 0 until helloRead.limit) {
assertEquals(fullRead[i], helloRead[i])
}
for (i in 0 until worldRead.limit) {
assertEquals(fullRead[6 + i], worldRead[i])
}
}
@Test
fun readWriteBufferPrimitivesTest() {
val text = "Hello world!"
val bytes = text.encodeToByteArray()
val wt = ArrayReadWriteBuffer(bytes)
wt.requestCapacity(4096)
wt.put("Tests")
val str1 = wt.writePosition
assertEquals("Tests world!", wt.getString(0, bytes.size))
assertEquals("Tests", wt.getString(0, str1))
wt.put(Int.MAX_VALUE)
assertEquals(Int.MAX_VALUE, wt.getInt(str1))
val pos = wt.writePosition
wt.put(Double.NEGATIVE_INFINITY)
assertEquals(Double.NEGATIVE_INFINITY, wt.getDouble(pos))
val jap = " are really すごい!".encodeToByteArray()
wt.writePosition = str1
wt.put(jap)
assertEquals("Tests are really すごい!", wt.getString())
}
@Test
fun readWriteBufferGrowthTest() {
val a = ArrayReadWriteBuffer(1)
assertEquals(1, a.capacity)
a.put(0.toByte())
assertEquals(1, a.capacity)
assertFailsWith(IndexOutOfBoundsException::class) { a.put(0xFF.toShort()) }
a.requestCapacity(8)
a.writePosition = 0
a.put(0xFF.toShort())
assertEquals(8, a.capacity)
assertEquals(0xFF, a.getShort(0))
a.requestCapacity(8 + 12)
a.put(ByteArray(12) { it.toByte() })
// we grow as power or two, so 20 jumps to 32
assertEquals(32, a.capacity)
a.requestCapacity(513, false)
assertEquals(1024, a.capacity)
a.requestCapacity(234, false)
assertEquals(1024, a.capacity)
}
}

53
kotlin/flatbuffers-kotlin/src/commonTest/kotlin/com/google/flatbuffers/kotlin/ByteArrayTest.kt
View File

@@ -139,60 +139,9 @@ class ByteArrayTest {
val testSet = "∮ E⋅da = Q"
val encoded = testSet.encodeToByteArray()
val data = ByteArray(encoded.size)
data.setString(0, testSet)
data.setCharSequence(0, testSet)
assertArrayEquals(encoded, data)
assertEquals(testSet, data.getString(0, encoded.size))
}
}
fun <T> assertArrayEquals(expected: Array<out T>, actual: Array<out T>) =
assertTrue(expected contentEquals actual, arrayFailMessage(expected, actual))
fun assertArrayEquals(expected: IntArray, actual: IntArray) =
assertTrue(expected contentEquals actual, arrayFailMessage(expected, actual))
fun assertArrayEquals(expected: ShortArray, actual: ShortArray) =
assertTrue(expected contentEquals actual, arrayFailMessage(expected, actual))
fun assertArrayEquals(expected: LongArray, actual: LongArray) =
assertTrue(expected contentEquals actual, arrayFailMessage(expected, actual))
fun assertArrayEquals(expected: ByteArray, actual: ByteArray) =
assertTrue(expected contentEquals actual, arrayFailMessage(expected, actual))
fun assertArrayEquals(expected: DoubleArray, actual: DoubleArray) =
assertTrue(expected contentEquals actual, arrayFailMessage(expected, actual))
fun assertArrayEquals(expected: FloatArray, actual: FloatArray) =
assertTrue(expected contentEquals actual, arrayFailMessage(expected, actual))
fun <T> arrayFailMessage(expected: Array<out T>, actual: Array<out T>): String =
failMessage(expected.contentToString(), actual.contentToString())
fun arrayFailMessage(expected: IntArray, actual: IntArray): String =
failMessage(expected.contentToString(), actual.contentToString())
fun arrayFailMessage(expected: ShortArray, actual: ShortArray): String =
failMessage(expected.contentToString(), actual.contentToString())
fun arrayFailMessage(expected: LongArray, actual: LongArray): String =
failMessage(expected.contentToString(), actual.contentToString())
fun failMessage(expected: String, actual: String): String =
"Expected: $expected\nActual: $actual"
fun arrayFailMessage(expected: FloatArray, actual: FloatArray): String {
return "Expected: ${expected.contentToString()}\nActual: ${actual.contentToString()}"
}
fun arrayFailMessage(expected: DoubleArray, actual: DoubleArray): String {
return "Expected: ${expected.contentToString()}\nActual: ${actual.contentToString()}"
}
fun arrayFailMessage(expected: BooleanArray, actual: BooleanArray): String {
return "Expected: ${expected.contentToString()}\nActual: ${actual.contentToString()}"
}
fun arrayFailMessage(expected: ByteArray, actual: ByteArray): String {
return "Expected: ${expected.contentToString()}\nActual: ${actual.contentToString()}"
}

575
kotlin/flatbuffers-kotlin/src/commonTest/kotlin/com/google/flatbuffers/kotlin/FlatBufferBuilderTest.kt Normal file
View File

@@ -0,0 +1,575 @@
/*
* Copyright 2021 Google Inc. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
@file:Suppress("UNCHECKED_CAST")
package com.google.flatbuffers.kotlin
import Attacker
import AttackerOffsetArray
import CharacterEArray
import dictionaryLookup.LongFloatEntry
import dictionaryLookup.LongFloatMap
import Movie
import dictionaryLookup.LongFloatEntryOffsetArray
import myGame.example.*
import myGame.example.Test.Companion.createTest
import optionalScalars.OptionalByte
import optionalScalars.ScalarStuff
import kotlin.test.Test
import kotlin.test.assertEquals
@ExperimentalUnsignedTypes
class FlatBufferBuilderTest {
@Test
fun testSingleTable() {
val fbb = FlatBufferBuilder()
val name = fbb.createString("Frodo")
val invValues = ubyteArrayOf(10u, 11u, 12u, 13u, 14u)
val inv = Monster.createInventoryVector(fbb, invValues)
Monster.startMonster(fbb)
Monster.addPos(
fbb, Vec3.createVec3(
fbb, 1.0f, 2.0f, 3.0f, 3.0,
Color.Green, 5.toShort(), 6.toByte()
)
)
Monster.addHp(fbb, 80.toShort())
Monster.addName(fbb, name)
Monster.addMana(fbb, 150)
Monster.addInventory(fbb, inv)
Monster.addTestType(fbb, AnyE.Monster)
Monster.addTestbool(fbb, true)
Monster.addTesthashu32Fnv1(fbb, (Int.MAX_VALUE + 1L).toUInt())
val root = Monster.endMonster(fbb)
fbb.finish(root)
val monster = Monster.asRoot(fbb.dataBuffer())
assertEquals(monster.name, "Frodo")
assertEquals(monster.mana, 150.toShort())
assertEquals(monster.hp, 80)
val pos = monster.pos!!
assertEquals(monster.inventory(0), invValues[0])
assertEquals(monster.inventory(1), invValues[1])
assertEquals(monster.inventory(2), invValues[2])
assertEquals(monster.inventory(3), invValues[3])
assertEquals(pos.x, 1.0f)
assertEquals(pos.y, 2.0f)
assertEquals(pos.z, 3.0f)
assertEquals(pos.test1, 3.0)
assertEquals(pos.test2, Color.Green)
assertEquals(pos.test3!!.a, 5.toShort())
assertEquals(pos.test3!!.b, 6.toByte())
val inventoryBuffer = monster.inventoryAsBuffer()
assertEquals(invValues.size, inventoryBuffer.limit)
for (i in invValues.indices) {
assertEquals(invValues[i], inventoryBuffer.getUByte(i))
}
}
@Test
fun testSortedVector() {
val fbb = FlatBufferBuilder()
val names = arrayOf(fbb.createString("Frodo"), fbb.createString("Barney"), fbb.createString("Wilma"))
val monsters = MonsterOffsetArray(3) {
Monster.startMonster(fbb)
Monster.addName(fbb, names[it])
Monster.endMonster(fbb)
}
val ary = Monster.createTestarrayoftablesVector(fbb, monsters)
Monster.startMonster(fbb)
Monster.addName(fbb, names[0])
Monster.addTestarrayoftables(fbb, ary)
val root = Monster.endMonster(fbb)
fbb.finish(root)
val a = Monster.asRoot(fbb.dataBuffer())
assertEquals(a.name, "Frodo")
assertEquals(a.testarrayoftablesLength, 3)
val monster0 = a.testarrayoftables(0)!!
val monster1 = a.testarrayoftables(1)!!
val monster2 = a.testarrayoftables(2)!!
assertEquals(monster0.name, "Frodo")
assertEquals(monster1.name, "Barney")
assertEquals(monster2.name, "Wilma")
// test AsBuffer feature
}
@Test
fun testCreateBufferVector() {
val fbb = FlatBufferBuilder(16)
val str = fbb.createString("MyMonster")
val inventory = ubyteArrayOf(0u, 1u, 2u, 3u, 4u, 5u, 6u, 88u, 99u, 122u, 1u)
val vec = Monster.createInventoryVector(fbb, inventory)
Monster.startMonster(fbb)
Monster.addInventory(fbb, vec)
Monster.addName(fbb, str)
val monster1 = Monster.endMonster(fbb)
Monster.finishMonsterBuffer(fbb, monster1)
val monsterObject: Monster = Monster.asRoot(fbb.dataBuffer())
val iBuffer = monsterObject.inventoryAsBuffer()
assertEquals(monsterObject.inventoryLength, inventory.size)
assertEquals(iBuffer.limit, inventory.size)
for (i in inventory.indices) {
assertEquals(inventory[i], monsterObject.inventory(i))
assertEquals(inventory[i], iBuffer.getUByte(i))
}
}
@Test
fun testCreateUninitializedVector() {
val fbb = FlatBufferBuilder(16)
val str = fbb.createString("MyMonster")
val inventory = byteArrayOf(10, 11, 12, 13, 14)
val uninitializedBuffer = fbb.createUnintializedVector(1, inventory.size, 1)
for (i in inventory) {
uninitializedBuffer.put(i)
}
val vec = fbb.endVector<UByte>()
Monster.startMonster(fbb)
Monster.addInventory(fbb, vec)
Monster.addName(fbb, str)
val monster1 = Monster.endMonster(fbb)
Monster.finishMonsterBuffer(fbb, monster1)
val monsterObject: Monster = Monster.asRoot(fbb.dataBuffer())
assertEquals(inventory[1].toUByte(), monsterObject.inventory(1))
assertEquals(inventory.size, monsterObject.inventoryLength)
val inventoryBuffer = monsterObject.inventoryAsBuffer()
assertEquals(inventory[1].toInt().toUByte(), inventoryBuffer.getUByte(1))
assertEquals(inventory.size, inventoryBuffer.limit)
}
@Test
fun testBuilderBasics() {
val fbb = FlatBufferBuilder()
val names = arrayOf(fbb.createString("Frodo"), fbb.createString("Barney"), fbb.createString("Wilma"))
val off = Array<Offset<Monster>>(3) { Offset(0) }
Monster.startMonster(fbb)
Monster.addName(fbb, names[0])
off[0] = Monster.endMonster(fbb)
Monster.startMonster(fbb)
Monster.addName(fbb, names[1])
off[1] = Monster.endMonster(fbb)
Monster.startMonster(fbb)
Monster.addName(fbb, names[2])
off[2] = Monster.endMonster(fbb)
val sortMons = fbb.createSortedVectorOfTables(Monster(), off)
// We set up the same values as monsterdata.json:
val inv = Monster.createInventoryVector(fbb, byteArrayOf(0,1,2,3,4).toUByteArray())
val fred = fbb.createString("Fred")
Monster.startMonster(fbb)
Monster.addName(fbb, fred)
val mon2 = Monster.endMonster(fbb)
Monster.startTest4Vector(fbb, 2)
createTest(fbb, 10.toShort(), 20.toByte())
createTest(fbb, 30.toShort(), 40.toByte())
val test4 = fbb.endVector<myGame.example.Test>()
val strings = StringOffsetArray(2) { fbb.createString("test$it") }
val testArrayOfString =
Monster.createTestarrayofstringVector(fbb, strings)
Monster.startMonster(fbb)
Monster.addName(fbb, names[0])
Monster.addPos(fbb, Vec3.createVec3(
fbb, 1.0f, 2.0f, 3.0f, 3.0,
Color.Green, 5.toShort(), 6.toByte()
))
Monster.addHp(fbb, 80)
Monster.addMana(fbb, 150)
Monster.addInventory(fbb, inv)
Monster.addTestType(fbb, AnyE.Monster)
Monster.addTest(fbb, mon2.toUnion())
Monster.addTest4(fbb, test4)
Monster.addTestarrayofstring(fbb, testArrayOfString)
Monster.addTestbool(fbb, true)
Monster.addTesthashu32Fnv1(fbb, (Int.MAX_VALUE + 1L).toUInt())
Monster.addTestarrayoftables(fbb, sortMons)
val mon = Monster.endMonster(fbb)
Monster.finishMonsterBuffer(fbb, mon)
//Attempt to mutate Monster fields and check whether the buffer has been mutated properly
// revert to original values after testing
val monster = Monster.asRoot(fbb.dataBuffer())
// mana is optional and does not exist in the buffer so the mutation should fail
// the mana field should retain its default value
assertEquals(monster.mana, 150.toShort())
assertEquals(monster.hp, 80)
// Accessing a vector of sorted by the key tables
assertEquals(monster.testarrayoftables(0)!!.name, "Barney")
assertEquals(monster.testarrayoftables(1)!!.name, "Frodo")
assertEquals(monster.testarrayoftables(2)!!.name, "Wilma")
// Example of searching for a table by the key
assertEquals(monster.testarrayoftablesByKey("Frodo")!!.name, "Frodo")
assertEquals(monster.testarrayoftablesByKey("Barney")!!.name, "Barney")
assertEquals(monster.testarrayoftablesByKey("Wilma")!!.name, "Wilma")
for (i in 0 until monster.inventoryLength) {
assertEquals(monster.inventory(i), (i).toUByte())
}
// get a struct field and edit one of its fields
val pos2 = monster.pos!!
assertEquals(pos2.x, 1.0f)
assertEquals(pos2.test2, Color.Green)
}
@Test
fun testVectorOfUnions() {
val fbb = FlatBufferBuilder()
val swordAttackDamage = 1
val attacker = Attacker.createAttacker(fbb, swordAttackDamage).toUnion()
val attackers = UnionOffsetArray(1) { attacker }
val characters = CharacterEArray(1)
characters[0] = CharacterE.MuLan.value
Movie.finishMovieBuffer(
fbb,
Movie.createMovie(
fbb,
CharacterE.MuLan,
attacker,
Movie.createCharactersTypeVector(fbb, characters),
Movie.createCharactersVector(fbb, attackers)
)
)
val movie: Movie = Movie.asRoot(fbb.dataBuffer())
assertEquals(movie.charactersTypeLength, 1)
assertEquals(movie.charactersLength, 1)
assertEquals(movie.charactersType(0), CharacterE.MuLan)
assertEquals((movie.characters(Attacker(), 0) as Attacker).swordAttackDamage, swordAttackDamage)
}
@Test
fun TestVectorOfBytes() {
val fbb = FlatBufferBuilder(16)
var str = fbb.createString("ByteMonster")
val data = ubyteArrayOf(0u, 1u, 2u, 3u, 4u, 5u, 6u, 7u, 8u, 9u)
var offset = Monster.createInventoryVector(fbb, data)
Monster.startMonster(fbb)
Monster.addName(fbb, str)
Monster.addInventory(fbb, offset)
var monster1 = Monster.endMonster(fbb)
Monster.finishMonsterBuffer(fbb, monster1)
val monsterObject = Monster.asRoot(fbb.dataBuffer())
assertEquals("ByteMonster", monsterObject.name)
assertEquals(data.size, monsterObject.inventoryLength)
assertEquals(monsterObject.inventory(4), data[4])
offset = fbb.createByteVector(data.toByteArray()) as VectorOffset<UByte> // TODO: fix me
str = fbb.createString("ByteMonster")
Monster.startMonster(fbb)
Monster.addName(fbb, str)
Monster.addInventory(fbb, offset)
monster1 = Monster.endMonster(fbb)
Monster.finishMonsterBuffer(fbb, monster1)
val monsterObject2 = Monster.asRoot(fbb.dataBuffer())
assertEquals(monsterObject2.inventoryLength, data.size)
for (i in data.indices) {
assertEquals(monsterObject2.inventory(i), data[i])
}
fbb.clear()
offset = fbb.createByteVector(data.toByteArray(), 3, 4) as VectorOffset<UByte>
str = fbb.createString("ByteMonster")
Monster.startMonster(fbb)
Monster.addName(fbb, str)
Monster.addInventory(fbb, offset)
monster1 = Monster.endMonster(fbb)
Monster.finishMonsterBuffer(fbb, monster1)
val monsterObject3 = Monster.asRoot(fbb.dataBuffer())
assertEquals(monsterObject3.inventoryLength, 4)
assertEquals(monsterObject3.inventory(0), data[3])
fbb.clear()
offset = Monster.createInventoryVector(fbb, data)
str = fbb.createString("ByteMonster")
Monster.startMonster(fbb)
Monster.addName(fbb, str)
Monster.addInventory(fbb, offset)
monster1 = Monster.endMonster(fbb)
Monster.finishMonsterBuffer(fbb, monster1)
val monsterObject4 = Monster.asRoot(fbb.dataBuffer())
assertEquals(monsterObject4.inventoryLength, data.size)
assertEquals(monsterObject4.inventory(8), 8u)
fbb.clear()
val largeData = ByteArray(1024)
offset = fbb.createByteVector(largeData) as VectorOffset<UByte> //TODO: fix me
str = fbb.createString("ByteMonster")
Monster.startMonster(fbb)
Monster.addName(fbb, str)
Monster.addInventory(fbb, offset)
monster1 = Monster.endMonster(fbb)
Monster.finishMonsterBuffer(fbb, monster1)
val monsterObject5 = Monster.asRoot(fbb.dataBuffer())
assertEquals(monsterObject5.inventoryLength, largeData.size)
assertEquals(monsterObject5.inventory(25), largeData[25].toUByte())
fbb.clear()
var bb = ArrayReadBuffer(largeData, 512)
offset = fbb.createByteVector(bb) as VectorOffset<UByte> //TODO: fix me
str = fbb.createString("ByteMonster")
Monster.startMonster(fbb)
Monster.addName(fbb, str)
Monster.addInventory(fbb, offset)
monster1 = Monster.endMonster(fbb)
Monster.finishMonsterBuffer(fbb, monster1)
val monsterObject6 = Monster.asRoot(fbb.dataBuffer())
assertEquals(monsterObject6.inventoryLength, 512)
assertEquals(monsterObject6.inventory(0), largeData[0].toUByte())
fbb.clear()
bb = ArrayReadBuffer(largeData, largeData.size - 216)
val stringBuffer = ArrayReadBuffer("AlreadyBufferedString".encodeToByteArray())
offset = fbb.createByteVector(bb) as VectorOffset<UByte> //TODO: fix me
str = fbb.createString(stringBuffer)
Monster.startMonster(fbb)
Monster.addName(fbb, str)
Monster.addInventory(fbb, offset)
monster1 = Monster.endMonster(fbb)
Monster.finishMonsterBuffer(fbb, monster1)
val monsterObject7 = Monster.asRoot(fbb.dataBuffer())
assertEquals(monsterObject7.inventoryLength, 216)
assertEquals("AlreadyBufferedString", monsterObject7.name)
}
@Test
fun testEnums() {
assertEquals(Color.name(Color.Red), "Red")
assertEquals(Color.name(Color.Blue), "Blue")
assertEquals(AnyE.name(AnyE.None), "NONE")
assertEquals(AnyE.name(AnyE.Monster), "Monster")
}
@Test
fun testSharedStringPool() {
val fb = FlatBufferBuilder(1)
val testString = "My string"
val offset = fb.createSharedString(testString)
for (i in 0..9) {
assertEquals(offset, fb.createSharedString(testString))
}
}
@Test
fun testScalarOptional() {
val fbb = FlatBufferBuilder(1)
ScalarStuff.startScalarStuff(fbb)
var pos = ScalarStuff.endScalarStuff(fbb)
fbb.finish(pos)
var scalarStuff: ScalarStuff = ScalarStuff.asRoot(fbb.dataBuffer())
assertEquals(scalarStuff.justI8, 0.toByte())
assertEquals(scalarStuff.maybeI8, null)
assertEquals(scalarStuff.defaultI8, 42.toByte())
assertEquals(scalarStuff.justU8, 0u)
assertEquals(scalarStuff.maybeU8, null)
assertEquals(scalarStuff.defaultU8, 42u)
assertEquals(scalarStuff.justI16, 0.toShort())
assertEquals(scalarStuff.maybeI16, null)
assertEquals(scalarStuff.defaultI16, 42.toShort())
assertEquals(scalarStuff.justU16, 0u)
assertEquals(scalarStuff.maybeU16, null)
assertEquals(scalarStuff.defaultU16, 42u)
assertEquals(scalarStuff.justI32, 0)
assertEquals(scalarStuff.maybeI32, null)
assertEquals(scalarStuff.defaultI32, 42)
assertEquals(scalarStuff.justU32, 0u)
assertEquals(scalarStuff.maybeU32, null)
assertEquals(scalarStuff.defaultU32, 42u)
assertEquals(scalarStuff.justI64, 0L)
assertEquals(scalarStuff.maybeI64, null)
assertEquals(scalarStuff.defaultI64, 42L)
assertEquals(scalarStuff.justU64, 0UL)
assertEquals(scalarStuff.maybeU64, null)
assertEquals(scalarStuff.defaultU64, 42UL)
assertEquals(scalarStuff.justF32, 0.0f)
assertEquals(scalarStuff.maybeF32, null)
assertEquals(scalarStuff.defaultF32, 42.0f)
assertEquals(scalarStuff.justF64, 0.0)
assertEquals(scalarStuff.maybeF64, null)
assertEquals(scalarStuff.defaultF64, 42.0)
assertEquals(scalarStuff.justBool, false)
assertEquals(scalarStuff.maybeBool, null)
assertEquals(scalarStuff.defaultBool, true)
assertEquals(scalarStuff.justEnum, OptionalByte.None)
assertEquals(scalarStuff.maybeEnum, null)
assertEquals(scalarStuff.defaultEnum, OptionalByte.One)
fbb.clear()
ScalarStuff.startScalarStuff(fbb)
ScalarStuff.addJustI8(fbb, 5.toByte())
ScalarStuff.addMaybeI8(fbb, 5.toByte())
ScalarStuff.addDefaultI8(fbb, 5.toByte())
ScalarStuff.addJustU8(fbb, 6u)
ScalarStuff.addMaybeU8(fbb, 6u)
ScalarStuff.addDefaultU8(fbb, 6u)
ScalarStuff.addJustI16(fbb, 7.toShort())
ScalarStuff.addMaybeI16(fbb, 7.toShort())
ScalarStuff.addDefaultI16(fbb, 7.toShort())
ScalarStuff.addJustU16(fbb, 8u)
ScalarStuff.addMaybeU16(fbb, 8u)
ScalarStuff.addDefaultU16(fbb, 8u)
ScalarStuff.addJustI32(fbb, 9)
ScalarStuff.addMaybeI32(fbb, 9)
ScalarStuff.addDefaultI32(fbb, 9)
ScalarStuff.addJustU32(fbb, 10u)
ScalarStuff.addMaybeU32(fbb, 10u)
ScalarStuff.addDefaultU32(fbb, 10u)
ScalarStuff.addJustI64(fbb, 11L)
ScalarStuff.addMaybeI64(fbb, 11L)
ScalarStuff.addDefaultI64(fbb, 11L)
ScalarStuff.addJustU64(fbb, 12UL)
ScalarStuff.addMaybeU64(fbb, 12UL)
ScalarStuff.addDefaultU64(fbb, 12UL)
ScalarStuff.addJustF32(fbb, 13.0f)
ScalarStuff.addMaybeF32(fbb, 13.0f)
ScalarStuff.addDefaultF32(fbb, 13.0f)
ScalarStuff.addJustF64(fbb, 14.0)
ScalarStuff.addMaybeF64(fbb, 14.0)
ScalarStuff.addDefaultF64(fbb, 14.0)
ScalarStuff.addJustBool(fbb, true)
ScalarStuff.addMaybeBool(fbb, true)
ScalarStuff.addDefaultBool(fbb, true)
ScalarStuff.addJustEnum(fbb, OptionalByte.Two)
ScalarStuff.addMaybeEnum(fbb, OptionalByte.Two)
ScalarStuff.addDefaultEnum(fbb, OptionalByte.Two)
pos = ScalarStuff.endScalarStuff(fbb)
fbb.finish(pos)
scalarStuff = ScalarStuff.asRoot(fbb.dataBuffer())
assertEquals(scalarStuff.justI8, 5.toByte())
assertEquals(scalarStuff.maybeI8, 5.toByte())
assertEquals(scalarStuff.defaultI8, 5.toByte())
assertEquals(scalarStuff.justU8, 6u)
assertEquals(scalarStuff.maybeU8, 6u)
assertEquals(scalarStuff.defaultU8, 6u)
assertEquals(scalarStuff.justI16, 7.toShort())
assertEquals(scalarStuff.maybeI16, 7.toShort())
assertEquals(scalarStuff.defaultI16, 7.toShort())
assertEquals(scalarStuff.justU16, 8u)
assertEquals(scalarStuff.maybeU16, 8u)
assertEquals(scalarStuff.defaultU16, 8u)
assertEquals(scalarStuff.justI32, 9)
assertEquals(scalarStuff.maybeI32, 9)
assertEquals(scalarStuff.defaultI32, 9)
assertEquals(scalarStuff.justU32, 10u)
assertEquals(scalarStuff.maybeU32, 10u)
assertEquals(scalarStuff.defaultU32, 10u)
assertEquals(scalarStuff.justI64, 11L)
assertEquals(scalarStuff.maybeI64, 11L)
assertEquals(scalarStuff.defaultI64, 11L)
assertEquals(scalarStuff.justU64, 12UL)
assertEquals(scalarStuff.maybeU64, 12UL)
assertEquals(scalarStuff.defaultU64, 12UL)
assertEquals(scalarStuff.justF32, 13.0f)
assertEquals(scalarStuff.maybeF32, 13.0f)
assertEquals(scalarStuff.defaultF32, 13.0f)
assertEquals(scalarStuff.justF64, 14.0)
assertEquals(scalarStuff.maybeF64, 14.0)
assertEquals(scalarStuff.defaultF64, 14.0)
assertEquals(scalarStuff.justBool, true)
assertEquals(scalarStuff.maybeBool, true)
assertEquals(scalarStuff.defaultBool, true)
assertEquals(scalarStuff.justEnum, OptionalByte.Two)
assertEquals(scalarStuff.maybeEnum, OptionalByte.Two)
assertEquals(scalarStuff.defaultEnum, OptionalByte.Two)
}
// @todo Seems like nesting code generation is broken for all generators.
// disabling test for now.
// @Test
// fun testNamespaceNesting() {
// // reference / manipulate these to verify compilation
// val fbb = FlatBufferBuilder(1)
// TableInNestedNS.startTableInNestedNS(fbb)
// TableInNestedNS.addFoo(fbb, 1234)
// val nestedTableOff = TableInNestedNS.endTableInNestedNs(fbb)
// TableInFirstNS.startTableInFirstNS(fbb)
// TableInFirstNS.addFooTable(fbb, nestedTableOff)
// TableInFirstNS.endTableInFirstNs(fbb)
// }
@Test
fun testNestedFlatBuffer() {
val nestedMonsterName = "NestedMonsterName"
val nestedMonsterHp: Short = 600
val nestedMonsterMana: Short = 1024
val fbb1 = FlatBufferBuilder(16)
val str1 = fbb1.createString(nestedMonsterName)
Monster.startMonster(fbb1)
Monster.addName(fbb1, str1)
Monster.addHp(fbb1, nestedMonsterHp)
Monster.addMana(fbb1, nestedMonsterMana)
val monster1 = Monster.endMonster(fbb1)
Monster.finishMonsterBuffer(fbb1, monster1)
val fbb1Bytes: ByteArray = fbb1.sizedByteArray()
val fbb2 = FlatBufferBuilder(16)
val str2 = fbb2.createString("My Monster")
val nestedBuffer = Monster.createTestnestedflatbufferVector(fbb2, fbb1Bytes.toUByteArray())
Monster.startMonster(fbb2)
Monster.addName(fbb2, str2)
Monster.addHp(fbb2, 50.toShort())
Monster.addMana(fbb2, 32.toShort())
Monster.addTestnestedflatbuffer(fbb2, nestedBuffer)
val monster = Monster.endMonster(fbb2)
Monster.finishMonsterBuffer(fbb2, monster)
// Now test the data extracted from the nested buffer
val mons = Monster.asRoot(fbb2.dataBuffer())
val nestedMonster = mons.testnestedflatbufferAsMonster
assertEquals(nestedMonsterMana, nestedMonster!!.mana)
assertEquals(nestedMonsterHp, nestedMonster.hp)
assertEquals(nestedMonsterName, nestedMonster.name)
}
@Test
fun testDictionaryLookup() {
val fbb = FlatBufferBuilder(16)
val lfIndex = LongFloatEntry.createLongFloatEntry(fbb, 0, 99.0f)
val vectorEntriesIdx = LongFloatMap.createEntriesVector(fbb, LongFloatEntryOffsetArray(1) { lfIndex })
val rootIdx = LongFloatMap.createLongFloatMap(fbb, vectorEntriesIdx)
LongFloatMap.finishLongFloatMapBuffer(fbb, rootIdx)
val map: LongFloatMap = LongFloatMap.asRoot(fbb.dataBuffer())
assertEquals(1, map.entriesLength)
val e: LongFloatEntry = map.entries(0)!!
assertEquals(0L, e.key)
assertEquals(99.0f, e.value)
val e2: LongFloatEntry = map.entriesByKey(0)!!
assertEquals(0L, e2.key)
assertEquals(99.0f, e2.value)
}
}

1
kotlin/flatbuffers-kotlin/src/commonTest/kotlin/com/google/flatbuffers/kotlin/FlexBuffersTest.kt
View File

@@ -21,6 +21,7 @@ import kotlin.test.Test
import kotlin.test.assertEquals
class FlexBuffersTest {
@Test
fun testWriteInt() {
val values = listOf(

1
kotlin/flatbuffers-kotlin/src/jvmMain/kotlin/com/google/flatbuffers/kotlin/ByteArray.kt
View File

@@ -13,6 +13,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
@file:JvmName("JVMByteArray")
@file:Suppress("NOTHING_TO_INLINE")

9
kotlin/flatbuffers-kotlin/src/jvmTest/kotlin/com/google/flatbuffers/kotlin/Utf8Test.kt
View File

@@ -22,11 +22,16 @@ class Utf8Test {
@Test
fun testUtf8EncodingDecoding() {
val utf8Lines = String(this.javaClass.classLoader.getResourceAsStream("utf8_sample.txt")!!.readBytes())
val classLoader = this.javaClass.classLoader
val utf8Lines = String(classLoader.getResourceAsStream("utf8_sample.txt")!!.readBytes())
.split("\n")
.filter { it.trim().isNotEmpty() }
val utf8Bytes = utf8Lines.map { s -> ByteArray(Utf8.encodedLength(s)).also { Utf8.encodeUtf8Array(s, it) } }
val utf8Bytes = utf8Lines.map {
s -> ByteArray(Utf8.encodedLength(s)).also {
Utf8.encodeUtf8Array(s, it)
}
}
utf8Bytes.indices.forEach {
assertArrayEquals(utf8Lines[it].encodeToByteArray(), utf8Bytes[it])
assertEquals(utf8Lines[it], Utf8.decodeUtf8Array(utf8Bytes[it]))

1
kotlin/flatbuffers-kotlin/src/nativeMain/kotlin/com/google/flatbuffers/kotlin/ByteArray.kt
View File

@@ -14,6 +14,7 @@
* limitations under the License.
*/
@file:Suppress("NOTHING_TO_INLINE")
package com.google.flatbuffers.kotlin
/**

12
kotlin/gradle.properties
View File

@@ -1,4 +1,6 @@
#Gradle
group = "com.google.flatbuffers"
version = "2.0.0-SNAPSHOT"
org.gradle.parallel=true
org.gradle.caching=true
@@ -7,8 +9,12 @@ org.gradle.caching=true
kotlin.code.style=official
#MPP
kotlin.mpp.enableGranularSourceSetsMetadata=true
kotlin.native.enableDependencyPropagation=false
kotlin.mpp.enableCompatibilityMetadataVariant=true
kotlin.js.compiler=ir
kotlin.native.ignoreDisabledTargets=true
kotlin.mpp.stability.nowarn=true
kotlin.incremental.multiplatform=true
kotlin.native.binary.memoryModel=experimental
kotlin.native.distribution.type=prebuilt
org.gradle.jvmargs="-XX:+HeapDumpOnOutOfMemoryError"

14
kotlin/gradle/libs.versions.toml
View File

@@ -1,7 +1,8 @@
[versions]
kotlin = "1.7.21"
kotlin = "1.8.21"
plugin-kotlin = "1.6.10"
plugin-gver = "0.42.0"
kotlinx-benchmark = "0.4.6"
kotlinx-benchmark = "0.4.8"
junit = "4.12"
gson = "2.8.5"
moshi-kotlin = "1.11.0"
@@ -11,9 +12,14 @@ kotlin-compiler = { module = "org.jetbrains.kotlin:kotlin-compiler", version.ref
moshi-kotlin = { module = "com.squareup.moshi:moshi-kotlin", version.ref = "moshi-kotlin" }
gson = { module = "com.google.code.gson:gson", version.ref = "gson" }
kotlinx-benchmark-runtime = { module = "org.jetbrains.kotlinx:kotlinx-benchmark-runtime", version.ref = "kotlinx-benchmark" }
plugin-gver = { module = "com.github.ben-manes:gradle-versions-plugin", version.ref = "plugin-gver" }
junit = { module="junit:junit", version.ref="junit"}
kotlin-allopen = { module = "org.jetbrains.kotlin:kotlin-allopen", version.ref = "kotlin"}
plugin-kotlin-gradle = { module = "org.jetbrains.kotlin:kotlin-gradle-plugin", version.ref = "kotlin" }
plugin-kotlinx-benchmark = { module="org.jetbrains.kotlinx:kotlinx-benchmark-plugin", version.ref="kotlinx-benchmark"}
plugin-jmhreport = { module = "gradle.plugin.io.morethan.jmhreport:gradle-jmh-report", version="0.9.0" }
plugin-download = { module = "de.undercouch:gradle-download-task", version = "5.3.0"}
junit = { module="junit:junit", version.ref="junit"}

1
src/BUILD.bazel
View File

@@ -123,6 +123,7 @@ cc_library(
"idl_gen_json_schema.h",
"idl_gen_kotlin.cpp",
"idl_gen_kotlin.h",
"idl_gen_kotlin_kmp.cpp",
"idl_gen_lobster.cpp",
"idl_gen_lobster.h",
"idl_gen_php.cpp",

5
src/flatc_main.cpp
View File

@@ -125,6 +125,11 @@ int main(int argc, const char *argv[]) {
"Generate Kotlin classes for tables/structs" },
flatbuffers::NewKotlinCodeGenerator());
flatc.RegisterCodeGenerator(
flatbuffers::FlatCOption{ "", "kotlin-kmp", "",
"Generate Kotlin multiplatform classes for tables/structs" },
flatbuffers::NewKotlinKMPCodeGenerator());
flatc.RegisterCodeGenerator(
flatbuffers::FlatCOption{ "", "lobster", "",
"Generate Lobster files for tables/structs" },

2
src/idl_gen_kotlin.h
View File

@@ -24,6 +24,8 @@ namespace flatbuffers {
// Constructs a new Kotlin code generator.
std::unique_ptr<CodeGenerator> NewKotlinCodeGenerator();
// Constructs a new Kotlin code generator.
std::unique_ptr<CodeGenerator> NewKotlinKMPCodeGenerator();
} // namespace flatbuffers
#endif // FLATBUFFERS_IDL_GEN_KOTLIN_H_

1623
src/idl_gen_kotlin_kmp.cpp Normal file
View File

File diff suppressed because it is too large Load Diff

5
src/idl_namer.h
View File

@@ -88,8 +88,9 @@ class IdlNamer : public Namer {
}
std::string Directories(const struct Namespace &ns,
SkipDir skips = SkipDir::None) const {
return Directories(ns.components, skips);
SkipDir skips = SkipDir::None,
Case input_case = Case::kUpperCamel) const {
return Directories(ns.components, skips, input_case);
}
// Legacy fields do not really follow the usual config and should be

9
src/idl_parser.cpp
View File

@@ -2679,9 +2679,10 @@ std::vector<IncludedFile> Parser::GetIncludedFiles() const {
bool Parser::SupportsOptionalScalars(const flatbuffers::IDLOptions &opts) {
static FLATBUFFERS_CONSTEXPR unsigned long supported_langs =
IDLOptions::kRust | IDLOptions::kSwift | IDLOptions::kLobster |
IDLOptions::kKotlin | IDLOptions::kCpp | IDLOptions::kJava |
IDLOptions::kCSharp | IDLOptions::kTs | IDLOptions::kBinary |
IDLOptions::kGo | IDLOptions::kPython | IDLOptions::kJson |
IDLOptions::kKotlin | IDLOptions::kKotlinKmp | IDLOptions::kCpp |
IDLOptions::kJava | IDLOptions::kCSharp | IDLOptions::kTs |
IDLOptions::kBinary | IDLOptions::kGo | IDLOptions::kPython |
IDLOptions::kJson |
IDLOptions::kNim;
unsigned long langs = opts.lang_to_generate;
return (langs > 0 && langs < IDLOptions::kMAX) && !(langs & ~supported_langs);
@@ -2702,7 +2703,7 @@ bool Parser::SupportsAdvancedUnionFeatures() const {
~(IDLOptions::kCpp | IDLOptions::kTs | IDLOptions::kPhp |
IDLOptions::kJava | IDLOptions::kCSharp | IDLOptions::kKotlin |
IDLOptions::kBinary | IDLOptions::kSwift | IDLOptions::kNim |
IDLOptions::kJson)) == 0;
IDLOptions::kJson | IDLOptions::kKotlinKmp)) == 0;
}
bool Parser::SupportsAdvancedArrayFeatures() const {

8
src/namer.h
View File

@@ -185,15 +185,19 @@ class Namer {
// right seperator. Output path prefixing and the trailing separator may be
// skiped using `skips`.
// Callers may want to use `EnsureDirExists` with the result.
// input_case is used to tell how to modify namespace. e.g. kUpperCamel will
// add a underscode between case changes, so MyGame turns into My_Game
// (depending also on the output_case).
virtual std::string Directories(const std::vector<std::string> &directories,
SkipDir skips = SkipDir::None) const {
SkipDir skips = SkipDir::None,
Case input_case = Case::kUpperCamel) const {
const bool skip_output_path =
(skips & SkipDir::OutputPath) != SkipDir::None;
const bool skip_trailing_seperator =
(skips & SkipDir::TrailingPathSeperator) != SkipDir::None;
std::string result = skip_output_path ? "" : config_.output_path;
for (auto d = directories.begin(); d != directories.end(); d++) {
result += ConvertCase(*d, config_.directories, Case::kUpperCamel);
result += ConvertCase(*d, config_.directories, input_case);
result.push_back(kPathSeparator);
}
if (skip_trailing_seperator && !result.empty()) result.pop_back();

2
tests/KotlinTest.sh
View File

@@ -34,7 +34,7 @@ fi
all_kt_files=`find . -name "*.kt" -print`
# Compile java FlatBuffer library
javac ${testdir}/../java/com/google/flatbuffers/*.java -d $targetdir
javac ${testdir}/../java/src/main/java/com/google/flatbuffers/*.java -d $targetdir
# Compile Kotlin files
kotlinc $all_kt_files -classpath $targetdir -include-runtime -d $targetdir
# Make jar