20 Commits
Author SHA1 Message Date
rboullardandClaude Sonnet 5 ae18a062b7 Add Sasquatch: mmap'd, load-only Game-mode asset backend, plus parallel batch loading
Bigfoot / Sonarqube (push) Skipped
Bigfoot / Build & Test Debug with ./ConanProfiles/host/clang (Unity Build: OFF) (push) Successful in 6m12s
Bigfoot / Build & Test Debug with ./ConanProfiles/host/clang (Unity Build: ON) (push) Successful in 5m49s
Bigfoot / Build & Test Debug with ./ConanProfiles/host/clang_asan (Unity Build: OFF) (push) Successful in 6m33s
Bigfoot / Build & Test Debug with ./ConanProfiles/host/clang_asan (Unity Build: ON) (push) Successful in 6m50s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/host/clang (Unity Build: OFF) (push) Successful in 6m48s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/host/clang_asan (Unity Build: OFF) (push) Failing after 32s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/host/clang_asan (Unity Build: ON) (push) Failing after 0s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/host/clang (Unity Build: ON) (push) Successful in 6m40s
Bigfoot / Build & Test Release with ./ConanProfiles/host/clang (Unity Build: OFF) (push) Canceled after 0s
Bigfoot / Build & Test Release with ./ConanProfiles/host/clang (Unity Build: ON) (push) Successful in 15m18s
Bigfoot / Clang Format Checks (push) Failing after 10s
Bigfoot / Build & Test Release with ./ConanProfiles/host/clang_asan (Unity Build: ON) (push) Successful in 16m5s
Bigfoot / Build & Test Release with ./ConanProfiles/host/clang_asan (Unity Build: OFF) (push) Successful in 16m28s
Introduces a dual-backend SerializationManager: Tool mode keeps reading/writing
BigFile over SQL as before, Game mode now reads a Sasquatch file instead - a
flattened, mmap'd counterpart to a BigFile produced ahead of time by
Sasquatch::WriteFrom or the new SasquatchPacker CLI tool. The backend is picked
at compile time via a new BIGFOOT_TOOL_MODE option (mirrors the existing
BIGFOOT_OPTIMIZED/BIGFOOT_NOT_OPTIMIZED pattern); SaveAsset only exists in Tool
mode, since a running game never writes back to its own asset pack.

Also adds SerializationManager::LoadAssets (Game-mode only): loads a batch of
UUIDs by decoding them across multiple threads instead of one at a time,
layered by HardReference depth so a reference target is always fully
registered before anything decodes a reference to it. This required making
HardReference/SoftReference resolution genuinely thread-safe (it mutates
ContainerHandle ref counts and AssetLibrary's UUID index/unreferenced-set from
inside Unpack()) via atomic ref counts and targeted mutexes, validated by
dedicated concurrency tests exercising real thread contention.

BigFile itself is faster too: prepared statements are now pooled per query
text instead of recompiled on every Request (pooled rather than cached-as-one
because LoadAsset recurses and a naive cache would corrupt an in-flight
statement), and PRAGMA mmap_size is enabled.

Benchmarked against real Asset<T> objects (AssetsForBenchmarking) through the
full Pack/Unpack/HardReference pipeline: Sasquatch's raw storage layer hits
~31 GiB/s (mmap ceiling), and the new parallel LoadAssets reaches ~18 GiB/s
end-to-end versus ~1.3 GiB/s for the existing sequential LoadAsset loop.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-22 17:43:21 +02:00
rboullard 33331ea529 flush logs on all asserts 2026-08-21 23:01:25 +02:00
rboullard 8a36af6bcb Update profiles
Bigfoot / Sonarqube (push) Skipped
Bigfoot / Build & Test Debug with ./ConanProfiles/host/clang_asan (Unity Build: OFF) (push) Successful in 6m16s
Bigfoot / Build & Test Debug with ./ConanProfiles/host/clang (Unity Build: OFF) (push) Successful in 5m47s
Bigfoot / Build & Test Debug with ./ConanProfiles/host/clang (Unity Build: ON) (push) Successful in 5m35s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/host/clang (Unity Build: ON) (push) Successful in 6m22s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/host/clang_asan (Unity Build: OFF) (push) Successful in 7m25s
Bigfoot / Build & Test Debug with ./ConanProfiles/host/clang_asan (Unity Build: ON) (push) Successful in 6m7s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/host/clang (Unity Build: OFF) (push) Successful in 6m17s
Bigfoot / Build & Test Release with ./ConanProfiles/host/clang (Unity Build: OFF) (push) Successful in 6m10s
Bigfoot / Build & Test Release with ./ConanProfiles/host/clang_asan (Unity Build: OFF) (push) Successful in 7m17s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/host/clang_asan (Unity Build: ON) (push) Successful in 7m21s
Bigfoot / Build & Test Release with ./ConanProfiles/host/clang (Unity Build: ON) (push) Successful in 6m7s
Bigfoot / Build & Test Release with ./ConanProfiles/host/clang_asan (Unity Build: ON) (push) Successful in 6m55s
Bigfoot / Clang Format Checks (push) Successful in 9s
2026-08-19 10:09:58 +02:00
rboullard c6ccdf8214 Cleanup
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 7m24s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 6m8s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 7m2s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 6m18s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 6m24s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 7m32s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 6m6s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 6m59s
Bigfoot / Clang Format Checks (push) Successful in 8s
Bigfoot / Sonarqube (push) Skipped
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 6m6s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 5m40s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 5m33s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 6m15s
2026-08-18 18:25:34 +02:00
rboullard d9f4a0e896 Do not allow creating ReferenceBase
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: OFF) (push) Canceled after 0s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: ON) (push) Canceled after 0s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Canceled after 0s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Canceled after 0s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: OFF) (push) Canceled after 0s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: ON) (push) Canceled after 0s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Canceled after 0s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Canceled after 0s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: OFF) (push) Canceled after 0s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: ON) (push) Canceled after 0s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Canceled after 0s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Canceled after 0s
Bigfoot / Clang Format Checks (push) Canceled after 0s
Bigfoot / Sonarqube (push) Skipped
2026-08-18 18:12:05 +02:00
rboullard 1d572b9d09 Simplify asserts
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: OFF) (push) Canceled after 0s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: ON) (push) Canceled after 0s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Canceled after 0s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Canceled after 0s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: OFF) (push) Canceled after 0s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: ON) (push) Canceled after 0s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Canceled after 0s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Canceled after 0s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: OFF) (push) Canceled after 0s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: ON) (push) Canceled after 0s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Canceled after 0s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Canceled after 0s
Bigfoot / Clang Format Checks (push) Canceled after 0s
Bigfoot / Sonarqube (push) Skipped
2026-08-18 17:41:24 +02:00
rboullard 870bfe57e6 Cleanup
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 6m15s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 5m48s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 5m39s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 6m29s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 6m20s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 6m25s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 6m18s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 1m10s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 7m28s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 6m16s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 3m55s
Bigfoot / Clang Format Checks (push) Canceled after 0s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Canceled after 2m13s
Bigfoot / Sonarqube (push) Skipped
2026-08-18 17:30:39 +02:00
rboullard 9978ff358b Format + API cleanup
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 8m15s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 6m28s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 6m22s
Bigfoot / Clang Format Checks (push) Successful in 8s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 7m25s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 7m1s
Bigfoot / Sonarqube (push) Skipped
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 5m44s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 6m30s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 5m52s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 6m24s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 6m17s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 6m35s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 7m49s
2026-08-18 17:11:57 +02:00
rboullard 8be97bacb5 Full test for reference Copy/Move
Bigfoot / Sonarqube (push) Skipped
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 5m39s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 6m25s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 6m6s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 7m28s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 7m32s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 6m31s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 7m12s
Bigfoot / Clang Format Checks (push) Failing after 8s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 5m44s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 6m25s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 6m18s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 6m27s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 7m8s
2026-08-18 17:01:26 +02:00
rboullard cea3b8a904 SoftRef/HardRef count testing
Bigfoot / Sonarqube (push) Skipped
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 6m17s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 5m46s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 5m35s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 6m14s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 6m25s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 6m18s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 7m30s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 7m27s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 6m19s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 6m21s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 7m6s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 7m6s
Bigfoot / Clang Format Checks (push) Failing after 8s
2026-08-18 16:53:56 +02:00
rboullard 1d2dafff48 msvc flags update
Bigfoot / Sonarqube (push) Skipped
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 5m43s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 6m12s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 6m23s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 5m41s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 6m8s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 6m22s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 7m34s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 7m21s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 6m16s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 6m12s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 6m58s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 7m7s
Bigfoot / Clang Format Checks (push) Failing after 8s
2026-08-18 15:49:50 +02:00
rboullard 17ab916b11 Remove unused method
Bigfoot / Sonarqube (push) Skipped
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 5m45s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 5m50s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 6m14s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 6m11s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 6m25s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 7m28s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 6m17s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 7m21s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 6m13s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 6m8s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 6m59s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 6m57s
Bigfoot / Clang Format Checks (push) Failing after 8s
2026-08-18 15:23:49 +02:00
rboullard 3c306dd2c7 basic testing for reference
Bigfoot / Sonarqube (push) Skipped
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 5m52s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 5m46s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 6m18s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 6m14s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 7m17s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 6m25s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 8m7s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 7m31s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 6m24s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 6m18s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 7m30s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 7m6s
Bigfoot / Clang Format Checks (push) Failing after 8s
2026-08-18 14:59:42 +02:00
rboullard a1e4119311 Fix compilation
Bigfoot / Sonarqube (push) Skipped
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 5m49s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 5m50s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 6m11s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 6m5s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 6m17s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 6m19s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 7m38s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 7m26s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 6m11s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 6m6s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 6m59s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 6m59s
Bigfoot / Clang Format Checks (push) Successful in 8s
2026-08-18 10:23:23 +02:00
rboullard 1bae20b79c Fix clang-format
Bigfoot / Sonarqube (push) Skipped
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 5m39s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 7m16s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: ON) (push) Failing after 5m57s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 5m45s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 6m7s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 6m1s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 6m19s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 6m13s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 7m25s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: OFF) (push) Failing after 5m59s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Failing after 6m43s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Failing after 6m44s
Bigfoot / Clang Format Checks (push) Successful in 9s
2026-08-18 00:52:22 +02:00
rboullard 39d2ddbe49 ContainerHandle test
Bigfoot / Sonarqube (push) Skipped
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 6m13s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: OFF) (push) Successful in 5m34s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 5m33s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 5m55s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 5m54s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 6m2s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 7m9s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Successful in 7m13s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: OFF) (push) Failing after 5m49s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: ON) (push) Failing after 5m52s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Failing after 6m34s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Failing after 6m34s
Bigfoot / Clang Format Checks (push) Failing after 0s
2026-08-17 18:32:02 +02:00
rboullard 6bc25a776e format
Bigfoot / Sonarqube (push) Skipped
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: OFF) (push) Failing after 10m27s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Successful in 6m3s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: OFF) (push) Canceled after 0s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: ON) (push) Canceled after 0s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Canceled after 0s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Canceled after 0s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: OFF) (push) Canceled after 0s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: ON) (push) Canceled after 0s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Canceled after 0s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Canceled after 0s
Bigfoot / Clang Format Checks (push) Canceled after 0s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Canceled after 2m34s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: ON) (push) Successful in 5m30s
2026-08-17 18:08:27 +02:00
rboullard 96c4353a8a StableSlotMap
Bigfoot / Sonarqube (push) Skipped
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: OFF) (push) Failing after 2m4s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: ON) (push) Failing after 2m34s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Failing after 8m3s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Failing after 6m46s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: OFF) (push) Canceled after 0s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: ON) (push) Canceled after 0s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Canceled after 0s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Canceled after 0s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: ON) (push) Canceled after 0s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Canceled after 0s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Canceled after 0s
Bigfoot / Clang Format Checks (push) Canceled after 0s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: OFF) (push) Canceled after 1m18s
2026-08-17 17:25:49 +02:00
rboullard 9733860ecd Fix compilation
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Failing after 0s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Failing after 0s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: OFF) (push) Failing after 1s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: OFF) (push) Failing after 0s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: ON) (push) Failing after 0s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: ON) (push) Failing after 0s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Failing after 0s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: OFF) (push) Failing after 0s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Failing after 0s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: ON) (push) Failing after 0s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Failing after 0s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Failing after 0s
Bigfoot / Clang Format Checks (push) Failing after 0s
2026-08-17 02:32:42 +02:00
rboullard 2062b8169f BigFile
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Failing after 0s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: OFF) (push) Failing after 0s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Failing after 0s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Failing after 0s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: ON) (push) Failing after 0s
Bigfoot / Clang Format Checks (push) Failing after 0s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: OFF) (push) Failing after 0s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang (Unity Build: ON) (push) Failing after 0s
Bigfoot / Build & Test Debug with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Failing after 0s
Bigfoot / Build & Test RelWithDebInfo with ./ConanProfiles/clang (Unity Build: ON) (push) Failing after 0s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: OFF) (push) Failing after 0s
Bigfoot / Build & Test Release with ./ConanProfiles/clang (Unity Build: ON) (push) Failing after 0s
Bigfoot / Build & Test Release with ./ConanProfiles/clang_asan (Unity Build: OFF) (push) Failing after 0s
2026-08-17 01:16:13 +02:00
172 changed files with 4896 additions and 9690 deletions
+7 -28
View File
@@ -127,37 +127,16 @@ IncludeCategories:
Priority: 2
- Regex: '^<Utils/'
Priority: 3
- Regex: '^<SasquatchBaker/'
Priority: 4
- Regex: '^<EngineTests/'
Priority: 5
Priority: 4
- Regex: '^<SystemTests/'
Priority: 6
Priority: 5
- Regex: '^<UtilsTests/'
Priority: 7
- Regex: '^<AssetsForTesting/'
Priority: 8
- Regex: '^<SasquatchBakerTests/'
Priority: 9
- Regex: '^<EngineBenchmarks/'
Priority: 10
- Regex: '^<AssetsForBenchmarking/'
Priority: 11
- Regex: '^<SasquatchBakerBenchmarks/'
Priority: 12
Priority: 6
- Regex: '^<[a-zA-Z0-9_/]+\.h>'
Priority: 13
Priority: 7
- Regex: '^<[a-zA-Z0-9_/]+\.hpp>'
Priority: 13
Priority: 7
- Regex: '^<[a-zA-Z0-9_/]+>'
Priority: 14
MainIncludeChar: AngleBracket
Macros:
- 'BIGFOOT_OPTIMIZED_ONLY(x)=x'
- 'BIGFOOT_OPTIMIZED_ONLY(x,y)=x,y'
- 'BIGFOOT_NOT_OPTIMIZED_ONLY(x)=x'
- 'BIGFOOT_NOT_OPTIMIZED_ONLY(x,y)=x,y'
- 'BIGFOOT_TOOLMODE_ONLY(x)=x'
- 'BIGFOOT_TOOLMODE_ONLY(x,y)=x,y'
- 'BIGFOOT_GAME_ONLY(x)=x'
- 'BIGFOOT_GAME_ONLY(x,y)=x,y'
Priority: 8
MainIncludeChar: AngleBracket
+1 -5
View File
@@ -11,13 +11,9 @@ Checks: >
portability-*,
-modernize-use-trailing-return-type,
-modernize-use-auto,
-readability-avoid-const-params-in-decls,
-cppcoreguidelines-macro-usage,
-cppcoreguidelines-avoid-do-while,
-cppcoreguidelines-pro-bounds-avoid-unchecked-container-access,
-cppcoreguidelines-pro-bounds-pointer-arithmetic,
-cppcoreguidelines-pro-type-reinterpret-cast
-cppcoreguidelines-avoid-do-while
ExcludeHeaderFilterRegex: '_generated.*'
+11 -14
View File
@@ -18,10 +18,7 @@ jobs:
build_type: ["Debug", "RelWithDebInfo", "Release"]
unity_build: ["ON", "OFF"]
conan_profile: ["./ConanProfiles/host/clang", "./ConanProfiles/host/clang_asan"]
variant:
- { value: "True", name: "toolmode" }
- { value: "False", name: "game" }
name: "Build & Test ${{ matrix.build_type }} with ${{ matrix.conan_profile }} (Unity Build: ${{ matrix.unity_build }}, Variant: ${{ matrix.variant.name }})"
name: "Build & Test ${{ matrix.build_type }} with ${{ matrix.conan_profile }} (Unity Build: ${{ matrix.unity_build }})"
steps:
- name: Install Node.js
run: apt-get update && apt-get install -y nodejs
@@ -33,18 +30,18 @@ jobs:
- name: Build
run: |
conan install . --remote=bigfootpackages -pr:h=${{ matrix.conan_profile }} -pr:b=./ConanProfiles/build/clang --build=missing -s build_type=${{ matrix.build_type }} -o bigfoot/*:build_tests=True -o bigfoot/*:build_benchmarks=True -o bigfoot/*:tracy=False -o bigfoot/*:vulkan=True -o bigfoot/*:toolmode=${{ matrix.variant.value }} -of build-${{ matrix.variant.name }}
. ./build-${{ matrix.variant.name }}/build/${{ matrix.build_type }}/generators/conanbuild.sh
cmake -S . -B ./build-${{ matrix.variant.name }}/${{ matrix.build_type }} --toolchain ./build-${{ matrix.variant.name }}/build/${{ matrix.build_type }}/generators/conan_toolchain.cmake -DCMAKE_BUILD_TYPE=${{ matrix.build_type }} -DCMAKE_UNITY_BUILD=${{ matrix.unity_build }} -G "Ninja"
cmake --build build-${{ matrix.variant.name }}/${{ matrix.build_type }} --parallel $(nproc)
. ./build-${{ matrix.variant.name }}/build/${{ matrix.build_type }}/generators/deactivate_conanbuild.sh
conan install . --remote=bigfootpackages -pr:h=${{ matrix.conan_profile }} -pr:b=./ConanProfiles/build/clang --build=* -s build_type=${{ matrix.build_type }} -o bigfoot/*:build_tests=True -o bigfoot/*:build_benchmarks=True -o bigfoot/*:tracy=False -o bigfoot/*:vulkan=True
. ./build/${{ matrix.build_type }}/generators/conanbuild.sh
cmake -S . -B ./build/${{ matrix.build_type }} --toolchain ./build/${{ matrix.build_type }}/generators/conan_toolchain.cmake -DCMAKE_BUILD_TYPE=${{ matrix.build_type }} -DCMAKE_UNITY_BUILD=${{ matrix.unity_build }} -G "Ninja"
cmake --build build/${{ matrix.build_type }} --parallel $(nproc)
. ./build/${{ matrix.build_type }}/generators/deactivate_conanbuild.sh
- name: Unit Tests
run: |
cd ./build-${{ matrix.variant.name }}/${{ matrix.build_type }}
. ../../build-${{ matrix.variant.name }}/build/${{ matrix.build_type }}/generators/conanbuild.sh
cd ./build/${{ matrix.build_type }}
. ../../build/${{ matrix.build_type }}/generators/conanbuild.sh
xvfb-run ctest . --output-on-failure
. ../../build-${{ matrix.variant.name }}/build/${{ matrix.build_type }}/generators/deactivate_conanbuild.sh
. ../../build/${{ matrix.build_type }}/generators/deactivate_conanbuild.sh
clang-format:
runs-on: ubuntu-latest
@@ -60,6 +57,6 @@ jobs:
uses: actions/checkout@v6
with:
submodules: recursive
- name: Clang Format Checks
run: chmod +x format.sh && ./format.sh --check Bigfoot
run: chmod +x format.sh && ./format.sh --check Bigfoot
+12 -59
View File
@@ -5,10 +5,6 @@ on:
branches:
- main
- Development
pull_request:
branches:
- main
- Development
jobs:
build-and-test:
@@ -17,14 +13,7 @@ jobs:
container:
image: git.romainboullard.com/bigfootdev/linuxcppbuilder:development
strategy:
fail-fast: false
matrix:
variant:
- { value: "True", name: "toolmode", projectKey: "Bigfoot-Tool", projectName: "Bigfoot Tool" }
- { value: "False", name: "game", projectKey: "Bigfoot-Game", projectName: "Bigfoot Game" }
name: "Sonarqube (${{ matrix.variant.name }})"
name: "Sonarqube"
steps:
- name: Install Node.js
run: apt-get update && apt-get install -y nodejs
@@ -34,64 +23,28 @@ jobs:
with:
fetch-depth: 0
submodules: recursive
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Read version
id: version
run: echo "version=$(cat VERSION)" >> "$GITHUB_OUTPUT"
- name: Build
run: |
conan install . --remote=bigfootpackages -pr:h=./ConanProfiles/host/clang_coverage -pr:b=./ConanProfiles/build/clang --build=missing -s build_type=Debug -o bigfoot/*:build_tests=True -o bigfoot/*:build_benchmarks=True -o bigfoot/*:tracy=False -o bigfoot/*:toolmode=${{ matrix.variant.value }} -o bigfoot/*:vulkan=True -of build-${{ matrix.variant.name }}
. ./build-${{ matrix.variant.name }}/build/Debug/generators/conanbuild.sh
cmake -S . -B ./build-${{ matrix.variant.name }}/Debug --toolchain ./build-${{ matrix.variant.name }}/build/Debug/generators/conan_toolchain.cmake -DCMAKE_BUILD_TYPE=Debug -G "Ninja"
cmake --build build-${{ matrix.variant.name }}/Debug --parallel $(nproc)
. ./build-${{ matrix.variant.name }}/build/Debug/generators/deactivate_conanbuild.sh
- name: Run tests
run: |
cd ./build-${{ matrix.variant.name }}/Debug
. ../../build-${{ matrix.variant.name }}/build/Debug/generators/conanbuild.sh
LLVM_PROFILE_FILE="%p.profraw%c" xvfb-run ctest --output-on-failure
. ../../build-${{ matrix.variant.name }}/build/Debug/generators/deactivate_conanbuild.sh
conan install . --remote=bigfootpackages -pr:h=./ConanProfiles/host/clang_coverage -pr:b=./ConanProfiles/build/clang --build=* -s build_type=Debug -o bigfoot/*:build_tests=True -o bigfoot/*:build_benchmarks=True -o bigfoot/*:tracy=False -o bigfoot/*:build_tools=True -o bigfoot/*:vulkan=True
. ./build/Debug/generators/conanbuild.sh
cmake -S . -B ./build/Debug --toolchain ./build/Debug/generators/conan_toolchain.cmake -DCMAKE_BUILD_TYPE=Debug -G "Ninja"
cmake --build build/Debug --parallel $(nproc)
. ./build/Debug/generators/deactivate_conanbuild.sh
- name: Clang-Tidy
run: run-clang-tidy -p ./build-${{ matrix.variant.name }}/Debug/ > tidy_result.txt
run: run-clang-tidy -p ./build/Debug/ >> tidy_result.txt
- name: Infer
run: infer run --compilation-database build-${{ matrix.variant.name }}/Debug/compile_commands.json
- name: Generate coverage report
shell: bash
run: |
# gcovr's own LLVM-JSON reader only understands coverage-mapping format
# 2.x; this image's clang 22 llvm-cov emits format 3.x, which gcovr
# rejects outright ("Wrong major version"). Bypassing gcovr's JSON
# path entirely: llvm-cov's lcov exporter is a separate, unaffected
# code path, so merge -> export lcov -> convert lcov to Cobertura.
llvm-profdata merge -sparse $(find build-${{ matrix.variant.name }}/Debug -name "*.profraw") -o merged.profdata
BINARIES=($(find build-${{ matrix.variant.name }}/Debug -maxdepth 8 -type f -executable -iname "*Tests"))
OBJECT_ARGS=$(printf -- '-object %s ' "${BINARIES[@]:1}")
llvm-cov export -format=lcov -instr-profile=merged.profdata "${BINARIES[0]}" $OBJECT_ARGS > coverage.lcov
# lcov_cobertura isn't preinstalled on the image; inject it into
# gcovr's existing pipx venv and expose its CLI on PATH.
pipx inject gcovr lcov_cobertura --include-apps
lcov_cobertura coverage.lcov -b . -o coverage.xml
run: infer run --compilation-database build/Debug/compile_commands.json
- name: SonarQube Scan
if: github.head_ref == 'main' || github.ref_name == 'main'
uses: SonarSource/sonarqube-scan-action@v7.0.0
with:
args: >
-Dsonar.projectKey=${{ matrix.variant.projectKey }}
"-Dsonar.projectName=${{ matrix.variant.projectName }}"
-Dsonar.projectVersion=${{ steps.version.outputs.version }}
-Dsonar.cxx.jsonCompilationDatabase=./build-${{ matrix.variant.name }}/Debug/compile_commands.json
-Dsonar.cxx.xunit.reportPaths=build-${{ matrix.variant.name }}/Debug/TestResults/*.xml
-Dsonar.qualitygate.wait=true
${{ github.event_name == 'pull_request' &&
format('-Dsonar.pullrequest.key={0} -Dsonar.pullrequest.branch={1} -Dsonar.pullrequest.base={2} -Dsonar.analysis.sqbot={3}',
github.event.pull_request.number, github.head_ref, github.base_ref, github.event.pull_request.head.sha)
|| format('-Dsonar.branch.name={0}', github.ref_name) }}
-Dsonar.cxx.jsonCompilationDatabase=./build/Debug/compile_commands.json
-Dsonar.verbose=true
env:
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
SONAR_HOST_URL: ${{ secrets.SONAR_HOST }}
SONAR_HOST_URL: ${{ secrets.SONAR_HOST }}
-5
View File
@@ -1,5 +0,0 @@
InheritParentConfig: true
Checks: >
-cppcoreguidelines-non-private-member-variables-in-classes,
-cppcoreguidelines-avoid-magic-numbers,
-readability-magic-numbers
@@ -1,11 +0,0 @@
include "Engine/Asset/Reference.fbs";
namespace Flat.Bigfoot;
table BenchmarkAsset
{
payload: [ubyte];
refs: [HardReference] (native_inline, bigfoot_ref: "::Bigfoot::BenchmarkAssetReferred");
}
root_type BenchmarkAsset;
@@ -1,25 +0,0 @@
/*********************************************************************
* \file BenchmarkAsset.hpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#ifndef BIGFOOT_ASSETSFORBENCHMARKING_ASSET_BENCHMARKASSET_HPP
#define BIGFOOT_ASSETSFORBENCHMARKING_ASSET_BENCHMARKASSET_HPP
#include <Engine/Asset/Asset.hpp>
#include <AssetsForBenchmarking/Asset/BenchmarkAsset_generated.hpp>
namespace Bigfoot
{
struct BenchmarkAssetTraits
{
using FLAT = ::Flat::Bigfoot::BenchmarkAsset;
};
class BenchmarkAsset: public Asset<BenchmarkAssetTraits>
{
};
} // namespace Bigfoot
#endif
@@ -1,7 +0,0 @@
namespace Flat.Bigfoot;
table BenchmarkAssetReferred
{
}
root_type BenchmarkAssetReferred;
@@ -1,25 +0,0 @@
/*********************************************************************
* \file BenchmarkAssetReferred.hpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#ifndef BIGFOOT_ASSETSFORBENCHMARKING_ASSET_BENCHMARKASSETREFERRED_HPP
#define BIGFOOT_ASSETSFORBENCHMARKING_ASSET_BENCHMARKASSETREFERRED_HPP
#include <Engine/Asset/Asset.hpp>
#include <AssetsForBenchmarking/Asset/BenchmarkAssetReferred_generated.hpp>
namespace Bigfoot
{
struct BenchmarkAssetReferredTraits
{
using FLAT = ::Flat::Bigfoot::BenchmarkAssetReferred;
};
class BenchmarkAssetReferred: public Asset<BenchmarkAssetReferredTraits>
{
};
} // namespace Bigfoot
#endif
@@ -1,172 +0,0 @@
// automatically generated by the FlatBuffers compiler, do not modify
#ifndef FLATBUFFERS_GENERATED_BENCHMARKASSETREFERRED_FLAT_BIGFOOT_H_
#define FLATBUFFERS_GENERATED_BENCHMARKASSETREFERRED_FLAT_BIGFOOT_H_
#include "flatbuffers/flatbuffers.h"
// Ensure the included flatbuffers.h is the same version as when this file was
// generated, otherwise it may not be compatible.
static_assert(FLATBUFFERS_VERSION_MAJOR == 25 &&
FLATBUFFERS_VERSION_MINOR == 12 &&
FLATBUFFERS_VERSION_REVISION == 19,
"Non-compatible flatbuffers version included");
// Ensure the included flatbuffers.h is Bigfoot's fork - stock
// flatbuffers (or a differently versioned Bigfoot fork) is not compatible.
#ifndef FLATBUFFERS_BIGFOOT_VERSION
#error "This file requires Bigfoot's flatbuffers fork - stock flatbuffers is not compatible"
#endif
static_assert(FLATBUFFERS_BIGFOOT_VERSION == 1,
"Non-compatible Bigfoot flatbuffers fork version included");
#include "EASTL/unique_ptr.h"
#include "EASTL/string.h"
#include "EASTL/vector.h"
namespace Flat {
namespace Bigfoot {
struct BenchmarkAssetReferred;
struct BenchmarkAssetReferredBuilder;
struct BenchmarkAssetReferredT;
inline const ::flatbuffers::TypeTable *BenchmarkAssetReferredTypeTable();
struct BenchmarkAssetReferredT : public ::flatbuffers::NativeTable {
typedef BenchmarkAssetReferred TableType;
static FLATBUFFERS_CONSTEXPR_CPP11 const char *GetFullyQualifiedName() {
return "Flat.Bigfoot.BenchmarkAssetReferredT";
}
};
struct BenchmarkAssetReferred FLATBUFFERS_FINAL_CLASS : private ::flatbuffers::Table {
typedef BenchmarkAssetReferredT NativeTableType;
typedef BenchmarkAssetReferredBuilder Builder;
struct Traits;
static const ::flatbuffers::TypeTable *MiniReflectTypeTable() {
return BenchmarkAssetReferredTypeTable();
}
static FLATBUFFERS_CONSTEXPR_CPP11 const char *GetFullyQualifiedName() {
return "Flat.Bigfoot.BenchmarkAssetReferred";
}
template <bool B = false>
bool Verify(::flatbuffers::VerifierTemplate<B> &verifier) const {
return VerifyTableStart(verifier) &&
verifier.EndTable();
}
BenchmarkAssetReferredT *UnPack(const ::flatbuffers::resolver_function_t *_resolver = nullptr) const;
void UnPackTo(BenchmarkAssetReferredT *_o, const ::flatbuffers::resolver_function_t *_resolver = nullptr) const;
static ::flatbuffers::Offset<BenchmarkAssetReferred> Pack(::flatbuffers::FlatBufferBuilder &_fbb, const BenchmarkAssetReferredT* _o, const ::flatbuffers::rehasher_function_t *_rehasher = nullptr);
};
struct BenchmarkAssetReferredBuilder {
typedef BenchmarkAssetReferred Table;
::flatbuffers::FlatBufferBuilder &fbb_;
::flatbuffers::uoffset_t start_;
explicit BenchmarkAssetReferredBuilder(::flatbuffers::FlatBufferBuilder &_fbb)
: fbb_(_fbb) {
start_ = fbb_.StartTable();
}
::flatbuffers::Offset<BenchmarkAssetReferred> Finish() {
const auto end = fbb_.EndTable(start_);
auto o = ::flatbuffers::Offset<BenchmarkAssetReferred>(end);
return o;
}
};
inline ::flatbuffers::Offset<BenchmarkAssetReferred> CreateBenchmarkAssetReferred(
::flatbuffers::FlatBufferBuilder &_fbb) {
BenchmarkAssetReferredBuilder builder_(_fbb);
return builder_.Finish();
}
struct BenchmarkAssetReferred::Traits {
using type = BenchmarkAssetReferred;
static auto constexpr Create = CreateBenchmarkAssetReferred;
};
::flatbuffers::Offset<BenchmarkAssetReferred> CreateBenchmarkAssetReferred(::flatbuffers::FlatBufferBuilder &_fbb, const BenchmarkAssetReferredT *_o, const ::flatbuffers::rehasher_function_t *_rehasher = nullptr);
inline BenchmarkAssetReferredT *BenchmarkAssetReferred::UnPack(const ::flatbuffers::resolver_function_t *_resolver) const {
auto _o = std::make_unique<BenchmarkAssetReferredT>();
UnPackTo(_o.get(), _resolver);
return _o.release();
}
inline void BenchmarkAssetReferred::UnPackTo(BenchmarkAssetReferredT *_o, const ::flatbuffers::resolver_function_t *_resolver) const {
(void)_o;
(void)_resolver;
}
inline ::flatbuffers::Offset<BenchmarkAssetReferred> CreateBenchmarkAssetReferred(::flatbuffers::FlatBufferBuilder &_fbb, const BenchmarkAssetReferredT *_o, const ::flatbuffers::rehasher_function_t *_rehasher) {
return BenchmarkAssetReferred::Pack(_fbb, _o, _rehasher);
}
inline ::flatbuffers::Offset<BenchmarkAssetReferred> BenchmarkAssetReferred::Pack(::flatbuffers::FlatBufferBuilder &_fbb, const BenchmarkAssetReferredT* _o, const ::flatbuffers::rehasher_function_t *_rehasher) {
(void)_rehasher;
(void)_o;
struct _VectorArgs { ::flatbuffers::FlatBufferBuilder *__fbb; const BenchmarkAssetReferredT* __o; const ::flatbuffers::rehasher_function_t *__rehasher; } _va = { &_fbb, _o, _rehasher}; (void)_va;
return Flat::Bigfoot::CreateBenchmarkAssetReferred(
_fbb);
}
inline const ::flatbuffers::TypeTable *BenchmarkAssetReferredTypeTable() {
static const ::flatbuffers::TypeTable tt = {
::flatbuffers::ST_TABLE, 0, nullptr, nullptr, nullptr, nullptr, nullptr
};
return &tt;
}
inline const Flat::Bigfoot::BenchmarkAssetReferred *GetBenchmarkAssetReferred(const void *buf) {
return ::flatbuffers::GetRoot<Flat::Bigfoot::BenchmarkAssetReferred>(buf);
}
inline const Flat::Bigfoot::BenchmarkAssetReferred *GetSizePrefixedBenchmarkAssetReferred(const void *buf) {
return ::flatbuffers::GetSizePrefixedRoot<Flat::Bigfoot::BenchmarkAssetReferred>(buf);
}
template <bool B = false>
inline bool VerifyBenchmarkAssetReferredBuffer(
::flatbuffers::VerifierTemplate<B> &verifier) {
return verifier.template VerifyBuffer<Flat::Bigfoot::BenchmarkAssetReferred>(nullptr);
}
template <bool B = false>
inline bool VerifySizePrefixedBenchmarkAssetReferredBuffer(
::flatbuffers::VerifierTemplate<B> &verifier) {
return verifier.template VerifySizePrefixedBuffer<Flat::Bigfoot::BenchmarkAssetReferred>(nullptr);
}
inline const char *BenchmarkAssetReferredExtension() {
return "bfbs";
}
inline void FinishBenchmarkAssetReferredBuffer(
::flatbuffers::FlatBufferBuilder &fbb,
::flatbuffers::Offset<Flat::Bigfoot::BenchmarkAssetReferred> root) {
fbb.Finish(root);
}
inline void FinishSizePrefixedBenchmarkAssetReferredBuffer(
::flatbuffers::FlatBufferBuilder &fbb,
::flatbuffers::Offset<Flat::Bigfoot::BenchmarkAssetReferred> root) {
fbb.FinishSizePrefixed(root);
}
inline eastl::unique_ptr<Flat::Bigfoot::BenchmarkAssetReferredT> UnPackBenchmarkAssetReferred(
const void *buf,
const ::flatbuffers::resolver_function_t *res = nullptr) {
return eastl::unique_ptr<Flat::Bigfoot::BenchmarkAssetReferredT>(GetBenchmarkAssetReferred(buf)->UnPack(res));
}
inline eastl::unique_ptr<Flat::Bigfoot::BenchmarkAssetReferredT> UnPackSizePrefixedBenchmarkAssetReferred(
const void *buf,
const ::flatbuffers::resolver_function_t *res = nullptr) {
return eastl::unique_ptr<Flat::Bigfoot::BenchmarkAssetReferredT>(GetSizePrefixedBenchmarkAssetReferred(buf)->UnPack(res));
}
} // namespace Bigfoot
} // namespace Flat
#endif // FLATBUFFERS_GENERATED_BENCHMARKASSETREFERRED_FLAT_BIGFOOT_H_
@@ -1,238 +0,0 @@
// automatically generated by the FlatBuffers compiler, do not modify
#ifndef FLATBUFFERS_GENERATED_BENCHMARKASSET_FLAT_BIGFOOT_H_
#define FLATBUFFERS_GENERATED_BENCHMARKASSET_FLAT_BIGFOOT_H_
#include "flatbuffers/flatbuffers.h"
// Ensure the included flatbuffers.h is the same version as when this file was
// generated, otherwise it may not be compatible.
static_assert(FLATBUFFERS_VERSION_MAJOR == 25 &&
FLATBUFFERS_VERSION_MINOR == 12 &&
FLATBUFFERS_VERSION_REVISION == 19,
"Non-compatible flatbuffers version included");
// Ensure the included flatbuffers.h is Bigfoot's fork - stock
// flatbuffers (or a differently versioned Bigfoot fork) is not compatible.
#ifndef FLATBUFFERS_BIGFOOT_VERSION
#error "This file requires Bigfoot's flatbuffers fork - stock flatbuffers is not compatible"
#endif
static_assert(FLATBUFFERS_BIGFOOT_VERSION == 1,
"Non-compatible Bigfoot flatbuffers fork version included");
#include "System/UUID/UUID.hpp"
#include "Engine/Asset/Reference.hpp"
#include "Engine/Asset/Reference_generated.hpp"
#include "EASTL/unique_ptr.h"
#include "EASTL/string.h"
#include "EASTL/vector.h"
namespace Flat {
namespace Bigfoot {
struct BenchmarkAsset;
struct BenchmarkAssetBuilder;
struct BenchmarkAssetT;
} // namespace Bigfoot
} // namespace Flat
namespace Bigfoot {
class BenchmarkAssetReferred;
} // namespace Bigfoot
namespace flatbuffers {
Flat::Bigfoot::HardReference PackHardReferenceBenchmarkAssetReferred(const ::Bigfoot::HardReference<::Bigfoot::BenchmarkAssetReferred>& p_asset);
::Bigfoot::HardReference<::Bigfoot::BenchmarkAssetReferred> UnPackHardReferenceBenchmarkAssetReferred(const Flat::Bigfoot::HardReference& p_asset);
} // namespace flatbuffers
namespace Flat {
namespace Bigfoot {
inline const ::flatbuffers::TypeTable *BenchmarkAssetTypeTable();
struct BenchmarkAssetT : public ::flatbuffers::NativeTable {
typedef BenchmarkAsset TableType;
static FLATBUFFERS_CONSTEXPR_CPP11 const char *GetFullyQualifiedName() {
return "Flat.Bigfoot.BenchmarkAssetT";
}
eastl::vector<uint8_t> payload{};
eastl::vector<::Bigfoot::HardReference<::Bigfoot::BenchmarkAssetReferred>> refs{};
};
struct BenchmarkAsset FLATBUFFERS_FINAL_CLASS : private ::flatbuffers::Table {
typedef BenchmarkAssetT NativeTableType;
typedef BenchmarkAssetBuilder Builder;
struct Traits;
static const ::flatbuffers::TypeTable *MiniReflectTypeTable() {
return BenchmarkAssetTypeTable();
}
static FLATBUFFERS_CONSTEXPR_CPP11 const char *GetFullyQualifiedName() {
return "Flat.Bigfoot.BenchmarkAsset";
}
enum FlatBuffersVTableOffset FLATBUFFERS_VTABLE_UNDERLYING_TYPE {
VT_PAYLOAD = 4,
VT_REFS = 6
};
const ::flatbuffers::Vector<uint8_t> *payload() const {
return GetPointer<const ::flatbuffers::Vector<uint8_t> *>(VT_PAYLOAD);
}
const ::flatbuffers::Vector<const Flat::Bigfoot::HardReference *> *refs() const {
return GetPointer<const ::flatbuffers::Vector<const Flat::Bigfoot::HardReference *> *>(VT_REFS);
}
template <bool B = false>
bool Verify(::flatbuffers::VerifierTemplate<B> &verifier) const {
return VerifyTableStart(verifier) &&
VerifyOffset(verifier, VT_PAYLOAD) &&
verifier.VerifyVector(payload()) &&
VerifyOffset(verifier, VT_REFS) &&
verifier.VerifyVector(refs()) &&
verifier.EndTable();
}
BenchmarkAssetT *UnPack(const ::flatbuffers::resolver_function_t *_resolver = nullptr) const;
void UnPackTo(BenchmarkAssetT *_o, const ::flatbuffers::resolver_function_t *_resolver = nullptr) const;
static ::flatbuffers::Offset<BenchmarkAsset> Pack(::flatbuffers::FlatBufferBuilder &_fbb, const BenchmarkAssetT* _o, const ::flatbuffers::rehasher_function_t *_rehasher = nullptr);
};
struct BenchmarkAssetBuilder {
typedef BenchmarkAsset Table;
::flatbuffers::FlatBufferBuilder &fbb_;
::flatbuffers::uoffset_t start_;
void add_payload(::flatbuffers::Offset<::flatbuffers::Vector<uint8_t>> payload) {
fbb_.AddOffset(BenchmarkAsset::VT_PAYLOAD, payload);
}
void add_refs(::flatbuffers::Offset<::flatbuffers::Vector<const Flat::Bigfoot::HardReference *>> refs) {
fbb_.AddOffset(BenchmarkAsset::VT_REFS, refs);
}
explicit BenchmarkAssetBuilder(::flatbuffers::FlatBufferBuilder &_fbb)
: fbb_(_fbb) {
start_ = fbb_.StartTable();
}
::flatbuffers::Offset<BenchmarkAsset> Finish() {
const auto end = fbb_.EndTable(start_);
auto o = ::flatbuffers::Offset<BenchmarkAsset>(end);
return o;
}
};
inline ::flatbuffers::Offset<BenchmarkAsset> CreateBenchmarkAsset(
::flatbuffers::FlatBufferBuilder &_fbb,
::flatbuffers::Offset<::flatbuffers::Vector<uint8_t>> payload = 0,
::flatbuffers::Offset<::flatbuffers::Vector<const Flat::Bigfoot::HardReference *>> refs = 0) {
BenchmarkAssetBuilder builder_(_fbb);
builder_.add_refs(refs);
builder_.add_payload(payload);
return builder_.Finish();
}
struct BenchmarkAsset::Traits {
using type = BenchmarkAsset;
static auto constexpr Create = CreateBenchmarkAsset;
};
::flatbuffers::Offset<BenchmarkAsset> CreateBenchmarkAsset(::flatbuffers::FlatBufferBuilder &_fbb, const BenchmarkAssetT *_o, const ::flatbuffers::rehasher_function_t *_rehasher = nullptr);
inline BenchmarkAssetT *BenchmarkAsset::UnPack(const ::flatbuffers::resolver_function_t *_resolver) const {
auto _o = std::make_unique<BenchmarkAssetT>();
UnPackTo(_o.get(), _resolver);
return _o.release();
}
inline void BenchmarkAsset::UnPackTo(BenchmarkAssetT *_o, const ::flatbuffers::resolver_function_t *_resolver) const {
(void)_o;
(void)_resolver;
{ auto _e = payload(); if (_e) { _o->payload.resize(_e->size()); std::copy(_e->begin(), _e->end(), _o->payload.begin()); } }
{ auto _e = refs(); if (_e) { _o->refs.resize(_e->size()); for (::flatbuffers::uoffset_t _i = 0; _i < _e->size(); _i++) { _o->refs[_i] = ::flatbuffers::UnPackHardReferenceBenchmarkAssetReferred(*_e->Get(_i)); } } else { _o->refs.resize(0); } }
}
inline ::flatbuffers::Offset<BenchmarkAsset> CreateBenchmarkAsset(::flatbuffers::FlatBufferBuilder &_fbb, const BenchmarkAssetT *_o, const ::flatbuffers::rehasher_function_t *_rehasher) {
return BenchmarkAsset::Pack(_fbb, _o, _rehasher);
}
inline ::flatbuffers::Offset<BenchmarkAsset> BenchmarkAsset::Pack(::flatbuffers::FlatBufferBuilder &_fbb, const BenchmarkAssetT* _o, const ::flatbuffers::rehasher_function_t *_rehasher) {
(void)_rehasher;
(void)_o;
struct _VectorArgs { ::flatbuffers::FlatBufferBuilder *__fbb; const BenchmarkAssetT* __o; const ::flatbuffers::rehasher_function_t *__rehasher; } _va = { &_fbb, _o, _rehasher}; (void)_va;
auto _payload = _fbb.CreateVector(_o->payload.data(), _o->payload.size());
auto _refs = _fbb.CreateVectorOfNativeStructs<Flat::Bigfoot::HardReference, ::Bigfoot::HardReference<::Bigfoot::BenchmarkAssetReferred>>(_o->refs.data(), _o->refs.size(), ::flatbuffers::PackHardReferenceBenchmarkAssetReferred);
return Flat::Bigfoot::CreateBenchmarkAsset(
_fbb,
_payload,
_refs);
}
inline const ::flatbuffers::TypeTable *BenchmarkAssetTypeTable() {
static const ::flatbuffers::TypeCode type_codes[] = {
{ ::flatbuffers::ET_UCHAR, 1, -1 },
{ ::flatbuffers::ET_SEQUENCE, 1, 0 }
};
static const ::flatbuffers::TypeFunction type_refs[] = {
Flat::Bigfoot::HardReferenceTypeTable
};
static const ::flatbuffers::TypeTable tt = {
::flatbuffers::ST_TABLE, 2, type_codes, type_refs, nullptr, nullptr, nullptr
};
return &tt;
}
inline const Flat::Bigfoot::BenchmarkAsset *GetBenchmarkAsset(const void *buf) {
return ::flatbuffers::GetRoot<Flat::Bigfoot::BenchmarkAsset>(buf);
}
inline const Flat::Bigfoot::BenchmarkAsset *GetSizePrefixedBenchmarkAsset(const void *buf) {
return ::flatbuffers::GetSizePrefixedRoot<Flat::Bigfoot::BenchmarkAsset>(buf);
}
template <bool B = false>
inline bool VerifyBenchmarkAssetBuffer(
::flatbuffers::VerifierTemplate<B> &verifier) {
return verifier.template VerifyBuffer<Flat::Bigfoot::BenchmarkAsset>(nullptr);
}
template <bool B = false>
inline bool VerifySizePrefixedBenchmarkAssetBuffer(
::flatbuffers::VerifierTemplate<B> &verifier) {
return verifier.template VerifySizePrefixedBuffer<Flat::Bigfoot::BenchmarkAsset>(nullptr);
}
inline const char *BenchmarkAssetExtension() {
return "bfbs";
}
inline void FinishBenchmarkAssetBuffer(
::flatbuffers::FlatBufferBuilder &fbb,
::flatbuffers::Offset<Flat::Bigfoot::BenchmarkAsset> root) {
fbb.Finish(root);
}
inline void FinishSizePrefixedBenchmarkAssetBuffer(
::flatbuffers::FlatBufferBuilder &fbb,
::flatbuffers::Offset<Flat::Bigfoot::BenchmarkAsset> root) {
fbb.FinishSizePrefixed(root);
}
inline eastl::unique_ptr<Flat::Bigfoot::BenchmarkAssetT> UnPackBenchmarkAsset(
const void *buf,
const ::flatbuffers::resolver_function_t *res = nullptr) {
return eastl::unique_ptr<Flat::Bigfoot::BenchmarkAssetT>(GetBenchmarkAsset(buf)->UnPack(res));
}
inline eastl::unique_ptr<Flat::Bigfoot::BenchmarkAssetT> UnPackSizePrefixedBenchmarkAsset(
const void *buf,
const ::flatbuffers::resolver_function_t *res = nullptr) {
return eastl::unique_ptr<Flat::Bigfoot::BenchmarkAssetT>(GetSizePrefixedBenchmarkAsset(buf)->UnPack(res));
}
} // namespace Bigfoot
} // namespace Flat
namespace flatbuffers {
#ifndef FLATBUFFERS_BIGFOOT_REF_HARDREFERENCEBENCHMARKASSETREFERRED
#define FLATBUFFERS_BIGFOOT_REF_HARDREFERENCEBENCHMARKASSETREFERRED
inline Flat::Bigfoot::HardReference PackHardReferenceBenchmarkAssetReferred(const ::Bigfoot::HardReference<::Bigfoot::BenchmarkAssetReferred>& p_asset) { return {Pack(p_asset.GetUUID())}; }
inline ::Bigfoot::HardReference<::Bigfoot::BenchmarkAssetReferred> UnPackHardReferenceBenchmarkAssetReferred(const Flat::Bigfoot::HardReference& p_asset) { return {UnPack(p_asset.uuid())}; }
#endif // FLATBUFFERS_BIGFOOT_REF_HARDREFERENCEBENCHMARKASSETREFERRED
} // namespace flatbuffers
#endif // FLATBUFFERS_GENERATED_BENCHMARKASSET_FLAT_BIGFOOT_H_
+2 -6
View File
@@ -1,7 +1,3 @@
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/AssetsForBenchmarking)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/Engine)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/Utils)
if(${TOOLMODE})
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/Tools)
endif()
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/Misc)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/Engine)
@@ -1,175 +0,0 @@
/*********************************************************************
* \file AssetContainer.cpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#include <Engine/Asset/AssetContainer.hpp>
#include <AssetsForBenchmarking/Asset/BenchmarkAsset.hpp>
#include <benchmark/benchmark.h>
#include <effolkronium/random.hpp>
#include <cstdint>
namespace Bigfoot
{
namespace
{
using Random = effolkronium::random_static;
} // namespace
class AssetContainerFixture: public benchmark::Fixture
{
protected:
[[nodiscard]]
static eastl::vector<UUID> InsertRandomAssets(AssetContainer<BenchmarkAsset>& p_container,
const std::uint32_t p_count)
{
eastl::vector<UUID> uuids;
uuids.reserve(p_count);
for (std::uint32_t i = 0; i < p_count; ++i)
{
uuids.push_back(p_container.Insert()->GetAsset()->GetHeader().uuid);
}
Random::shuffle(uuids);
return uuids;
}
AssetContainer<BenchmarkAsset> m_container;
};
/****************************************************************************************/
BENCHMARK_DEFINE_F(AssetContainerFixture, Insert)(benchmark::State& state)
{
const auto count = static_cast<std::uint32_t>(state.range(0));
for (auto _: state)
{
AssetContainer<BenchmarkAsset> container;
for (std::uint32_t i = 0; i < count; ++i)
{
benchmark::DoNotOptimize(container.Insert());
}
benchmark::DoNotOptimize(container);
benchmark::ClobberMemory();
}
state.SetItemsProcessed(state.iterations() * count);
state.SetBytesProcessed(static_cast<std::int64_t>(state.iterations() * count * sizeof(BenchmarkAsset)));
}
BENCHMARK_REGISTER_F(AssetContainerFixture, Insert)->Name("AssetContainer/Insert")->Arg(512)->Arg(8192)->Arg(131'072);
/****************************************************************************************/
BENCHMARK_DEFINE_F(AssetContainerFixture, Get)(benchmark::State& state)
{
const auto count = static_cast<std::uint32_t>(state.range(0));
const eastl::vector<UUID> uuids = InsertRandomAssets(m_container, count);
eastl::vector<ContainerHandle*> handles;
handles.reserve(count);
for (const UUID& uuid: uuids)
{
handles.push_back(m_container.AcquireContainerHandle(uuid));
}
for (auto _: state)
{
std::uint64_t sum = 0;
for (ContainerHandle* const handle: handles)
{
if (const auto* const asset = static_cast<const BenchmarkAsset*>(handle->GetAsset()))
{
sum += asset->GetHeader().type_id;
}
}
benchmark::DoNotOptimize(sum);
}
state.SetItemsProcessed(state.iterations() * count);
}
BENCHMARK_REGISTER_F(AssetContainerFixture, Get)->Name("AssetContainer/Get")->Arg(512)->Arg(8192)->Arg(131'072);
/****************************************************************************************/
BENCHMARK_DEFINE_F(AssetContainerFixture, Has)(benchmark::State& state)
{
const auto count = static_cast<std::uint32_t>(state.range(0));
const eastl::vector<UUID> uuids = InsertRandomAssets(m_container, count);
for (auto _: state)
{
bool result = true;
for (const UUID& uuid: uuids)
{
result = result && m_container.Has(uuid);
}
benchmark::DoNotOptimize(result);
}
state.SetItemsProcessed(state.iterations() * count);
}
BENCHMARK_REGISTER_F(AssetContainerFixture, Has)->Name("AssetContainer/Has")->Arg(512)->Arg(8192)->Arg(131'072);
/****************************************************************************************/
BENCHMARK_DEFINE_F(AssetContainerFixture, AcquireContainerHandle)(benchmark::State& state)
{
const auto count = static_cast<std::uint32_t>(state.range(0));
const eastl::vector<UUID> uuids = InsertRandomAssets(m_container, count);
for (auto _: state)
{
std::uint64_t sum = 0;
for (const UUID& uuid: uuids)
{
sum += m_container.AcquireContainerHandle(uuid)->GetHardRefCount();
}
benchmark::DoNotOptimize(sum);
}
state.SetItemsProcessed(state.iterations() * count);
}
BENCHMARK_REGISTER_F(AssetContainerFixture, AcquireContainerHandle)
->Name("AssetContainer/AcquireContainerHandle")
->Arg(512)
->Arg(8192)
->Arg(131'072);
/****************************************************************************************/
BENCHMARK_DEFINE_F(AssetContainerFixture, Remove)(benchmark::State& state)
{
const auto count = static_cast<std::uint32_t>(state.range(0));
eastl::vector<UUID> uuids;
for (auto _: state)
{
state.PauseTiming();
uuids = InsertRandomAssets(m_container, count);
state.ResumeTiming();
for (const UUID& uuid: uuids)
{
m_container.Remove(uuid);
}
}
state.SetItemsProcessed(state.iterations() * count);
state.SetBytesProcessed(static_cast<std::int64_t>(state.iterations() * count * sizeof(BenchmarkAsset)));
}
BENCHMARK_REGISTER_F(AssetContainerFixture, Remove)->Name("AssetContainer/Remove")->Arg(512)->Arg(8192)->Arg(131'072);
} // namespace Bigfoot
@@ -1,187 +0,0 @@
/*********************************************************************
* \file Reference.cpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#include <Engine/Asset/Reference.hpp>
#include <Engine/Asset/AssetLibrary.hpp>
#include <AssetsForBenchmarking/Asset/BenchmarkAsset.hpp>
#include <EASTL/optional.h>
#include <EASTL/vector.h>
#include <benchmark/benchmark.h>
#include <cstddef>
#include <utility>
namespace Bigfoot
{
template<template<class> class REFERENCE>
class ReferenceFixture: public benchmark::Fixture
{
protected:
using ReferenceType = REFERENCE<BenchmarkAsset>;
void SetUp(benchmark::State&) override
{
SetActiveAssetLibrary(&m_library);
std::ignore = m_library.Insert(&m_container);
m_uuid = CreateAsset<BenchmarkAsset>()->GetHeader().uuid;
m_reference.emplace(m_uuid);
}
void TearDown(benchmark::State&) override
{
SetActiveAssetLibrary(&g_assetLibrary);
}
AssetLibrary m_library;
AssetContainer<BenchmarkAsset> m_container;
UUID m_uuid;
eastl::optional<ReferenceType> m_reference;
};
/****************************************************************************************/
BENCHMARK_TEMPLATE_DEFINE_F(ReferenceFixture, HardReferenceConstruct, HardReference)(benchmark::State& state)
{
for (auto _: state)
{
ReferenceType reference {m_uuid};
benchmark::DoNotOptimize(reference);
}
state.SetItemsProcessed(state.iterations());
}
BENCHMARK_REGISTER_F(ReferenceFixture, HardReferenceConstruct)->Name("Reference/HardReference/Construct");
BENCHMARK_TEMPLATE_DEFINE_F(ReferenceFixture, SoftReferenceConstruct, SoftReference)(benchmark::State& state)
{
for (auto _: state)
{
ReferenceType reference {m_uuid};
benchmark::DoNotOptimize(reference);
}
state.SetItemsProcessed(state.iterations());
}
BENCHMARK_REGISTER_F(ReferenceFixture, SoftReferenceConstruct)->Name("Reference/SoftReference/Construct");
/****************************************************************************************/
BENCHMARK_TEMPLATE_DEFINE_F(ReferenceFixture, HardReferenceGet, HardReference)(benchmark::State& state)
{
for (auto _: state)
{
benchmark::DoNotOptimize(m_reference->Get());
}
state.SetItemsProcessed(state.iterations());
}
BENCHMARK_REGISTER_F(ReferenceFixture, HardReferenceGet)->Name("Reference/HardReference/Get");
BENCHMARK_TEMPLATE_DEFINE_F(ReferenceFixture, SoftReferenceGet, SoftReference)(benchmark::State& state)
{
for (auto _: state)
{
benchmark::DoNotOptimize(m_reference->Get());
}
state.SetItemsProcessed(state.iterations());
}
BENCHMARK_REGISTER_F(ReferenceFixture, SoftReferenceGet)->Name("Reference/SoftReference/Get");
/****************************************************************************************/
BENCHMARK_TEMPLATE_DEFINE_F(ReferenceFixture, HardReferenceCopy, HardReference)(benchmark::State& state)
{
for (auto _: state)
{
ReferenceType copy {*m_reference};
benchmark::DoNotOptimize(copy);
}
state.SetItemsProcessed(state.iterations());
}
BENCHMARK_REGISTER_F(ReferenceFixture, HardReferenceCopy)->Name("Reference/HardReference/Copy");
BENCHMARK_TEMPLATE_DEFINE_F(ReferenceFixture, SoftReferenceCopy, SoftReference)(benchmark::State& state)
{
for (auto _: state)
{
ReferenceType copy {*m_reference};
benchmark::DoNotOptimize(copy);
}
state.SetItemsProcessed(state.iterations());
}
BENCHMARK_REGISTER_F(ReferenceFixture, SoftReferenceCopy)->Name("Reference/SoftReference/Copy");
/****************************************************************************************/
BENCHMARK_TEMPLATE_DEFINE_F(ReferenceFixture, HardReferenceMove, HardReference)(benchmark::State& state)
{
constexpr std::size_t BATCH_SIZE = 128;
eastl::vector<ReferenceType> batch;
batch.reserve(BATCH_SIZE);
for (auto _: state)
{
state.PauseTiming();
batch.clear();
for (std::size_t i = 0; i < BATCH_SIZE; ++i)
{
batch.emplace_back(m_uuid);
}
state.ResumeTiming();
for (ReferenceType& reference: batch)
{
ReferenceType moved {std::move(reference)};
benchmark::DoNotOptimize(moved);
}
}
state.SetItemsProcessed(state.iterations() * BATCH_SIZE);
}
BENCHMARK_REGISTER_F(ReferenceFixture, HardReferenceMove)->Name("Reference/HardReference/Move");
BENCHMARK_TEMPLATE_DEFINE_F(ReferenceFixture, SoftReferenceMove, SoftReference)(benchmark::State& state)
{
constexpr std::size_t BATCH_SIZE = 128;
eastl::vector<ReferenceType> batch;
batch.reserve(BATCH_SIZE);
for (auto _: state)
{
state.PauseTiming();
batch.clear();
for (std::size_t i = 0; i < BATCH_SIZE; ++i)
{
batch.emplace_back(m_uuid);
}
state.ResumeTiming();
for (ReferenceType& reference: batch)
{
ReferenceType moved {std::move(reference)};
benchmark::DoNotOptimize(moved);
}
}
state.SetItemsProcessed(state.iterations() * BATCH_SIZE);
}
BENCHMARK_REGISTER_F(ReferenceFixture, SoftReferenceMove)->Name("Reference/SoftReference/Move");
} // namespace Bigfoot
@@ -1,312 +0,0 @@
/*********************************************************************
* \file BigFile.cpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#include <Engine/BigFile/BigFile.hpp>
#include <System/File/File.hpp>
#include <System/UUID/UUID.hpp>
#include <EngineBenchmarks/BigFileInfo_generated.hpp>
#include <AssetsForBenchmarking/Asset/BenchmarkAsset.hpp>
#include <AssetsForBenchmarking/Asset/BenchmarkAssetReferred.hpp>
#include <EASTL/optional.h>
#include <EASTL/span.h>
#include <EASTL/vector.h>
#include <benchmark/benchmark.h>
#include <effolkronium/random.hpp>
#include <cstddef>
#include <cstdint>
namespace Bigfoot
{
#ifdef BIGFOOT_TOOLMODE
using Random = effolkronium::random_static;
class BigFileFixture: public benchmark::Fixture
{
protected:
static constexpr std::uint32_t ASSET_COUNT = 20'000;
static constexpr std::uint32_t REFS_PER_ASSET = 4;
static constexpr std::size_t ASSET_PAYLOAD_SIZE = 256;
void SetUp([[maybe_unused]] benchmark::State& p_state) override
{
// The DB is populated exactly once for the entire benchmark process.
// SetUp is called before every calibration round (Google Benchmark doubles
// iteration counts until it reaches a stable measurement), so rebuilding
// the DB each time would dominate the total benchmark time.
m_uuids = BuildData();
m_bigFile.emplace(File {BIGFILE_ENGINEBENCHMARKS_LOCATION});
}
void TearDown([[maybe_unused]] benchmark::State& p_state) override
{
m_bigFile.reset();
}
[[nodiscard]]
eastl::vector<UUID> RandomBatch(const std::size_t p_batchSize) const
{
eastl::vector<UUID> batch;
batch.reserve(p_batchSize);
for (std::size_t i = 0; i < p_batchSize; ++i)
{
batch.push_back(m_uuids[Random::get<std::size_t>(0, m_uuids.size() - 1)]);
}
return batch;
}
[[nodiscard]]
eastl::vector<eastl::vector<UUID>> RandomBatchPool(const std::size_t p_batchSize,
const std::size_t p_poolSize) const
{
eastl::vector<eastl::vector<UUID>> pool;
pool.reserve(p_poolSize);
for (std::size_t i = 0; i < p_poolSize; ++i)
{
pool.push_back(RandomBatch(p_batchSize));
}
return pool;
}
eastl::optional<BigFile> m_bigFile;
eastl::span<const UUID> m_uuids;
private:
static const eastl::vector<UUID>& BuildData()
{
static const eastl::vector<UUID> s_uuids = []()
{
eastl::vector<UUID> uuids;
eastl::vector<eastl::vector<std::byte>> flatbuffers;
eastl::vector<eastl::vector<UUID>> hardReferences;
uuids.reserve(ASSET_COUNT);
flatbuffers.reserve(ASSET_COUNT);
hardReferences.reserve(ASSET_COUNT);
for (std::uint32_t i = 0; i < ASSET_COUNT; ++i)
{
BenchmarkAsset asset;
asset.SetName(asset.GetHeader().uuid.ToString());
asset.GetAsset().payload.assign(ASSET_PAYLOAD_SIZE, std::uint8_t {0xAB});
for (std::uint32_t j = 0; j < REFS_PER_ASSET; ++j)
{
BenchmarkAssetReferred assetReferred;
assetReferred.SetName(assetReferred.GetHeader().uuid.ToString());
asset.GetAsset().refs.emplace_back(assetReferred.GetHeader().uuid);
}
eastl::vector<std::byte> flatbuffer = asset.Pack();
hardReferences.push_back(asset.GetHardReferences(flatbuffer));
flatbuffers.push_back(std::move(flatbuffer));
uuids.push_back(asset.GetHeader().uuid);
}
BigFile bigFile {File {BIGFILE_ENGINEBENCHMARKS_LOCATION}};
bigFile.Clear();
bigFile.SaveAssetFlatbuffers(uuids, flatbuffers, hardReferences);
return uuids;
}();
return s_uuids;
}
};
/****************************************************************************************/
BENCHMARK_DEFINE_F(BigFileFixture, GetAssetFlatbuffer)(benchmark::State& state)
{
std::int64_t bytesProcessed = 0;
for (auto _: state)
{
eastl::vector<std::byte> flatbuffer = m_bigFile->GetAssetFlatbuffer(
m_uuids[Random::get<std::size_t>(0, m_uuids.size() - 1)]);
benchmark::DoNotOptimize(flatbuffer);
bytesProcessed += static_cast<std::int64_t>(flatbuffer.size());
}
state.SetItemsProcessed(state.iterations());
state.SetBytesProcessed(bytesProcessed);
}
BENCHMARK_REGISTER_F(BigFileFixture, GetAssetFlatbuffer)->Name("BigFile/GetAssetFlatbuffer")->UseRealTime();
/****************************************************************************************/
BENCHMARK_DEFINE_F(BigFileFixture, GetAssetFlatbuffers)(benchmark::State& state)
{
const auto batchSize = static_cast<std::size_t>(state.range(0));
constexpr std::size_t BATCH_POOL_SIZE = 64;
const eastl::vector<eastl::vector<UUID>> batchPool = RandomBatchPool(batchSize, BATCH_POOL_SIZE);
std::int64_t bytesProcessed = 0;
std::size_t batchIndex = 0;
for (auto _: state)
{
eastl::vector<eastl::vector<std::byte>> flatbuffers = m_bigFile->GetAssetFlatbuffers(batchPool[batchIndex]);
benchmark::DoNotOptimize(flatbuffers);
for (const eastl::vector<std::byte>& flatbuffer: flatbuffers)
{
bytesProcessed += static_cast<std::int64_t>(flatbuffer.size());
}
batchIndex = (batchIndex + 1) % BATCH_POOL_SIZE;
}
state.SetItemsProcessed(state.iterations() * batchSize);
state.SetBytesProcessed(bytesProcessed);
}
BENCHMARK_REGISTER_F(BigFileFixture, GetAssetFlatbuffers)
->Name("BigFile/GetAssetFlatbuffers")
->Arg(1)
->Arg(8)
->Arg(64)
->Arg(512)
->Arg(4096)
->UseRealTime();
/****************************************************************************************/
BENCHMARK_DEFINE_F(BigFileFixture, GetAssetHardReferences)(benchmark::State& state)
{
for (auto _: state)
{
benchmark::DoNotOptimize(
m_bigFile->GetAssetHardReferences(m_uuids[Random::get<std::size_t>(0, m_uuids.size() - 1)]));
}
state.SetItemsProcessed(state.iterations());
state.SetBytesProcessed(static_cast<std::int64_t>(state.iterations()) * REFS_PER_ASSET * UUID::UUID_BYTE_SIZE);
}
BENCHMARK_REGISTER_F(BigFileFixture, GetAssetHardReferences)->Name("BigFile/GetAssetHardReferences")->UseRealTime();
/****************************************************************************************/
BENCHMARK_DEFINE_F(BigFileFixture, GetAssetsHardReferences)(benchmark::State& state)
{
const auto batchSize = static_cast<std::size_t>(state.range(0));
constexpr std::size_t BATCH_POOL_SIZE = 64;
const eastl::vector<eastl::vector<UUID>> batchPool = RandomBatchPool(batchSize, BATCH_POOL_SIZE);
std::size_t batchIndex = 0;
for (auto _: state)
{
benchmark::DoNotOptimize(m_bigFile->GetAssetsHardReferences(batchPool[batchIndex]));
batchIndex = (batchIndex + 1) % BATCH_POOL_SIZE;
}
state.SetItemsProcessed(state.iterations() * batchSize);
state.SetBytesProcessed(static_cast<std::int64_t>(state.iterations()) * batchSize * REFS_PER_ASSET *
UUID::UUID_BYTE_SIZE);
}
BENCHMARK_REGISTER_F(BigFileFixture, GetAssetsHardReferences)
->Name("BigFile/GetAssetsHardReferences")
->Arg(1)
->Arg(8)
->Arg(64)
->Arg(512)
->Arg(4096)
->UseRealTime();
/****************************************************************************************/
BENCHMARK_DEFINE_F(BigFileFixture, SaveAssetFlatbuffer)(benchmark::State& state)
{
std::int64_t bytesProcessed = 0;
for (auto _: state)
{
state.PauseTiming();
BenchmarkAsset asset;
asset.SetName(asset.GetHeader().uuid.ToString());
asset.GetAsset().payload.assign(ASSET_PAYLOAD_SIZE, std::uint8_t {0xAB});
for (std::uint32_t j = 0; j < REFS_PER_ASSET; ++j)
{
asset.GetAsset().refs.emplace_back(m_uuids[Random::get<std::size_t>(0, m_uuids.size() - 1)]);
}
const eastl::vector<std::byte> flatbuffer = asset.Pack();
const eastl::vector<UUID> hardReferences = asset.GetHardReferences(flatbuffer);
bytesProcessed += static_cast<std::int64_t>(flatbuffer.size());
state.ResumeTiming();
m_bigFile->SaveAssetFlatbuffer(asset.GetHeader().uuid, flatbuffer, hardReferences);
}
state.SetItemsProcessed(state.iterations());
state.SetBytesProcessed(bytesProcessed);
}
BENCHMARK_REGISTER_F(BigFileFixture, SaveAssetFlatbuffer)->Name("BigFile/SaveAssetFlatbuffer")->UseRealTime();
/****************************************************************************************/
BENCHMARK_DEFINE_F(BigFileFixture, SaveAssetFlatbuffers)(benchmark::State& state)
{
const auto batchSize = static_cast<std::size_t>(state.range(0));
std::int64_t bytesProcessed = 0;
for (auto _: state)
{
state.PauseTiming();
eastl::vector<UUID> uuids;
eastl::vector<eastl::vector<std::byte>> flatbuffers;
eastl::vector<eastl::vector<UUID>> hardReferences;
uuids.reserve(batchSize);
flatbuffers.reserve(batchSize);
hardReferences.reserve(batchSize);
for (std::size_t i = 0; i < batchSize; ++i)
{
BenchmarkAsset asset;
asset.SetName(asset.GetHeader().uuid.ToString());
asset.GetAsset().payload.assign(ASSET_PAYLOAD_SIZE, std::uint8_t {0xAB});
for (std::uint32_t j = 0; j < REFS_PER_ASSET; ++j)
{
asset.GetAsset().refs.emplace_back(m_uuids[Random::get<std::size_t>(0, m_uuids.size() - 1)]);
}
eastl::vector<std::byte> flatbuffer = asset.Pack();
uuids.push_back(asset.GetHeader().uuid);
bytesProcessed += static_cast<std::int64_t>(flatbuffer.size());
hardReferences.push_back(asset.GetHardReferences(flatbuffer));
flatbuffers.push_back(std::move(flatbuffer));
}
state.ResumeTiming();
m_bigFile->SaveAssetFlatbuffers(uuids, flatbuffers, hardReferences);
}
state.SetItemsProcessed(state.iterations() * batchSize);
state.SetBytesProcessed(bytesProcessed);
}
BENCHMARK_REGISTER_F(BigFileFixture, SaveAssetFlatbuffers)
->Name("BigFile/SaveAssetFlatbuffers")
->Arg(1)
->Arg(8)
->Arg(64)
->Arg(512)
->Arg(4096)
->UseRealTime();
#endif
} // namespace Bigfoot
@@ -0,0 +1,465 @@
/*********************************************************************
* \file Sasquatch.cpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#include <AssetsForBenchmarking/Asset/BenchAssetLeaf.hpp>
#include <AssetsForBenchmarking/Asset/BenchAssetRoot.hpp>
#include <Engine/Asset/AssetContainer.hpp>
#include <Engine/Asset/AssetLibrary.hpp>
#include <Engine/Asset/Reference.hpp>
#include <Engine/Asset/SerializationManager.hpp>
#include <Engine/BigFile/BigFile.hpp>
#include <Engine/Sasquatch/Sasquatch.hpp>
#include <EngineBenchmarks/BigFileInfo_generated.hpp>
#include <System/File.hpp>
#include <System/UUID/UUID.hpp>
#include <EASTL/span.h>
#include <EASTL/string.h>
#include <EASTL/vector.h>
#include <benchmark/benchmark.h>
#include <algorithm>
#include <cstdint>
#include <cstring>
#include <filesystem>
#include <fstream>
#include <optional>
#include <random>
#include <string>
#include <tuple>
#include <vector>
// Tool-mode only: populates via SerializationManager::SaveAsset, which only exists in Tool mode (see
// SerializationManager.hpp). Building EngineBenchmarks against BIGFOOT_GAME (see
// SasquatchLoadAssets.cpp, in this same directory) compiles this file down to nothing.
#if defined BIGFOOT_TOOL
// Compares the production BigFile/SQLite path against the production Sasquatch path, loading real
// Asset<T> objects (see AssetsForBenchmarking) through the real AssetContainer/SerializationManager
// pipeline - Pack()/Unpack(), HardReference resolution and all - at asset sizes big enough that per-asset
// fixed overhead (hashing, SQL parsing, small allocations) stops dominating and storage throughput does.
// Sasquatch_RawRead skips the AssetContainer/flatbuffers layer entirely to show the ceiling that layer
// costs against - see the class comment on SerializationManager for why Sasquatch has no BigFile-style
// prepared-statement-reuse concern (it's a single mmap, not a series of SQL round-trips).
namespace Bigfoot
{
namespace
{
[[nodiscard]]
eastl::string UniqueName(const UUID& p_uuid)
{
const std::string uuidString = p_uuid.ToString();
return eastl::string {uuidString.data(), uuidString.size()};
}
void ClearBigFile(BigFile& p_bigFile)
{
// Cascades to Asset/AssetHardReference/AssetSoftReference (see BigFileSchema.sql's ON DELETE CASCADE):
// the fixture BigFile is shared and persistent across every Arg() this benchmark runs with, so each
// run clears out whatever the previous one left behind. VACUUM reclaims the freed pages: without it,
// repeated DELETE+repopulate cycles across every Arg()/repetition only ever grow the file, and query
// planning cost on a several-hundred-MB file swamps everything else being measured.
BigFile::Request deleteRequest {p_bigFile, "DELETE FROM AssetHeader;"};
std::ignore = deleteRequest.Execute();
BigFile::Request vacuumRequest {p_bigFile, "VACUUM;"};
std::ignore = vacuumRequest.Execute();
}
[[nodiscard]]
std::filesystem::path TempPath(const std::string_view p_name)
{
return std::filesystem::temp_directory_path() / p_name;
}
[[nodiscard]]
File ToFile(const std::filesystem::path& p_path)
{
const std::string pathString = p_path.string();
return File {eastl::string_view {pathString.c_str(), pathString.size()}};
}
// Sums every 8 bytes of the payload, so the loop can't be optimized away and every byte actually gets
// faulted in/read - a stride of 8 rather than 1 so the reduction itself stays cheap enough not to become
// the bottleneck we're trying to measure around it.
[[nodiscard]]
std::uint64_t Touch(const eastl::span<const std::byte> p_bytes)
{
std::uint64_t sum = 0;
std::size_t i = 0;
for (; i + sizeof(std::uint64_t) <= p_bytes.size(); i += sizeof(std::uint64_t))
{
std::uint64_t chunk;
std::memcpy(&chunk, p_bytes.data() + i, sizeof(chunk));
sum += chunk;
}
for (; i < p_bytes.size(); ++i)
{
sum += static_cast<std::uint8_t>(p_bytes[i]);
}
return sum;
}
[[nodiscard]]
std::uint64_t Touch(const eastl::vector<std::uint8_t>& p_payload)
{
return Touch(eastl::span<const std::byte> {reinterpret_cast<const std::byte*>(p_payload.data()),
p_payload.size()});
}
// Builds p_count BenchAssetRoot/BenchAssetLeaf pairs, each leaf carrying a p_payloadSize-byte payload, and
// saves each one via SerializationManager::SaveAsset - the real Pack()/BigFile write path - then unloads
// them (SaveAndUnload, same as Tests/Engine/Asset/SerializationManager.cpp's fixture) so the benchmarked
// LoadAsset calls have real work to do instead of trivially finding everything already Has()-true.
void PopulateAssets(SerializationManager& p_manager,
AssetContainer<BenchAssetLeaf>& p_leafContainer,
AssetContainer<BenchAssetRoot>& p_rootContainer,
const std::uint32_t p_count,
const std::size_t p_payloadSize,
std::mt19937_64& p_rng,
std::vector<UUID>& p_outRootUUIDs,
std::vector<UUID>& p_outLeafUUIDs)
{
std::uniform_int_distribution<int> byteDist {0, 255};
p_outRootUUIDs.reserve(p_count);
p_outLeafUUIDs.reserve(p_count);
for (std::uint32_t i = 0; i < p_count; ++i)
{
BenchAssetLeaf leaf;
leaf.SetName(UniqueName(leaf.GetHeader().uuid));
leaf.GetAsset().payload.resize(p_payloadSize);
for (std::uint8_t& byte: leaf.GetAsset().payload)
{
byte = static_cast<std::uint8_t>(byteDist(p_rng));
}
const UUID leafUUID = leaf.GetHeader().uuid;
std::ignore = p_leafContainer.Insert(std::move(leaf));
p_manager.SaveAsset(leafUUID);
p_leafContainer.Remove(leafUUID);
BenchAssetRoot root;
root.SetName(UniqueName(root.GetHeader().uuid));
root.GetAsset().leaf = HardReference<BenchAssetLeaf> {leafUUID};
const UUID rootUUID = root.GetHeader().uuid;
std::ignore = p_rootContainer.Insert(std::move(root));
p_manager.SaveAsset(rootUUID);
p_rootContainer.Remove(rootUUID);
p_outRootUUIDs.push_back(rootUUID);
p_outLeafUUIDs.push_back(leafUUID);
}
}
// SerializationManager's LoadAsset body is picked at compile time by BIGFOOT_TOOL/BIGFOOT_GAME (see
// SerializationManager.hpp): this benchmark binary is built Tool-mode, so only the BigFile-backed body
// exists in it - there is no compiled Game-mode/Sasquatch LoadAsset to call here. This mirrors just enough
// of it (no cycle guard: this fixture's data is a strict two-level root->leaf tree, never cyclic) to
// exercise Sasquatch::Find() and AssetContainerBase::Insert() the same way a Game build would.
void LoadFromSasquatch(Sasquatch& p_sasquatch, AssetLibrary& p_library, const UUID& p_uuid)
{
if (!p_uuid)
{
return;
}
const std::optional<Sasquatch::Entry> entry = p_sasquatch.Find(p_uuid);
if (!entry.has_value())
{
return;
}
AssetContainerBase* container = p_library.GetAssetContainer(entry->m_typeID);
if (container == nullptr || container->Has(p_uuid))
{
return;
}
for (std::size_t offset = 0; offset < entry->m_hardReferenceUUIDs.size(); offset += UUID::UUID_BYTE_SIZE)
{
const eastl::span<const std::byte> uuidBytes = entry->m_hardReferenceUUIDs.subspan(offset, UUID::UUID_BYTE_SIZE);
LoadFromSasquatch(
p_sasquatch, p_library,
UUID {std::span<const std::byte, UUID::UUID_BYTE_SIZE> {uuidBytes.data(), UUID::UUID_BYTE_SIZE}});
}
container->Insert(entry->m_bytes);
}
[[nodiscard]]
std::uint64_t TouchLoadedRoot(AssetContainer<BenchAssetRoot>& p_rootContainer, const UUID& p_uuid)
{
const auto* root = static_cast<const BenchAssetRoot*>(p_rootContainer.AcquireContainerHandle(p_uuid)->GetAsset());
if (root == nullptr)
{
return 0;
}
const BenchAssetLeaf* leaf = root->GetAsset().leaf.Get();
return leaf == nullptr ? 0 : Touch(leaf->GetAsset().payload);
}
void RunBigFileBenchmark(benchmark::State& state, const bool p_shuffleLookupOrder)
{
const auto count = static_cast<std::uint32_t>(state.range(0));
const auto payloadSize = static_cast<std::size_t>(state.range(1));
BigFile bigFile {File {BIGFILE_ENGINEBENCHMARKS_LOCATION}};
SerializationManager manager {bigFile};
AssetLibrary library;
AssetContainer<BenchAssetLeaf> leafContainer;
AssetContainer<BenchAssetRoot> rootContainer;
SetActiveAssetLibrary(&library);
std::ignore = library.Insert(&leafContainer);
std::ignore = library.Insert(&rootContainer);
ClearBigFile(bigFile);
std::mt19937_64 rng {0x5EEDULL};
std::vector<UUID> rootUUIDs;
std::vector<UUID> leafUUIDs;
PopulateAssets(manager, leafContainer, rootContainer, count, payloadSize, rng, rootUUIDs, leafUUIDs);
std::vector<UUID> lookupOrder = rootUUIDs;
if (p_shuffleLookupOrder)
{
std::shuffle(lookupOrder.begin(), lookupOrder.end(), rng);
}
for (auto _: state)
{
std::uint64_t sum = 0;
for (const UUID& uuid: lookupOrder)
{
manager.LoadAsset(uuid);
sum += TouchLoadedRoot(rootContainer, uuid);
}
benchmark::DoNotOptimize(sum);
state.PauseTiming();
for (const UUID& uuid: rootUUIDs)
{
rootContainer.Remove(uuid);
}
for (const UUID& uuid: leafUUIDs)
{
leafContainer.Remove(uuid);
}
state.ResumeTiming();
}
state.SetItemsProcessed(state.iterations() * lookupOrder.size());
state.SetBytesProcessed(static_cast<std::int64_t>(state.iterations()) * count * static_cast<std::int64_t>(payloadSize));
SetActiveAssetLibrary(&g_assetLibrary);
}
void RunSasquatchBenchmark(benchmark::State& state, const bool p_shuffleLookupOrder)
{
const auto count = static_cast<std::uint32_t>(state.range(0));
const auto payloadSize = static_cast<std::size_t>(state.range(1));
const std::filesystem::path sasquatchPath = TempPath("BigfootAssetBench.sasquatch");
BigFile bigFile {File {BIGFILE_ENGINEBENCHMARKS_LOCATION}};
SerializationManager manager {bigFile};
AssetLibrary library;
AssetContainer<BenchAssetLeaf> leafContainer;
AssetContainer<BenchAssetRoot> rootContainer;
SetActiveAssetLibrary(&library);
std::ignore = library.Insert(&leafContainer);
std::ignore = library.Insert(&rootContainer);
ClearBigFile(bigFile);
std::mt19937_64 rng {0x5EEDULL};
std::vector<UUID> rootUUIDs;
std::vector<UUID> leafUUIDs;
PopulateAssets(manager, leafContainer, rootContainer, count, payloadSize, rng, rootUUIDs, leafUUIDs);
Sasquatch::WriteFrom(bigFile, ToFile(sasquatchPath));
std::vector<UUID> lookupOrder = rootUUIDs;
if (p_shuffleLookupOrder)
{
std::shuffle(lookupOrder.begin(), lookupOrder.end(), rng);
}
{
Sasquatch sasquatch {ToFile(sasquatchPath)};
for (auto _: state)
{
std::uint64_t sum = 0;
for (const UUID& uuid: lookupOrder)
{
LoadFromSasquatch(sasquatch, library, uuid);
sum += TouchLoadedRoot(rootContainer, uuid);
}
benchmark::DoNotOptimize(sum);
state.PauseTiming();
for (const UUID& uuid: rootUUIDs)
{
rootContainer.Remove(uuid);
}
for (const UUID& uuid: leafUUIDs)
{
leafContainer.Remove(uuid);
}
state.ResumeTiming();
}
state.SetItemsProcessed(state.iterations() * lookupOrder.size());
state.SetBytesProcessed(static_cast<std::int64_t>(state.iterations()) * count *
static_cast<std::int64_t>(payloadSize));
}
SetActiveAssetLibrary(&g_assetLibrary);
std::filesystem::remove(sasquatchPath);
}
// Bypasses AssetContainer/flatbuffers entirely: just Sasquatch::Find() (hash lookup into the index built
// at construction) plus a direct read of the returned span. This is the ceiling the full pipeline above
// (Unpack + HardReference resolution + AssetContainer::Insert) is measured against - the gap between the
// two is exactly what that layer costs.
void Sasquatch_RawRead(benchmark::State& state)
{
const auto count = static_cast<std::uint32_t>(state.range(0));
const auto payloadSize = static_cast<std::size_t>(state.range(1));
const std::filesystem::path sasquatchPath = TempPath("BigfootAssetBench.sasquatch");
BigFile bigFile {File {BIGFILE_ENGINEBENCHMARKS_LOCATION}};
SerializationManager manager {bigFile};
AssetLibrary library;
AssetContainer<BenchAssetLeaf> leafContainer;
AssetContainer<BenchAssetRoot> rootContainer;
SetActiveAssetLibrary(&library);
std::ignore = library.Insert(&leafContainer);
std::ignore = library.Insert(&rootContainer);
ClearBigFile(bigFile);
std::mt19937_64 rng {0x5EEDULL};
std::vector<UUID> rootUUIDs;
std::vector<UUID> leafUUIDs;
PopulateAssets(manager, leafContainer, rootContainer, count, payloadSize, rng, rootUUIDs, leafUUIDs);
Sasquatch::WriteFrom(bigFile, ToFile(sasquatchPath));
SetActiveAssetLibrary(&g_assetLibrary);
std::vector<UUID> lookupOrder = leafUUIDs;
std::shuffle(lookupOrder.begin(), lookupOrder.end(), rng);
{
Sasquatch sasquatch {ToFile(sasquatchPath)};
for (auto _: state)
{
std::uint64_t sum = 0;
for (const UUID& uuid: lookupOrder)
{
if (const std::optional<Sasquatch::Entry> entry = sasquatch.Find(uuid))
{
sum += Touch(entry->m_bytes);
}
}
benchmark::DoNotOptimize(sum);
}
state.SetItemsProcessed(state.iterations() * lookupOrder.size());
state.SetBytesProcessed(static_cast<std::int64_t>(state.iterations()) * count *
static_cast<std::int64_t>(payloadSize));
}
std::filesystem::remove(sasquatchPath);
}
void BigFile_RandomLookup(benchmark::State& state)
{
RunBigFileBenchmark(state, true);
}
void BigFile_FullScan(benchmark::State& state)
{
RunBigFileBenchmark(state, false);
}
void Sasquatch_RandomLookup(benchmark::State& state)
{
RunSasquatchBenchmark(state, true);
}
void Sasquatch_FullScan(benchmark::State& state)
{
RunSasquatchBenchmark(state, false);
}
// Not a real benchmark: writes a persistent .sasquatch file plus a manifest of its root/leaf UUID pairs
// (one "root leaf" line per pair) to well-known temp paths, for
// Benchmarks/Engine/BigFile/SasquatchLoadAssets.cpp's Game-mode-only LoadAsset/LoadAssets comparison to
// read - Game mode never writes, so it can't produce its own fixture data or know the UUIDs without being
// told. Run once via --benchmark_filter=GenerateLoadAssetsFixture before running that Game-mode build.
void Sasquatch_GenerateLoadAssetsFixture(benchmark::State& state)
{
static constexpr std::uint32_t COUNT = 5'000;
static constexpr std::size_t PAYLOAD_SIZE = 262'144;
for (auto _: state)
{
BigFile bigFile {File {BIGFILE_ENGINEBENCHMARKS_LOCATION}};
SerializationManager manager {bigFile};
AssetLibrary library;
AssetContainer<BenchAssetLeaf> leafContainer;
AssetContainer<BenchAssetRoot> rootContainer;
SetActiveAssetLibrary(&library);
std::ignore = library.Insert(&leafContainer);
std::ignore = library.Insert(&rootContainer);
ClearBigFile(bigFile);
std::mt19937_64 rng {0x5EEDULL};
std::vector<UUID> rootUUIDs;
std::vector<UUID> leafUUIDs;
PopulateAssets(manager, leafContainer, rootContainer, COUNT, PAYLOAD_SIZE, rng, rootUUIDs, leafUUIDs);
Sasquatch::WriteFrom(bigFile, ToFile(TempPath("BigfootLoadAssetsFixture.sasquatch")));
std::ofstream manifest {TempPath("BigfootLoadAssetsFixture.manifest")};
for (std::size_t i = 0; i < rootUUIDs.size(); ++i)
{
manifest << rootUUIDs[i].ToString() << ' ' << leafUUIDs[i].ToString() << '\n';
}
SetActiveAssetLibrary(&g_assetLibrary);
}
}
} // namespace
// {count, payloadSize} - total data volume kept in the low hundreds of MB so population (real,
// transactional SaveAsset calls on the BigFile side) stays fast: 2000x64KiB=128MiB, 200x1MiB=200MiB,
// 20x16MiB=320MiB. Small-to-large so per-asset fixed overhead's share of the total is visible shrinking.
#define BIGFOOT_ASSET_SIZE_ARGS \
Args({2'000, 65'536})->Args({200, 1'048'576})->Args({20, 16'777'216})
BENCHMARK(BigFile_RandomLookup)->Name("BigFile/RandomLookup")->BIGFOOT_ASSET_SIZE_ARGS;
BENCHMARK(Sasquatch_RandomLookup)->Name("Sasquatch/RandomLookup")->BIGFOOT_ASSET_SIZE_ARGS;
BENCHMARK(BigFile_FullScan)->Name("BigFile/FullScan")->BIGFOOT_ASSET_SIZE_ARGS;
BENCHMARK(Sasquatch_FullScan)->Name("Sasquatch/FullScan")->BIGFOOT_ASSET_SIZE_ARGS;
BENCHMARK(Sasquatch_RawRead)->Name("Sasquatch/RawRead")->BIGFOOT_ASSET_SIZE_ARGS;
BENCHMARK(Sasquatch_GenerateLoadAssetsFixture)->Name("GenerateLoadAssetsFixture")->Iterations(1);
#undef BIGFOOT_ASSET_SIZE_ARGS
} // namespace Bigfoot
#endif
@@ -0,0 +1,182 @@
/*********************************************************************
* \file SasquatchLoadAssets.cpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#if !defined BIGFOOT_TOOL
#include <AssetsForBenchmarking/Asset/BenchAssetLeaf.hpp>
#include <AssetsForBenchmarking/Asset/BenchAssetRoot.hpp>
#include <Engine/Asset/AssetContainer.hpp>
#include <Engine/Asset/AssetLibrary.hpp>
#include <Engine/Asset/SerializationManager.hpp>
#include <Engine/EngineAssertHandler.hpp>
#include <Engine/Sasquatch/Sasquatch.hpp>
#include <System/File.hpp>
#include <System/UUID/UUID.hpp>
#include <EASTL/string_view.h>
#include <EASTL/vector.h>
#include <benchmark/benchmark.h>
#include <cstdint>
#include <filesystem>
#include <fstream>
#include <string>
#include <tuple>
#include <vector>
// Game-mode-only: compares the existing, single-threaded LoadAsset (called once per root, in a loop)
// against the new, parallel-decode LoadAssets (one call, the whole batch) - see SerializationManager -
// against real Asset<T> objects. Reads the fixture Sasquatch_GenerateLoadAssetsFixture (Tool mode, see
// Sasquatch.cpp in this directory) writes out, since a Game build never saves and so can't produce its
// own: run that once first via a Tool-mode EngineBenchmarks build.
namespace Bigfoot
{
namespace
{
[[nodiscard]]
File ToFile(const std::filesystem::path& p_path)
{
const std::string pathString = p_path.string();
return File {eastl::string_view {pathString.c_str(), pathString.size()}};
}
struct ManifestEntry
{
UUID m_root;
UUID m_leaf;
};
[[nodiscard]]
std::vector<ManifestEntry> ReadManifest()
{
const std::filesystem::path path = std::filesystem::temp_directory_path() / "BigfootLoadAssetsFixture.manifest";
std::ifstream manifest {path};
CRITICAL_ASSERT(EngineAssertHandler, manifest.is_open(),
"Missing LoadAssets fixture manifest at {} - run "
"EngineBenchmarks --benchmark_filter=GenerateLoadAssetsFixture from a Tool-mode build first",
path.string());
std::vector<ManifestEntry> entries;
std::string rootString;
std::string leafString;
while (manifest >> rootString >> leafString)
{
entries.push_back({UUID {rootString}, UUID {leafString}});
}
return entries;
}
[[nodiscard]]
std::uint64_t Touch(const eastl::vector<std::uint8_t>& p_payload)
{
return p_payload.empty() ? 0 : p_payload.back();
}
[[nodiscard]]
std::uint64_t TouchLoadedRoot(AssetContainer<BenchAssetRoot>& p_rootContainer, const UUID& p_uuid)
{
const auto* root = static_cast<const BenchAssetRoot*>(p_rootContainer.AcquireContainerHandle(p_uuid)->GetAsset());
if (root == nullptr)
{
return 0;
}
const BenchAssetLeaf* leaf = root->GetAsset().leaf.Get();
return leaf == nullptr ? 0 : Touch(leaf->GetAsset().payload);
}
void RunLoadAssetsBenchmark(benchmark::State& state, const bool p_parallel)
{
Sasquatch sasquatch {ToFile(std::filesystem::temp_directory_path() / "BigfootLoadAssetsFixture.sasquatch")};
const std::vector<ManifestEntry> manifest = ReadManifest();
SerializationManager manager {sasquatch};
AssetLibrary library;
AssetContainer<BenchAssetLeaf> leafContainer;
AssetContainer<BenchAssetRoot> rootContainer;
SetActiveAssetLibrary(&library);
std::ignore = library.Insert(&leafContainer);
std::ignore = library.Insert(&rootContainer);
eastl::vector<UUID> rootUUIDs;
rootUUIDs.reserve(manifest.size());
for (const ManifestEntry& entry: manifest)
{
rootUUIDs.push_back(entry.m_root);
}
std::uintmax_t totalBytes = 0;
for (const ManifestEntry& entry: manifest)
{
if (const std::optional<Sasquatch::Entry> leaf = sasquatch.Find(entry.m_leaf))
{
totalBytes += leaf->m_bytes.size();
}
}
for (auto _: state)
{
std::uint64_t sum = 0;
if (p_parallel)
{
manager.LoadAssets(eastl::span<const UUID> {rootUUIDs.data(), rootUUIDs.size()});
for (const UUID& uuid: rootUUIDs)
{
sum += TouchLoadedRoot(rootContainer, uuid);
}
}
else
{
for (const UUID& uuid: rootUUIDs)
{
manager.LoadAsset(uuid);
sum += TouchLoadedRoot(rootContainer, uuid);
}
}
benchmark::DoNotOptimize(sum);
state.PauseTiming();
for (const ManifestEntry& entry: manifest)
{
rootContainer.Remove(entry.m_root);
leafContainer.Remove(entry.m_leaf);
}
state.ResumeTiming();
}
state.SetItemsProcessed(state.iterations() * manifest.size());
state.SetBytesProcessed(static_cast<std::int64_t>(state.iterations()) * static_cast<std::int64_t>(totalBytes));
SetActiveAssetLibrary(&g_assetLibrary);
}
void Sasquatch_LoadAsset_Sequential(benchmark::State& state)
{
RunLoadAssetsBenchmark(state, false);
}
void Sasquatch_LoadAssets_Parallel(benchmark::State& state)
{
RunLoadAssetsBenchmark(state, true);
}
} // namespace
// UseRealTime(): LoadAssets does its decoding on worker threads, not the calling one, so the default CPU-
// time measurement (which only counts the calling thread) would wildly understate the real cost - wall
// time is what actually matters for "how long until these assets are usable."
BENCHMARK(Sasquatch_LoadAsset_Sequential)
->Name("Sasquatch/LoadAsset_Sequential")
->Unit(benchmark::kMillisecond)
->UseRealTime();
BENCHMARK(Sasquatch_LoadAssets_Parallel)
->Name("Sasquatch/LoadAssets_Parallel")
->Unit(benchmark::kMillisecond)
->UseRealTime();
} // namespace Bigfoot
#endif
-3
View File
@@ -6,9 +6,6 @@ bigfoot_create_package_benchmarks(
target_link_libraries(${PROJECT_NAME}
PRIVATE
effolkronium_random
$<LINK_LIBRARY:WHOLE_ARCHIVE,AssetsForBenchmarking>)
bigfoot_create_bigfile("Benchmarks/Bigfoot")
bigfoot_create_sasquatch_info()
@@ -9,14 +9,11 @@
#include <EASTL/string_view.h>
#if defined BIGFOOT_TOOLMODE
namespace Bigfoot
{
/*
* BigFile location
*/
constexpr eastl::string_view BIGFILE_ENGINEBENCHMARKS_LOCATION {"./EngineBenchmarks.db"};
constexpr eastl::string_view BIGFILE_ENGINEBENCHMARKS_LOCATION {"./EngineBenchmarks-bigfile.db"};
} // namespace Bigfoot
#endif
#endif
@@ -1,22 +0,0 @@
// AUTO-GENERATED DO NOT TOUCH
/*********************************************************************
* \file SasquatchInfo.generated.hpp
*
*********************************************************************/
#ifndef BIGFOOT_ENGINEBENCHMARKS_SASQUATCHINFO_GENERATED_HPP
#define BIGFOOT_ENGINEBENCHMARKS_SASQUATCHINFO_GENERATED_HPP
#include <EASTL/string_view.h>
#if defined BIGFOOT_GAME
namespace Bigfoot
{
/*
* Sasquatch location
*/
constexpr eastl::string_view SASQUATCH_ENGINEBENCHMARKS_LOCATION {"./EngineBenchmarks.sasquatch"};
} // namespace Bigfoot
#endif
#endif
@@ -1,245 +0,0 @@
/*********************************************************************
* \file Sasquatch.cpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#include <Engine/Sasquatch/Sasquatch.hpp>
#include <System/File/File.hpp>
#include <System/UUID/UUID.hpp>
#include <EngineBenchmarks/SasquatchInfo_generated.hpp>
#include <AssetsForBenchmarking/Asset/BenchmarkAsset.hpp>
#include <AssetsForBenchmarking/Asset/BenchmarkAssetReferred.hpp>
#include <EASTL/optional.h>
#include <EASTL/span.h>
#include <EASTL/vector.h>
#include <benchmark/benchmark.h>
#include <effolkronium/random.hpp>
#include <cstddef>
#include <cstdint>
namespace Bigfoot
{
#ifdef BIGFOOT_GAME
using Random = effolkronium::random_static;
class SasquatchFixture: public benchmark::Fixture
{
protected:
static constexpr std::uint32_t ASSET_COUNT = 20'000;
static constexpr std::uint32_t REFS_PER_ASSET = 4;
static constexpr std::size_t ASSET_PAYLOAD_SIZE = 256;
void SetUp([[maybe_unused]] benchmark::State& p_state) override
{
// The Sasquatch file is built exactly once for the entire benchmark process.
// SetUp is called before every calibration round (Google Benchmark doubles
// iteration counts until it reaches a stable measurement), so rebuilding
// the file each time would dominate the total benchmark time.
m_uuids = BuildData();
m_sasquatch.emplace(File {SASQUATCH_ENGINEBENCHMARKS_LOCATION});
}
void TearDown([[maybe_unused]] benchmark::State& p_state) override
{
m_sasquatch.reset();
}
[[nodiscard]]
eastl::vector<UUID> RandomBatch(const std::size_t p_batchSize) const
{
eastl::vector<UUID> batch;
batch.reserve(p_batchSize);
for (std::size_t i = 0; i < p_batchSize; ++i)
{
batch.push_back(m_uuids[Random::get<std::size_t>(0, m_uuids.size() - 1)]);
}
return batch;
}
[[nodiscard]]
eastl::vector<eastl::vector<UUID>> RandomBatchPool(const std::size_t p_batchSize,
const std::size_t p_poolSize) const
{
eastl::vector<eastl::vector<UUID>> pool;
pool.reserve(p_poolSize);
for (std::size_t i = 0; i < p_poolSize; ++i)
{
pool.push_back(RandomBatch(p_batchSize));
}
return pool;
}
eastl::optional<Sasquatch> m_sasquatch;
eastl::span<const UUID> m_uuids;
private:
static const eastl::vector<UUID>& BuildData()
{
static const eastl::vector<UUID> s_uuids = []()
{
File {SASQUATCH_ENGINEBENCHMARKS_LOCATION}.Delete();
eastl::vector<UUID> uuids;
eastl::vector<eastl::vector<std::byte>> flatbuffers;
eastl::vector<eastl::vector<UUID>> hardReferences;
uuids.reserve(ASSET_COUNT);
flatbuffers.reserve(ASSET_COUNT);
hardReferences.reserve(ASSET_COUNT);
for (std::uint32_t i = 0; i < ASSET_COUNT; ++i)
{
BenchmarkAsset asset;
asset.SetName(asset.GetHeader().uuid.ToString());
asset.GetAsset().payload.assign(ASSET_PAYLOAD_SIZE, std::uint8_t {0xAB});
for (std::uint32_t j = 0; j < REFS_PER_ASSET; ++j)
{
BenchmarkAssetReferred assetReferred;
assetReferred.SetName(assetReferred.GetHeader().uuid.ToString());
asset.GetAsset().refs.emplace_back(assetReferred.GetHeader().uuid);
}
eastl::vector<std::byte> flatbuffer = asset.Pack();
hardReferences.push_back(asset.GetHardReferences(flatbuffer));
flatbuffers.push_back(std::move(flatbuffer));
uuids.push_back(asset.GetHeader().uuid);
}
std::size_t totalFlatbufferBytes = 0;
std::size_t totalHardReferencesBytes = 0;
for (std::size_t i = 0; i < uuids.size(); ++i)
{
totalFlatbufferBytes += flatbuffers[i].size();
totalHardReferencesBytes += hardReferences[i].size() * UUID::UUID_BYTE_SIZE;
}
SasquatchWriter writer {File {SASQUATCH_ENGINEBENCHMARKS_LOCATION},
uuids.size(),
totalFlatbufferBytes,
totalHardReferencesBytes};
for (std::size_t i = 0; i < uuids.size(); ++i)
{
writer.WriteAsset(uuids[i], flatbuffers[i], hardReferences[i]);
}
return uuids;
}();
return s_uuids;
}
};
/****************************************************************************************/
BENCHMARK_DEFINE_F(SasquatchFixture, GetAssetHardReferences)(benchmark::State& state)
{
for (auto _: state)
{
benchmark::DoNotOptimize(
m_sasquatch->GetAssetHardReferences(m_uuids[Random::get<std::size_t>(0, m_uuids.size() - 1)]));
}
state.SetItemsProcessed(state.iterations());
state.SetBytesProcessed(static_cast<std::int64_t>(state.iterations()) * REFS_PER_ASSET * UUID::UUID_BYTE_SIZE);
}
BENCHMARK_REGISTER_F(SasquatchFixture, GetAssetHardReferences)->Name("Sasquatch/GetAssetHardReferences")->UseRealTime();
/****************************************************************************************/
BENCHMARK_DEFINE_F(SasquatchFixture, GetAssetsHardReferences)(benchmark::State& state)
{
const auto batchSize = static_cast<std::size_t>(state.range(0));
constexpr std::size_t BATCH_POOL_SIZE = 64;
const eastl::vector<eastl::vector<UUID>> batchPool = RandomBatchPool(batchSize, BATCH_POOL_SIZE);
std::size_t batchIndex = 0;
for (auto _: state)
{
benchmark::DoNotOptimize(m_sasquatch->GetAssetsHardReferences(batchPool[batchIndex]));
batchIndex = (batchIndex + 1) % BATCH_POOL_SIZE;
}
state.SetItemsProcessed(state.iterations() * batchSize);
state.SetBytesProcessed(static_cast<std::int64_t>(state.iterations()) * batchSize * REFS_PER_ASSET *
UUID::UUID_BYTE_SIZE);
}
BENCHMARK_REGISTER_F(SasquatchFixture, GetAssetsHardReferences)
->Name("Sasquatch/GetAssetsHardReferences")
->Arg(1)
->Arg(8)
->Arg(64)
->Arg(512)
->Arg(4096)
->UseRealTime();
/****************************************************************************************/
BENCHMARK_DEFINE_F(SasquatchFixture, GetAssetFlatbuffer)(benchmark::State& state)
{
std::int64_t bytesProcessed = 0;
for (auto _: state)
{
eastl::vector<std::byte> flatbuffer = m_sasquatch->GetAssetFlatbuffer(
m_uuids[Random::get<std::size_t>(0, m_uuids.size() - 1)]);
benchmark::DoNotOptimize(flatbuffer);
bytesProcessed += static_cast<std::int64_t>(flatbuffer.size());
}
state.SetItemsProcessed(state.iterations());
state.SetBytesProcessed(bytesProcessed);
}
BENCHMARK_REGISTER_F(SasquatchFixture, GetAssetFlatbuffer)->Name("Sasquatch/GetAssetFlatbuffer")->UseRealTime();
/****************************************************************************************/
BENCHMARK_DEFINE_F(SasquatchFixture, GetAssetFlatbuffers)(benchmark::State& state)
{
const auto batchSize = static_cast<std::size_t>(state.range(0));
constexpr std::size_t BATCH_POOL_SIZE = 64;
const eastl::vector<eastl::vector<UUID>> batchPool = RandomBatchPool(batchSize, BATCH_POOL_SIZE);
std::int64_t bytesProcessed = 0;
std::size_t batchIndex = 0;
for (auto _: state)
{
eastl::vector<eastl::vector<std::byte>> flatbuffers = m_sasquatch->GetAssetFlatbuffers(batchPool[batchIndex]);
benchmark::DoNotOptimize(flatbuffers);
for (const eastl::vector<std::byte>& flatbuffer: flatbuffers)
{
bytesProcessed += static_cast<std::int64_t>(flatbuffer.size());
}
batchIndex = (batchIndex + 1) % BATCH_POOL_SIZE;
}
state.SetItemsProcessed(state.iterations() * batchSize);
state.SetBytesProcessed(bytesProcessed);
}
BENCHMARK_REGISTER_F(SasquatchFixture, GetAssetFlatbuffers)
->Name("Sasquatch/GetAssetFlatbuffers")
->Arg(1)
->Arg(8)
->Arg(64)
->Arg(512)
->Arg(4096)
->UseRealTime();
#endif
} // namespace Bigfoot
@@ -1,391 +0,0 @@
/*********************************************************************
* \file SerializationManager.cpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#include <Engine/SerializationManager/SerializationManager.hpp>
#include <Engine/Asset/AssetLibrary.hpp>
#include <Engine/DataAccessor/DataAccessor.hpp>
#if defined BIGFOOT_TOOLMODE
#include <Engine/BigFile/BigFile.hpp>
#include <EngineBenchmarks/BigFileInfo_generated.hpp>
#elif defined BIGFOOT_GAME
#include <Engine/Sasquatch/Sasquatch.hpp>
#include <EngineBenchmarks/SasquatchInfo_generated.hpp>
#endif
#include <System/File/File.hpp>
#include <System/UUID/UUID.hpp>
#include <AssetsForBenchmarking/Asset/BenchmarkAsset.hpp>
#include <AssetsForBenchmarking/Asset/BenchmarkAssetReferred.hpp>
#include <EASTL/optional.h>
#include <EASTL/span.h>
#include <EASTL/vector.h>
#include <ankerl/unordered_dense.h>
#include <benchmark/benchmark.h>
#include <effolkronium/random.hpp>
#include <cstddef>
#include <cstdint>
#include <tuple>
namespace Bigfoot
{
using Random = effolkronium::random_static;
class SerializationManagerFixture: public benchmark::Fixture
{
protected:
static constexpr std::uint32_t ROOT_ASSET_COUNT = 20'000;
static constexpr std::uint32_t REFS_PER_ROOT = 4;
static constexpr std::size_t ASSET_PAYLOAD_SIZE = 256;
void SetUp([[maybe_unused]] benchmark::State& p_state) override
{
m_previousDataAccessor = g_dataAccessor;
m_previousAssetLibrary = g_activeAssetLibrary;
// The data store is populated exactly once for the entire benchmark process.
// SetUp is called before every calibration round (Google Benchmark doubles
// iteration counts until it reaches a stable measurement), so rebuilding
// the data store each time would dominate the total benchmark time.
static const SharedData& s_data = BuildData();
m_rootUuids = s_data.rootUuids;
m_leafUuids = s_data.leafUuids;
m_rootTotalBytes = &s_data.rootTotalBytes;
#if defined BIGFOOT_TOOLMODE
m_bigFile.emplace(File {BIGFILE_ENGINEBENCHMARKS_LOCATION});
SetDataAccessor(&*m_bigFile);
#elif defined BIGFOOT_GAME
m_sasquatch.emplace(File {SASQUATCH_ENGINEBENCHMARKS_LOCATION});
SetDataAccessor(&*m_sasquatch);
#endif
}
void TearDown([[maybe_unused]] benchmark::State& p_state) override
{
#if defined BIGFOOT_TOOLMODE
m_bigFile.reset();
#elif defined BIGFOOT_GAME
m_sasquatch.reset();
#endif
SetDataAccessor(m_previousDataAccessor);
SetActiveAssetLibrary(m_previousAssetLibrary);
}
/*
* Registers a fresh, empty AssetLibrary as the active one, so that every root and leaf asset appears unloaded
*
* \param p_library The library to register
* \param p_leafContainer The container to register leaves into
* \param p_rootContainer The container to register roots into
*/
static void ResetActiveLibrary(AssetLibrary& p_library,
AssetContainer<BenchmarkAssetReferred>& p_leafContainer,
AssetContainer<BenchmarkAsset>& p_rootContainer)
{
std::ignore = p_library.Insert(&p_leafContainer);
std::ignore = p_library.Insert(&p_rootContainer);
SetActiveAssetLibrary(&p_library);
}
[[nodiscard]]
eastl::vector<UUID> RandomRootBatch(const std::size_t p_batchSize) const
{
eastl::vector<UUID> batch;
batch.reserve(p_batchSize);
for (std::size_t i = 0; i < p_batchSize; ++i)
{
batch.push_back(m_rootUuids[Random::get<std::size_t>(0, m_rootUuids.size() - 1)]);
}
return batch;
}
/*
* Total flatbuffer bytes (root + its leaves) that loading p_roots will actually read, deduplicated the same way
* SerializationManager::LoadAssets deduplicates its own transitive discovery
*
* \param p_roots The root assets that would be loaded
* \return The total flatbuffer bytes across the unique roots
*/
[[nodiscard]]
std::int64_t TotalBytes(const eastl::vector<UUID>& p_roots) const
{
ankerl::unordered_dense::set<UUID> unique;
unique.reserve(p_roots.size());
for (const UUID& uuid: p_roots)
{
unique.insert(uuid);
}
std::int64_t bytes = 0;
for (const UUID& uuid: unique)
{
bytes += static_cast<std::int64_t>(m_rootTotalBytes->at(uuid));
}
return bytes;
}
DataAccessor* m_previousDataAccessor = nullptr;
AssetLibrary* m_previousAssetLibrary = nullptr;
eastl::span<const UUID> m_rootUuids;
eastl::span<const UUID> m_leafUuids;
const ankerl::unordered_dense::map<UUID, std::size_t>* m_rootTotalBytes = nullptr;
#if defined BIGFOOT_TOOLMODE
eastl::optional<BigFile> m_bigFile;
#elif defined BIGFOOT_GAME
eastl::optional<Sasquatch> m_sasquatch;
#endif
private:
struct SharedData
{
eastl::vector<UUID> rootUuids;
eastl::vector<UUID> leafUuids;
ankerl::unordered_dense::map<UUID, std::size_t> rootTotalBytes;
};
static const SharedData& BuildData()
{
static const SharedData s_data = []()
{
SharedData data;
data.rootUuids.reserve(ROOT_ASSET_COUNT);
data.leafUuids.reserve(ROOT_ASSET_COUNT * REFS_PER_ROOT);
data.rootTotalBytes.reserve(ROOT_ASSET_COUNT);
eastl::vector<UUID> uuids;
eastl::vector<eastl::vector<std::byte>> flatbuffers;
eastl::vector<eastl::vector<UUID>> hardReferences;
uuids.reserve(ROOT_ASSET_COUNT * (1 + REFS_PER_ROOT));
flatbuffers.reserve(uuids.capacity());
hardReferences.reserve(uuids.capacity());
for (std::uint32_t i = 0; i < ROOT_ASSET_COUNT; ++i)
{
BenchmarkAsset root;
root.SetName(root.GetHeader().uuid.ToString());
root.GetAsset().payload.assign(ASSET_PAYLOAD_SIZE, std::uint8_t {0xAB});
std::size_t totalBytes = 0;
for (std::uint32_t j = 0; j < REFS_PER_ROOT; ++j)
{
BenchmarkAssetReferred leaf;
leaf.SetName(leaf.GetHeader().uuid.ToString());
root.GetAsset().refs.emplace_back(leaf.GetHeader().uuid);
eastl::vector<std::byte> leafFlatbuffer = leaf.Pack();
totalBytes += leafFlatbuffer.size();
data.leafUuids.push_back(leaf.GetHeader().uuid);
uuids.push_back(leaf.GetHeader().uuid);
hardReferences.push_back(leaf.GetHardReferences(leafFlatbuffer));
flatbuffers.push_back(std::move(leafFlatbuffer));
}
eastl::vector<std::byte> rootFlatbuffer = root.Pack();
totalBytes += rootFlatbuffer.size();
data.rootTotalBytes.emplace(root.GetHeader().uuid, totalBytes);
data.rootUuids.push_back(root.GetHeader().uuid);
uuids.push_back(root.GetHeader().uuid);
hardReferences.push_back(root.GetHardReferences(rootFlatbuffer));
flatbuffers.push_back(std::move(rootFlatbuffer));
}
#if defined BIGFOOT_TOOLMODE
BigFile bigFile {File { BIGFILE_ENGINEBENCHMARKS_LOCATION }};
bigFile.Clear();
bigFile.SaveAssetFlatbuffers(uuids, flatbuffers, hardReferences);
#elif defined BIGFOOT_GAME
std::size_t totalFlatbufferBytes = 0;
std::size_t totalHardReferencesBytes = 0;
for (std::size_t i = 0; i < uuids.size(); ++i)
{
totalFlatbufferBytes += flatbuffers[i].size();
totalHardReferencesBytes += hardReferences[i].size() * UUID::UUID_BYTE_SIZE;
}
File {SASQUATCH_ENGINEBENCHMARKS_LOCATION}.Delete();
SasquatchWriter writer {File {SASQUATCH_ENGINEBENCHMARKS_LOCATION},
uuids.size(),
totalFlatbufferBytes,
totalHardReferencesBytes};
for (std::size_t i = 0; i < uuids.size(); ++i)
{
writer.WriteAsset(uuids[i], flatbuffers[i], hardReferences[i]);
}
#endif
return data;
}();
return s_data;
}
};
/****************************************************************************************/
BENCHMARK_DEFINE_F(SerializationManagerFixture, LoadAsset)(benchmark::State& state)
{
std::int64_t bytesProcessed = 0;
for (auto _: state)
{
state.PauseTiming();
AssetLibrary library;
// leafContainer must outlive rootContainer: the loaded root's HardReferences release into leafContainer's
// handles on destruction, so declaring it second would tear it down first and use-after-free those handles.
AssetContainer<BenchmarkAssetReferred> leafContainer;
AssetContainer<BenchmarkAsset> rootContainer;
ResetActiveLibrary(library, leafContainer, rootContainer);
const UUID uuid = m_rootUuids[Random::get<std::size_t>(0, m_rootUuids.size() - 1)];
const std::size_t uuidBytes = m_rootTotalBytes->at(uuid);
state.ResumeTiming();
SerializationManager::LoadAsset(uuid);
bytesProcessed += static_cast<std::int64_t>(uuidBytes);
}
state.SetItemsProcessed(state.iterations() * (1 + REFS_PER_ROOT));
state.SetBytesProcessed(bytesProcessed);
}
BENCHMARK_REGISTER_F(SerializationManagerFixture, LoadAsset)->Name("SerializationManager/LoadAsset")->UseRealTime();
/****************************************************************************************/
BENCHMARK_DEFINE_F(SerializationManagerFixture, LoadAssets)(benchmark::State& state)
{
const auto batchSize = static_cast<std::size_t>(state.range(0));
std::int64_t bytesProcessed = 0;
for (auto _: state)
{
state.PauseTiming();
AssetLibrary library;
// leafContainer must outlive rootContainer, see the same comment in the LoadAsset benchmark above.
AssetContainer<BenchmarkAssetReferred> leafContainer;
AssetContainer<BenchmarkAsset> rootContainer;
ResetActiveLibrary(library, leafContainer, rootContainer);
const eastl::vector<UUID> batch = RandomRootBatch(batchSize);
const std::int64_t batchBytes = TotalBytes(batch);
state.ResumeTiming();
SerializationManager::LoadAssets(batch);
bytesProcessed += batchBytes;
}
state.SetItemsProcessed(state.iterations() * batchSize * (1 + REFS_PER_ROOT));
state.SetBytesProcessed(bytesProcessed);
}
BENCHMARK_REGISTER_F(SerializationManagerFixture, LoadAssets)
->Name("SerializationManager/LoadAssets")
->Arg(1)
->Arg(8)
->Arg(64)
->Arg(512)
->Arg(4096)
->UseRealTime();
/****************************************************************************************/
#if defined BIGFOOT_TOOLMODE
BENCHMARK_DEFINE_F(SerializationManagerFixture, SaveAsset)(benchmark::State& state)
{
AssetLibrary library;
AssetContainer<BenchmarkAsset> container;
std::ignore = library.Insert(&container);
SetActiveAssetLibrary(&library);
std::int64_t bytesProcessed = 0;
for (auto _: state)
{
state.PauseTiming();
auto* asset = CreateAsset<BenchmarkAsset>();
asset->SetName(asset->GetHeader().uuid.ToString());
asset->GetAsset().payload.assign(ASSET_PAYLOAD_SIZE, std::uint8_t {0xAB});
for (std::uint32_t j = 0; j < REFS_PER_ROOT; ++j)
{
asset->GetAsset().refs.emplace_back(m_leafUuids[Random::get<std::size_t>(0, m_leafUuids.size() - 1)]);
}
const UUID uuid = asset->GetHeader().uuid;
const std::size_t assetBytes = asset->Pack().size();
bytesProcessed += static_cast<std::int64_t>(assetBytes);
state.ResumeTiming();
benchmark::DoNotOptimize(SerializationManager::SaveAsset(uuid));
}
state.SetItemsProcessed(state.iterations());
state.SetBytesProcessed(bytesProcessed);
}
BENCHMARK_REGISTER_F(SerializationManagerFixture, SaveAsset)->Name("SerializationManager/SaveAsset")->UseRealTime();
/****************************************************************************************/
BENCHMARK_DEFINE_F(SerializationManagerFixture, SaveAssets)(benchmark::State& state)
{
const auto batchSize = static_cast<std::size_t>(state.range(0));
AssetLibrary library;
AssetContainer<BenchmarkAsset> container;
std::ignore = library.Insert(&container);
SetActiveAssetLibrary(&library);
std::int64_t bytesProcessed = 0;
for (auto _: state)
{
state.PauseTiming();
eastl::vector<UUID> uuids;
uuids.reserve(batchSize);
std::size_t batchBytes = 0;
for (std::size_t i = 0; i < batchSize; ++i)
{
auto* asset = CreateAsset<BenchmarkAsset>();
asset->SetName(asset->GetHeader().uuid.ToString());
asset->GetAsset().payload.assign(ASSET_PAYLOAD_SIZE, std::uint8_t {0xAB});
for (std::uint32_t j = 0; j < REFS_PER_ROOT; ++j)
{
asset->GetAsset().refs.emplace_back(m_leafUuids[Random::get<std::size_t>(0, m_leafUuids.size() - 1)]);
}
batchBytes += asset->Pack().size();
uuids.push_back(asset->GetHeader().uuid);
}
state.ResumeTiming();
benchmark::DoNotOptimize(SerializationManager::SaveAssets(uuids));
bytesProcessed += static_cast<std::int64_t>(batchBytes);
}
state.SetItemsProcessed(state.iterations() * batchSize);
state.SetBytesProcessed(bytesProcessed);
}
BENCHMARK_REGISTER_F(SerializationManagerFixture, SaveAssets)
->Name("SerializationManager/SaveAssets")
->Arg(1)
->Arg(8)
->Arg(64)
->Arg(512)
->UseRealTime();
#endif
} // namespace Bigfoot
@@ -1,11 +1,11 @@
/*********************************************************************
* \file BenchmarkAsset.cpp
* \file BenchAssetLeaf.cpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#include <AssetsForBenchmarking/Asset/BenchmarkAsset.hpp>
#include <AssetsForBenchmarking/Asset/BenchAssetLeaf.hpp>
#include <Engine/Asset/AssetHelper.hpp>
BIGFOOT_ASSET(BenchmarkAsset, ::Bigfoot::BenchmarkAsset)
BIGFOOT_ASSET(BenchAssetLeaf, ::Bigfoot::BenchAssetLeaf)
@@ -1,11 +1,11 @@
/*********************************************************************
* \file BenchmarkAssetReferred.cpp
* \file BenchAssetRoot.cpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#include <AssetsForBenchmarking/Asset/BenchmarkAssetReferred.hpp>
#include <AssetsForBenchmarking/Asset/BenchAssetRoot.hpp>
#include <Engine/Asset/AssetHelper.hpp>
BIGFOOT_ASSET(BenchmarkAssetReferred, ::Bigfoot::BenchmarkAssetReferred)
BIGFOOT_ASSET(BenchAssetRoot, ::Bigfoot::BenchAssetRoot)
@@ -7,11 +7,9 @@ set(BigfootPublicDependencies)
set(BigfootPrivateDependencies
Engine)
bigfoot_create_package_lib(
bigfoot_create_package_tests_helper_lib(
"${PublicDependencies}"
"${PrivateDependencies}"
"${BigfootPublicDependencies}"
"${BigfootPrivateDependencies}"
"")
set_target_properties(${PROJECT_NAME} PROPERTIES FOLDER Benchmarks/Bigfoot)
@@ -0,0 +1,8 @@
namespace Flat.Bigfoot;
table BenchAssetLeaf
{
payload: [ubyte];
}
root_type BenchAssetLeaf;
@@ -0,0 +1,40 @@
/*********************************************************************
* \file BenchAssetLeaf.hpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#ifndef BIGFOOT_ASSETSFORBENCHMARKING_ASSET_BENCHASSETLEAF_HPP
#define BIGFOOT_ASSETSFORBENCHMARKING_ASSET_BENCHASSETLEAF_HPP
#include <Engine/Asset/Asset.hpp>
#include <AssetsForBenchmarking/Asset/BenchAssetLeaf_generated.hpp>
namespace Bigfoot
{
struct BenchAssetLeafTraits
{
using FLAT = ::Flat::Bigfoot::BenchAssetLeaf;
};
/*
* A leaf asset with a variable-size byte payload - stands in for a self-contained game asset (a texture,
* a mesh, ...) so BigFile/Sasquatch benchmarks measure real Asset<T> Pack()/Unpack() round-trips instead
* of hand-rolled blobs.
*/
class BenchAssetLeaf: public Asset<BenchAssetLeafTraits>
{
public:
BenchAssetLeaf() = default;
BenchAssetLeaf(const BenchAssetLeaf& p_asset) = delete;
BenchAssetLeaf(BenchAssetLeaf&& p_asset) = default;
~BenchAssetLeaf() = default;
BenchAssetLeaf& operator=(const BenchAssetLeaf& p_asset) = delete;
BenchAssetLeaf& operator=(BenchAssetLeaf&& p_asset) = default;
};
} // namespace Bigfoot
#endif
@@ -0,0 +1,192 @@
// automatically generated by the FlatBuffers compiler, do not modify
#ifndef FLATBUFFERS_GENERATED_BENCHASSETLEAF_FLAT_BIGFOOT_H_
#define FLATBUFFERS_GENERATED_BENCHASSETLEAF_FLAT_BIGFOOT_H_
#include "flatbuffers/flatbuffers.h"
// Ensure the included flatbuffers.h is the same version as when this file was
// generated, otherwise it may not be compatible.
static_assert(FLATBUFFERS_VERSION_MAJOR == 25 &&
FLATBUFFERS_VERSION_MINOR == 12 &&
FLATBUFFERS_VERSION_REVISION == 19,
"Non-compatible flatbuffers version included");
// Ensure the included flatbuffers.h is Bigfoot's fork - stock
// flatbuffers (or a differently versioned Bigfoot fork) is not compatible.
#ifndef FLATBUFFERS_BIGFOOT_VERSION
#error "This file requires Bigfoot's flatbuffers fork - stock flatbuffers is not compatible"
#endif
static_assert(FLATBUFFERS_BIGFOOT_VERSION == 1,
"Non-compatible Bigfoot flatbuffers fork version included");
#include "EASTL/unique_ptr.h"
#include "EASTL/string.h"
#include "EASTL/vector.h"
namespace Flat {
namespace Bigfoot {
struct BenchAssetLeaf;
struct BenchAssetLeafBuilder;
struct BenchAssetLeafT;
inline const ::flatbuffers::TypeTable *BenchAssetLeafTypeTable();
struct BenchAssetLeafT : public ::flatbuffers::NativeTable {
typedef BenchAssetLeaf TableType;
static FLATBUFFERS_CONSTEXPR_CPP11 const char *GetFullyQualifiedName() {
return "Flat.Bigfoot.BenchAssetLeafT";
}
eastl::vector<uint8_t> payload{};
};
struct BenchAssetLeaf FLATBUFFERS_FINAL_CLASS : private ::flatbuffers::Table {
typedef BenchAssetLeafT NativeTableType;
typedef BenchAssetLeafBuilder Builder;
struct Traits;
static const ::flatbuffers::TypeTable *MiniReflectTypeTable() {
return BenchAssetLeafTypeTable();
}
static FLATBUFFERS_CONSTEXPR_CPP11 const char *GetFullyQualifiedName() {
return "Flat.Bigfoot.BenchAssetLeaf";
}
enum FlatBuffersVTableOffset FLATBUFFERS_VTABLE_UNDERLYING_TYPE {
VT_PAYLOAD = 4
};
const ::flatbuffers::Vector<uint8_t> *payload() const {
return GetPointer<const ::flatbuffers::Vector<uint8_t> *>(VT_PAYLOAD);
}
template <bool B = false>
bool Verify(::flatbuffers::VerifierTemplate<B> &verifier) const {
return VerifyTableStart(verifier) &&
VerifyOffset(verifier, VT_PAYLOAD) &&
verifier.VerifyVector(payload()) &&
verifier.EndTable();
}
BenchAssetLeafT *UnPack(const ::flatbuffers::resolver_function_t *_resolver = nullptr) const;
void UnPackTo(BenchAssetLeafT *_o, const ::flatbuffers::resolver_function_t *_resolver = nullptr) const;
static ::flatbuffers::Offset<BenchAssetLeaf> Pack(::flatbuffers::FlatBufferBuilder &_fbb, const BenchAssetLeafT* _o, const ::flatbuffers::rehasher_function_t *_rehasher = nullptr);
};
struct BenchAssetLeafBuilder {
typedef BenchAssetLeaf Table;
::flatbuffers::FlatBufferBuilder &fbb_;
::flatbuffers::uoffset_t start_;
void add_payload(::flatbuffers::Offset<::flatbuffers::Vector<uint8_t>> payload) {
fbb_.AddOffset(BenchAssetLeaf::VT_PAYLOAD, payload);
}
explicit BenchAssetLeafBuilder(::flatbuffers::FlatBufferBuilder &_fbb)
: fbb_(_fbb) {
start_ = fbb_.StartTable();
}
::flatbuffers::Offset<BenchAssetLeaf> Finish() {
const auto end = fbb_.EndTable(start_);
auto o = ::flatbuffers::Offset<BenchAssetLeaf>(end);
return o;
}
};
inline ::flatbuffers::Offset<BenchAssetLeaf> CreateBenchAssetLeaf(
::flatbuffers::FlatBufferBuilder &_fbb,
::flatbuffers::Offset<::flatbuffers::Vector<uint8_t>> payload = 0) {
BenchAssetLeafBuilder builder_(_fbb);
builder_.add_payload(payload);
return builder_.Finish();
}
struct BenchAssetLeaf::Traits {
using type = BenchAssetLeaf;
static auto constexpr Create = CreateBenchAssetLeaf;
};
::flatbuffers::Offset<BenchAssetLeaf> CreateBenchAssetLeaf(::flatbuffers::FlatBufferBuilder &_fbb, const BenchAssetLeafT *_o, const ::flatbuffers::rehasher_function_t *_rehasher = nullptr);
inline BenchAssetLeafT *BenchAssetLeaf::UnPack(const ::flatbuffers::resolver_function_t *_resolver) const {
auto _o = std::make_unique<BenchAssetLeafT>();
UnPackTo(_o.get(), _resolver);
return _o.release();
}
inline void BenchAssetLeaf::UnPackTo(BenchAssetLeafT *_o, const ::flatbuffers::resolver_function_t *_resolver) const {
(void)_o;
(void)_resolver;
{ auto _e = payload(); if (_e) { _o->payload.resize(_e->size()); std::copy(_e->begin(), _e->end(), _o->payload.begin()); } }
}
inline ::flatbuffers::Offset<BenchAssetLeaf> CreateBenchAssetLeaf(::flatbuffers::FlatBufferBuilder &_fbb, const BenchAssetLeafT *_o, const ::flatbuffers::rehasher_function_t *_rehasher) {
return BenchAssetLeaf::Pack(_fbb, _o, _rehasher);
}
inline ::flatbuffers::Offset<BenchAssetLeaf> BenchAssetLeaf::Pack(::flatbuffers::FlatBufferBuilder &_fbb, const BenchAssetLeafT* _o, const ::flatbuffers::rehasher_function_t *_rehasher) {
(void)_rehasher;
(void)_o;
struct _VectorArgs { ::flatbuffers::FlatBufferBuilder *__fbb; const BenchAssetLeafT* __o; const ::flatbuffers::rehasher_function_t *__rehasher; } _va = { &_fbb, _o, _rehasher}; (void)_va;
auto _payload = _fbb.CreateVector(_o->payload.data(), _o->payload.size());
return Flat::Bigfoot::CreateBenchAssetLeaf(
_fbb,
_payload);
}
inline const ::flatbuffers::TypeTable *BenchAssetLeafTypeTable() {
static const ::flatbuffers::TypeCode type_codes[] = {
{ ::flatbuffers::ET_UCHAR, 1, -1 }
};
static const ::flatbuffers::TypeTable tt = {
::flatbuffers::ST_TABLE, 1, type_codes, nullptr, nullptr, nullptr, nullptr
};
return &tt;
}
inline const Flat::Bigfoot::BenchAssetLeaf *GetBenchAssetLeaf(const void *buf) {
return ::flatbuffers::GetRoot<Flat::Bigfoot::BenchAssetLeaf>(buf);
}
inline const Flat::Bigfoot::BenchAssetLeaf *GetSizePrefixedBenchAssetLeaf(const void *buf) {
return ::flatbuffers::GetSizePrefixedRoot<Flat::Bigfoot::BenchAssetLeaf>(buf);
}
template <bool B = false>
inline bool VerifyBenchAssetLeafBuffer(
::flatbuffers::VerifierTemplate<B> &verifier) {
return verifier.template VerifyBuffer<Flat::Bigfoot::BenchAssetLeaf>(nullptr);
}
template <bool B = false>
inline bool VerifySizePrefixedBenchAssetLeafBuffer(
::flatbuffers::VerifierTemplate<B> &verifier) {
return verifier.template VerifySizePrefixedBuffer<Flat::Bigfoot::BenchAssetLeaf>(nullptr);
}
inline const char *BenchAssetLeafExtension() {
return "bfbs";
}
inline void FinishBenchAssetLeafBuffer(
::flatbuffers::FlatBufferBuilder &fbb,
::flatbuffers::Offset<Flat::Bigfoot::BenchAssetLeaf> root) {
fbb.Finish(root);
}
inline void FinishSizePrefixedBenchAssetLeafBuffer(
::flatbuffers::FlatBufferBuilder &fbb,
::flatbuffers::Offset<Flat::Bigfoot::BenchAssetLeaf> root) {
fbb.FinishSizePrefixed(root);
}
inline eastl::unique_ptr<Flat::Bigfoot::BenchAssetLeafT> UnPackBenchAssetLeaf(
const void *buf,
const ::flatbuffers::resolver_function_t *res = nullptr) {
return eastl::unique_ptr<Flat::Bigfoot::BenchAssetLeafT>(GetBenchAssetLeaf(buf)->UnPack(res));
}
inline eastl::unique_ptr<Flat::Bigfoot::BenchAssetLeafT> UnPackSizePrefixedBenchAssetLeaf(
const void *buf,
const ::flatbuffers::resolver_function_t *res = nullptr) {
return eastl::unique_ptr<Flat::Bigfoot::BenchAssetLeafT>(GetSizePrefixedBenchAssetLeaf(buf)->UnPack(res));
}
} // namespace Bigfoot
} // namespace Flat
#endif // FLATBUFFERS_GENERATED_BENCHASSETLEAF_FLAT_BIGFOOT_H_
@@ -0,0 +1,10 @@
include "Engine/Asset/Reference.fbs";
namespace Flat.Bigfoot;
table BenchAssetRoot
{
leaf: HardReference (native_inline, bigfoot_ref: "::Bigfoot::BenchAssetLeaf");
}
root_type BenchAssetRoot;
@@ -0,0 +1,40 @@
/*********************************************************************
* \file BenchAssetRoot.hpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#ifndef BIGFOOT_ASSETSFORBENCHMARKING_ASSET_BENCHASSETROOT_HPP
#define BIGFOOT_ASSETSFORBENCHMARKING_ASSET_BENCHASSETROOT_HPP
#include <Engine/Asset/Asset.hpp>
#include <AssetsForBenchmarking/Asset/BenchAssetRoot_generated.hpp>
namespace Bigfoot
{
struct BenchAssetRootTraits
{
using FLAT = ::Flat::Bigfoot::BenchAssetRoot;
};
/*
* A small "pointer" asset holding a HardReference to a BenchAssetLeaf - stands in for e.g. a Material
* referencing a Texture, so LoadAsset's recursive HardReference walk is actually exercised (a
* BenchAssetLeaf alone never triggers it).
*/
class BenchAssetRoot: public Asset<BenchAssetRootTraits>
{
public:
BenchAssetRoot() = default;
BenchAssetRoot(const BenchAssetRoot& p_asset) = delete;
BenchAssetRoot(BenchAssetRoot&& p_asset) = default;
~BenchAssetRoot() = default;
BenchAssetRoot& operator=(const BenchAssetRoot& p_asset) = delete;
BenchAssetRoot& operator=(BenchAssetRoot&& p_asset) = default;
};
} // namespace Bigfoot
#endif
@@ -0,0 +1,221 @@
// automatically generated by the FlatBuffers compiler, do not modify
#ifndef FLATBUFFERS_GENERATED_BENCHASSETROOT_FLAT_BIGFOOT_H_
#define FLATBUFFERS_GENERATED_BENCHASSETROOT_FLAT_BIGFOOT_H_
#include "flatbuffers/flatbuffers.h"
// Ensure the included flatbuffers.h is the same version as when this file was
// generated, otherwise it may not be compatible.
static_assert(FLATBUFFERS_VERSION_MAJOR == 25 &&
FLATBUFFERS_VERSION_MINOR == 12 &&
FLATBUFFERS_VERSION_REVISION == 19,
"Non-compatible flatbuffers version included");
// Ensure the included flatbuffers.h is Bigfoot's fork - stock
// flatbuffers (or a differently versioned Bigfoot fork) is not compatible.
#ifndef FLATBUFFERS_BIGFOOT_VERSION
#error "This file requires Bigfoot's flatbuffers fork - stock flatbuffers is not compatible"
#endif
static_assert(FLATBUFFERS_BIGFOOT_VERSION == 1,
"Non-compatible Bigfoot flatbuffers fork version included");
#include "System/UUID/UUID.hpp"
#include "Engine/Asset/Reference.hpp"
#include "Engine/Asset/Reference_generated.hpp"
#include "EASTL/unique_ptr.h"
#include "EASTL/string.h"
#include "EASTL/vector.h"
namespace Flat {
namespace Bigfoot {
struct BenchAssetRoot;
struct BenchAssetRootBuilder;
struct BenchAssetRootT;
} // namespace Bigfoot
} // namespace Flat
namespace Bigfoot {
class BenchAssetLeaf;
} // namespace Bigfoot
namespace flatbuffers {
Flat::Bigfoot::HardReference PackHardReferenceBenchAssetLeaf(const ::Bigfoot::HardReference<::Bigfoot::BenchAssetLeaf>& p_asset);
::Bigfoot::HardReference<::Bigfoot::BenchAssetLeaf> UnPackHardReferenceBenchAssetLeaf(const Flat::Bigfoot::HardReference& p_asset);
} // namespace flatbuffers
namespace Flat {
namespace Bigfoot {
inline const ::flatbuffers::TypeTable *BenchAssetRootTypeTable();
struct BenchAssetRootT : public ::flatbuffers::NativeTable {
typedef BenchAssetRoot TableType;
static FLATBUFFERS_CONSTEXPR_CPP11 const char *GetFullyQualifiedName() {
return "Flat.Bigfoot.BenchAssetRootT";
}
::Bigfoot::HardReference<::Bigfoot::BenchAssetLeaf> leaf{};
};
struct BenchAssetRoot FLATBUFFERS_FINAL_CLASS : private ::flatbuffers::Table {
typedef BenchAssetRootT NativeTableType;
typedef BenchAssetRootBuilder Builder;
struct Traits;
static const ::flatbuffers::TypeTable *MiniReflectTypeTable() {
return BenchAssetRootTypeTable();
}
static FLATBUFFERS_CONSTEXPR_CPP11 const char *GetFullyQualifiedName() {
return "Flat.Bigfoot.BenchAssetRoot";
}
enum FlatBuffersVTableOffset FLATBUFFERS_VTABLE_UNDERLYING_TYPE {
VT_LEAF = 4
};
const Flat::Bigfoot::HardReference *leaf() const {
return GetStruct<const Flat::Bigfoot::HardReference *>(VT_LEAF);
}
template <bool B = false>
bool Verify(::flatbuffers::VerifierTemplate<B> &verifier) const {
return VerifyTableStart(verifier) &&
VerifyField<Flat::Bigfoot::HardReference>(verifier, VT_LEAF, 1) &&
verifier.EndTable();
}
BenchAssetRootT *UnPack(const ::flatbuffers::resolver_function_t *_resolver = nullptr) const;
void UnPackTo(BenchAssetRootT *_o, const ::flatbuffers::resolver_function_t *_resolver = nullptr) const;
static ::flatbuffers::Offset<BenchAssetRoot> Pack(::flatbuffers::FlatBufferBuilder &_fbb, const BenchAssetRootT* _o, const ::flatbuffers::rehasher_function_t *_rehasher = nullptr);
};
struct BenchAssetRootBuilder {
typedef BenchAssetRoot Table;
::flatbuffers::FlatBufferBuilder &fbb_;
::flatbuffers::uoffset_t start_;
void add_leaf(const Flat::Bigfoot::HardReference *leaf) {
fbb_.AddStruct(BenchAssetRoot::VT_LEAF, leaf);
}
explicit BenchAssetRootBuilder(::flatbuffers::FlatBufferBuilder &_fbb)
: fbb_(_fbb) {
start_ = fbb_.StartTable();
}
::flatbuffers::Offset<BenchAssetRoot> Finish() {
const auto end = fbb_.EndTable(start_);
auto o = ::flatbuffers::Offset<BenchAssetRoot>(end);
return o;
}
};
inline ::flatbuffers::Offset<BenchAssetRoot> CreateBenchAssetRoot(
::flatbuffers::FlatBufferBuilder &_fbb,
const Flat::Bigfoot::HardReference *leaf = nullptr) {
BenchAssetRootBuilder builder_(_fbb);
builder_.add_leaf(leaf);
return builder_.Finish();
}
struct BenchAssetRoot::Traits {
using type = BenchAssetRoot;
static auto constexpr Create = CreateBenchAssetRoot;
};
::flatbuffers::Offset<BenchAssetRoot> CreateBenchAssetRoot(::flatbuffers::FlatBufferBuilder &_fbb, const BenchAssetRootT *_o, const ::flatbuffers::rehasher_function_t *_rehasher = nullptr);
inline BenchAssetRootT *BenchAssetRoot::UnPack(const ::flatbuffers::resolver_function_t *_resolver) const {
auto _o = std::make_unique<BenchAssetRootT>();
UnPackTo(_o.get(), _resolver);
return _o.release();
}
inline void BenchAssetRoot::UnPackTo(BenchAssetRootT *_o, const ::flatbuffers::resolver_function_t *_resolver) const {
(void)_o;
(void)_resolver;
{ auto _e = leaf(); if (_e) _o->leaf = ::flatbuffers::UnPackHardReferenceBenchAssetLeaf(*_e); }
}
inline ::flatbuffers::Offset<BenchAssetRoot> CreateBenchAssetRoot(::flatbuffers::FlatBufferBuilder &_fbb, const BenchAssetRootT *_o, const ::flatbuffers::rehasher_function_t *_rehasher) {
return BenchAssetRoot::Pack(_fbb, _o, _rehasher);
}
inline ::flatbuffers::Offset<BenchAssetRoot> BenchAssetRoot::Pack(::flatbuffers::FlatBufferBuilder &_fbb, const BenchAssetRootT* _o, const ::flatbuffers::rehasher_function_t *_rehasher) {
(void)_rehasher;
(void)_o;
struct _VectorArgs { ::flatbuffers::FlatBufferBuilder *__fbb; const BenchAssetRootT* __o; const ::flatbuffers::rehasher_function_t *__rehasher; } _va = { &_fbb, _o, _rehasher}; (void)_va;
auto _leaf = ::flatbuffers::PackHardReferenceBenchAssetLeaf(_o->leaf);
return Flat::Bigfoot::CreateBenchAssetRoot(
_fbb,
&_leaf);
}
inline const ::flatbuffers::TypeTable *BenchAssetRootTypeTable() {
static const ::flatbuffers::TypeCode type_codes[] = {
{ ::flatbuffers::ET_SEQUENCE, 0, 0 }
};
static const ::flatbuffers::TypeFunction type_refs[] = {
Flat::Bigfoot::HardReferenceTypeTable
};
static const ::flatbuffers::TypeTable tt = {
::flatbuffers::ST_TABLE, 1, type_codes, type_refs, nullptr, nullptr, nullptr
};
return &tt;
}
inline const Flat::Bigfoot::BenchAssetRoot *GetBenchAssetRoot(const void *buf) {
return ::flatbuffers::GetRoot<Flat::Bigfoot::BenchAssetRoot>(buf);
}
inline const Flat::Bigfoot::BenchAssetRoot *GetSizePrefixedBenchAssetRoot(const void *buf) {
return ::flatbuffers::GetSizePrefixedRoot<Flat::Bigfoot::BenchAssetRoot>(buf);
}
template <bool B = false>
inline bool VerifyBenchAssetRootBuffer(
::flatbuffers::VerifierTemplate<B> &verifier) {
return verifier.template VerifyBuffer<Flat::Bigfoot::BenchAssetRoot>(nullptr);
}
template <bool B = false>
inline bool VerifySizePrefixedBenchAssetRootBuffer(
::flatbuffers::VerifierTemplate<B> &verifier) {
return verifier.template VerifySizePrefixedBuffer<Flat::Bigfoot::BenchAssetRoot>(nullptr);
}
inline const char *BenchAssetRootExtension() {
return "bfbs";
}
inline void FinishBenchAssetRootBuffer(
::flatbuffers::FlatBufferBuilder &fbb,
::flatbuffers::Offset<Flat::Bigfoot::BenchAssetRoot> root) {
fbb.Finish(root);
}
inline void FinishSizePrefixedBenchAssetRootBuffer(
::flatbuffers::FlatBufferBuilder &fbb,
::flatbuffers::Offset<Flat::Bigfoot::BenchAssetRoot> root) {
fbb.FinishSizePrefixed(root);
}
inline eastl::unique_ptr<Flat::Bigfoot::BenchAssetRootT> UnPackBenchAssetRoot(
const void *buf,
const ::flatbuffers::resolver_function_t *res = nullptr) {
return eastl::unique_ptr<Flat::Bigfoot::BenchAssetRootT>(GetBenchAssetRoot(buf)->UnPack(res));
}
inline eastl::unique_ptr<Flat::Bigfoot::BenchAssetRootT> UnPackSizePrefixedBenchAssetRoot(
const void *buf,
const ::flatbuffers::resolver_function_t *res = nullptr) {
return eastl::unique_ptr<Flat::Bigfoot::BenchAssetRootT>(GetSizePrefixedBenchAssetRoot(buf)->UnPack(res));
}
} // namespace Bigfoot
} // namespace Flat
namespace flatbuffers {
#ifndef FLATBUFFERS_BIGFOOT_REF_HARDREFERENCEBENCHASSETLEAF
#define FLATBUFFERS_BIGFOOT_REF_HARDREFERENCEBENCHASSETLEAF
inline Flat::Bigfoot::HardReference PackHardReferenceBenchAssetLeaf(const ::Bigfoot::HardReference<::Bigfoot::BenchAssetLeaf>& p_asset) { return {Pack(p_asset.GetUUID())}; }
inline ::Bigfoot::HardReference<::Bigfoot::BenchAssetLeaf> UnPackHardReferenceBenchAssetLeaf(const Flat::Bigfoot::HardReference& p_asset) { return {UnPack(p_asset.uuid())}; }
#endif // FLATBUFFERS_BIGFOOT_REF_HARDREFERENCEBENCHASSETLEAF
} // namespace flatbuffers
#endif // FLATBUFFERS_GENERATED_BENCHASSETROOT_FLAT_BIGFOOT_H_
+1
View File
@@ -0,0 +1 @@
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/AssetsForBenchmarking)
-1
View File
@@ -1 +0,0 @@
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/SasquatchBaker)
@@ -1,11 +0,0 @@
get_filename_component(PackageName ${CMAKE_CURRENT_SOURCE_DIR} NAME)
project(${PackageName}Benchmarks)
bigfoot_create_package_benchmarks(
"Tools")
target_link_libraries(${PROJECT_NAME}
PRIVATE
$<LINK_LIBRARY:WHOLE_ARCHIVE,AssetsForBenchmarking>)
bigfoot_create_bigfile("Benchmarks/Tools")
@@ -1,22 +0,0 @@
// AUTO-GENERATED DO NOT TOUCH
/*********************************************************************
* \file BigFileInfo.generated.hpp
*
*********************************************************************/
#ifndef BIGFOOT_SASQUATCHBAKERBENCHMARKS_BIGFILEINFO_GENERATED_HPP
#define BIGFOOT_SASQUATCHBAKERBENCHMARKS_BIGFILEINFO_GENERATED_HPP
#include <EASTL/string_view.h>
#if defined BIGFOOT_TOOLMODE
namespace Bigfoot
{
/*
* BigFile location
*/
constexpr eastl::string_view BIGFILE_SASQUATCHBAKERBENCHMARKS_LOCATION {"./SasquatchBakerBenchmarks.db"};
} // namespace Bigfoot
#endif
#endif
@@ -1,87 +0,0 @@
/*********************************************************************
* \file SasquatchBaker.cpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#include <SasquatchBaker/SasquatchBaker.hpp>
#include <Engine/BigFile/BigFile.hpp>
#include <Engine/Sasquatch/Sasquatch.hpp>
#include <System/File/File.hpp>
#include <AssetsForBenchmarking/Asset/BenchmarkAsset.hpp>
#include <SasquatchBakerBenchmarks/BigFileInfo_generated.hpp>
#include <EASTL/vector.h>
#include <benchmark/benchmark.h>
#include <cstddef>
#include <cstdint>
namespace Bigfoot
{
class SasquatchBakerFixture: public benchmark::Fixture
{
protected:
static constexpr std::uint32_t ASSET_COUNT = 20'000;
static constexpr std::size_t ASSET_PAYLOAD_SIZE = 256;
void SetUp(benchmark::State&) override
{
m_bigFile.Clear();
m_outputFile.Delete();
eastl::vector<UUID> uuids;
eastl::vector<eastl::vector<std::byte>> flatbuffers;
eastl::vector<eastl::vector<UUID>> hardReferences;
uuids.reserve(ASSET_COUNT);
flatbuffers.reserve(ASSET_COUNT);
hardReferences.reserve(ASSET_COUNT);
for (std::uint32_t i = 0; i < ASSET_COUNT; ++i)
{
BenchmarkAsset asset;
asset.SetName(asset.GetHeader().uuid.ToString());
asset.GetAsset().payload.assign(ASSET_PAYLOAD_SIZE, std::uint8_t {0xAB});
const eastl::vector<std::byte> flatbuffer = asset.Pack();
flatbuffers.push_back(flatbuffer);
hardReferences.push_back(asset.GetHardReferences(flatbuffer));
uuids.push_back(asset.GetHeader().uuid);
}
m_bigFile.SaveAssetFlatbuffers(uuids, flatbuffers, hardReferences);
}
void TearDown(benchmark::State&) override
{
m_outputFile.Delete();
}
static constexpr eastl::string_view SASQUATCH_SASQUATCHBAKERBENCHMARKS_LOCATION {
"./SasquatchBakerBenchmarks.sasquatch"};
BigFile m_bigFile {File {BIGFILE_SASQUATCHBAKERBENCHMARKS_LOCATION}};
File m_outputFile {SASQUATCH_SASQUATCHBAKERBENCHMARKS_LOCATION};
};
/****************************************************************************************/
BENCHMARK_DEFINE_F(SasquatchBakerFixture, Bake)(benchmark::State& state)
{
std::int64_t bytesProcessed = 0;
for (auto _: state)
{
Bake(m_bigFile, m_outputFile);
bytesProcessed += static_cast<std::int64_t>(m_outputFile.GetSize());
}
state.SetItemsProcessed(state.iterations() * ASSET_COUNT);
state.SetBytesProcessed(bytesProcessed);
}
BENCHMARK_REGISTER_F(SasquatchBakerFixture, Bake)->Name("SasquatchBaker/Bake")->Unit(benchmark::kMillisecond);
} // namespace Bigfoot
@@ -1,92 +0,0 @@
/*********************************************************************
* \file Graph.cpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#include <Utils/Algorithm/Graph.hpp>
#include <ankerl/unordered_dense.h>
#include <benchmark/benchmark.h>
#include <effolkronium/random.hpp>
#include <cstdint>
namespace Bigfoot
{
using Random = effolkronium::random_static;
class LayeredTopologicalSortFixture: public benchmark::Fixture
{
protected:
/*
* Nodes are grouped into layers, and every node in a layer depends on up to DEPENDENCY_MAX_COUNT
* distinct nodes randomly picked from the previous layer.
*/
void SetUp(benchmark::State& state) override
{
Random::seed(0x5EEDU);
const auto nodeCount = static_cast<std::uint32_t>(state.range(0));
m_graph.clear();
m_nodes.clear();
m_graph.reserve(nodeCount);
m_nodes.reserve(nodeCount);
for (std::uint32_t node = 0; node < nodeCount; ++node)
{
m_nodes.insert(node);
eastl::vector<std::uint32_t>& edges = m_graph[node];
const std::uint32_t layerStart = (node / NODE_COUNT_PER_LAYER) * NODE_COUNT_PER_LAYER;
if (layerStart == 0)
{
continue;
}
const std::uint32_t previousLayerStart = layerStart - NODE_COUNT_PER_LAYER;
ankerl::unordered_dense::set<std::uint32_t> picked;
for (std::uint32_t i = 0; i < DEPENDENCY_MAX_COUNT; ++i)
{
const std::uint32_t dependency = Random::get<std::uint32_t>(previousLayerStart, layerStart - 1);
if (picked.insert(dependency).second)
{
edges.push_back(dependency);
}
}
}
}
static constexpr std::uint32_t NODE_COUNT_PER_LAYER = 64;
static constexpr std::uint32_t DEPENDENCY_MAX_COUNT = 4;
ankerl::unordered_dense::map<std::uint32_t, eastl::vector<std::uint32_t>> m_graph;
ankerl::unordered_dense::set<std::uint32_t> m_nodes;
};
/****************************************************************************************/
BENCHMARK_DEFINE_F(LayeredTopologicalSortFixture, LayeredTopologicalSortBench)(benchmark::State& state)
{
const auto nodeCount = static_cast<std::uint32_t>(state.range(0));
for (auto _: state)
{
eastl::vector<eastl::vector<std::uint32_t>> waves;
eastl::vector<std::uint32_t> cyclicNodes;
benchmark::DoNotOptimize(LayeredTopologicalSort(m_graph, m_nodes, waves, cyclicNodes));
benchmark::DoNotOptimize(waves);
benchmark::ClobberMemory();
}
state.SetItemsProcessed(state.iterations() * nodeCount);
}
BENCHMARK_REGISTER_F(LayeredTopologicalSortFixture, LayeredTopologicalSortBench)
->Name("Bigfoot::LayeredTopologicalSort")
->Arg(512)
->Arg(8192)
->Arg(262'144)
->Arg(4'194'304);
} // namespace Bigfoot
@@ -2,17 +2,16 @@
#include <ankerl/unordered_dense.h>
#include <benchmark/benchmark.h>
#include <effolkronium/random.hpp>
#include <algorithm>
#include <cstdint>
#include <random>
#include <type_traits>
#include <utility>
#include <vector>
namespace Bigfoot
{
using Random = effolkronium::random_static;
struct MyComplexStruct
{
std::uint32_t m_actualValueForBench = 0; // field summed by the Iterate benchmark
@@ -138,7 +137,6 @@ void Insert(benchmark::State& state)
}
state.SetItemsProcessed(state.iterations() * count);
state.SetBytesProcessed(static_cast<std::int64_t>(state.iterations() * count * sizeof(MyComplexStruct)));
}
template<class ADAPTOR>
@@ -149,7 +147,7 @@ void Remove(benchmark::State& state)
ADAPTOR adaptor;
std::vector<typename ADAPTOR::Key> keys;
keys.reserve(count);
Random::seed(0x5EEDU);
std::mt19937 rng(0x5EEDU);
for (auto _: state)
{
@@ -160,7 +158,7 @@ void Remove(benchmark::State& state)
{
keys.push_back(adaptor.Add(MyComplexStruct {i}));
}
Random::shuffle(keys);
std::shuffle(keys.begin(), keys.end(), rng);
state.ResumeTiming();
for (const typename ADAPTOR::Key key: keys)
@@ -172,7 +170,6 @@ void Remove(benchmark::State& state)
}
state.SetItemsProcessed(state.iterations() * count);
state.SetBytesProcessed(static_cast<std::int64_t>(state.iterations() * count * sizeof(MyComplexStruct)));
}
template<class ADAPTOR>
@@ -187,8 +184,8 @@ void Get(benchmark::State& state)
{
keys.push_back(adaptor.Add(MyComplexStruct {i}));
}
Random::seed(0x5EEDU);
Random::shuffle(keys);
std::mt19937 rng(0x5EEDU);
std::shuffle(keys.begin(), keys.end(), rng);
for (auto _: state)
{
@@ -204,7 +201,6 @@ void Get(benchmark::State& state)
}
state.SetItemsProcessed(state.iterations() * count);
state.SetBytesProcessed(static_cast<std::int64_t>(state.iterations() * count * sizeof(MyComplexStruct)));
}
template<class ADAPTOR>
@@ -230,7 +226,6 @@ void Iterate(benchmark::State& state)
}
state.SetItemsProcessed(state.iterations() * count);
state.SetBytesProcessed(static_cast<std::int64_t>(state.iterations() * count * sizeof(MyComplexStruct)));
}
#define BIGFOOT_REGISTER_BENCHMARKS(ADAPTOR, NAME) \
+1 -1
View File
@@ -1,6 +1,6 @@
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/Utils)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/System)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/Engine)
if(${TOOLMODE})
if(BIGFOOT_TOOL_MODE)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/Tools)
endif()
@@ -1,63 +0,0 @@
/*********************************************************************
* \file AssetBase.cpp
*
* \author Romain BOULLARD
* \date April 2026
*********************************************************************/
#include <Engine/Asset/AssetBase.hpp>
#include <Engine/Asset/Reference_generated.hpp>
namespace Bigfoot
{
void AssetBase::HardReferenceVisitor::Field([[maybe_unused]] size_t p_fieldIndex,
[[maybe_unused]] size_t p_setIndex,
const flatbuffers::ElementaryType p_type,
const bool p_isVector,
const flatbuffers::TypeTable* p_typeTable,
[[maybe_unused]] const char* p_name,
const std::uint8_t* p_val)
{
if (!p_isVector)
{
Visit(p_type, p_typeTable, p_val);
}
}
/****************************************************************************************/
void AssetBase::HardReferenceVisitor::Element([[maybe_unused]] size_t p_index,
const flatbuffers::ElementaryType p_type,
const flatbuffers::TypeTable* p_typeTable,
const std::uint8_t* p_val)
{
Visit(p_type, p_typeTable, p_val);
}
/****************************************************************************************/
eastl::vector<UUID> AssetBase::HardReferenceVisitor::GetUUIDs() const
{
eastl::vector<UUID> uuids;
uuids.reserve(m_uuids.size());
std::copy(m_uuids.begin(), m_uuids.end(), eastl::back_inserter(uuids));
return uuids;
}
/****************************************************************************************/
void AssetBase::HardReferenceVisitor::Visit(const flatbuffers::ElementaryType p_type,
const flatbuffers::TypeTable* p_typeTable,
const std::uint8_t* p_val)
{
if (p_val == nullptr || p_type != flatbuffers::ET_SEQUENCE ||
p_typeTable != ::Flat::Bigfoot::HardReference::MiniReflectTypeTable())
{
return;
}
const ::Flat::Bigfoot::HardReference* hardReference = reinterpret_cast<const ::Flat::Bigfoot::HardReference*>(
p_val);
m_uuids.emplace(flatbuffers::UnPack(hardReference->uuid()));
}
} // namespace Bigfoot
+18 -161
View File
@@ -5,184 +5,41 @@
* \date August 2026
*********************************************************************/
#include <Engine/Asset/AssetContainer.hpp>
#include <Engine/Asset/AssetLibrary.hpp>
namespace Bigfoot
{
ContainerHandle::ContainerHandle(AssetContainerBase* p_assetContainer, const UUID p_uuid):
m_assetContainer(p_assetContainer),
m_uuid(p_uuid)
AssetContainerBase::Entry& AssetContainerBase::GetOrCreateEntry(const UUID& p_uuid)
{
}
const std::lock_guard lock {m_entryMutex};
/****************************************************************************************/
std::uint32_t ContainerHandle::GetHardRefCount() const
{
return m_hardRefCount;
}
/****************************************************************************************/
void ContainerHandle::IncrementHardRefCount()
{
if (m_hardRefCount == 0)
auto [it, inserted] = m_uuidToEntry.try_emplace(p_uuid);
if (inserted)
{
m_assetContainer->CancelRemoval(m_uuid);
it->second.m_handleSlotKey = m_handles.Insert();
if (m_owningLibrary != nullptr)
{
m_owningLibrary->RegisterUUID(p_uuid, this);
}
}
++m_hardRefCount;
}
/****************************************************************************************/
void ContainerHandle::DecrementHardRefCount()
{
--m_hardRefCount;
if (m_hardRefCount == 0)
{
m_assetContainer->QueueForRemoval(m_uuid);
}
}
/****************************************************************************************/
std::uint32_t ContainerHandle::GetSoftRefCount() const
{
return m_softRefCount;
}
/****************************************************************************************/
void ContainerHandle::IncrementSoftRefCount()
{
++m_softRefCount;
}
/****************************************************************************************/
void ContainerHandle::DecrementSoftRefCount()
{
--m_softRefCount;
}
/****************************************************************************************/
AssetBase* ContainerHandle::GetAsset()
{
return m_asset;
}
/****************************************************************************************/
const AssetBase* ContainerHandle::GetAsset() const
{
return m_asset;
}
/****************************************************************************************/
void ContainerHandle::SetAsset(AssetBase* p_asset)
{
m_asset = p_asset;
}
/****************************************************************************************/
bool AssetContainerBase::Has(const UUID& p_uuid) const
{
const Entry* entry = FindEntry(p_uuid);
return entry != nullptr && entry->m_assetSlotKey.Valid();
}
/****************************************************************************************/
ContainerHandle* AssetContainerBase::AcquireContainerHandle(const UUID& p_uuid)
{
return GetOrCreateEntry(p_uuid).m_handle;
return it->second;
}
/****************************************************************************************/
void AssetContainerBase::Remove(const UUID& p_uuid)
{
if (Entry* entry = FindEntry(p_uuid); entry != nullptr)
if (Entry* entry = FindEntry(p_uuid); entry != nullptr && entry->m_handleSlotKey.Valid())
{
if (entry->m_assetSlotKey.Valid())
{
entry->m_handle->SetAsset(nullptr);
RemoveAsset(entry->m_assetSlotKey);
entry->m_assetSlotKey = SlotMapKey {};
}
m_handles.Remove(entry->m_handleSlotKey);
m_uuidToEntry.erase(p_uuid);
if (m_owningLibrary != nullptr)
{
m_owningLibrary->UnregisterUUID(p_uuid);
}
}
}
/****************************************************************************************/
void AssetContainerBase::QueueForRemoval(const UUID& p_uuid)
{
m_pendingRemovalUuids.push_back(p_uuid);
}
/****************************************************************************************/
void AssetContainerBase::CancelRemoval(const UUID& p_uuid)
{
auto* const pendingIt = eastl::find(m_pendingRemovalUuids.begin(), m_pendingRemovalUuids.end(), p_uuid);
if (pendingIt != m_pendingRemovalUuids.end())
{
m_pendingRemovalUuids.erase_unsorted(pendingIt);
}
}
/****************************************************************************************/
void AssetContainerBase::ProcessPendingRemovals()
{
eastl::vector<UUID> toRemove;
toRemove.swap(m_pendingRemovalUuids);
for (const UUID& uuid: toRemove)
{
Remove(uuid);
}
}
/****************************************************************************************/
bool AssetContainerBase::IsMarkedForRemove(const UUID& p_uuid)
{
return eastl::find(m_pendingRemovalUuids.begin(), m_pendingRemovalUuids.end(), p_uuid) !=
m_pendingRemovalUuids.end();
}
/****************************************************************************************/
AssetContainerBase::Entry* AssetContainerBase::FindEntry(const UUID& p_uuid)
{
const auto entryIt = m_uuidToEntry.find(p_uuid);
return entryIt != m_uuidToEntry.end() ? &entryIt->second : nullptr;
}
/****************************************************************************************/
const AssetContainerBase::Entry* AssetContainerBase::FindEntry(const UUID& p_uuid) const
{
const auto entryIt = m_uuidToEntry.find(p_uuid);
return entryIt != m_uuidToEntry.end() ? &entryIt->second : nullptr;
}
/****************************************************************************************/
AssetContainerBase::Entry& AssetContainerBase::GetOrCreateEntry(const UUID& p_uuid)
{
auto [entryIt, inserted] = m_uuidToEntry.try_emplace(p_uuid);
if (inserted)
{
entryIt->second.m_handleSlotKey = m_handles.Insert(this, p_uuid);
entryIt->second.m_handle = m_handles.Get(entryIt->second.m_handleSlotKey);
}
return entryIt->second;
}
} // namespace Bigfoot
@@ -8,77 +8,6 @@
namespace Bigfoot
{
AssetContainerBase* AssetLibrary::GetAssetContainer(const AssetTypeID p_typeID)
{
if (const auto container = m_assetContainers.find(p_typeID); container != m_assetContainers.end())
{
return container->second;
}
return nullptr;
}
/****************************************************************************************/
const AssetContainerBase* AssetLibrary::GetAssetContainer(const AssetTypeID p_typeID) const
{
if (const auto container = m_assetContainers.find(p_typeID); container != m_assetContainers.end())
{
return container->second;
}
return nullptr;
}
/****************************************************************************************/
const AssetContainerBase* AssetLibrary::GetAssetContainer(const UUID& p_uuid) const
{
for (const auto [typeId, container]: m_assetContainers)
{
if (container->Has(p_uuid))
{
return container;
}
}
return nullptr;
}
/****************************************************************************************/
AssetContainerBase* AssetLibrary::GetAssetContainer(const UUID& p_uuid)
{
for (const auto [typeId, container]: m_assetContainers)
{
if (container->Has(p_uuid))
{
return container;
}
}
return nullptr;
}
/****************************************************************************************/
bool AssetLibrary::IsLoaded(const UUID& p_uuid) const
{
return GetAssetContainer(p_uuid) != nullptr;
}
/****************************************************************************************/
void AssetLibrary::ProcessPendingRemovals()
{
for (const auto [typeId, container]: m_assetContainers)
{
container->ProcessPendingRemovals();
}
}
/****************************************************************************************/
void SetActiveAssetLibrary(AssetLibrary* p_assetLibrary)
{
g_activeAssetLibrary = p_assetLibrary;
@@ -0,0 +1,479 @@
/*********************************************************************
* \file SerializationManager.cpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#include <Engine/Asset/SerializationManager.hpp>
#include <Engine/EngineAssertHandler.hpp>
#include <tuple>
#if defined BIGFOOT_TOOL
#include <Engine/Asset/Reference_generated.hpp>
#include <flatbuffers/minireflect.h>
#else
#include <algorithm>
#include <thread>
#include <vector>
#endif
namespace Bigfoot
{
#if defined BIGFOOT_TOOL
namespace
{
// Walks a flatbuffer generically (via its TypeTable) and collects the UUID of every HardReference and
// SoftReference struct field it finds, at any depth, sorted into the two lists by comparing each field's
// TypeTable against Flat::Bigfoot::HardReferenceTypeTable()/SoftReferenceTypeTable().
//
// This works uniformly for scalar reference fields, vectors of references, and polymorphic
// HardReference<AssetBase>/SoftReference<AssetBase> fields alike: on the wire, a HardReference (or
// SoftReference) is always the same { uuid: UUID } struct regardless of the C++ template argument used on
// the native side.
class ReferenceCollector: public flatbuffers::IterationVisitor
{
public:
void Field(const std::size_t /*p_fieldIndex*/,
const std::size_t /*p_setIndex*/,
const flatbuffers::ElementaryType p_type,
const bool p_isVector,
const flatbuffers::TypeTable* p_typeTable,
const char* /*p_name*/,
const std::uint8_t* p_value) override
{
// Vector fields are handled per-element via Element(): p_value here would only be the address
// of the field's offset slot, not usable as reference data.
if (!p_isVector)
{
TryCollect(p_type, p_typeTable, p_value);
}
}
void Element(const std::size_t /*p_index*/,
const flatbuffers::ElementaryType p_type,
const flatbuffers::TypeTable* p_typeTable,
const std::uint8_t* p_value) override
{
TryCollect(p_type, p_typeTable, p_value);
}
SerializationManager::ReferenceUUIDs m_references;
private:
void TryCollect(const flatbuffers::ElementaryType p_type,
const flatbuffers::TypeTable* p_typeTable,
const std::uint8_t* p_value)
{
if (p_value == nullptr || p_type != flatbuffers::ET_SEQUENCE)
{
return;
}
// A HardReference/SoftReference struct is exactly a UUID (uuid: UUID, at struct offset 0), stored
// inline: its address directly holds the 16 raw UUID bytes.
const std::span<const std::byte, UUID::UUID_BYTE_SIZE> uuidBytes {
reinterpret_cast<const std::byte*>(p_value), UUID::UUID_BYTE_SIZE};
if (p_typeTable == ::Flat::Bigfoot::HardReferenceTypeTable())
{
m_references.m_hard.push_back(UUID {uuidBytes});
}
else if (p_typeTable == ::Flat::Bigfoot::SoftReferenceTypeTable())
{
m_references.m_soft.push_back(UUID {uuidBytes});
}
}
};
} // namespace
SerializationManager::SerializationManager(BigFile& p_bigFile):
m_bigFile(&p_bigFile)
{
}
#else
SerializationManager::SerializationManager(Sasquatch& p_sasquatch):
m_sasquatch(&p_sasquatch)
{
}
#endif
/****************************************************************************************/
void SerializationManager::LoadAsset(const UUID& p_uuid)
{
if (!p_uuid)
{
return;
}
#if defined BIGFOOT_TOOL
BigFile::Request assetRequest {*m_bigFile,
"SELECT AssetHeader.TypeID, Asset.Asset FROM Asset "
"JOIN AssetHeader ON AssetHeader.UUID = Asset.UUID "
"WHERE Asset.UUID = ?1;"};
assetRequest.Bind(1, UUIDToSpan(p_uuid));
if (!assetRequest.Step())
{
return;
}
const AssetTypeID typeID = assetRequest.Get(0);
const eastl::span<const std::byte> bytes = assetRequest.Get(1);
#else
const std::optional<Sasquatch::Entry> entry = m_sasquatch->Find(p_uuid);
if (!entry.has_value())
{
return;
}
const AssetTypeID typeID = entry->m_typeID;
const eastl::span<const std::byte> bytes = entry->m_bytes;
#endif
AssetContainerBase* container = g_activeAssetLibrary->GetAssetContainer(typeID);
CRITICAL_ASSERT(EngineAssertHandler, container != nullptr, "No AssetContainer registered for AssetTypeID {}", typeID);
if (container->Has(p_uuid) || !m_loadingInProgress.insert(p_uuid).second)
{
// Already loaded, or a HardReference cycle led back to a UUID an ancestor call is still loading:
// skip either way, the ancestor call already owns finishing that load.
return;
}
// Load every hard-referenced asset before constructing this one, so that this asset's own reference
// fields - notably HardReference<AssetBase>/SoftReference<AssetBase>, which can only resolve to an
// already-loaded asset - find their target immediately when they're constructed below.
#if defined BIGFOOT_TOOL
BigFile::Request hardReferenceRequest {*m_bigFile, "SELECT ReferencedUUID FROM AssetHardReference WHERE UUID = ?1;"};
hardReferenceRequest.Bind(1, UUIDToSpan(p_uuid));
while (hardReferenceRequest.Step())
{
LoadAsset(SpanToUUID(hardReferenceRequest.Get(0)));
}
#else
for (std::size_t offset = 0; offset < entry->m_hardReferenceUUIDs.size(); offset += UUID::UUID_BYTE_SIZE)
{
LoadAsset(SpanToUUID(entry->m_hardReferenceUUIDs.subspan(offset, UUID::UUID_BYTE_SIZE)));
}
#endif
container->Insert(bytes);
m_loadingInProgress.erase(p_uuid);
}
/****************************************************************************************/
#if !defined BIGFOOT_TOOL
namespace
{
struct PendingLoad
{
AssetContainerBase* m_container;
eastl::span<const std::byte> m_bytes;
eastl::vector<UUID> m_hardReferences;
};
// Decodes every UUID in p_layer in parallel - striped across workers so per-item cost variance (a big
// texture next to a tiny material) balances out rather than one worker being left with all the heavy
// items - then registers them one at a time. See LoadAssets for why decode and register are split like
// this at all.
void DecodeAndInsertLayer(const ankerl::unordered_dense::segmented_map<UUID, PendingLoad>& p_pending,
const eastl::vector<UUID>& p_layer)
{
eastl::vector<eastl::unique_ptr<AssetBase>> decoded(p_layer.size());
const auto decodeOne = [&](const std::size_t p_index)
{
const PendingLoad& load = p_pending.find(p_layer[p_index])->second;
decoded[p_index] = load.m_container->DecodeOffContainer(load.m_bytes);
};
// Not worth spinning up threads to decode a handful of assets - the spawn/join overhead would dwarf
// the work itself.
static constexpr std::size_t PARALLEL_THRESHOLD = 32;
const std::size_t threadCount =
std::min<std::size_t>(std::max<unsigned int>(1, std::thread::hardware_concurrency()), p_layer.size());
if (threadCount <= 1 || p_layer.size() < PARALLEL_THRESHOLD)
{
for (std::size_t i = 0; i < p_layer.size(); ++i)
{
decodeOne(i);
}
}
else
{
std::vector<std::thread> workers;
workers.reserve(threadCount);
for (std::size_t worker = 0; worker < threadCount; ++worker)
{
workers.emplace_back(
[&, worker]()
{
for (std::size_t i = worker; i < p_layer.size(); i += threadCount)
{
decodeOne(i);
}
});
}
for (std::thread& worker: workers)
{
worker.join();
}
}
for (std::size_t i = 0; i < p_layer.size(); ++i)
{
p_pending.find(p_layer[i])->second.m_container->InsertDecoded(std::move(decoded[i]));
}
}
} // namespace
void SerializationManager::LoadAssets(const eastl::span<const UUID> p_uuids)
{
// Phase 1: discover every UUID that needs loading - p_uuids plus everything they transitively hold a
// HardReference to - skipping anything already loaded, exactly like LoadAsset's own recursion (just
// breadth-first with an explicit stack instead of the call stack, since layering needs the whole set
// up front).
ankerl::unordered_dense::segmented_map<UUID, PendingLoad> pending;
eastl::vector<UUID> toVisit {p_uuids.begin(), p_uuids.end()};
while (!toVisit.empty())
{
const UUID uuid = toVisit.back();
toVisit.pop_back();
if (!uuid || pending.contains(uuid))
{
continue;
}
const std::optional<Sasquatch::Entry> entry = m_sasquatch->Find(uuid);
if (!entry.has_value())
{
continue;
}
AssetContainerBase* container = g_activeAssetLibrary->GetAssetContainer(entry->m_typeID);
CRITICAL_ASSERT(EngineAssertHandler, container != nullptr, "No AssetContainer registered for AssetTypeID {}",
entry->m_typeID);
if (container->Has(uuid))
{
continue;
}
PendingLoad load;
load.m_container = container;
load.m_bytes = entry->m_bytes;
for (std::size_t offset = 0; offset < entry->m_hardReferenceUUIDs.size(); offset += UUID::UUID_BYTE_SIZE)
{
const UUID referenced = SpanToUUID(entry->m_hardReferenceUUIDs.subspan(offset, UUID::UUID_BYTE_SIZE));
load.m_hardReferences.push_back(referenced);
toVisit.push_back(referenced);
}
pending.emplace(uuid, std::move(load));
}
// Phase 2: repeatedly peel off a "ready" layer - every still-pending UUID whose HardReferences are all
// no longer pending (either outside this batch entirely and already loaded, or registered in an
// earlier layer this call) - decode that layer in parallel, then register it serially before moving to
// the next layer. A cycle within the batch would mean no UUID is ever ready; that should never happen
// (see LoadAssets' doc comment) so it asserts rather than looping forever.
ankerl::unordered_dense::segmented_set<UUID> remaining;
for (const auto& [uuid, load]: pending)
{
remaining.insert(uuid);
}
eastl::vector<UUID> layer;
while (!remaining.empty())
{
layer.clear();
for (const UUID& uuid: remaining)
{
bool ready = true;
for (const UUID& referenced: pending.find(uuid)->second.m_hardReferences)
{
if (remaining.contains(referenced))
{
ready = false;
break;
}
}
if (ready)
{
layer.push_back(uuid);
}
}
CRITICAL_ASSERT(EngineAssertHandler, !layer.empty(), "Cycle detected among HardReferences during LoadAssets");
DecodeAndInsertLayer(pending, layer);
for (const UUID& uuid: layer)
{
remaining.erase(uuid);
}
}
}
#endif
/****************************************************************************************/
#if defined BIGFOOT_TOOL
void SerializationManager::SaveAsset(const UUID& p_uuid)
{
if (!p_uuid)
{
return;
}
AssetContainerBase* container = g_activeAssetLibrary->FindContainerForUUID(p_uuid);
AssetBase* asset = container != nullptr ? container->AcquireContainerHandle(p_uuid)->GetAsset() : nullptr;
CRITICAL_ASSERT(EngineAssertHandler, asset != nullptr, "SaveAsset: no asset currently loaded for UUID {}", p_uuid);
const ::Flat::Bigfoot::AssetHeaderT& header = asset->GetHeader();
const eastl::vector<std::byte> bytes = asset->Pack();
const ::Flat::Bigfoot::Asset* outerAsset = flatbuffers::GetRoot<::Flat::Bigfoot::Asset>(bytes.data());
const ReferenceUUIDs references =
CollectReferenceUUIDs(outerAsset->inner_asset()->data(), asset->GetInnerTypeTable());
const eastl::span<const std::byte> uuidBytes = UUIDToSpan(header.uuid);
m_bigFile->BeginTransaction();
BigFile::Request headerRequest {*m_bigFile,
"INSERT INTO AssetHeader (UUID, Name, TypeID, TypeName, Version) "
"VALUES (?1, ?2, ?3, ?4, ?5) "
"ON CONFLICT(UUID) DO UPDATE SET "
"Name = excluded.Name, TypeID = excluded.TypeID, TypeName = excluded.TypeName, "
"Version = excluded.Version;"};
headerRequest.Bind(1, uuidBytes);
headerRequest.Bind(2, eastl::string_view {header.name.data(), header.name.size()});
headerRequest.Bind(3, header.type_id);
headerRequest.Bind(4, eastl::string_view {header.type_name.data(), header.type_name.size()});
headerRequest.Bind(5, header.version);
std::ignore = headerRequest.Execute();
BigFile::Request assetRequest {*m_bigFile,
"INSERT INTO Asset (UUID, Asset) VALUES (?1, ?2) "
"ON CONFLICT(UUID) DO UPDATE SET Asset = excluded.Asset;"};
assetRequest.Bind(1, uuidBytes);
assetRequest.Bind(2, eastl::span<const std::byte> {bytes.data(), bytes.size()});
std::ignore = assetRequest.Execute();
BigFile::Request clearHardReferencesRequest {*m_bigFile, "DELETE FROM AssetHardReference WHERE UUID = ?1;"};
clearHardReferencesRequest.Bind(1, uuidBytes);
std::ignore = clearHardReferencesRequest.Execute();
BigFile::Request clearSoftReferencesRequest {*m_bigFile, "DELETE FROM AssetSoftReference WHERE UUID = ?1;"};
clearSoftReferencesRequest.Bind(1, uuidBytes);
std::ignore = clearSoftReferencesRequest.Execute();
InsertReferences("INSERT INTO AssetHardReference (UUID, ReferencedUUID) VALUES (?1, ?2) ON CONFLICT DO NOTHING;",
uuidBytes,
references.m_hard);
InsertReferences("INSERT INTO AssetSoftReference (UUID, ReferencedUUID) VALUES (?1, ?2) ON CONFLICT DO NOTHING;",
uuidBytes,
references.m_soft);
m_bigFile->CommitTransaction();
}
/****************************************************************************************/
eastl::unique_ptr<::Flat::Bigfoot::AssetHeaderT> SerializationManager::GetAssetHeader(const UUID& p_uuid) const
{
if (!p_uuid)
{
return nullptr;
}
BigFile::Request request {*m_bigFile, "SELECT Name, TypeID, TypeName, Version FROM AssetHeader WHERE UUID = ?1;"};
request.Bind(1, UUIDToSpan(p_uuid));
if (!request.Step())
{
return nullptr;
}
auto header = eastl::make_unique<::Flat::Bigfoot::AssetHeaderT>();
header->uuid = p_uuid;
const eastl::string_view name = request.Get(0);
header->name = eastl::string {name.data(), name.size()};
header->type_id = request.Get(1);
const eastl::string_view typeName = request.Get(2);
header->type_name = eastl::string {typeName.data(), typeName.size()};
header->version = request.Get(3);
return header;
}
/****************************************************************************************/
#endif
void SerializationManager::RemoveUnreferencedAssets()
{
for (const UUID& uuid: g_activeAssetLibrary->TakeUnreferencedUUIDs())
{
AssetContainerBase* container = g_activeAssetLibrary->FindContainerForUUID(uuid);
if (container != nullptr && container->AcquireContainerHandle(uuid)->GetHardRefCount() == 0)
{
container->Remove(uuid);
}
}
}
/****************************************************************************************/
eastl::span<const std::byte> SerializationManager::UUIDToSpan(const UUID& p_uuid)
{
const std::span<const std::byte, UUID::UUID_BYTE_SIZE> bytes = p_uuid.ToBytes();
return eastl::span<const std::byte> {bytes.data(), bytes.size()};
}
/****************************************************************************************/
UUID SerializationManager::SpanToUUID(const eastl::span<const std::byte> p_bytes)
{
ASSERT(EngineAssertHandler, p_bytes.size() == UUID::UUID_BYTE_SIZE, "Invalid UUID blob size");
return UUID {std::span<const std::byte, UUID::UUID_BYTE_SIZE> {p_bytes.data(), UUID::UUID_BYTE_SIZE}};
}
#if defined BIGFOOT_TOOL
/****************************************************************************************/
SerializationManager::ReferenceUUIDs SerializationManager::CollectReferenceUUIDs(
const std::uint8_t* p_buffer, const flatbuffers::TypeTable* p_typeTable)
{
ReferenceCollector collector;
flatbuffers::IterateFlatBuffer(p_buffer, p_typeTable, &collector);
return std::move(collector.m_references);
}
/****************************************************************************************/
void SerializationManager::InsertReferences(const eastl::string_view p_insertQuery,
const eastl::span<const std::byte> p_uuidBytes,
const eastl::vector<UUID>& p_referencedUUIDs)
{
for (const UUID& referencedUUID: p_referencedUUIDs)
{
BigFile::Request request {*m_bigFile, p_insertQuery};
request.Bind(1, p_uuidBytes);
request.Bind(2, UUIDToSpan(referencedUUID));
std::ignore = request.Execute();
}
}
#endif
} // namespace Bigfoot
+83 -582
View File
@@ -8,13 +8,8 @@
#include <Engine/EngineAssertHandler.hpp>
#include <Utils/Format.hpp>
#include <EASTL/algorithm.h>
namespace Bigfoot
{
#ifdef BIGFOOT_TOOLMODE
BigFile::BigFile(const File& p_file)
{
[[maybe_unused]]
@@ -25,19 +20,12 @@ BigFile::BigFile(const File& p_file)
p_file.ToAbsolute().GetPath(),
sqlite3_errmsg(m_db));
// Lets SQLite read pages straight out of a memory mapping instead of read()-ing them into its own page
// cache - a plain read-perf win for the Asset blob column in particular, which dwarfs every other
// column. Not persisted (PRAGMA, not a schema change): every connection has to set this itself.
[[maybe_unused]]
const int synchronousResult = sqlite3_exec(m_db, "PRAGMA synchronous = NORMAL;", nullptr, nullptr, nullptr);
ASSERT(EngineAssertHandler,
synchronousResult == SQLITE_OK,
"Failed to set synchronous pragma: {}",
sqlite3_errmsg(m_db));
[[maybe_unused]]
const int foreignKeysResult = sqlite3_exec(m_db, "PRAGMA foreign_keys = ON;", nullptr, nullptr, nullptr);
ASSERT(EngineAssertHandler,
foreignKeysResult == SQLITE_OK,
"Failed to set foreign_keys pragma: {}",
sqlite3_errmsg(m_db));
const int mmapResult = sqlite3_exec(m_db, "PRAGMA mmap_size=1073741824;", nullptr, nullptr, nullptr);
ASSERT(EngineAssertHandler, mmapResult == SQLITE_OK, "Failed to set BigFile mmap_size: {}", sqlite3_errmsg(m_db));
}
/****************************************************************************************/
@@ -69,21 +57,19 @@ void BigFile::RollbackTransaction()
/****************************************************************************************/
void BigFile::Clear()
{
BeginTransaction();
BigFile::Request {*this, "DELETE FROM AssetHardReference;"}.Execute();
BigFile::Request {*this, "DELETE FROM Asset;"}.Execute();
BigFile::Request {*this, "DELETE FROM AssetMetadata;"}.Execute();
CommitTransaction();
}
/****************************************************************************************/
BigFile::~BigFile()
{
for (auto& [query, statements]: m_statementPool)
{
for (sqlite3_stmt* const statement: statements)
{
[[maybe_unused]]
const int result = sqlite3_finalize(statement);
CRITICAL_ASSERT(EngineAssertHandler, result == SQLITE_OK, "Failed to finalize pooled statement: {}",
sqlite3_errmsg(m_db));
}
}
[[maybe_unused]]
const int result = sqlite3_close_v2(m_db);
CRITICAL_ASSERT(EngineAssertHandler, result == SQLITE_OK, "Failed to close BigFile DB: {}", sqlite3_errmsg(m_db));
@@ -91,527 +77,52 @@ BigFile::~BigFile()
/****************************************************************************************/
eastl::vector<std::byte> BigFile::GetAssetFlatbuffer(const UUID& p_uuid) const
sqlite3_stmt* BigFile::AcquireStatement(const eastl::string_view p_query) const
{
BigFile::Request assetRequest {*this, "SELECT Asset FROM Asset WHERE UUID = ?1;"};
assetRequest.Bind(1, p_uuid.ToBytes());
if (!assetRequest.Step())
eastl::vector<sqlite3_stmt*>& pool = m_statementPool[std::string {p_query.data(), p_query.size()}];
if (!pool.empty())
{
return {};
sqlite3_stmt* const statement = pool.back();
pool.pop_back();
[[maybe_unused]]
const int resetResult = sqlite3_reset(statement);
ASSERT(EngineAssertHandler, resetResult == SQLITE_OK, "Failed to reset pooled statement: {}",
sqlite3_errmsg(m_db));
[[maybe_unused]]
const int clearResult = sqlite3_clear_bindings(statement);
ASSERT(EngineAssertHandler, clearResult == SQLITE_OK, "Failed to clear pooled statement bindings: {}",
sqlite3_errmsg(m_db));
return statement;
}
const eastl::span<const std::byte> flatbuffer = assetRequest.Get(0);
return eastl::vector<std::byte> {flatbuffer.begin(), flatbuffer.end()};
sqlite3_stmt* statement = nullptr;
[[maybe_unused]]
const int result =
sqlite3_prepare_v2(m_db, p_query.data(), static_cast<std::uint32_t>(p_query.size()), &statement, nullptr);
CRITICAL_ASSERT(EngineAssertHandler, result == SQLITE_OK, "Failed to create statement from BigFile DB: {}",
sqlite3_errmsg(m_db));
return statement;
}
/****************************************************************************************/
eastl::vector<eastl::vector<std::byte>> BigFile::GetAssetFlatbuffers(const eastl::span<const UUID> p_uuids) const
void BigFile::ReleaseStatement(const eastl::string_view p_query, sqlite3_stmt* const p_statement) const
{
eastl::vector<eastl::vector<std::byte>> flatbuffers(p_uuids.size());
if (p_uuids.empty())
{
return flatbuffers;
}
ankerl::unordered_dense::map<UUID, eastl::vector<std::size_t>> uuidToIndices;
uuidToIndices.reserve(p_uuids.size());
for (std::size_t i = 0; i < p_uuids.size(); ++i)
{
uuidToIndices[p_uuids[i]].push_back(i);
}
eastl::vector<UUID> distinctUUIDs;
distinctUUIDs.reserve(uuidToIndices.size());
for (const auto& [uuid, indices]: uuidToIndices)
{
distinctUUIDs.push_back(uuid);
}
constexpr std::size_t PARAMS_PER_ROW = 1;
const std::size_t maxRowsPerStatement = static_cast<std::size_t>(
sqlite3_limit(m_db, SQLITE_LIMIT_VARIABLE_NUMBER, -1)) /
PARAMS_PER_ROW;
for (std::size_t chunkStart = 0; chunkStart < distinctUUIDs.size(); chunkStart += maxRowsPerStatement)
{
const std::size_t chunkSize = eastl::min(maxRowsPerStatement, distinctUUIDs.size() - chunkStart);
eastl::string placeholders;
placeholders.reserve(chunkSize * 2);
for (std::size_t i = 0; i < chunkSize; ++i)
{
if (i > 0)
{
placeholders += ", ";
}
placeholders += "?";
}
const eastl::string sql = Format("SELECT UUID, Asset FROM Asset WHERE UUID IN ({});", placeholders);
BigFile::Request assetRequest {*this, sql};
for (std::size_t i = 0; i < chunkSize; ++i)
{
assetRequest.Bind(static_cast<std::uint32_t>(i + 1), distinctUUIDs[chunkStart + i].ToBytes());
}
while (assetRequest.Step())
{
const std::span<const std::byte, UUID::UUID_BYTE_SIZE> rawUUID {
static_cast<std::span<const std::byte>>(assetRequest.Get(0))};
const UUID uuid {rawUUID};
const auto indicesIt = uuidToIndices.find(uuid);
ASSERT(EngineAssertHandler,
indicesIt != uuidToIndices.end(),
"Returned UUID {} not found in requested UUIDs",
uuid);
const eastl::span<const std::byte> flatbuffer = assetRequest.Get(1);
for (const std::size_t index: indicesIt->second)
{
flatbuffers[index].assign(flatbuffer.begin(), flatbuffer.end());
}
}
}
return flatbuffers;
}
/****************************************************************************************/
eastl::vector<UUID> BigFile::GetAssetHardReferences(const UUID& p_uuid) const
{
BigFile::Request hardReferencesRequest {*this, "SELECT HardReference FROM AssetHardReference WHERE UUID = ?1;"};
hardReferencesRequest.Bind(1, p_uuid.ToBytes());
eastl::vector<UUID> hardReferences;
while (hardReferencesRequest.Step())
{
const std::span<const std::byte, UUID::UUID_BYTE_SIZE> rawUUID {
static_cast<std::span<const std::byte>>(hardReferencesRequest.Get(0))};
hardReferences.emplace_back(rawUUID);
}
return hardReferences;
}
/****************************************************************************************/
eastl::vector<eastl::vector<UUID>> BigFile::GetAssetsHardReferences(eastl::span<const UUID> p_uuids) const
{
eastl::vector<eastl::vector<UUID>> hardReferences(p_uuids.size());
if (p_uuids.empty())
{
return hardReferences;
}
ankerl::unordered_dense::map<UUID, eastl::vector<std::size_t>> uuidToIndices;
uuidToIndices.reserve(p_uuids.size());
for (std::size_t i = 0; i < p_uuids.size(); ++i)
{
uuidToIndices[p_uuids[i]].push_back(i);
}
eastl::vector<UUID> distinctUUIDs;
distinctUUIDs.reserve(uuidToIndices.size());
for (const auto& [uuid, indices]: uuidToIndices)
{
distinctUUIDs.push_back(uuid);
}
constexpr std::size_t PARAMS_PER_ROW = 1;
const std::size_t maxRowsPerStatement = static_cast<std::size_t>(
sqlite3_limit(m_db, SQLITE_LIMIT_VARIABLE_NUMBER, -1)) /
PARAMS_PER_ROW;
for (std::size_t chunkStart = 0; chunkStart < distinctUUIDs.size(); chunkStart += maxRowsPerStatement)
{
const std::size_t chunkSize = eastl::min(maxRowsPerStatement, distinctUUIDs.size() - chunkStart);
eastl::string placeholders;
placeholders.reserve(chunkSize * 2);
for (std::size_t i = 0; i < chunkSize; ++i)
{
if (i > 0)
{
placeholders += ", ";
}
placeholders += "?";
}
const eastl::string sql = Format("SELECT UUID, HardReference FROM AssetHardReference WHERE UUID IN ({});",
placeholders);
BigFile::Request hardReferencesRequest {*this, sql};
for (std::size_t i = 0; i < chunkSize; ++i)
{
hardReferencesRequest.Bind(static_cast<std::uint32_t>(i + 1), distinctUUIDs[chunkStart + i].ToBytes());
}
while (hardReferencesRequest.Step())
{
const std::span<const std::byte, UUID::UUID_BYTE_SIZE> rawOwnerUUID {
static_cast<std::span<const std::byte>>(hardReferencesRequest.Get(0))};
const UUID ownerUUID {rawOwnerUUID};
const auto indicesIt = uuidToIndices.find(ownerUUID);
ASSERT(EngineAssertHandler,
indicesIt != uuidToIndices.end(),
"Returned UUID {} not found in requested UUIDs",
ownerUUID);
const std::span<const std::byte, UUID::UUID_BYTE_SIZE> rawHardReference {
static_cast<std::span<const std::byte>>(hardReferencesRequest.Get(1))};
for (const std::size_t index: indicesIt->second)
{
hardReferences[index].emplace_back(rawHardReference);
}
}
}
return hardReferences;
}
/****************************************************************************************/
eastl::optional<BigFile::AssetTime> BigFile::GetAssetTime(const UUID& p_uuid) const
{
BigFile::Request timeRequest {*this, "SELECT CreateTime, ModificationTime FROM AssetMetadata WHERE UUID = ?1;"};
timeRequest.Bind(1, p_uuid.ToBytes());
if (!timeRequest.Step())
{
return eastl::nullopt;
}
return AssetTime {.m_creationTime = Time {static_cast<std::uint64_t>(timeRequest.Get(0))},
.m_modificationTime = Time {static_cast<std::uint64_t>(timeRequest.Get(1))}};
}
/****************************************************************************************/
eastl::vector<eastl::optional<BigFile::AssetTime>> BigFile::GetAssetsTime(const eastl::span<const UUID> p_uuids) const
{
eastl::vector<eastl::optional<AssetTime>> times(p_uuids.size());
if (p_uuids.empty())
{
return times;
}
ankerl::unordered_dense::map<UUID, eastl::vector<std::size_t>> uuidToIndices;
uuidToIndices.reserve(p_uuids.size());
for (std::size_t i = 0; i < p_uuids.size(); ++i)
{
uuidToIndices[p_uuids[i]].push_back(i);
}
eastl::vector<UUID> distinctUUIDs;
distinctUUIDs.reserve(uuidToIndices.size());
for (const auto& [uuid, indices]: uuidToIndices)
{
distinctUUIDs.push_back(uuid);
}
constexpr std::size_t PARAMS_PER_ROW = 1;
const std::size_t maxRowsPerStatement = static_cast<std::size_t>(
sqlite3_limit(m_db, SQLITE_LIMIT_VARIABLE_NUMBER, -1)) /
PARAMS_PER_ROW;
for (std::size_t chunkStart = 0; chunkStart < distinctUUIDs.size(); chunkStart += maxRowsPerStatement)
{
const std::size_t chunkSize = eastl::min(maxRowsPerStatement, distinctUUIDs.size() - chunkStart);
eastl::string placeholders;
placeholders.reserve(chunkSize * 2);
for (std::size_t i = 0; i < chunkSize; ++i)
{
if (i > 0)
{
placeholders += ", ";
}
placeholders += "?";
}
const eastl::string sql = Format(
"SELECT UUID, CreateTime, ModificationTime FROM AssetMetadata WHERE UUID IN ({});",
placeholders);
BigFile::Request timeRequest {*this, sql};
for (std::size_t i = 0; i < chunkSize; ++i)
{
timeRequest.Bind(static_cast<std::uint32_t>(i + 1), distinctUUIDs[chunkStart + i].ToBytes());
}
while (timeRequest.Step())
{
const std::span<const std::byte, UUID::UUID_BYTE_SIZE> rawUUID {
static_cast<std::span<const std::byte>>(timeRequest.Get(0))};
const UUID uuid {rawUUID};
const auto indicesIt = uuidToIndices.find(uuid);
ASSERT(EngineAssertHandler,
indicesIt != uuidToIndices.end(),
"Returned UUID {} not found in requested UUIDs",
uuid);
const AssetTime assetTime {.m_creationTime = Time {static_cast<std::uint64_t>(timeRequest.Get(1))},
.m_modificationTime = Time {static_cast<std::uint64_t>(timeRequest.Get(2))}};
for (const std::size_t index: indicesIt->second)
{
times[index] = assetTime;
}
}
}
return times;
}
/****************************************************************************************/
void BigFile::SaveAssetFlatbuffer(const UUID& p_uuid,
const eastl::span<const std::byte> p_flatbuffer,
const eastl::span<const UUID> p_hardReferences)
{
BeginTransaction();
BigFile::Request metadataRequest {*this,
"INSERT INTO AssetMetadata (UUID) VALUES (?1) "
"ON CONFLICT(UUID) DO NOTHING;"};
metadataRequest.Bind(1, p_uuid.ToBytes());
metadataRequest.Execute();
BigFile::Request assetRequest {*this,
"INSERT INTO Asset (UUID, Asset) VALUES (?1, ?2) "
"ON CONFLICT(UUID) DO UPDATE SET Asset = excluded.Asset;"};
assetRequest.Bind(1, p_uuid.ToBytes());
assetRequest.Bind(2, p_flatbuffer);
assetRequest.Execute();
BigFile::Request deleteHardReferencesRequest {*this, "DELETE FROM AssetHardReference WHERE UUID = ?1;"};
deleteHardReferencesRequest.Bind(1, p_uuid.ToBytes());
deleteHardReferencesRequest.Execute();
constexpr std::size_t HARDREF_PARAMS_PER_ROW = 2;
constexpr std::size_t HARDREF_PLACEHOLDER_RESERVE_CHARS_PER_ROW = 8;
const std::size_t maxHardRefRowsPerStatement = static_cast<std::size_t>(
sqlite3_limit(m_db, SQLITE_LIMIT_VARIABLE_NUMBER, -1)) /
HARDREF_PARAMS_PER_ROW;
for (std::size_t chunkStart = 0; chunkStart < p_hardReferences.size(); chunkStart += maxHardRefRowsPerStatement)
{
const std::size_t chunkSize = eastl::min(maxHardRefRowsPerStatement, p_hardReferences.size() - chunkStart);
eastl::string hardReferencePlaceholders;
hardReferencePlaceholders.reserve(chunkSize * HARDREF_PLACEHOLDER_RESERVE_CHARS_PER_ROW);
for (std::size_t i = 0; i < chunkSize; ++i)
{
if (i > 0)
{
hardReferencePlaceholders += ", ";
}
hardReferencePlaceholders += "(?, ?)";
}
const eastl::string hardReferenceSQL = Format("INSERT INTO AssetHardReference (UUID, HardReference) VALUES {} "
"ON CONFLICT(UUID, HardReference) DO NOTHING;",
hardReferencePlaceholders);
BigFile::Request hardReferenceRequest {*this, hardReferenceSQL};
std::uint32_t bindIndex = 1;
for (std::size_t i = 0; i < chunkSize; ++i)
{
hardReferenceRequest.Bind(bindIndex++, p_uuid.ToBytes());
hardReferenceRequest.Bind(bindIndex++, p_hardReferences[chunkStart + i].ToBytes());
}
hardReferenceRequest.Execute();
}
CommitTransaction();
}
/****************************************************************************************/
void BigFile::SaveAssetFlatbuffers(const eastl::span<const UUID> p_uuids,
const eastl::span<const eastl::vector<std::byte>> p_flatbuffers,
const eastl::span<const eastl::vector<UUID>> p_hardReferences)
{
ASSERT(EngineAssertHandler,
p_uuids.size() == p_flatbuffers.size() && p_uuids.size() == p_hardReferences.size(),
"p_metadatas and p_flatbuffers and p_hardReferences must have the same size");
if (p_uuids.empty())
{
return;
}
BeginTransaction();
constexpr std::size_t METADATA_PARAMS_PER_ROW = 4;
constexpr std::size_t METADATA_PLACEHOLDER_RESERVE_CHARS_PER_ROW = 14;
constexpr std::size_t ASSET_PLACEHOLDER_RESERVE_CHARS_PER_ROW = 8;
const std::size_t maxRowsPerStatement = static_cast<std::size_t>(
sqlite3_limit(m_db, SQLITE_LIMIT_VARIABLE_NUMBER, -1)) /
METADATA_PARAMS_PER_ROW;
for (std::size_t chunkStart = 0; chunkStart < p_uuids.size(); chunkStart += maxRowsPerStatement)
{
const std::size_t chunkSize = eastl::min(maxRowsPerStatement, p_uuids.size() - chunkStart);
eastl::string metadataPlaceholders;
metadataPlaceholders.reserve(chunkSize * METADATA_PLACEHOLDER_RESERVE_CHARS_PER_ROW);
for (std::size_t i = 0; i < chunkSize; ++i)
{
if (i > 0)
{
metadataPlaceholders += ", ";
}
metadataPlaceholders += "(?)";
}
const eastl::string metadataSQL = Format(
"INSERT INTO AssetMetadata (UUID) VALUES {} ON CONFLICT(UUID) DO NOTHING;",
metadataPlaceholders);
BigFile::Request metadataRequest {*this, metadataSQL};
std::uint32_t metadataBindIndex = 1;
for (std::size_t i = 0; i < chunkSize; ++i)
{
metadataRequest.Bind(metadataBindIndex++, p_uuids[chunkStart + i].ToBytes());
}
metadataRequest.Execute();
eastl::string assetPlaceholders;
assetPlaceholders.reserve(chunkSize * ASSET_PLACEHOLDER_RESERVE_CHARS_PER_ROW);
for (std::size_t i = 0; i < chunkSize; ++i)
{
if (i > 0)
{
assetPlaceholders += ", ";
}
assetPlaceholders += "(?, ?)";
}
const eastl::string assetSQL = Format(
"INSERT INTO Asset (UUID, Asset) VALUES {} ON CONFLICT(UUID) DO UPDATE SET Asset = excluded.Asset;",
assetPlaceholders);
BigFile::Request assetRequest {*this, assetSQL};
std::uint32_t assetBindIndex = 1;
for (std::size_t i = 0; i < chunkSize; ++i)
{
assetRequest.Bind(assetBindIndex++, p_uuids[chunkStart + i].ToBytes());
assetRequest.Bind(assetBindIndex++, p_flatbuffers[chunkStart + i]);
}
assetRequest.Execute();
// Replace hard references for every asset in this chunk with its current set.
eastl::string deleteInPlaceholders;
deleteInPlaceholders.reserve(chunkSize * 3);
for (std::size_t i = 0; i < chunkSize; ++i)
{
if (i > 0)
{
deleteInPlaceholders += ", ";
}
deleteInPlaceholders += "?";
}
const eastl::string deleteHardReferencesSQL = Format("DELETE FROM AssetHardReference WHERE UUID IN ({});",
deleteInPlaceholders);
BigFile::Request deleteHardReferencesRequest {*this, deleteHardReferencesSQL};
for (std::size_t i = 0; i < chunkSize; ++i)
{
deleteHardReferencesRequest.Bind(static_cast<std::uint32_t>(i + 1), p_uuids[chunkStart + i].ToBytes());
}
deleteHardReferencesRequest.Execute();
eastl::vector<eastl::pair<UUID, UUID>> hardReferencePairs;
for (std::size_t i = 0; i < chunkSize; ++i)
{
for (const UUID& hardReference: p_hardReferences[chunkStart + i])
{
hardReferencePairs.emplace_back(p_uuids[chunkStart + i], hardReference);
}
}
constexpr std::size_t HARDREF_PARAMS_PER_ROW = 2;
constexpr std::size_t HARDREF_PLACEHOLDER_RESERVE_CHARS_PER_ROW = 8;
const std::size_t maxHardRefRowsPerStatement = static_cast<std::size_t>(
sqlite3_limit(m_db, SQLITE_LIMIT_VARIABLE_NUMBER, -1)) /
HARDREF_PARAMS_PER_ROW;
for (std::size_t hardRefChunkStart = 0; hardRefChunkStart < hardReferencePairs.size();
hardRefChunkStart += maxHardRefRowsPerStatement)
{
const std::size_t hardRefChunkSize = eastl::min(maxHardRefRowsPerStatement,
hardReferencePairs.size() - hardRefChunkStart);
eastl::string hardReferencePlaceholders;
hardReferencePlaceholders.reserve(hardRefChunkSize * HARDREF_PLACEHOLDER_RESERVE_CHARS_PER_ROW);
for (std::size_t i = 0; i < hardRefChunkSize; ++i)
{
if (i > 0)
{
hardReferencePlaceholders += ", ";
}
hardReferencePlaceholders += "(?, ?)";
}
const eastl::string hardReferenceSQL = Format(
"INSERT INTO AssetHardReference (UUID, HardReference) VALUES {} "
"ON CONFLICT(UUID, HardReference) DO NOTHING;",
hardReferencePlaceholders);
BigFile::Request hardReferenceRequest {*this, hardReferenceSQL};
std::uint32_t bindIndex = 1;
for (std::size_t i = 0; i < hardRefChunkSize; ++i)
{
const auto& [assetUUID, hardReference] = hardReferencePairs[hardRefChunkStart + i];
hardReferenceRequest.Bind(bindIndex++, assetUUID.ToBytes());
hardReferenceRequest.Bind(bindIndex++, hardReference.ToBytes());
}
hardReferenceRequest.Execute();
}
}
CommitTransaction();
}
/****************************************************************************************/
bool BigFile::Has(const UUID& p_uuid) const
{
BigFile::Request request {*this, "SELECT 1 FROM AssetMetadata WHERE UUID = ?1;"};
request.Bind(1, p_uuid.ToBytes());
return request.Step();
m_statementPool[std::string {p_query.data(), p_query.size()}].push_back(p_statement);
}
/****************************************************************************************/
BigFile::Request::Request(const BigFile& p_bigFile, const eastl::string_view p_request):
m_db(p_bigFile.m_db)
m_db(p_bigFile.m_db),
m_owner(&p_bigFile),
m_queryText(p_request),
m_statement(p_bigFile.AcquireStatement(p_request))
{
[[maybe_unused]]
const int result = sqlite3_prepare_v2(m_db,
p_request.data(),
static_cast<int>(p_request.size()),
&m_statement,
nullptr);
CRITICAL_ASSERT(EngineAssertHandler,
result == SQLITE_OK,
"Failed to create statement from BigFile DB: {}",
sqlite3_errmsg(m_db));
}
/****************************************************************************************/
@@ -619,11 +130,11 @@ BigFile::Request::Request(const BigFile& p_bigFile, const eastl::string_view p_r
void BigFile::Request::Bind(const std::uint32_t p_index, const std::int32_t p_value)
{
ASSERT(EngineAssertHandler,
(p_index >= 1U) && (p_index <= static_cast<std::uint32_t>(sqlite3_bind_parameter_count(m_statement))),
(p_index >= 1) && (p_index <= static_cast<std::uint32_t>(sqlite3_bind_parameter_count(m_statement))),
"Invalid index for statement");
[[maybe_unused]]
const int result = sqlite3_bind_int(m_statement, static_cast<int>(p_index), p_value);
const int result = sqlite3_bind_int(m_statement, p_index, p_value);
ASSERT(EngineAssertHandler, result == SQLITE_OK, "Failed to bind value for statement: {}", sqlite3_errmsg(m_db));
}
@@ -632,11 +143,11 @@ void BigFile::Request::Bind(const std::uint32_t p_index, const std::int32_t p_va
void BigFile::Request::Bind(const std::uint32_t p_index, const std::uint32_t p_value)
{
ASSERT(EngineAssertHandler,
(p_index >= 1U) && (p_index <= static_cast<std::uint32_t>(sqlite3_bind_parameter_count(m_statement))),
(p_index >= 1) && (p_index <= static_cast<std::uint32_t>(sqlite3_bind_parameter_count(m_statement))),
"Invalid index for statement");
[[maybe_unused]]
const int result = sqlite3_bind_int(m_statement, static_cast<int>(p_index), static_cast<int>(p_value));
const int result = sqlite3_bind_int(m_statement, p_index, p_value);
ASSERT(EngineAssertHandler, result == SQLITE_OK, "Failed to bind value for statement: {}", sqlite3_errmsg(m_db));
}
@@ -645,11 +156,11 @@ void BigFile::Request::Bind(const std::uint32_t p_index, const std::uint32_t p_v
void BigFile::Request::Bind(const std::uint32_t p_index, const std::int64_t p_value)
{
ASSERT(EngineAssertHandler,
(p_index >= 1U) && (p_index <= static_cast<std::uint32_t>(sqlite3_bind_parameter_count(m_statement))),
(p_index >= 1) && (p_index <= static_cast<std::uint32_t>(sqlite3_bind_parameter_count(m_statement))),
"Invalid index for statement");
[[maybe_unused]]
const int result = sqlite3_bind_int64(m_statement, static_cast<int>(p_index), p_value);
const int result = sqlite3_bind_int64(m_statement, p_index, p_value);
ASSERT(EngineAssertHandler, result == SQLITE_OK, "Failed to bind value for statement: {}", sqlite3_errmsg(m_db));
}
@@ -658,11 +169,11 @@ void BigFile::Request::Bind(const std::uint32_t p_index, const std::int64_t p_va
void BigFile::Request::Bind(const std::uint32_t p_index, const std::uint64_t p_value)
{
ASSERT(EngineAssertHandler,
(p_index >= 1U) && (p_index <= static_cast<std::uint32_t>(sqlite3_bind_parameter_count(m_statement))),
(p_index >= 1) && (p_index <= static_cast<std::uint32_t>(sqlite3_bind_parameter_count(m_statement))),
"Invalid index for statement");
[[maybe_unused]]
const int result = sqlite3_bind_int64(m_statement, static_cast<int>(p_index), std::bit_cast<std::int64_t>(p_value));
const int result = sqlite3_bind_int64(m_statement, p_index, std::bit_cast<std::int64_t>(p_value));
ASSERT(EngineAssertHandler, result == SQLITE_OK, "Failed to bind value for statement: {}", sqlite3_errmsg(m_db));
}
@@ -671,11 +182,11 @@ void BigFile::Request::Bind(const std::uint32_t p_index, const std::uint64_t p_v
void BigFile::Request::Bind(const std::uint32_t p_index, const float p_value)
{
ASSERT(EngineAssertHandler,
(p_index >= 1U) && (p_index <= static_cast<std::uint32_t>(sqlite3_bind_parameter_count(m_statement))),
(p_index >= 1) && (p_index <= static_cast<std::uint32_t>(sqlite3_bind_parameter_count(m_statement))),
"Invalid index for statement");
[[maybe_unused]]
const int result = sqlite3_bind_double(m_statement, static_cast<int>(p_index), p_value);
const int result = sqlite3_bind_double(m_statement, p_index, p_value);
ASSERT(EngineAssertHandler, result == SQLITE_OK, "Failed to bind value for statement: {}", sqlite3_errmsg(m_db));
}
@@ -684,11 +195,11 @@ void BigFile::Request::Bind(const std::uint32_t p_index, const float p_value)
void BigFile::Request::Bind(const std::uint32_t p_index, const double p_value)
{
ASSERT(EngineAssertHandler,
(p_index >= 1U) && (p_index <= static_cast<std::uint32_t>(sqlite3_bind_parameter_count(m_statement))),
(p_index >= 1) && (p_index <= static_cast<std::uint32_t>(sqlite3_bind_parameter_count(m_statement))),
"Invalid index for statement");
[[maybe_unused]]
const int result = sqlite3_bind_double(m_statement, static_cast<int>(p_index), p_value);
const int result = sqlite3_bind_double(m_statement, p_index, p_value);
ASSERT(EngineAssertHandler, result == SQLITE_OK, "Failed to bind value for statement: {}", sqlite3_errmsg(m_db));
}
@@ -697,14 +208,14 @@ void BigFile::Request::Bind(const std::uint32_t p_index, const double p_value)
void BigFile::Request::Bind(const std::uint32_t p_index, const eastl::string_view p_value, const CopyValue p_copy)
{
ASSERT(EngineAssertHandler,
(p_index >= 1U) && (p_index <= static_cast<std::uint32_t>(sqlite3_bind_parameter_count(m_statement))),
(p_index >= 1) && (p_index <= static_cast<std::uint32_t>(sqlite3_bind_parameter_count(m_statement))),
"Invalid index for statement");
[[maybe_unused]]
const int result = sqlite3_bind_text(m_statement,
static_cast<int>(p_index),
p_index,
p_value.data(),
static_cast<int>(p_value.size()),
static_cast<std::uint32_t>(p_value.size()),
p_copy ? SQLITE_TRANSIENT : SQLITE_STATIC);
ASSERT(EngineAssertHandler, result == SQLITE_OK, "Failed to bind value for statement: {}", sqlite3_errmsg(m_db));
}
@@ -716,15 +227,19 @@ void BigFile::Request::Bind(const std::uint32_t p_index,
const CopyValue p_copy)
{
ASSERT(EngineAssertHandler,
(p_index >= 1U) && (p_index <= static_cast<std::uint32_t>(sqlite3_bind_parameter_count(m_statement))),
(p_index >= 1) && (p_index <= static_cast<std::uint32_t>(sqlite3_bind_parameter_count(m_statement))),
"Invalid index for statement");
// sqlite3_bind_blob treats a NULL data pointer as binding SQL NULL rather than an empty blob, even
// when the size is 0 - which eastl::span::data() commonly is for an empty span. Bind through a
// dummy non-null address in that case so an empty span always binds an empty (non-NULL) blob.
static constexpr std::byte EMPTY_BLOB_SENTINEL {};
const std::byte* data = p_value.empty() ? &EMPTY_BLOB_SENTINEL : p_value.data();
[[maybe_unused]]
const int result = sqlite3_bind_blob(m_statement,
static_cast<int>(p_index),
p_value.data(),
static_cast<int>(p_value.size()),
p_copy ? SQLITE_TRANSIENT : SQLITE_STATIC);
const int result =
sqlite3_bind_blob(m_statement, p_index, data, static_cast<std::uint32_t>(p_value.size()),
p_copy ? SQLITE_TRANSIENT : SQLITE_STATIC);
ASSERT(EngineAssertHandler, result == SQLITE_OK, "Failed to bind value for statement: {}", sqlite3_errmsg(m_db));
}
@@ -764,69 +279,58 @@ BigFile::Request::Column::Column(const Request& p_request, const std::uint32_t p
BigFile::Request::Column::operator std::int32_t() const
{
return sqlite3_column_int(m_statement, static_cast<int>(m_index));
return sqlite3_column_int(m_statement, m_index);
}
/****************************************************************************************/
BigFile::Request::Column::operator std::uint32_t() const
{
return sqlite3_column_int(m_statement, static_cast<int>(m_index));
return sqlite3_column_int(m_statement, m_index);
}
/****************************************************************************************/
BigFile::Request::Column::operator std::int64_t() const
{
return sqlite3_column_int64(m_statement, static_cast<int>(m_index));
return sqlite3_column_int64(m_statement, m_index);
}
/****************************************************************************************/
BigFile::Request::Column::operator std::uint64_t() const
{
return std::bit_cast<std::uint64_t>(sqlite3_column_int64(m_statement, static_cast<int>(m_index)));
return std::bit_cast<std::uint64_t>(sqlite3_column_int64(m_statement, m_index));
}
/****************************************************************************************/
BigFile::Request::Column::operator float() const
{
return static_cast<float>(sqlite3_column_double(m_statement, static_cast<int>(m_index)));
return static_cast<float>(sqlite3_column_double(m_statement, m_index));
}
/****************************************************************************************/
BigFile::Request::Column::operator double() const
{
return sqlite3_column_double(m_statement, static_cast<int>(m_index));
return sqlite3_column_double(m_statement, m_index);
}
/****************************************************************************************/
BigFile::Request::Column::operator eastl::string_view() const
{
return eastl::string_view {
reinterpret_cast<const char*>(sqlite3_column_text(m_statement, static_cast<int>(m_index))),
static_cast<std::size_t>(sqlite3_column_bytes(m_statement, static_cast<int>(m_index)))};
return eastl::string_view {reinterpret_cast<const char*>(sqlite3_column_text(m_statement, m_index)),
static_cast<std::size_t>(sqlite3_column_bytes(m_statement, m_index))};
}
/****************************************************************************************/
BigFile::Request::Column::operator eastl::span<const std::byte>() const
{
return eastl::span<const std::byte> {
static_cast<const std::byte*>(sqlite3_column_blob(m_statement, static_cast<int>(m_index))),
static_cast<std::size_t>(sqlite3_column_bytes(m_statement, static_cast<int>(m_index)))};
}
/****************************************************************************************/
BigFile::Request::Column::operator std::span<const std::byte>() const
{
return std::span<const std::byte> {
static_cast<const std::byte*>(sqlite3_column_blob(m_statement, static_cast<int>(m_index))),
static_cast<std::size_t>(sqlite3_column_bytes(m_statement, static_cast<int>(m_index)))};
return eastl::span<const std::byte> {static_cast<const std::byte*>(sqlite3_column_blob(m_statement, m_index)),
static_cast<std::size_t>(sqlite3_column_bytes(m_statement, m_index))};
}
/****************************************************************************************/
@@ -843,9 +347,6 @@ BigFile::Request::Column BigFile::Request::Get(const std::uint32_t p_index) cons
BigFile::Request::~Request()
{
[[maybe_unused]]
const int result = sqlite3_finalize(m_statement);
CRITICAL_ASSERT(EngineAssertHandler, result == SQLITE_OK, "Failed to finalize statement: {}", sqlite3_errmsg(m_db));
m_owner->ReleaseStatement(m_queryText, m_statement);
}
#endif
} // namespace Bigfoot
@@ -1,38 +1,74 @@
PRAGMA journal_mode=WAL;
PRAGMA foreign_keys = ON;
CREATE TABLE IF NOT EXISTS AssetMetadata (
UUID BLOB NOT NULL,
CREATE TABLE IF NOT EXISTS AssetHeader (
UUID BLOB NOT NULL UNIQUE,
Name TEXT NOT NULL UNIQUE,
TypeID INTEGER NOT NULL,
TypeName TEXT NOT NULL,
Version INTEGER NOT NULL DEFAULT 0,
CreateTime INTEGER NOT NULL DEFAULT(CAST(unixepoch('subsec') * 1000000 AS INTEGER)),
ModificationTime INTEGER NOT NULL DEFAULT(CAST(unixepoch('subsec') * 1000000 AS INTEGER)),
CreateTime INTEGER NOT NULL DEFAULT(CAST(unixepoch('subsec') AS INTEGER) * 1000000),
ModificationTime INTEGER NOT NULL DEFAULT(CAST(unixepoch('subsec') AS INTEGER) * 1000000),
PRIMARY KEY(UUID)
) WITHOUT ROWID;
);
CREATE TRIGGER IF NOT EXISTS AssetHeader_UpdateTime
AFTER UPDATE OF Name, TypeID, TypeName, Version ON AssetHeader FOR EACH ROW
BEGIN
UPDATE AssetHeader
SET ModificationTime = CAST(unixepoch('subsec') AS INTEGER) * 1000000
WHERE UUID = NEW.UUID;
END;
CREATE TABLE IF NOT EXISTS Asset (
UUID BLOB NOT NULL,
UUID BLOB NOT NULL UNIQUE,
Asset BLOB NOT NULL,
PRIMARY KEY(UUID),
FOREIGN KEY(UUID) REFERENCES AssetMetadata(UUID) ON DELETE CASCADE
) WITHOUT ROWID;
FOREIGN KEY(UUID) REFERENCES AssetHeader(UUID) ON DELETE CASCADE
);
CREATE TRIGGER IF NOT EXISTS Asset_UpdateTime
AFTER UPDATE OF Asset ON Asset FOR EACH ROW
BEGIN
UPDATE AssetMetadata
SET ModificationTime = CAST(unixepoch('subsec') * 1000000 AS INTEGER)
UPDATE AssetHeader
SET ModificationTime = CAST(unixepoch('subsec') AS INTEGER) * 1000000
WHERE UUID = NEW.UUID;
END;
-- Every asset another asset points at via a HardReference field, one row per (referrer, referenced) pair.
-- Populated by SerializationManager on save (discovered generically via flatbuffers mini-reflection, see
-- SerializationManager.cpp), so that SerializationManager::LoadAsset can recursively load an asset's
-- HardReferences without having to deserialize the asset's own bytes first. Also directly queryable/
-- debuggable in SQL, e.g.:
-- SELECT ReferencedUUID FROM AssetHardReference WHERE UUID = ? -- what does this asset reference?
-- SELECT UUID FROM AssetHardReference WHERE ReferencedUUID = ? -- what references this asset?
-- SELECT AssetHeader.Name FROM AssetHardReference JOIN AssetHeader
-- ON AssetHeader.UUID = AssetHardReference.ReferencedUUID WHERE ... -- with human-readable names
CREATE TABLE IF NOT EXISTS AssetHardReference (
UUID BLOB NOT NULL,
HardReference BLOB NOT NULL,
UUID BLOB NOT NULL,
ReferencedUUID BLOB NOT NULL,
PRIMARY KEY(UUID, HardReference),
FOREIGN KEY(UUID) REFERENCES AssetMetadata(UUID)
ON DELETE CASCADE
PRIMARY KEY(UUID, ReferencedUUID),
FOREIGN KEY(UUID) REFERENCES AssetHeader(UUID) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS AssetHardReference_ReferencedUUID ON AssetHardReference(HardReference);
-- For "what references this asset" lookups: the primary key above only indexes UUID -> ReferencedUUID
-- efficiently, not the reverse.
CREATE INDEX IF NOT EXISTS AssetHardReference_ReferencedUUID ON AssetHardReference(ReferencedUUID);
-- Same as AssetHardReference, but for SoftReference fields. Purely informational: SoftReferences are
-- never eagerly loaded, so LoadAsset never reads this table - it exists so the full reference graph,
-- including optional/lazy links, is queryable and debuggable directly in SQL too.
CREATE TABLE IF NOT EXISTS AssetSoftReference (
UUID BLOB NOT NULL,
ReferencedUUID BLOB NOT NULL,
PRIMARY KEY(UUID, ReferencedUUID),
FOREIGN KEY(UUID) REFERENCES AssetHeader(UUID) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS AssetSoftReference_ReferencedUUID ON AssetSoftReference(ReferencedUUID);
@@ -1,15 +0,0 @@
/*********************************************************************
* \file DataAccessor.cpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#include <Engine/DataAccessor/DataAccessor.hpp>
namespace Bigfoot
{
void SetDataAccessor(DataAccessor* p_dataAccessor)
{
g_dataAccessor = p_dataAccessor;
}
} // namespace Bigfoot
@@ -10,6 +10,7 @@ table AssetHeader
name: string (required, id: 1);
type_id: AssetTypeID (required, native_inline, id: 2);
type_name: string (required, id: 3);
version: uint = 0 (id: 4);
}
table Asset
@@ -6,113 +6,110 @@
*********************************************************************/
#ifndef BIGFOOT_ENGINE_ASSET_ASSET_HPP
#define BIGFOOT_ENGINE_ASSET_ASSET_HPP
#include <Engine/Asset/AssetBase.hpp>
#include <Engine/Asset/AssetConcepts.hpp>
#include <Engine/Asset/AssetContainer.hpp>
#include <Engine/Asset/AssetLibrary.hpp>
#include <Engine/Asset/Asset_generated.hpp>
#include <Engine/EngineAssertHandler.hpp>
#include <EASTL/span.h>
#include <EASTL/vector.h>
#include <ankerl/unordered_dense.h>
#include <flatbuffers/minireflect.h>
#include <flatbuffers/reflection.h>
#include <rapidhash.h>
namespace Bigfoot
{
class AssetBase
{
public:
virtual ~AssetBase() = default;
/*
* Packs this asset's header and data into the flatbuffer byte representation used throughout the
* BigFile.
*
* \return The packed bytes
*/
[[nodiscard]]
virtual eastl::vector<std::byte> Pack() const = 0;
/*
* Get the asset's header
*
* \return The header
*/
[[nodiscard]]
virtual const ::Flat::Bigfoot::AssetHeaderT& GetHeader() const = 0;
/*
* Get the TypeTable describing this asset's own (inner) flatbuffer type, as generated by flatc's
* --reflect-types (see FLAT_ASSET::MiniReflectTypeTable()). Lets code that only has an AssetBase&
* (e.g. SerializationManager::SaveAsset) walk an asset's packed bytes generically - to find its
* HardReferences, for instance - without needing to know its concrete type at compile time.
*
* \return The TypeTable for this asset's own flatbuffer type
*/
[[nodiscard]]
virtual const flatbuffers::TypeTable* GetInnerTypeTable() const = 0;
protected:
AssetBase() = default;
AssetBase(const AssetBase& p_asset) = default;
AssetBase(AssetBase&& p_asset) = default;
AssetBase& operator=(const AssetBase& p_asset) = default;
AssetBase& operator=(AssetBase&& p_asset) = default;
};
template<class ASSET>
requires AssetTraits<ASSET>
class Asset: public AssetBase
{
public:
using FLAT_ASSET = ASSET::FLAT;
using FLAT_ASSET = typename ASSET::FLAT;
/**
* Constructor
*
* Will default initialize header
*
*/
Asset()
{
m_header.type_id = GetTypeID();
m_header.type_name = eastl::string {GetTypeName()};
}
/**
* Get the header of the asset
*
* \return The header of the asset
*/
[[nodiscard]]
const ::Flat::Bigfoot::AssetHeaderT& GetHeader() const override
{
return m_header;
}
Asset(const Asset& p_asset) = default;
Asset(Asset&& p_asset) = default;
/**
* Set the name of the asset
*
* \param p_name The name of the asset
*/
void SetName(const eastl::string_view p_name)
{
m_header.name = p_name;
}
~Asset() = default;
/**
* Get the asset
/*
* Overwrites this asset's header and data by unpacking p_flatbuffer, as produced by Pack(). Meant to
* be called right after default-construction (see AssetContainer::Insert(span<const byte>)): unlike a
* "from bytes" constructor, this lets the caller construct the asset in place first and only register
* it (by its real UUID, read from p_flatbuffer) once its data is actually in place.
*
* \return The asset
* \param p_flatbuffer The flatbuffer to unpack, as produced by Pack()
*/
[[nodiscard]]
const FLAT_ASSET::NativeTableType& GetAsset() const
{
return m_asset;
}
/**
* Get the asset
*
* \return The asset
*/
[[nodiscard]]
FLAT_ASSET::NativeTableType& GetAsset()
{
return m_asset;
}
/**
* Unpack a flatbuffer to the asset
*
* \param p_flatbuffer The flatbuffer to unpack
*/
void Unpack(const eastl::span<const std::byte> p_flatbuffer) override
void Unpack(const eastl::span<const std::byte> p_flatbuffer)
{
#ifdef BIGFOOT_NOT_OPTIMIZED
flatbuffers::Verifier verifier {reinterpret_cast<const std::uint8_t*>(p_flatbuffer.data()),
p_flatbuffer.size()};
flatbuffers::Verifier verifier {std::bit_cast<std::uint8_t*>(p_flatbuffer.data()), p_flatbuffer.size()};
const bool assetVerified = ::Flat::Bigfoot::VerifyAssetBuffer(verifier);
CRITICAL_ASSERT(EngineAssertHandler, assetVerified, "Given asset could not be verified!");
#endif
const ::Flat::Bigfoot::Asset* asset = flatbuffers::GetRoot<::Flat::Bigfoot::Asset>(p_flatbuffer.data());
#ifdef BIGFOOT_NOT_OPTIMIZED
flatbuffers::Verifier innerAssetVerifier {asset->inner_asset()->data(), asset->inner_asset()->size()};
const bool innerAssetVerified = innerAssetVerifier.VerifyBuffer<FLAT_ASSET>();
CRITICAL_ASSERT(EngineAssertHandler, innerAssetVerified, "Given asset could not be verified!");
#else
const ::Flat::Bigfoot::Asset* asset = flatbuffers::GetRoot<::Flat::Bigfoot::Asset>(p_flatbuffer.data());
#endif
asset->header()->UnPackTo(&m_header);
flatbuffers::GetRoot<FLAT_ASSET>(asset->inner_asset()->data())->UnPackTo(&m_asset, nullptr);
}
/**
* Pack the asset to a flatbuffer
/*
* Packs this asset's header and data into the flatbuffer byte representation used throughout the
* BigFile - the format Unpack() (and AssetContainer::Insert(span<const byte>)) expects.
*
* \return The packed asset
* \return The packed bytes
*/
[[nodiscard]]
eastl::vector<std::byte> Pack() const override
@@ -120,98 +117,73 @@ class Asset: public AssetBase
flatbuffers::FlatBufferBuilder innerAssetFbb;
innerAssetFbb.Finish(FLAT_ASSET::Pack(innerAssetFbb, &m_asset));
::Flat::Bigfoot::AssetT asset;
asset.header = eastl::make_unique<::Flat::Bigfoot::AssetHeaderT>(m_header);
asset.inner_asset = eastl::vector<std::uint8_t> {innerAssetFbb.GetBufferPointer(),
innerAssetFbb.GetBufferPointer() + innerAssetFbb.GetSize()};
flatbuffers::FlatBufferBuilder assetFbb;
const auto header = ::Flat::Bigfoot::AssetHeader::Pack(assetFbb, &m_header);
const auto innerAsset = assetFbb.CreateVector(innerAssetFbb.GetBufferPointer(), innerAssetFbb.GetSize());
assetFbb.Finish(::Flat::Bigfoot::CreateAsset(assetFbb, header, innerAsset));
assetFbb.Finish(::Flat::Bigfoot::Asset::Pack(assetFbb, &asset));
return eastl::vector<std::byte> {
reinterpret_cast<std::byte*>(assetFbb.GetBufferPointer()),
reinterpret_cast<std::byte*>(assetFbb.GetBufferPointer() + assetFbb.GetSize())};
return eastl::vector<std::byte> {std::bit_cast<std::byte*>(assetFbb.GetBufferPointer()),
std::bit_cast<std::byte*>(assetFbb.GetBufferPointer() + assetFbb.GetSize())};
}
/**
* Get the HardReferences the flatbuffer has
*
* \param p_flatbuffer The flatbuffer
* \return The UUIDs of the HardReferences
*/
[[nodiscard]]
eastl::vector<UUID> GetHardReferences(const eastl::span<const std::byte> p_flatbuffer) const override
const ::Flat::Bigfoot::AssetHeaderT& GetHeader() const override
{
#ifdef BIGFOOT_NOT_OPTIMIZED
flatbuffers::Verifier verifier {reinterpret_cast<const std::uint8_t*>(p_flatbuffer.data()),
p_flatbuffer.size()};
const bool assetVerified = ::Flat::Bigfoot::VerifyAssetBuffer(verifier);
CRITICAL_ASSERT(EngineAssertHandler, assetVerified, "Given asset could not be verified!");
#endif
const ::Flat::Bigfoot::Asset* asset = flatbuffers::GetRoot<::Flat::Bigfoot::Asset>(p_flatbuffer.data());
#ifdef BIGFOOT_NOT_OPTIMIZED
flatbuffers::Verifier innerAssetVerifier {asset->inner_asset()->data(), asset->inner_asset()->size()};
const bool innerAssetVerified = innerAssetVerifier.VerifyBuffer<FLAT_ASSET>();
CRITICAL_ASSERT(EngineAssertHandler, innerAssetVerified, "Given asset could not be verified!");
#endif
HardReferenceVisitor visitor;
flatbuffers::IterateFlatBuffer(asset->inner_asset()->data(), FLAT_ASSET::MiniReflectTypeTable(), &visitor);
return visitor.GetUUIDs();
return m_header;
}
/**
* Get the AssetTypeID
*
* \return The AssetTypeIPD
*/
[[nodiscard]]
static AssetTypeID GetTypeID()
const flatbuffers::TypeTable* GetInnerTypeTable() const override
{
static const AssetTypeID typeID = rapidhash(FLAT_ASSET::GetFullyQualifiedName(),
std::strlen(FLAT_ASSET::GetFullyQualifiedName()));
return FLAT_ASSET::MiniReflectTypeTable();
}
void SetName(const eastl::string_view p_name)
{
m_header.name = p_name;
}
void SetVersion(const std::uint32_t p_version)
{
m_header.version = p_version;
}
[[nodiscard]]
const typename FLAT_ASSET::NativeTableType& GetAsset() const
{
return m_asset;
}
[[nodiscard]]
typename FLAT_ASSET::NativeTableType& GetAsset()
{
return m_asset;
}
[[nodiscard]]
static std::uint64_t GetTypeID()
{
static const std::uint64_t typeID = rapidhash(FLAT_ASSET::GetFullyQualifiedName(),
std::strlen(FLAT_ASSET::GetFullyQualifiedName()));
return typeID;
}
/**
* Get the type name
*
* \return The type name
*/
[[nodiscard]]
static eastl::string_view GetTypeName()
{
return FLAT_ASSET::GetFullyQualifiedName();
}
Asset& operator=(const Asset& p_asset) = default;
Asset& operator=(Asset&& p_asset) = default;
private:
/**
* The asset
*/
FLAT_ASSET::NativeTableType m_asset;
/**
* The asset header
*/
typename FLAT_ASSET::NativeTableType m_asset;
::Flat::Bigfoot::AssetHeaderT m_header;
};
/**
* Create an asset and insert it into its corresponding AssetContainer
*
* \tparam ASSET The Asset type
* \return The created asset, nullptr if no AssetContainer is registered for ASSET
*/
template<class ASSET>
requires AssetType<ASSET>
[[nodiscard]]
ASSET* CreateAsset()
{
AssetContainerBase* container = g_activeAssetLibrary->GetAssetContainer(ASSET::GetTypeID());
CRITICAL_ASSERT(EngineAssertHandler, container != nullptr, "No AssetContainer registered for this asset type!");
ContainerHandle* handle = container->Insert();
return static_cast<ASSET*>(handle->GetAsset());
}
} // namespace Bigfoot
#endif
@@ -1,111 +0,0 @@
/*********************************************************************
* \file AssetBase.hpp
*
* \author Romain BOULLARD
* \date April 2026
*********************************************************************/
#ifndef BIGFOOT_ENGINE_ASSET_ASSETBASE_HPP
#define BIGFOOT_ENGINE_ASSET_ASSETBASE_HPP
#include <Engine/Asset/Asset_generated.hpp>
#include <System/UUID/UUID.hpp>
#include <EASTL/span.h>
#include <EASTL/vector.h>
#include <ankerl/unordered_dense.h>
#include <flatbuffers/minireflect.h>
namespace Bigfoot
{
class AssetBase
{
public:
virtual ~AssetBase() = default;
/**
* Get the header of the asset
*
* \return The header of the asset
*/
[[nodiscard]]
virtual const ::Flat::Bigfoot::AssetHeaderT& GetHeader() const = 0;
/**
* Get the HardReferences the flatbuffer has
*
* \param p_flatbuffer The flatbuffer
* \return The UUIDs of the HardReferences
*/
[[nodiscard]]
virtual eastl::vector<UUID> GetHardReferences(const eastl::span<const std::byte> p_flatbuffer) const = 0;
/**
* Unpack a flatbuffer to the asset
*
* \param p_flatbuffer The flatbuffer to unpack
*/
virtual void Unpack(const eastl::span<const std::byte> p_flatbuffer) = 0;
/**
* Pack the asset to a flatbuffer
*
* \return The packed asset
*/
[[nodiscard]]
virtual eastl::vector<std::byte> Pack() const = 0;
AssetBase(const AssetBase& p_asset) = delete;
AssetBase& operator=(const AssetBase& p_asset) = delete;
protected:
AssetBase() = default;
AssetBase(AssetBase&& p_asset) = default;
AssetBase& operator=(AssetBase&& p_asset) = default;
class HardReferenceVisitor: public flatbuffers::IterationVisitor
{
public:
void Field(size_t p_fieldIndex,
size_t p_setIndex,
const flatbuffers::ElementaryType p_type,
const bool p_isVector,
const flatbuffers::TypeTable* p_typeTable,
const char* p_name,
const std::uint8_t* p_val) override;
void Element(size_t p_index,
const flatbuffers::ElementaryType p_type,
const flatbuffers::TypeTable* p_typeTable,
const std::uint8_t* p_val) override;
/**
* Get the visited HardReference
*
* \return The visited HardReferences
*/
[[nodiscard]]
eastl::vector<UUID> GetUUIDs() const;
private:
/**
* Visit an element in the flatbuffer
*
* \param p_type The element type
* \param p_typeTable The element typetable
* \param p_val The element
*/
void Visit(const flatbuffers::ElementaryType p_type,
const flatbuffers::TypeTable* p_typeTable,
const std::uint8_t* p_val);
/**
* The HardReferences visited so far
*/
ankerl::unordered_dense::set<UUID> m_uuids;
};
};
} // namespace Bigfoot
#endif
@@ -1,43 +0,0 @@
/*********************************************************************
* \file AssetConcepts.hpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#ifndef BIGFOOT_ENGINE_ASSET_ASSETCONCEPTS_HPP
#define BIGFOOT_ENGINE_ASSET_ASSETCONCEPTS_HPP
#include <Engine/Asset/AssetTypeID.hpp>
#include <System/UUID/UUID.hpp>
#include <flatbuffers/minireflect.h>
#include <concepts>
namespace Bigfoot
{
class AssetBase;
/**
* The traits struct passed to Asset<TRAITS> (e.g. AssetATraits).
* Must name the generated flatbuffers table type for the asset.
*/
template<class TRAITS>
concept AssetTraits = requires {
typename TRAITS::FLAT;
typename TRAITS::FLAT::NativeTableType;
{ TRAITS::FLAT::GetFullyQualifiedName() } -> std::convertible_to<const char*>;
{ TRAITS::FLAT::MiniReflectTypeTable() } -> std::convertible_to<const flatbuffers::TypeTable*>;
};
/**
* A concrete Asset type (e.g. AssetA), usable with AssetContainer, AssetLibrary and References.
*/
template<class ASSET>
concept AssetType = std::derived_from<ASSET, AssetBase> && !std::same_as<ASSET, AssetBase> &&
requires(const ASSET& p_asset) {
{ ASSET::GetTypeID() } -> std::convertible_to<AssetTypeID>;
{ p_asset.GetHeader().uuid } -> std::convertible_to<UUID>;
};
} // namespace Bigfoot
#endif
@@ -6,33 +6,47 @@
*********************************************************************/
#ifndef BIGFOOT_ENGINE_ASSET_ASSETCONTAINER_HPP
#define BIGFOOT_ENGINE_ASSET_ASSETCONTAINER_HPP
#include <Engine/Asset/AssetBase.hpp>
#include <Engine/Asset/AssetConcepts.hpp>
#include <Engine/Asset/Asset.hpp>
#include <Engine/EngineAssertHandler.hpp>
#include <Utils/Containers/SlotMap.hpp>
#include <EASTL/span.h>
#include <EASTL/unique_ptr.h>
#include <ankerl/unordered_dense.h>
#include <atomic>
#include <cstdint>
#include <mutex>
#include <tuple>
namespace Bigfoot
{
class AssetContainerBase;
class AssetLibrary;
/**
* An indirection that allows References to immediately get the asset, without fetching the actual container
*/
class ContainerHandle
{
public:
/**
* Constructor
*
* \param p_assetContainer The originating assetcontainer
* \param p_uuid The UUID of tha asset
*/
ContainerHandle(AssetContainerBase* p_assetContainer, const UUID p_uuid);
ContainerHandle() = default;
// Copying/moving an atomic isn't allowed by default (and never actually happens: every ContainerHandle
// is default-constructed once in place by StableSlotMap::Insert() and never relocated afterwards -
// StableSlotMap's pages are exactly what makes that a stable guarantee) - these snapshot the counts
// rather than moving the atomics themselves, which would only be wrong if a copy/move genuinely raced
// with a concurrent Increment/Decrement, which nothing in this codebase does.
ContainerHandle(const ContainerHandle& p_handle):
m_hardRefCount(p_handle.m_hardRefCount.load(std::memory_order_relaxed)),
m_softRefCount(p_handle.m_softRefCount.load(std::memory_order_relaxed)),
m_asset(p_handle.m_asset)
{
}
ContainerHandle(ContainerHandle&& p_handle) noexcept:
ContainerHandle(p_handle)
{
}
~ContainerHandle() = default;
/*
* Get the HardRef count
@@ -40,19 +54,30 @@ class ContainerHandle
* \return The HardRef count.
*/
[[nodiscard]]
std::uint32_t GetHardRefCount() const;
std::uint32_t GetHardRefCount() const
{
return m_hardRefCount.load(std::memory_order_relaxed);
}
/*
* Increment the HardRef count
* Increment the HardRef count. Atomic: LoadAssets decodes a batch's HardReference fields (which
* increment the referenced asset's count) from multiple threads at once, possibly the same asset from
* more than one of them (e.g. several assets sharing one referenced texture).
*
*/
void IncrementHardRefCount();
void IncrementHardRefCount()
{
m_hardRefCount.fetch_add(1, std::memory_order_relaxed);
}
/*
* Decrement the HardRef count
* Decrement the HardRef count. See IncrementHardRefCount for why this is atomic.
*
*/
void DecrementHardRefCount();
void DecrementHardRefCount()
{
m_hardRefCount.fetch_sub(1, std::memory_order_relaxed);
}
/*
* Get the SoftRef count
@@ -60,19 +85,28 @@ class ContainerHandle
* \return The SoftRef count.
*/
[[nodiscard]]
std::uint32_t GetSoftRefCount() const;
std::uint32_t GetSoftRefCount() const
{
return m_softRefCount.load(std::memory_order_relaxed);
}
/*
* Increment the SoftRef count
* Increment the SoftRef count. See IncrementHardRefCount for why this is atomic.
*
*/
void IncrementSoftRefCount();
void IncrementSoftRefCount()
{
m_softRefCount.fetch_add(1, std::memory_order_relaxed);
}
/*
* Decrement the SoftRef count
* Decrement the SoftRef count. See IncrementHardRefCount for why this is atomic.
*
*/
void DecrementSoftRefCount();
void DecrementSoftRefCount()
{
m_softRefCount.fetch_sub(1, std::memory_order_relaxed);
}
/*
* Get the Asset
@@ -80,7 +114,10 @@ class ContainerHandle
* \return The asset, can be nullptr
*/
[[nodiscard]]
AssetBase* GetAsset();
AssetBase* GetAsset()
{
return m_asset;
}
/*
* Get the Asset
@@ -88,40 +125,52 @@ class ContainerHandle
* \return The asset
*/
[[nodiscard]]
const AssetBase* GetAsset() const;
const AssetBase* GetAsset() const
{
return m_asset;
}
/*
* Set the asset
*
* \param p_asset The asset
*/
void SetAsset(AssetBase* p_asset);
void SetAsset(AssetBase* p_asset)
{
m_asset = p_asset;
}
ContainerHandle& operator=(const ContainerHandle& p_handle)
{
if (this != &p_handle)
{
m_hardRefCount.store(p_handle.m_hardRefCount.load(std::memory_order_relaxed), std::memory_order_relaxed);
m_softRefCount.store(p_handle.m_softRefCount.load(std::memory_order_relaxed), std::memory_order_relaxed);
m_asset = p_handle.m_asset;
}
return *this;
}
ContainerHandle& operator=(ContainerHandle&& p_handle) noexcept
{
return *this = p_handle;
}
private:
/*
* The HardRef count
* The HardRef count. Atomic - see IncrementHardRefCount.
*/
std::uint32_t m_hardRefCount = 0;
std::atomic<std::uint32_t> m_hardRefCount = 0;
/*
* The SoftRef count
* The SoftRef count. Atomic - see IncrementHardRefCount.
*/
std::uint32_t m_softRefCount = 0;
std::atomic<std::uint32_t> m_softRefCount = 0;
/*
* The asset
*/
AssetBase* m_asset = nullptr;
/**
* The asset container
*/
AssetContainerBase* m_assetContainer = nullptr;
/**
* The UUID of the asset
*/
UUID m_uuid = UUID::NULL_UUID;
};
/****************************************************************************************/
@@ -134,23 +183,6 @@ class AssetContainerBase
public:
virtual ~AssetContainerBase() = default;
/*
* Inserts an Asset to the AssetContainer
*
* \param p_flatbuffer The flatbuffer to unpack to the asset
* \return The handle of the inserted asset
*/
[[nodiscard]]
virtual ContainerHandle* Insert(const eastl::span<const std::byte> p_flatbuffer) = 0;
/*
* Inserts an Asset to the AssetContainer
*
* \return The handle of the inserted asset
*/
[[nodiscard]]
virtual ContainerHandle* Insert() = 0;
/*
* Check if an Asset is in the AssetContainer or not
*
@@ -158,7 +190,11 @@ class AssetContainerBase
* \return True if in AssetContainer, false otherwise
*/
[[nodiscard]]
bool Has(const UUID& p_uuid) const;
bool Has(const UUID& p_uuid) const
{
const Entry* entry = FindEntry(p_uuid);
return entry != nullptr && entry->m_assetSlotKey.Valid();
}
/*
* Acquires a ContainerHandle for a given UUID
@@ -168,44 +204,69 @@ class AssetContainerBase
* \return A ContainerHandle for the UUID
*/
[[nodiscard]]
ContainerHandle* AcquireContainerHandle(const UUID& p_uuid);
ContainerHandle* AcquireContainerHandle(const UUID& p_uuid)
{
return m_handles.Get(GetOrCreateEntry(p_uuid).m_handleSlotKey);
}
/*
* Remove an Asset from the AssetContainer
*
* \param p_uuid The UUID to remove
*/
void Remove(const UUID& p_uuid);
virtual void Remove(const UUID& p_uuid);
/**
* Add an asset to be removed
/*
* Type-erased overload of AssetContainer::Insert, used by SerializationManager (which only has an
* AssetContainerBase*, the concrete ASSET being resolved at runtime via AssetTypeID) to construct and
* insert an asset of this container's concrete type from its serialized bytes. The asset's UUID is
* read from the bytes themselves (its header), same as AssetContainer::Insert.
*
* \param p_uuid UUID of the asset to remove
* \param p_bytes The serialized bytes of the asset, as produced by Asset<ASSET>::Save()
*/
void QueueForRemoval(const UUID& p_uuid);
virtual void Insert(const eastl::span<const std::byte> p_bytes) = 0;
/**
* Remove an asset from the pending remove queue
/*
* Type-erased, container-state-free half of Insert: constructs an asset of this container's concrete
* type and unpacks p_bytes into it, but never touches this container (no SlotMap/map mutation) - safe
* to call concurrently, from any number of threads, on the same AssetContainerBase, including
* concurrently with other DecodeOffContainer calls on completely different containers. Pairs with
* InsertDecoded, which does the (cheap, must-stay-single-threaded) other half. See
* SerializationManager::LoadAssets, which uses this pair to decode a batch of assets in parallel while
* only ever mutating a container from one thread at a time.
*
* \param p_uuid UUID of the asset to cancel removal
* \param p_bytes The serialized bytes of the asset, as produced by Asset<ASSET>::Save()
* \return The decoded asset, not yet part of this container
*/
void CancelRemoval(const UUID& p_uuid);
/**
* Process the assets that have been marked for remove
*
*/
void ProcessPendingRemovals();
[[nodiscard]]
bool IsMarkedForRemove(const UUID& p_uuid);
virtual eastl::unique_ptr<AssetBase> DecodeOffContainer(const eastl::span<const std::byte> p_bytes) const = 0;
/*
* Moves an asset decoded by DecodeOffContainer into this container, exactly as Insert(bytes) would have
* - not safe to call concurrently with anything else touching this container (Insert, Remove,
* DecodeOffContainer is fine, InsertDecoded is not).
*
* \param p_decoded An asset produced by this same container's DecodeOffContainer
*/
virtual void InsertDecoded(eastl::unique_ptr<AssetBase> p_decoded) = 0;
/*
* Set the AssetLibrary this container is registered into. Called once by AssetLibrary::Insert.
*
* Needed so that newly created UUID entries (see GetOrCreateEntry) can be published into the
* library's UUID -> AssetContainerBase index, which HardReference<AssetBase>/SoftReference<AssetBase>
* use to resolve a bare UUID to its container without knowing its concrete type ahead of time.
*
* \param p_library The owning AssetLibrary
*/
void SetOwningLibrary(AssetLibrary* p_library)
{
m_owningLibrary = p_library;
}
AssetContainerBase& operator=(const AssetContainerBase& p_container) = delete;
AssetContainerBase& operator=(AssetContainerBase&& p_container) = delete;
AssetContainerBase(const AssetContainerBase& p_container) = delete;
AssetContainerBase(AssetContainerBase&& p_container) = delete;
protected:
struct Entry
{
@@ -218,23 +279,12 @@ class AssetContainerBase
* SlotMapKey to the asset
*/
SlotMapKey m_assetSlotKey;
/*
* The handle itself, cached to avoid a SlotMap lookup on every access
*/
ContainerHandle* m_handle = nullptr;
};
AssetContainerBase() = default;
/*
* Finds an entry matching the Asset
*
* \param p_uuid The UUID to find
* \return Pointer to an Entry, nullptr otherwise
*/
[[nodiscard]]
Entry* FindEntry(const UUID& p_uuid);
AssetContainerBase(const AssetContainerBase& p_container) = delete;
AssetContainerBase(AssetContainerBase&& p_container) = delete;
/*
* Finds an entry matching the Asset
@@ -243,12 +293,35 @@ class AssetContainerBase
* \return Pointer to an Entry, nullptr otherwise
*/
[[nodiscard]]
const Entry* FindEntry(const UUID& p_uuid) const;
Entry* FindEntry(const UUID& p_uuid)
{
const auto it = m_uuidToEntry.find(p_uuid);
return it != m_uuidToEntry.end() ? &it->second : nullptr;
}
/*
* Finds an entry matching the Asset
*
* \param p_uuid The UUID to find
* \return Pointer to an Entry, nullptr otherwise
*/
[[nodiscard]]
const Entry* FindEntry(const UUID& p_uuid) const
{
const auto it = m_uuidToEntry.find(p_uuid);
return it != m_uuidToEntry.end() ? &it->second : nullptr;
}
/*
* Finds an entry matching the Asset
* Creates it if it does not exist
*
* Thread-safe against concurrent calls to itself (guarded by m_entryMutex): SerializationManager::
* LoadAssets resolves Hard/SoftReference fields - which call this indirectly via
* AcquireContainerHandle - from multiple threads at once while decoding a batch, and a reference can
* turn out to be the first thing that ever touches its target's entry (e.g. an unloaded
* SoftReference), so two threads can race to create the same entry.
*
* \param p_uuid The UUID to find
* \return The entry matching the Asset
*/
@@ -256,11 +329,16 @@ class AssetContainerBase
Entry& GetOrCreateEntry(const UUID& p_uuid);
/*
* Remove an asset
* Get a ContainerHandle
*
* \param p_key The key to the asset
* \param p_handleSlotKey The key to the container
* \return The container
*/
virtual void RemoveAsset(SlotMapKey p_slotKey) = 0;
[[nodiscard]]
ContainerHandle& FindContainerHandle(const SlotMapKey p_handleSlotKey)
{
return *m_handles.Get(p_handleSlotKey);
}
private:
/*
@@ -271,87 +349,160 @@ class AssetContainerBase
/*
* Every Asset added to the map
*/
ankerl::unordered_dense::map<UUID, Entry> m_uuidToEntry;
ankerl::unordered_dense::segmented_map<UUID, Entry> m_uuidToEntry;
/**
* Assets queued for removal
/*
* Guards m_uuidToEntry/m_handles against concurrent GetOrCreateEntry calls - see its comment.
*/
eastl::vector<UUID> m_pendingRemovalUuids;
std::mutex m_entryMutex;
/*
* The AssetLibrary this container is registered into, if any. Set via SetOwningLibrary.
*/
AssetLibrary* m_owningLibrary = nullptr;
};
/****************************************************************************************/
template<class ASSET>
requires AssetType<ASSET>
class AssetContainer: public AssetContainerBase
{
using SlotMap = StableSlotMap<ASSET>;
using SlotMapKey = SlotMap::SlotMapKey;
public:
/*
* Inserts an Asset to the AssetContainer
*
* \param p_flatbuffer The flatbuffer to unpack to the asset
* \return The handle of the inserted asset
*/
[[nodiscard]]
ContainerHandle* Insert(const eastl::span<const std::byte> p_flatbuffer) override
{
return InsertInternal(&p_flatbuffer);
}
AssetContainer() = default;
AssetContainer(const AssetContainer& p_container) = delete;
AssetContainer(AssetContainer&& p_container) = delete;
~AssetContainer() = default;
/*
* Inserts an Asset to the AssetContainer
*
* \return The handle of the inserted asset
* \tparam ARGS Arguments to construct the asset
* \param p_args The arguments
* \return The key to the inserted asset.
*/
template<class... ARGS>
[[nodiscard]]
SlotMapKey Insert(ARGS&&... p_args)
{
return Register(EmplaceUnregistered(std::forward<ARGS>(p_args)...));
}
/*
* Get the Asset
*
* \param p_key The key to the asset
* \return Pointer to the Asset
*/
[[nodiscard]]
ContainerHandle* Insert() override
ASSET* Get(const SlotMapKey p_key)
{
return InsertInternal(nullptr);
return m_assets.Get(p_key);
}
/*
* Get the Asset
*
* \param p_key The key to the asset
* \return Pointer to the Asset
*/
[[nodiscard]]
const ASSET* Get(const SlotMapKey p_key) const
{
return m_assets.Get(p_key);
}
/*
* Remove an Asset from the AssetContainer
*
* \param p_uuid The UUID to remove
*/
void Remove(const UUID& p_uuid) override
{
if (Entry* entry = FindEntry(p_uuid); entry != nullptr && entry->m_assetSlotKey.Valid())
{
FindContainerHandle(entry->m_handleSlotKey).SetAsset(nullptr);
m_assets.Remove(entry->m_assetSlotKey);
entry->m_assetSlotKey = SlotMapKey {};
}
AssetContainerBase::Remove(p_uuid);
}
void Insert(const eastl::span<const std::byte> p_bytes) override
{
// Construct a default asset in place first (a placeholder, random UUID), fill it in from p_bytes
// (which sets its real UUID), and only then register it - registering before Unpack would index
// the entry under the placeholder UUID instead of the asset's real one.
const SlotMapKey slotKey = EmplaceUnregistered();
m_assets.Get(slotKey)->Unpack(p_bytes);
std::ignore = Register(slotKey);
}
[[nodiscard]]
eastl::unique_ptr<AssetBase> DecodeOffContainer(const eastl::span<const std::byte> p_bytes) const override
{
auto asset = eastl::make_unique<ASSET>();
asset->Unpack(p_bytes);
return asset;
}
void InsertDecoded(eastl::unique_ptr<AssetBase> p_decoded) override
{
// Move rather than re-construct: the caller already paid for Unpack() via DecodeOffContainer, this
// should be nothing more than placing an already-built object into a slot.
eastl::unique_ptr<ASSET> decoded {static_cast<ASSET*>(p_decoded.release())};
std::ignore = Register(EmplaceUnregistered(std::move(*decoded)));
}
AssetContainer& operator=(const AssetContainer& p_container) = delete;
AssetContainer& operator=(AssetContainer&& p_container) = delete;
private:
/*
* Inserts an Asset to the AssetContainer, unpacking it from a flatbuffer if given one
* Constructs an Asset in place from p_args, without registering it under any UUID yet. Split out from
* Insert so that Insert(span<const byte>) can fill in a default-constructed asset (via Asset::Unpack,
* which may change its UUID) before registration happens - see Register.
*
* \param p_flatbuffer The flatbuffer to unpack to the asset, nullptr to default-construct the asset
* \return The handle of the inserted asset
* \tparam ARGS Arguments to construct the asset
* \param p_args The arguments
* \return The key of the constructed, not-yet-registered asset
*/
template<class... ARGS>
[[nodiscard]]
SlotMapKey EmplaceUnregistered(ARGS&&... p_args)
{
return m_assets.Insert(std::forward<ARGS>(p_args)...);
}
/*
* Registers an asset previously constructed via EmplaceUnregistered under its current UUID: creates
* or reuses the entry for that UUID and wires up its ContainerHandle. If another asset is already
* registered under the same UUID, p_slotKey's asset is discarded and the existing slot's key is
* returned instead.
*
* \param p_slotKey The key of a not-yet-registered asset, from EmplaceUnregistered
* \return The key of the now-registered asset (p_slotKey, or an existing slot on UUID collision)
*/
[[nodiscard]]
ContainerHandle* InsertInternal(const eastl::span<const std::byte>* p_flatbuffer)
SlotMapKey Register(const SlotMapKey p_slotKey)
{
SlotMapKey slotKey = m_assets.Insert();
ASSET* asset = m_assets.Get(slotKey);
if (p_flatbuffer != nullptr)
{
asset->Unpack(*p_flatbuffer);
}
ASSET* asset = m_assets.Get(p_slotKey);
Entry& entry = GetOrCreateEntry(asset->GetHeader().uuid);
if (entry.m_assetSlotKey.Valid())
{
m_assets.Remove(slotKey);
slotKey = entry.m_assetSlotKey;
}
else
{
entry.m_assetSlotKey = slotKey;
entry.m_handle->SetAsset(asset);
m_assets.Remove(p_slotKey);
return entry.m_assetSlotKey;
}
return entry.m_handle;
}
/*
* Remove an asset
*
* \param p_key The key to the asset
*/
void RemoveAsset(SlotMapKey p_key) override
{
m_assets.Remove(p_key);
entry.m_assetSlotKey = p_slotKey;
FindContainerHandle(entry.m_handleSlotKey).SetAsset(asset);
return p_slotKey;
}
/*
@@ -19,13 +19,6 @@
return AssetType::GetTypeID(); \
} \
AssetContainer<AssetType> g_containerFor##AssetName; \
const bool g_insertedContainerFor##AssetName = []() \
{ \
const bool inserted = g_assetLibrary.Insert(&g_containerFor##AssetName); \
CRITICAL_ASSERT(EngineAssertHandler, \
inserted, \
"AssetTypeID collision or duplicate registration for asset type '" #AssetName "'"); \
return inserted; \
}(); \
const bool g_insertedContainerFor##AssetName = g_assetLibrary.Insert(&g_containerFor##AssetName); \
}
#endif
@@ -6,12 +6,14 @@
*********************************************************************/
#ifndef BIGFOOT_ENGINE_ASSET_ASSETLIBRARY_HPP
#define BIGFOOT_ENGINE_ASSET_ASSETLIBRARY_HPP
#include <Engine/Asset/AssetConcepts.hpp>
#include <Engine/Asset/AssetContainer.hpp>
#include <Engine/Asset/AssetTypeID.hpp>
#include <ankerl/unordered_dense.h>
#include <mutex>
#include <utility>
namespace Bigfoot
{
class AssetLibrary
@@ -19,9 +21,20 @@ class AssetLibrary
public:
AssetLibrary() = default;
AssetLibrary(const AssetLibrary& p_library) = default;
// std::mutex isn't copyable, so this can't be = default: copies every member except the mutexes, which
// a copy just default-constructs its own instances of (a mutex protects one specific object's
// internals, copying a lock along with the data it guards would be meaningless).
AssetLibrary(const AssetLibrary& p_library):
m_assetContainers(p_library.m_assetContainers),
m_containerByUUID(p_library.m_containerByUUID),
m_unreferencedUUIDs(p_library.m_unreferencedUUIDs)
{
}
AssetLibrary(AssetLibrary&& p_library) = delete;
~AssetLibrary() = default;
/**
* Insert an AssetContainer pointer to the library
*
@@ -30,7 +43,6 @@ class AssetLibrary
* \return The AssetContainer for the Asset type. Nullptr if not found
*/
template<class ASSET>
requires AssetType<ASSET>
[[nodiscard]]
bool Insert(AssetContainer<ASSET>* p_assetContainer)
{
@@ -41,6 +53,7 @@ class AssetLibrary
}
const auto inserted = m_assetContainers.emplace(typeID, p_assetContainer);
p_assetContainer->SetOwningLibrary(this);
return inserted.second;
}
@@ -52,7 +65,15 @@ class AssetLibrary
* \return The AssetContainer for the AssetType. Nullptr if not found
*/
[[nodiscard]]
AssetContainerBase* GetAssetContainer(const AssetTypeID p_typeID);
AssetContainerBase* GetAssetContainer(const AssetTypeID p_typeID)
{
if (const auto container = m_assetContainers.find(p_typeID); container != m_assetContainers.end())
{
return container->second;
}
return nullptr;
}
/**
* Get the AssetContainer for a given AssetType
@@ -61,49 +82,147 @@ class AssetLibrary
* \return The AssetContainer for the AssetType. Nullptr if not found
*/
[[nodiscard]]
const AssetContainerBase* GetAssetContainer(const AssetTypeID p_typeID) const;
const AssetContainerBase* GetAssetContainer(const AssetTypeID p_typeID) const
{
if (const auto container = m_assetContainers.find(p_typeID); container != m_assetContainers.end())
{
return container->second;
}
return nullptr;
}
/**
* Get the AssetContainer for a given UUID
* Find whichever registered AssetContainer currently owns a given UUID, regardless of its concrete
* type. Used to resolve HardReference<AssetBase>/SoftReference<AssetBase> (polymorphic references,
* which only carry a UUID - the concrete type isn't known statically) generically.
*
* \param p_uuid The asset
* \return The AssetContainer for the UUID. Nullptr if not found
* Only finds UUIDs that already have an entry in some container (see AssetContainerBase::
* GetOrCreateEntry / RegisterUUID) - unlike GetAssetContainer(AssetTypeID), there is no "reserve a
* slot for a UUID we haven't seen yet" here, since which container to reserve it in isn't knowable
* without the concrete type.
*
* \param p_uuid The UUID to find
* \return The AssetContainer currently holding p_uuid. Nullptr if none does
*/
[[nodiscard]]
const AssetContainerBase* GetAssetContainer(const UUID& p_uuid) const;
AssetContainerBase* FindContainerForUUID(const UUID& p_uuid)
{
const std::lock_guard lock {m_containerByUUIDMutex};
if (const auto container = m_containerByUUID.find(p_uuid); container != m_containerByUUID.end())
{
return container->second;
}
return nullptr;
}
/**
* Get the AssetContainer for a given UUID
* Publish that p_uuid now lives in p_container, so that FindContainerForUUID can find it. Called by
* AssetContainerBase::GetOrCreateEntry whenever it creates a new entry for a UUID - not meant to be
* called directly.
*
* \param p_uuid The asset
* \return The AssetContainer for the UUID. Nullptr if not found
* \param p_uuid The UUID that now has an entry in p_container
* \param p_container The container the UUID's entry lives in
*/
void RegisterUUID(const UUID& p_uuid, AssetContainerBase* p_container)
{
const std::lock_guard lock {m_containerByUUIDMutex};
m_containerByUUID[p_uuid] = p_container;
}
/**
* Undo RegisterUUID. Called by AssetContainerBase::Remove - not meant to be called directly.
*
* \param p_uuid The UUID to forget
*/
void UnregisterUUID(const UUID& p_uuid)
{
const std::lock_guard lock {m_containerByUUIDMutex};
m_containerByUUID.erase(p_uuid);
}
/**
* Mark p_uuid as a candidate for removal: nothing holds a HardReference to it anymore (its
* ContainerHandle's hard ref count just reached zero). Called by HardReference when the last
* HardReference pointing at an asset is released - not meant to be called directly. See
* SerializationManager::RemoveUnreferencedAssets, which consumes these via TakeUnreferencedUUIDs.
*
* \param p_uuid The UUID whose HardReference count just reached zero
*/
void MarkUnreferenced(const UUID& p_uuid)
{
const std::lock_guard lock {m_unreferencedMutex};
m_unreferencedUUIDs.insert(p_uuid);
}
/**
* Undo MarkUnreferenced: p_uuid is no longer a removal candidate. Called by HardReference whenever a
* new HardReference to p_uuid is acquired (even if it was never marked - erasing an absent key is a
* harmless no-op) - not meant to be called directly.
*
* \param p_uuid The UUID that just gained a HardReference
*/
void UnmarkUnreferenced(const UUID& p_uuid)
{
const std::lock_guard lock {m_unreferencedMutex};
m_unreferencedUUIDs.erase(p_uuid);
}
/**
* Take (and clear) every UUID marked via MarkUnreferenced since the last call. A UUID coming out of
* here is only a candidate: it may have gained a new HardReference since being marked, so callers
* should re-check before actually removing anything (see SerializationManager::RemoveUnreferencedAssets).
*
* \return The UUIDs marked as unreferenced since the last call
*/
[[nodiscard]]
AssetContainerBase* GetAssetContainer(const UUID& p_uuid);
ankerl::unordered_dense::segmented_set<UUID> TakeUnreferencedUUIDs()
{
const std::lock_guard lock {m_unreferencedMutex};
ankerl::unordered_dense::segmented_set<UUID> uuids = std::move(m_unreferencedUUIDs);
m_unreferencedUUIDs.clear();
return uuids;
}
/**
* Check if an Asset is already loaded in the library
*
* \param p_uuid The asset to check
* \return True if already loaded, false otherwise
*/
[[nodiscard]]
bool IsLoaded(const UUID& p_uuid) const;
/**
* Loop through the AssetContainers to remove the pending assets marked for removal
*
*/
void ProcessPendingRemovals();
AssetLibrary& operator=(const AssetLibrary& p_library) = default;
AssetLibrary& operator=(const AssetLibrary& p_library)
{
if (this != &p_library)
{
m_assetContainers = p_library.m_assetContainers;
{
const std::lock_guard lock {m_containerByUUIDMutex};
m_containerByUUID = p_library.m_containerByUUID;
}
const std::lock_guard lock {m_unreferencedMutex};
m_unreferencedUUIDs = p_library.m_unreferencedUUIDs;
}
return *this;
}
AssetLibrary& operator=(AssetLibrary&& p_library) = delete;
~AssetLibrary() = default;
private:
// All the asset containers
ankerl::unordered_dense::map<AssetTypeID, AssetContainerBase*> m_assetContainers;
ankerl::unordered_dense::segmented_map<AssetTypeID, AssetContainerBase*> m_assetContainers;
// Guards m_containerByUUID: SerializationManager::LoadAssets resolves Hard/SoftReference<AssetBase>
// fields (which call RegisterUUID/FindContainerForUUID) from multiple threads at once while decoding a
// batch, and different AssetContainers - each with their own AssetContainerBase::m_entryMutex - can
// both be publishing a newly-seen UUID into this single shared map at the same time.
std::mutex m_containerByUUIDMutex;
// Every UUID that has an entry in some container, and which container that is. See FindContainerForUUID.
ankerl::unordered_dense::segmented_map<UUID, AssetContainerBase*> m_containerByUUID;
// Guards m_unreferencedUUIDs only: SerializationManager::LoadAssets resolves HardReference fields
// (which call MarkUnreferenced/UnmarkUnreferenced) from multiple threads at once while decoding a
// batch. m_assetContainers/m_containerByUUID need no such guard - they only ever mutate during
// AssetLibrary::Insert/RegisterUUID/UnregisterUUID, which LoadAssets always does single-threaded (see
// AssetContainerBase::InsertDecoded).
std::mutex m_unreferencedMutex;
// UUIDs whose HardReference count reached zero since the last TakeUnreferencedUUIDs. See MarkUnreferenced.
ankerl::unordered_dense::segmented_set<UUID> m_unreferencedUUIDs;
};
/****************************************************************************************/
@@ -15,10 +15,13 @@ using AssetTypeID = std::uint64_t;
/****************************************************************************************/
namespace Flat::Bigfoot
namespace Flat
{
namespace Bigfoot
{
struct AssetTypeID;
} // namespace Flat::Bigfoot
}
} // namespace Flat
/****************************************************************************************/
@@ -53,6 +53,7 @@ struct AssetHeaderT : public ::flatbuffers::NativeTable {
eastl::string name{};
::Bigfoot::AssetTypeID type_id{};
eastl::string type_name{};
uint32_t version = 0;
};
struct AssetHeader FLATBUFFERS_FINAL_CLASS : private ::flatbuffers::Table {
@@ -69,7 +70,8 @@ struct AssetHeader FLATBUFFERS_FINAL_CLASS : private ::flatbuffers::Table {
VT_UUID = 4,
VT_NAME = 6,
VT_TYPE_ID = 8,
VT_TYPE_NAME = 10
VT_TYPE_NAME = 10,
VT_VERSION = 12
};
const Flat::Bigfoot::UUID *uuid() const {
return GetStruct<const Flat::Bigfoot::UUID *>(VT_UUID);
@@ -83,6 +85,9 @@ struct AssetHeader FLATBUFFERS_FINAL_CLASS : private ::flatbuffers::Table {
const ::flatbuffers::String *type_name() const {
return GetPointer<const ::flatbuffers::String *>(VT_TYPE_NAME);
}
uint32_t version() const {
return GetField<uint32_t>(VT_VERSION, 0);
}
template <bool B = false>
bool Verify(::flatbuffers::VerifierTemplate<B> &verifier) const {
return VerifyTableStart(verifier) &&
@@ -92,6 +97,7 @@ struct AssetHeader FLATBUFFERS_FINAL_CLASS : private ::flatbuffers::Table {
VerifyFieldRequired<Flat::Bigfoot::AssetTypeID>(verifier, VT_TYPE_ID, 8) &&
VerifyOffsetRequired(verifier, VT_TYPE_NAME) &&
verifier.VerifyString(type_name()) &&
VerifyField<uint32_t>(verifier, VT_VERSION, 4) &&
verifier.EndTable();
}
AssetHeaderT *UnPack(const ::flatbuffers::resolver_function_t *_resolver = nullptr) const;
@@ -115,6 +121,9 @@ struct AssetHeaderBuilder {
void add_type_name(::flatbuffers::Offset<::flatbuffers::String> type_name) {
fbb_.AddOffset(AssetHeader::VT_TYPE_NAME, type_name);
}
void add_version(uint32_t version) {
fbb_.AddElement<uint32_t>(AssetHeader::VT_VERSION, version, 0);
}
explicit AssetHeaderBuilder(::flatbuffers::FlatBufferBuilder &_fbb)
: fbb_(_fbb) {
start_ = fbb_.StartTable();
@@ -135,8 +144,10 @@ inline ::flatbuffers::Offset<AssetHeader> CreateAssetHeader(
const Flat::Bigfoot::UUID *uuid = nullptr,
::flatbuffers::Offset<::flatbuffers::String> name = 0,
const Flat::Bigfoot::AssetTypeID *type_id = nullptr,
::flatbuffers::Offset<::flatbuffers::String> type_name = 0) {
::flatbuffers::Offset<::flatbuffers::String> type_name = 0,
uint32_t version = 0) {
AssetHeaderBuilder builder_(_fbb);
builder_.add_version(version);
builder_.add_type_name(type_name);
builder_.add_type_id(type_id);
builder_.add_name(name);
@@ -156,7 +167,7 @@ struct AssetT : public ::flatbuffers::NativeTable {
static FLATBUFFERS_CONSTEXPR_CPP11 const char *GetFullyQualifiedName() {
return "Flat.Bigfoot.AssetT";
}
eastl::unique_ptr<Flat::Bigfoot::AssetHeaderT> header{new Flat::Bigfoot::AssetHeaderT()};
eastl::unique_ptr<Flat::Bigfoot::AssetHeaderT> header{};
eastl::vector<uint8_t> inner_asset{};
AssetT() = default;
AssetT(const AssetT &o);
@@ -251,6 +262,7 @@ inline void AssetHeader::UnPackTo(AssetHeaderT *_o, const ::flatbuffers::resolve
{ auto _e = name(); if (_e) _o->name = eastl::string(_e->c_str(), _e->size()); }
{ auto _e = type_id(); if (_e) _o->type_id = ::flatbuffers::UnPack(*_e); }
{ auto _e = type_name(); if (_e) _o->type_name = eastl::string(_e->c_str(), _e->size()); }
{ auto _e = version(); _o->version = _e; }
}
inline ::flatbuffers::Offset<AssetHeader> CreateAssetHeader(::flatbuffers::FlatBufferBuilder &_fbb, const AssetHeaderT *_o, const ::flatbuffers::rehasher_function_t *_rehasher) {
@@ -265,12 +277,14 @@ inline ::flatbuffers::Offset<AssetHeader> AssetHeader::Pack(::flatbuffers::FlatB
auto _name = _fbb.CreateString(_o->name);
auto _type_id = ::flatbuffers::Pack(_o->type_id);
auto _type_name = _fbb.CreateString(_o->type_name);
auto _version = _o->version;
return Flat::Bigfoot::CreateAssetHeader(
_fbb,
&_uuid,
_name,
&_type_id,
_type_name);
_type_name,
_version);
}
inline AssetT::AssetT(const AssetT &o)
@@ -318,14 +332,15 @@ inline const ::flatbuffers::TypeTable *AssetHeaderTypeTable() {
{ ::flatbuffers::ET_SEQUENCE, 0, 0 },
{ ::flatbuffers::ET_STRING, 0, -1 },
{ ::flatbuffers::ET_SEQUENCE, 0, 1 },
{ ::flatbuffers::ET_STRING, 0, -1 }
{ ::flatbuffers::ET_STRING, 0, -1 },
{ ::flatbuffers::ET_UINT, 0, -1 }
};
static const ::flatbuffers::TypeFunction type_refs[] = {
Flat::Bigfoot::UUIDTypeTable,
Flat::Bigfoot::AssetTypeIDTypeTable
};
static const ::flatbuffers::TypeTable tt = {
::flatbuffers::ST_TABLE, 4, type_codes, type_refs, nullptr, nullptr, nullptr
::flatbuffers::ST_TABLE, 5, type_codes, type_refs, nullptr, nullptr, nullptr
};
return &tt;
}
@@ -6,11 +6,11 @@
*********************************************************************/
#ifndef BIGFOOT_ENGINE_ASSET_REFERENCE_HPP
#define BIGFOOT_ENGINE_ASSET_REFERENCE_HPP
#include <Engine/Asset/AssetConcepts.hpp>
#include <Engine/Asset/AssetLibrary.hpp>
#include <System/UUID/UUID.hpp>
#include <type_traits>
#include <utility>
namespace Bigfoot
@@ -18,45 +18,25 @@ namespace Bigfoot
template<class ASSET>
AssetTypeID GetAssetTypeID();
template<>
AssetTypeID GetAssetTypeID<AssetBase>() = delete;
template<class DERIVED, class ASSET>
class ReferenceBase
{
public:
/**
* Get the UUID of the reference
*
* \return The UUID of the reference
*/
[[nodiscard]]
const UUID& GetUUID() const
{
return m_uuid;
}
/**
* Get the asset
*
* \return The asset, nullptr if it is not loaded
*/
[[nodiscard]]
const ASSET* Get() const
{
static_assert(AssetType<ASSET>, "ASSET must derive from AssetBase (and not be AssetBase itself)");
return m_handle ? static_cast<const ASSET*>(m_handle->GetAsset()) : nullptr;
}
/**
* Get the asset
*
* \return The asset, nullptr if it is not loaded
*/
[[nodiscard]]
ASSET* Get()
{
static_assert(AssetType<ASSET>, "ASSET must derive from AssetBase (and not be AssetBase itself)");
return m_handle ? static_cast<ASSET*>(m_handle->GetAsset()) : nullptr;
}
@@ -94,9 +74,7 @@ class ReferenceBase
return m_uuid == p_reference.m_uuid;
}
private:
friend DERIVED;
protected:
ReferenceBase() = default;
ReferenceBase(const UUID& p_uuid):
@@ -118,11 +96,7 @@ class ReferenceBase
{
}
/**
* Get the associated handle container for the asset
* Register the reference inside the handle
*
*/
private:
void Acquire()
{
if (!m_uuid)
@@ -130,44 +104,45 @@ class ReferenceBase
return;
}
if (AssetContainerBase* assetContainer = g_activeAssetLibrary->GetAssetContainer(GetAssetTypeID<ASSET>()))
// AssetBase has no AssetContainer of its own (it's not a concrete, loadable type): a
// Hard/SoftReference<AssetBase> can point at an asset of any type, so its container has to be
// found from the UUID alone rather than via the usual static GetAssetTypeID<ASSET>() lookup. This
// only finds assets that are already loaded somewhere - see AssetLibrary::FindContainerForUUID.
AssetContainerBase* assetContainer;
if constexpr (std::is_same_v<ASSET, AssetBase>)
{
assetContainer = g_activeAssetLibrary->FindContainerForUUID(m_uuid);
}
else
{
assetContainer = g_activeAssetLibrary->GetAssetContainer(GetAssetTypeID<ASSET>());
}
if (assetContainer)
{
if (!m_handle)
{
m_handle = assetContainer->AcquireContainerHandle(m_uuid);
}
DERIVED::IncrementCount(*m_handle);
DERIVED::OnCountIncremented(m_uuid, *m_handle);
}
}
/**
* Unregister the reference from the handle
*
*/
void Release()
{
if (m_handle)
{
DERIVED::DecrementCount(*m_handle);
DERIVED::OnCountDecremented(m_uuid, *m_handle);
m_handle = nullptr;
}
}
/**
* The UUID
*/
UUID m_uuid = UUID::NULL_UUID;
/**
* Handle for the asset
*/
ContainerHandle* m_handle = nullptr;
};
/*
* Owning reference
* Keeps the asset alive
*/
template<class ASSET>
class HardReference: public ReferenceBase<HardReference<ASSET>, ASSET>
{
@@ -182,26 +157,34 @@ class HardReference: public ReferenceBase<HardReference<ASSET>, ASSET>
}
private:
/**
* Increment the hard ref count of the handle
*
* \param p_handle The handle for the asset
*/
static void IncrementCount(ContainerHandle& p_handle)
{
p_handle.IncrementHardRefCount();
}
/**
* Decrement the hard ref count of the handle
*
* \param p_handle The handle for the asset
*/
static void DecrementCount(ContainerHandle& p_handle)
{
p_handle.DecrementHardRefCount();
}
// Registers p_uuid as a candidate for SerializationManager::RemoveUnreferencedAssets once the last
// HardReference pointing at it is released - instead of that having to scan every loaded asset to
// find the ones nothing holds a HardReference to anymore.
static void OnCountDecremented(const UUID& p_uuid, const ContainerHandle& p_handle)
{
if (p_handle.GetHardRefCount() == 0)
{
g_activeAssetLibrary->MarkUnreferenced(p_uuid);
}
}
// A new HardReference to p_uuid means it's no longer a candidate for removal, even if it was
// previously marked (its count had reached zero before, but has since been acquired again).
static void OnCountIncremented(const UUID& p_uuid, const ContainerHandle&)
{
g_activeAssetLibrary->UnmarkUnreferenced(p_uuid);
}
friend Base;
};
@@ -219,26 +202,27 @@ class SoftReference: public ReferenceBase<SoftReference<ASSET>, ASSET>
}
private:
/**
* Increment the soft ref count of the handle
*
* \param p_handle The handle for the asset
*/
static void IncrementCount(ContainerHandle& p_handle)
{
p_handle.IncrementSoftRefCount();
}
/**
* Decrement the hard ref count of the handle
*
* \param p_handle The handle for the asset
*/
static void DecrementCount(ContainerHandle& p_handle)
{
p_handle.DecrementSoftRefCount();
}
// SoftReferences don't keep an asset alive, so releasing one is never a reason to consider the asset
// for removal.
static void OnCountDecremented(const UUID&, const ContainerHandle&)
{
}
// SoftReferences don't affect removal eligibility - only HardReference does.
static void OnCountIncremented(const UUID&, const ContainerHandle&)
{
}
friend Base;
};
} // namespace Bigfoot
@@ -0,0 +1,226 @@
/*********************************************************************
* \file SerializationManager.hpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#ifndef BIGFOOT_ENGINE_ASSET_SERIALIZATIONMANAGER_HPP
#define BIGFOOT_ENGINE_ASSET_SERIALIZATIONMANAGER_HPP
#include <Engine/Asset/AssetLibrary.hpp>
#include <Engine/Asset/Asset_generated.hpp>
#if defined BIGFOOT_TOOL
#include <Engine/BigFile/BigFile.hpp>
#else
#include <Engine/Sasquatch/Sasquatch.hpp>
#endif
#include <System/UUID/UUID.hpp>
#include <EASTL/span.h>
#include <EASTL/unique_ptr.h>
#include <EASTL/vector.h>
#include <ankerl/unordered_dense.h>
#include <flatbuffers/flatbuffers.h>
#include <cstdint>
namespace Bigfoot
{
/*
* Bridges the in-memory AssetLibrary/AssetContainer world with an on-disk asset store: loads assets (and
* their hard-referenced assets, recursively) into the active AssetLibrary, and - Tool mode only - saves
* assets (along with the set of assets they hold a HardReference to) back out.
*
* The backend is picked at compile time: Tool mode (BIGFOOT_TOOL) reads/writes a BigFile directly over
* SQL; Game mode (BIGFOOT_GAME) only ever reads a Sasquatch, the mmap'd, load-only flattening of a BigFile
* produced ahead of time by Sasquatch::WriteFrom or the SasquatchPacker tool. SaveAsset does not exist in
* Game mode - a running game never writes back to its own asset pack.
*/
class SerializationManager
{
public:
#if defined BIGFOOT_TOOL
/*
* Constructor
*
* \param p_bigFile The BigFile to load assets from / save assets to
*/
explicit SerializationManager(BigFile& p_bigFile);
#else
/*
* Constructor
*
* \param p_sasquatch The Sasquatch to load assets from
*/
explicit SerializationManager(Sasquatch& p_sasquatch);
#endif
SerializationManager(const SerializationManager& p_manager) = default;
SerializationManager(SerializationManager&& p_manager) = default;
~SerializationManager() = default;
/*
* Loads an asset from the BigFile into the active AssetLibrary. Every asset it (transitively) holds a
* HardReference to is loaded first, recursively, so that by the time this asset itself is constructed
* its own reference fields - including polymorphic HardReference<AssetBase>/SoftReference<AssetBase>,
* which can only resolve to an already-loaded asset - immediately find their target. SoftReferences
* are never eagerly loaded.
*
* Does nothing if the UUID is null, is not present in the BigFile, or has already been loaded. Also
* safe against HardReference cycles: a UUID already in the middle of being loaded (an ancestor in the
* current recursion) is skipped rather than re-entered.
*
* \param p_uuid The UUID of the asset to load
*/
void LoadAsset(const UUID& p_uuid);
#if !defined BIGFOOT_TOOL
/*
* Loads every asset in p_uuids into the active AssetLibrary, and every asset they (transitively) hold
* a HardReference to - same end result as calling LoadAsset(uuid) once per uuid, except the CPU-heavy
* part (Asset::Unpack, which deserializes the flatbuffer and resolves the asset's own reference
* fields) runs across multiple threads instead of one at a time.
*
* Assets are grouped into dependency layers - an asset's layer is one past the deepest layer among
* what it HardReferences, so every HardReference target is fully registered before its referrer is
* even decoded - each layer is decoded in parallel (AssetContainerBase::DecodeOffContainer, which
* touches nothing but the asset being decoded) and then that layer is registered serially
* (AssetContainerBase::InsertDecoded) before moving on to the next. HardReference/SoftReference
* resolution during Unpack is safe to run concurrently across a layer (see AssetContainer.hpp's
* m_entryMutex and AssetLibrary.hpp's mutexes) but container registration itself is not, hence the
* layering.
*
* Same skip rules as LoadAsset: a null, not-present, or already-loaded UUID is silently skipped, and a
* UUID reachable through more than one path (shared by several requested assets, or requested directly
* as well as referenced) is only ever decoded once. Unlike LoadAsset, a HardReference cycle across the
* batch asserts rather than being silently skipped - Sasquatch-packed data should never contain one
* (WriteFrom always runs against a BigFile, and nothing writes a cyclic HardReference graph into a
* BigFile in the first place).
*
* \param p_uuids The UUIDs to load
*/
void LoadAssets(const eastl::span<const UUID> p_uuids);
#endif
#if defined BIGFOOT_TOOL
/*
* Saves the asset currently loaded under p_uuid (in the active AssetLibrary, found the same way
* HardReference<AssetBase>/SoftReference<AssetBase> resolve a bare UUID - see
* AssetLibrary::FindContainerForUUID) to the BigFile: inserts it if its UUID is not present in the
* BigFile yet, overwrites it otherwise.
*
* Every asset it holds a HardReference or SoftReference to is discovered generically via flatbuffers'
* mini-reflection (walking the packed bytes using AssetBase::GetInnerTypeTable()) and (re)written to
* the AssetHardReference/AssetSoftReference tables, so that LoadAsset can later recursively load its
* HardReferences without having to deserialize the asset itself again, and so the full reference graph
* (Hard and Soft alike) is queryable directly in SQL for debugging.
*
* Does nothing if the UUID is null. Asserts if no asset is currently loaded under p_uuid: unlike
* LoadAsset (which treats "nothing to load" as a legitimate, silent case), asking to save a UUID
* nothing is loaded for is a programmer error.
*
* Tool-only: a running game never writes back to its own asset pack, see the class comment.
*
* \param p_uuid The UUID of the (currently loaded) asset to save
*/
void SaveAsset(const UUID& p_uuid);
/*
* Reads just an asset's header/metadata (name, type, version) straight from the BigFile's AssetHeader
* table, without touching the (often much bigger) Asset blob column and without unpacking anything -
* cheap enough to call for e.g. an asset browser or a dependency listing, unlike LoadAsset.
*
* Tool-only: Sasquatch doesn't carry this metadata (Game mode never needs an asset browser), so this
* has no Game-mode equivalent.
*
* \param p_uuid The UUID of the asset whose header to read
* \return The header, or nullptr if p_uuid is null or not present in the BigFile
*/
[[nodiscard]]
eastl::unique_ptr<::Flat::Bigfoot::AssetHeaderT> GetAssetHeader(const UUID& p_uuid) const;
#endif
/*
* Removes every asset currently marked as unreferenced - nothing holds a HardReference to it anymore,
* see HardReference/AssetLibrary::MarkUnreferenced - from the active AssetLibrary. Each candidate's
* hard ref count is re-checked before actually removing it, since a new HardReference could have been
* acquired after it was marked, so this is safe to call at any time.
*
* This only unloads assets from memory: it does not touch the BigFile, an unreferenced asset can
* still be found again later via LoadAsset.
*/
void RemoveUnreferencedAssets();
SerializationManager& operator=(const SerializationManager& p_manager) = default;
SerializationManager& operator=(SerializationManager&& p_manager) = default;
#if defined BIGFOOT_TOOL
/*
* The UUIDs a single asset's HardReference and SoftReference fields point at, as discovered by
* CollectReferenceUUIDs. Public only so the reflection visitor implementing CollectReferenceUUIDs
* (an unrelated class, in SerializationManager.cpp's anonymous namespace) can fill one in directly.
*/
struct ReferenceUUIDs
{
eastl::vector<UUID> m_hard;
eastl::vector<UUID> m_soft;
};
#endif
private:
[[nodiscard]]
static eastl::span<const std::byte> UUIDToSpan(const UUID& p_uuid);
[[nodiscard]]
static UUID SpanToUUID(const eastl::span<const std::byte> p_bytes);
#if defined BIGFOOT_TOOL
/*
* Generically collects the UUIDs of every HardReference and SoftReference struct found while walking
* p_buffer (a standalone flatbuffer, e.g. the inner bytes of an Asset<ASSET>::Pack() buffer) using
* p_typeTable (obtained via AssetBase::GetInnerTypeTable()).
*
* \param p_buffer The flatbuffer to walk
* \param p_typeTable The TypeTable describing p_buffer's root type
* \return The UUIDs of every HardReference/SoftReference found
*/
[[nodiscard]]
static ReferenceUUIDs CollectReferenceUUIDs(const std::uint8_t* p_buffer,
const flatbuffers::TypeTable* p_typeTable);
/*
* Writes one AssetHardReference/AssetSoftReference row per UUID in p_referencedUUIDs, all under
* p_uuidBytes. p_insertQuery picks which of the two tables - it must be an INSERT INTO
* AssetHardReference/AssetSoftReference (UUID, ReferencedUUID) VALUES (?1, ?2) ... statement, ?1 bound
* to the referrer's UUID and ?2 to each referenced UUID in turn.
*
* \param p_insertQuery The INSERT statement to run once per referenced UUID
* \param p_uuidBytes The referrer's UUID, as bound to ?1
* \param p_referencedUUIDs The UUIDs to write one row for each, bound to ?2 in turn
*/
void InsertReferences(const eastl::string_view p_insertQuery,
const eastl::span<const std::byte> p_uuidBytes,
const eastl::vector<UUID>& p_referencedUUIDs);
/*
* The BigFile assets are loaded from / saved to
*/
BigFile* m_bigFile;
#else
/*
* The Sasquatch assets are loaded from
*/
Sasquatch* m_sasquatch;
#endif
/*
* UUIDs currently in the middle of being loaded (an active LoadAsset call is somewhere up the
* recursion stack for them). Used to detect and skip HardReference cycles instead of recursing
* forever - see LoadAsset.
*/
ankerl::unordered_dense::segmented_set<UUID> m_loadingInProgress;
};
} // namespace Bigfoot
#endif
@@ -6,24 +6,20 @@
*********************************************************************/
#ifndef BIGFOOT_ENGINE_BIGFILE_BIGFILE_HPP
#define BIGFOOT_ENGINE_BIGFILE_BIGFILE_HPP
#include <Engine/DataAccessor/DataAccessor.hpp>
#include <System/File/File.hpp>
#include <System/Time/Time.hpp>
#include <System/File.hpp>
#include <Utils/TaggedType.hpp>
#include <EASTL/optional.h>
#include <EASTL/span.h>
#include <EASTL/string_view.h>
#include <EASTL/vector.h>
#include <ankerl/unordered_dense.h>
#include <sqlite3.h>
#include <span>
#include <string>
namespace Bigfoot
{
#ifdef BIGFOOT_TOOLMODE
class BigFile: public DataAccessor
class BigFile
{
public:
/**
@@ -54,107 +50,7 @@ class BigFile: public DataAccessor
*/
void RollbackTransaction();
/**
* Clear the BigFile
*
*/
void Clear();
~BigFile() override;
/**
* Get an Asset flatbuffer
*
* \param p_uuid The asset to get the flatbuffer of
* \return The flatbuffer, or an empty vector if the asset was not found
*/
[[nodiscard]]
eastl::vector<std::byte> GetAssetFlatbuffer(const UUID& p_uuid) const override;
/**
* Get several Asset flatbuffers
*
* \param p_uuids The assets to get the flatbuffers of
* \return The flatbuffers that were found
*/
[[nodiscard]]
eastl::vector<eastl::vector<std::byte>> GetAssetFlatbuffers(eastl::span<const UUID> p_uuids) const override;
/**
* Get the HardReferences of a given asset
*
* \param p_uuid The asset to get the HardReferences of
* \return The HardReferences of the asset
*/
[[nodiscard]]
eastl::vector<UUID> GetAssetHardReferences(const UUID& p_uuid) const override;
/**
* Get the HardReferences of given assets
*
* \param p_uuids The assets to get the HardReferences of
* \return The HardReferences of the assets
*/
[[nodiscard]]
eastl::vector<eastl::vector<UUID>> GetAssetsHardReferences(eastl::span<const UUID> p_uuids) const override;
struct AssetTime
{
Time m_creationTime;
Time m_modificationTime;
[[nodiscard]]
bool operator==(const AssetTime& p_other) const = default;
};
/**
* Get the creation and modification time of an asset
*
* \param p_uuid The asset to get the time of
* \return The time of the asset, nullopt if the asset was not found
*/
[[nodiscard]]
eastl::optional<AssetTime> GetAssetTime(const UUID& p_uuid) const;
/**
* Get the creation and modification time of several assets
*
* \param p_uuids The assets to get the time of
* \return The time of the assets. nullopt for assets not found
*/
[[nodiscard]]
eastl::vector<eastl::optional<AssetTime>> GetAssetsTime(eastl::span<const UUID> p_uuids) const;
/**
* Save an asset
*
* \param p_uuid The UUID
* \param p_flatbuffer The flatbuffer of the asset
* \param p_hardReferences The HardReferences the asset points to
*/
void SaveAssetFlatbuffer(const UUID& p_uuid,
const eastl::span<const std::byte> p_flatbuffer,
const eastl::span<const UUID> p_hardReferences) override;
/**
* Save several assets
*
* \param p_uuids The UUIDs
* \param p_flatbuffers The flatbuffers of the assets, matching p_metadatas by index
* \param p_hardReferences The HardReferences the asset points to
*/
void SaveAssetFlatbuffers(const eastl::span<const UUID> p_uuids,
const eastl::span<const eastl::vector<std::byte>> p_flatbuffers,
const eastl::span<const eastl::vector<UUID>> p_hardReferences) override;
/**
* Check if the Asset exists
*
* \param p_uuid The asset to check
* \param True if it exists, false otherwise
*/
[[nodiscard]]
bool Has(const UUID& p_uuid) const override;
~BigFile();
BigFile& operator=(const BigFile& p_bigFile) = delete;
BigFile& operator=(BigFile&& p_bigFile) = delete;
@@ -248,7 +144,7 @@ class BigFile: public DataAccessor
/*
* Step through the request
*
* \return True if a row is available to read via Get(), false once the statement is done
* \return True if the request can continue (get next row), false otherwise
*/
[[nodiscard]]
bool Step();
@@ -256,7 +152,7 @@ class BigFile: public DataAccessor
/*
* Execute the request once
*
* \return The number of rows modified by the statement
* The number of modified rows
*/
[[maybe_unused]]
std::uint32_t Execute();
@@ -272,6 +168,14 @@ class BigFile: public DataAccessor
*/
Column(const Request& p_request, const std::uint32_t p_index);
Column(const Column& p_column) = default;
Column(Column&& p_column) = default;
~Column() = default;
Column& operator=(const Column& p_column) = default;
Column& operator=(Column&& p_column) = default;
/*
* Get value as a int32
*
@@ -328,13 +232,6 @@ class BigFile: public DataAccessor
*/
operator eastl::span<const std::byte>() const;
/*
* Get value as a blob
*
* \return The value
*/
operator std::span<const std::byte>() const;
private:
/*
* The statement
@@ -366,19 +263,61 @@ class BigFile: public DataAccessor
*/
sqlite3* m_db;
/*
* The BigFile this statement was acquired from - kept only to return the statement to its pool
* (see BigFile::ReleaseStatement) on destruction.
*/
const BigFile* m_owner;
/*
* The query text this statement was prepared from - identifies which pool bucket to return it to.
*/
eastl::string_view m_queryText;
/*
* The statement
*/
sqlite3_stmt* m_statement = nullptr;
sqlite3_stmt* m_statement;
};
private:
/*
* Borrows a compiled statement for p_query: an idle one from m_statementPool if one's available,
* freshly prepared otherwise. Pooled per query text rather than cached as a single instance per text
* because SerializationManager::LoadAsset recurses: a nested call for the same query text must get
* its own statement, not reset the one an ancestor call further up the stack is still stepping
* through.
*
* \param p_query The query text to get a statement for
* \return A statement ready to bind and step, reset to its start state
*/
[[nodiscard]]
sqlite3_stmt* AcquireStatement(const eastl::string_view p_query) const;
/*
* Returns a statement borrowed via AcquireStatement to its pool, for the next AcquireStatement(p_query)
* to reuse instead of recompiling.
*
* \param p_query The query text p_statement was acquired for
* \param p_statement The statement to return
*/
void ReleaseStatement(const eastl::string_view p_query, sqlite3_stmt* p_statement) const;
/**
* The BigFile DB
*/
sqlite3* m_db = nullptr;
sqlite3* m_db;
/*
* Idle, already-compiled statements, keyed by their query text and reused by AcquireStatement/
* ReleaseStatement instead of recompiling identical SQL on every BigFile::Request. Mutable: acquiring/
* releasing a statement doesn't change what a BigFile logically contains, only this cache of it -
* every BigFile::Request constructor takes a const BigFile&. Keyed by std::string rather than
* eastl::string: no std::hash<eastl::string> specialization exists for ankerl::unordered_dense to key
* off of, and this cache is purely a private implementation detail.
*/
mutable ankerl::unordered_dense::segmented_map<std::string, eastl::vector<sqlite3_stmt*>> m_statementPool;
};
#endif
} // namespace Bigfoot
#endif
@@ -9,8 +9,6 @@
#include <EASTL/string_view.h>
#if defined BIGFOOT_TOOLMODE
namespace Bigfoot
{
/*
@@ -18,5 +16,4 @@ namespace Bigfoot
*/
constexpr eastl::string_view BIGFILE_@BIGFILE_NAME@_LOCATION {"@BIGFILE_LOCATION@"};
} // namespace Bigfoot
#endif
#endif
@@ -1,145 +0,0 @@
/*********************************************************************
* \file DataAccessor.hpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#ifndef BIGFOOT_ENGINE_DATAACCESSOR_DATAACCESSOR_HPP
#define BIGFOOT_ENGINE_DATAACCESSOR_DATAACCESSOR_HPP
#include <Engine/Asset/AssetTypeID.hpp>
#include <System/Time/Time.hpp>
#include <System/UUID/UUID.hpp>
#include <EASTL/span.h>
#include <EASTL/string.h>
#include <EASTL/vector.h>
namespace Bigfoot
{
class DataAccessor
{
public:
DataAccessor() = default;
DataAccessor(const DataAccessor& p_accessor) = default;
DataAccessor(DataAccessor&& p_accessor) = default;
virtual ~DataAccessor() = default;
/**
* Get an Asset flatbuffer
*
* \param p_uuid The asset to get the flatbuffer of
* \return The flatbuffer, or an empty vector if the asset was not found
*/
[[nodiscard]]
virtual eastl::vector<std::byte> GetAssetFlatbuffer(const UUID& p_uuid) const = 0;
/**
* Get several Asset flatbuffers
*
* \param p_uuids The assets to get the flatbuffers of
* \return The flatbuffers that were found
*/
[[nodiscard]]
virtual eastl::vector<eastl::vector<std::byte>> GetAssetFlatbuffers(eastl::span<const UUID> p_uuids) const = 0;
/**
* Get the HardReferences of a given asset
*
* \param p_uuid The asset to get the HardReferences of
* \return The HardReferences of the asset
*/
[[nodiscard]]
virtual eastl::vector<UUID> GetAssetHardReferences(const UUID& p_uuid) const = 0;
/**
* Get the HardReferences of given assets
*
* \param p_uuids The assets to get the HardReferences of
* \return The HardReferences of the assets
*/
[[nodiscard]]
virtual eastl::vector<eastl::vector<UUID>> GetAssetsHardReferences(eastl::span<const UUID> p_uuids) const = 0;
/**
* Save an asset
*
* \param p_uuid The UUID
* \param p_flatbuffer The flatbuffer of the asset
* \param p_hardReferences The HardReferences the asset points to
*/
virtual void SaveAssetFlatbuffer(const UUID& p_uuid,
const eastl::span<const std::byte> p_flatbuffer,
const eastl::span<const UUID> p_hardReferences) = 0;
/**
* Save several assets
*
* \param p_uuids The UUIDs
* \param p_flatbuffers The flatbuffers of the assets, matching p_metadatas by index
* \param p_hardReferences The HardReferences the asset points to
*/
virtual void SaveAssetFlatbuffers(const eastl::span<const UUID> p_uuids,
const eastl::span<const eastl::vector<std::byte>> p_flatbuffers,
const eastl::span<const eastl::vector<UUID>> p_hardReferences) = 0;
/**
* Check if the Asset exists
*
* \param p_uuid The asset to check
* \param True if it exists, false otherwise
*/
[[nodiscard]]
virtual bool Has(const UUID& p_uuid) const = 0;
DataAccessor& operator=(const DataAccessor& p_accessor) = default;
DataAccessor& operator=(DataAccessor&& p_accessor) = default;
};
/****************************************************************************************/
// The global data accessor for Bigfoot
inline DataAccessor* g_dataAccessor = nullptr;
/****************************************************************************************/
/*
* Set the data accessor
*
* \param p_dataAccessor The data accessor to set g_dataAccessor to.
*/
void SetDataAccessor(DataAccessor* p_dataAccessor);
/****************************************************************************************/
class ScopedDataAccessor
{
public:
ScopedDataAccessor(DataAccessor* p_dataAccessor):
m_previous(g_dataAccessor)
{
SetDataAccessor(p_dataAccessor);
}
ScopedDataAccessor(const ScopedDataAccessor& p_scopedDataAccessor) = delete;
ScopedDataAccessor(ScopedDataAccessor&& p_scopedDataAccessor) = delete;
~ScopedDataAccessor()
{
SetDataAccessor(m_previous);
}
ScopedDataAccessor& operator=(const ScopedDataAccessor& p_scopedDataAccessor) = delete;
ScopedDataAccessor& operator=(ScopedDataAccessor&& p_scopedDataAccessor) = delete;
private:
/*
* The previous data accessor.
*/
DataAccessor* m_previous = nullptr;
};
} // namespace Bigfoot
#endif
@@ -10,7 +10,7 @@
#include <Utils/Assert/AssertHandler.hpp>
#ifdef BIGFOOT_NOT_OPTIMIZED
#if defined BIGFOOT_NOT_OPTIMIZED
namespace Bigfoot
{
@@ -15,7 +15,7 @@ namespace Bigfoot
/*
* Logger
*/
inline Log::LoggerInfo ENGINE_LOGGER { .m_name = "ENGINE_LOGGER", .m_level = Flat::LogLevel::Trace};
inline Log::LoggerInfo ENGINE_LOGGER {"ENGINE_LOGGER", Flat::LogLevel::Trace};
} // namespace Bigfoot
#endif
#endif
@@ -6,238 +6,101 @@
*********************************************************************/
#ifndef BIGFOOT_ENGINE_SASQUATCH_SASQUATCH_HPP
#define BIGFOOT_ENGINE_SASQUATCH_SASQUATCH_HPP
#include <Engine/DataAccessor/DataAccessor.hpp>
#include <Engine/Asset/AssetTypeID.hpp>
#if defined BIGFOOT_TOOL
#include <Engine/BigFile/BigFile.hpp>
#endif
#include <System/File/File.hpp>
#include <System/File/MemoryMappedFile.hpp>
#include <System/File.hpp>
#include <System/UUID/UUID.hpp>
#include <Utils/TargetMacros.h>
#include <EASTL/span.h>
#include <EASTL/vector.h>
#include <ankerl/unordered_dense.h>
#include <EASTL/unique_ptr.h>
#include <cstddef>
#include <cstdint>
#include <type_traits>
#include <optional>
namespace Bigfoot
{
struct SasquatchHeader
{
static constexpr std::uint32_t MAGIC = 0x53515443; // 'SQTC'
static constexpr std::uint32_t VERSION = 1;
std::uint32_t m_magic = MAGIC;
std::uint32_t m_version = VERSION;
std::uint64_t m_assetCount = 0;
};
static_assert(std::is_trivially_copyable_v<SasquatchHeader>);
/****************************************************************************************/
struct SasquatchEntry
{
std::array<std::byte, UUID::UUID_BYTE_SIZE> m_uuid {};
std::size_t m_assetOffset = 0;
std::size_t m_assetSize = 0;
std::size_t m_hardReferencesOffset = 0;
std::size_t m_hardReferencesSize = 0;
};
static_assert(std::is_trivially_copyable_v<SasquatchEntry>);
/****************************************************************************************/
#if defined(BIGFOOT_TOOLMODE) || defined(BIGFOOT_BUILD_TESTS) || defined(BIGFOOT_BUILD_BENCHMARKS)
class SasquatchWriter
/*
* Read-only, mmap'd counterpart to a BigFile: every Asset/AssetHeader row (and the UUIDs it holds a
* HardReference to) flattened into one file, so a running game can look an asset up by UUID with no SQL,
* no allocation and no per-lookup syscall. See SerializationManager, which uses this as its Game-mode
* backend (BigFile is the Tool-mode one) - Game builds only ever construct one of these and call Find, they
* never write one (see WriteFrom, Tool-only).
*/
class Sasquatch
{
public:
/**
* Constructor
* Writes the header
/*
* Opens and maps p_file, and builds the UUID -> entry index Find() looks up into. p_file must already
* exist, produced ahead of time by WriteFrom (Tool mode) or the SasquatchPacker tool.
*
* \param p_file The file to write
* \param p_assetCount The number of assets that will be written
* \param p_totalFlatbufferBytes The sum of the sizes of every flatbuffer that will be written
* \param p_totalHardReferencesBytes The sum of the sizes of every asset hardreferences that will be written
* \param p_file The .sasquatch file to open
*/
SasquatchWriter(const File& p_file,
const std::uint64_t p_assetCount,
const std::size_t p_totalFlatbufferBytes,
const std::size_t p_totalHardReferencesBytes);
SasquatchWriter(const SasquatchWriter& p_writer) = delete;
SasquatchWriter(SasquatchWriter&& p_writer) = delete;
/**
* Destructor
* Writes the entries
*
*/
~SasquatchWriter();
/**
* Writes an asset to the file.
*
* \param p_uuid The UUID of the asset
* \param p_flatbuffer The flatbuffer of the asset
*/
void WriteAsset(const UUID& p_uuid,
eastl::span<const std::byte> p_flatbuffer,
eastl::span<const UUID> p_hardReferences);
SasquatchWriter& operator=(const SasquatchWriter& p_writer) = delete;
SasquatchWriter& operator=(SasquatchWriter&& p_writer) = delete;
private:
static constexpr std::uint64_t PREFETCH_BYTES = 4ULL * 1024 * 1024;
/**
* The number of assets expected
*/
std::uint64_t m_assetCount;
/**
* How far into m_file has already been prefetched
*/
std::size_t m_prefetchedUpTo = 0;
/**
* Current write position in the data region
*/
std::size_t m_currentDataOffset;
/**
* The Sasquatch file being written, mapped at its final size
*/
WritableMemoryMappedFile m_file;
/**
* The entries
*/
eastl::vector<SasquatchEntry> m_entries;
};
/****************************************************************************************/
class Sasquatch: public DataAccessor
{
public:
/**
* Constructor
*
* \param p_file The sasquatch file
*/
Sasquatch(const File& p_file);
explicit Sasquatch(const File& p_file);
Sasquatch(const Sasquatch& p_sasquatch) = delete;
Sasquatch(Sasquatch&& p_sasquatch) = delete;
~Sasquatch() override = default;
/**
* Get the header
*
* \return The header
*/
[[nodiscard]]
const SasquatchHeader& GetHeader() const;
/**
* Get an Asset flatbuffer
*
* \param p_uuid The asset to get the flatbuffer of
* \return The flatbuffer, or an empty vector if the asset was not found
*/
[[nodiscard]]
eastl::vector<std::byte> GetAssetFlatbuffer(const UUID& p_uuid) const override;
/**
* Get several Asset flatbuffers
*
* \param p_uuids The assets to get the flatbuffers of
* \return The flatbuffers that were found
*/
[[nodiscard]]
eastl::vector<eastl::vector<std::byte>> GetAssetFlatbuffers(eastl::span<const UUID> p_uuids) const override;
/**
* Get the HardReferences of a given asset
*
* \param p_uuid The asset to get the HardReferences of
* \return The HardReferences of the asset
*/
[[nodiscard]]
eastl::vector<UUID> GetAssetHardReferences(const UUID& p_uuid) const override;
/**
* Get the HardReferences of given assets
*
* \param p_uuids The assets to get the HardReferences of
* \return The HardReferences of the assets
*/
[[nodiscard]]
eastl::vector<eastl::vector<UUID>> GetAssetsHardReferences(eastl::span<const UUID> p_uuids) const override;
/**
* Sasquatch is read-only: this will always fail
*
*/
void SaveAssetFlatbuffer(const UUID& p_uuid,
eastl::span<const std::byte> p_flatbuffer,
eastl::span<const UUID> p_hardReferences) override;
/**
* Sasquatch is read-only: this will always fail
*
*/
void SaveAssetFlatbuffers(const eastl::span<const UUID> p_uuids,
eastl::span<const eastl::vector<std::byte>> p_flatbuffers,
eastl::span<const eastl::vector<UUID>> p_hardReferences) override;
/**
* Check if the Asset exists
*
* \param p_uuid The asset to check
* \param True if it exists, false otherwise
*/
[[nodiscard]]
bool Has(const UUID& p_uuid) const override;
~Sasquatch();
Sasquatch& operator=(const Sasquatch& p_sasquatch) = delete;
Sasquatch& operator=(Sasquatch&& p_sasquatch) = delete;
private:
/*
* A single asset, as found by Find(): a direct, zero-copy view into the mapped file.
*/
struct Entry
{
std::size_t m_assetOffset = 0;
std::size_t m_assetSize = 0;
/*
* The asset's type
*/
AssetTypeID m_typeID;
std::size_t m_hardReferencesOffset = 0;
std::size_t m_hardReferencesSize = 0;
/*
* The packed Asset flatbuffer bytes, as produced by Asset<ASSET>::Pack() - what
* AssetContainerBase::Insert(span<const byte>) expects.
*/
eastl::span<const std::byte> m_bytes;
/*
* The UUIDs this asset holds a HardReference to, concatenated UUID::UUID_BYTE_SIZE bytes apiece
* (see SerializationManager::SpanToUUID) - the Sasquatch equivalent of BigFileSchema.sql's
* AssetHardReference table, precomputed by WriteFrom so LoadAsset never has to touch m_bytes just
* to find them.
*/
eastl::span<const std::byte> m_hardReferenceUUIDs;
};
/**
* The Sasquatch file
/*
* Looks up an asset by UUID: a hash lookup into the index built at construction, then a direct read
* into the mapped file.
*
* \param p_uuid The UUID to find
* \return The asset, or std::nullopt if p_uuid is not present
*/
MemoryMappedFile m_file;
[[nodiscard]]
std::optional<Entry> Find(const UUID& p_uuid) const;
/**
* The sasquatch header
#if defined BIGFOOT_TOOL
/*
* Flattens every Asset/AssetHeader row in p_bigFile - and, for each, the UUIDs it holds a
* HardReference to (from AssetHardReference) - into a .sasquatch file at p_outFile. Tool-only: packing
* is a build-time step, not something a running game ever does (see SerializationManager, which never
* saves in Game mode).
*
* \param p_bigFile The BigFile to read every asset from
* \param p_outFile Where to write the .sasquatch file
*/
SasquatchHeader m_header;
/**
* UUID to entry
*/
ankerl::unordered_dense::map<UUID, Entry> m_entries;
};
static void WriteFrom(BigFile& p_bigFile, const File& p_outFile);
#endif
private:
struct Impl;
eastl::unique_ptr<Impl> m_impl;
};
} // namespace Bigfoot
#endif
@@ -1,22 +0,0 @@
@AUTO_GENERATED_COMMENT@
/*********************************************************************
* \file SasquatchInfo.generated.hpp
*
*********************************************************************/
#ifndef BIGFOOT_@SASQUATCH_NAME@_SASQUATCHINFO_GENERATED_HPP
#define BIGFOOT_@SASQUATCH_NAME@_SASQUATCHINFO_GENERATED_HPP
#include <EASTL/string_view.h>
#if defined BIGFOOT_GAME
namespace Bigfoot
{
/*
* Sasquatch location
*/
constexpr eastl::string_view SASQUATCH_@SASQUATCH_NAME@_LOCATION {"@SASQUATCH_LOCATION@"};
} // namespace Bigfoot
#endif
#endif
@@ -1,54 +0,0 @@
/*********************************************************************
* \file SerializationManager.hpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#ifndef BIGFOOT_ENGINE_SERIALIZATIONMANAGER_SERIALIZATIONMANAGER_HPP
#define BIGFOOT_ENGINE_SERIALIZATIONMANAGER_SERIALIZATIONMANAGER_HPP
#include <System/UUID/UUID.hpp>
#include <EASTL/span.h>
namespace Bigfoot::SerializationManager
{
/**
* Load an Asset, along with every HardReference it transitively points to
* Already loaded assets are skipped. Assets that are part of a cyclic dependency are left unloaded
*
* \param p_uuid The asset to load
*/
void LoadAsset(const UUID& p_uuid);
/**
* Load several Assets, along with every HardReference they transitively point to
* Already loaded assets are skipped. Assets that are part of a cyclic dependency are left unloaded
*
* \param p_uuids The assets to load
*/
void LoadAssets(const eastl::span<const UUID> p_uuids);
#ifdef BIGFOOT_TOOLMODE
/**
* Save an Asset
* Fails without saving if a cyclic dependency is detected starting from this asset
*
* \param p_uuid The asset to save
* \return True if the asset was saved, false otherwise
*/
[[nodiscard]]
bool SaveAsset(const UUID& p_uuid);
/**
* Save several Assets
* Fails without saving any of them if a cyclic dependency is detected starting from any of them
*
* \param p_uuids The assets to save
* \return True if the assets were saved, false otherwise
*/
[[nodiscard]]
bool SaveAssets(const eastl::span<const UUID> p_uuids);
#endif
} // namespace Bigfoot::SerializationManager
#endif
+246 -214
View File
@@ -4,263 +4,295 @@
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#include <Engine/EngineAssertHandler.hpp>
#include <Engine/Sasquatch/Sasquatch.hpp>
#include <Engine/EngineAssertHandler.hpp>
#include <ankerl/unordered_dense.h>
#include <algorithm>
#include <cstring>
#include <fstream>
#if defined BIGFOOT_TOOL
#include <EASTL/vector.h>
#endif
#if defined BIGFOOT_WINDOWS
#include <Windows.h>
#else
#include <fcntl.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <unistd.h>
#endif
namespace Bigfoot
{
#if defined(BIGFOOT_TOOLMODE) || defined(BIGFOOT_BUILD_TESTS) || defined(BIGFOOT_BUILD_BENCHMARKS)
SasquatchWriter::SasquatchWriter(const File& p_file,
const std::uint64_t p_assetCount,
const std::size_t p_totalFlatbufferBytes,
const std::size_t p_totalHardReferencesBytes):
m_assetCount(p_assetCount),
m_currentDataOffset(sizeof(SasquatchHeader) + (m_assetCount * sizeof(SasquatchEntry))),
m_file(p_file,
sizeof(SasquatchHeader) + (m_assetCount * sizeof(SasquatchEntry)) + p_totalFlatbufferBytes +
p_totalHardReferencesBytes)
namespace
{
m_entries.reserve(m_assetCount);
// "SASQUATC" read as a little-endian uint64 - just a format sentinel, never round-tripped through
// anything that cares about the actual text.
constexpr std::uint64_t SASQUATCH_MAGIC = 0x4354415551534153ULL;
SasquatchHeader header;
header.m_assetCount = m_assetCount;
std::memcpy(m_file.GetData().data(), &header, sizeof(SasquatchHeader));
}
/****************************************************************************************/
SasquatchWriter::~SasquatchWriter()
struct SasquatchHeader
{
ASSERT(EngineAssertHandler,
m_entries.size() == m_assetCount,
"SasquatchWriter expected {} assets, only {} were added",
m_assetCount,
m_entries.size());
std::uint64_t m_magic = SASQUATCH_MAGIC;
std::uint32_t m_version = 1;
std::uint32_t m_assetCount = 0;
};
m_file.Prefetch(sizeof(SasquatchHeader), m_entries.size() * sizeof(SasquatchEntry));
if (!m_entries.empty())
{
std::memcpy(m_file.GetData().data() + sizeof(SasquatchHeader),
m_entries.data(),
m_entries.size() * sizeof(SasquatchEntry));
}
}
/****************************************************************************************/
void SasquatchWriter::WriteAsset(const UUID& p_uuid,
eastl::span<const std::byte> p_flatbuffer,
eastl::span<const UUID> p_hardReferences)
// On-disk per-asset record. Offsets are absolute from the start of the file. m_hardReferenceOffset points
// at m_hardReferenceCount UUID::UUID_BYTE_SIZE-byte blocks back to back; m_blobOffset points at the packed
// Asset flatbuffer, as produced by Asset<ASSET>::Pack().
struct SasquatchEntry
{
ASSERT(EngineAssertHandler,
m_entries.size() < m_assetCount,
"SasquatchWriter can only hold {} assets",
m_assetCount);
std::byte m_uuid[UUID::UUID_BYTE_SIZE];
std::uint64_t m_typeID;
std::uint64_t m_blobOffset;
std::uint64_t m_blobSize;
std::uint64_t m_hardReferenceOffset;
std::uint64_t m_hardReferenceCount;
};
const std::size_t offset = m_currentDataOffset;
const eastl::span<std::byte> data = m_file.GetData();
ASSERT(EngineAssertHandler,
offset + p_flatbuffer.size() + (p_hardReferences.size() * UUID::UUID_BYTE_SIZE) <= data.size(),
"SasquatchWriter can only write {} bytes",
data.size());
const std::size_t writeEnd = offset + p_flatbuffer.size() + (p_hardReferences.size() * UUID::UUID_BYTE_SIZE);
if (writeEnd > m_prefetchedUpTo)
{
const std::size_t prefetchEnd = std::min(writeEnd + PREFETCH_BYTES, data.size());
m_file.Prefetch(m_prefetchedUpTo, prefetchEnd - m_prefetchedUpTo);
m_prefetchedUpTo = prefetchEnd;
}
SasquatchEntry entry;
std::memcpy(entry.m_uuid.data(), p_uuid.ToBytes().data(), UUID::UUID_BYTE_SIZE);
entry.m_assetOffset = offset;
entry.m_assetSize = p_flatbuffer.size();
entry.m_hardReferencesOffset = entry.m_assetOffset + entry.m_assetSize;
entry.m_hardReferencesSize = p_hardReferences.size() * UUID::UUID_BYTE_SIZE;
std::memcpy(data.data() + entry.m_assetOffset, p_flatbuffer.data(), entry.m_assetSize);
for (std::uint32_t i = 0; i < p_hardReferences.size(); ++i)
{
const std::span<const std::byte, UUID::UUID_BYTE_SIZE> rawUUID = p_hardReferences[i].ToBytes();
std::memcpy(data.data() + entry.m_hardReferencesOffset + (static_cast<std::size_t>(i) * UUID::UUID_BYTE_SIZE),
rawUUID.data(),
UUID::UUID_BYTE_SIZE);
}
m_currentDataOffset += entry.m_assetSize + entry.m_hardReferencesSize;
m_entries.push_back(entry);
}
/****************************************************************************************/
Sasquatch::Sasquatch(const File& p_file):
m_file(p_file)
#if defined BIGFOOT_WINDOWS
// File::GetPath() comes from std::filesystem::path::string(), which on Windows is the native narrow
// (ACP) encoding, not UTF-8 - convert with the matching codepage so non-ASCII paths round-trip correctly.
[[nodiscard]]
std::wstring ToWidePath(const eastl::string_view p_path)
{
const eastl::span<const std::byte> data = m_file.GetData();
CRITICAL_ASSERT(EngineAssertHandler,
data.size() >= sizeof(SasquatchHeader),
"Sasquatch file {} is smaller than its header",
p_file.GetPath());
std::memcpy(&m_header, data.data(), sizeof(SasquatchHeader));
CRITICAL_ASSERT(EngineAssertHandler,
m_header.m_magic == SasquatchHeader::MAGIC,
"Sasquatch file {} has an invalid magic number (got {} / expected {})",
p_file.GetPath(),
m_header.m_magic,
SasquatchHeader::MAGIC);
CRITICAL_ASSERT(EngineAssertHandler,
m_header.m_version == SasquatchHeader::VERSION,
"Sasquatch file {} has an unsupported version (got {} / expected {})",
p_file.GetPath(),
m_header.m_version,
SasquatchHeader::VERSION);
CRITICAL_ASSERT(EngineAssertHandler,
data.size() >= sizeof(SasquatchHeader) + (m_header.m_assetCount * sizeof(SasquatchEntry)),
"Sasquatch file {} index is truncated (got {} / expected at least {})",
p_file.GetPath(),
data.size(),
sizeof(SasquatchHeader) + (m_header.m_assetCount * sizeof(SasquatchEntry)));
eastl::vector<SasquatchEntry> entries(m_header.m_assetCount);
if (!entries.empty())
{
std::memcpy(entries.data(), data.data() + sizeof(SasquatchHeader), entries.size() * sizeof(SasquatchEntry));
}
m_entries.reserve(m_header.m_assetCount);
for (const SasquatchEntry& entry: entries)
{
CRITICAL_ASSERT(EngineAssertHandler,
entry.m_assetOffset + entry.m_assetSize + entry.m_hardReferencesSize <= data.size(),
"Sasquatch file {} has an out-of-bounds asset entry",
p_file.GetPath());
m_entries.emplace(UUID {entry.m_uuid},
Entry {.m_assetOffset = entry.m_assetOffset,
.m_assetSize = entry.m_assetSize,
.m_hardReferencesOffset = entry.m_hardReferencesOffset,
.m_hardReferencesSize = entry.m_hardReferencesSize});
}
}
/****************************************************************************************/
const SasquatchHeader& Sasquatch::GetHeader() const
{
return m_header;
}
/****************************************************************************************/
eastl::vector<std::byte> Sasquatch::GetAssetFlatbuffer(const UUID& p_uuid) const
{
const auto entryIt = m_entries.find(p_uuid);
if (entryIt == m_entries.end())
if (p_path.empty())
{
return {};
}
const eastl::span<const std::byte> data = m_file.GetData();
const eastl::span<const std::byte> bytes {data.data() + entryIt->second.m_assetOffset,
static_cast<std::size_t>(entryIt->second.m_assetSize)};
return eastl::vector<std::byte> {bytes.begin(), bytes.end()};
const int requiredSize = MultiByteToWideChar(CP_ACP, 0, p_path.data(), static_cast<int>(p_path.size()), nullptr, 0);
std::wstring wide(static_cast<std::size_t>(requiredSize), L'\0');
MultiByteToWideChar(CP_ACP, 0, p_path.data(), static_cast<int>(p_path.size()), wide.data(), requiredSize);
return wide;
}
#endif
} // namespace
struct Sasquatch::Impl
{
#if defined BIGFOOT_WINDOWS
HANDLE m_fileHandle = INVALID_HANDLE_VALUE;
HANDLE m_mapping = nullptr;
#else
int m_fileDescriptor = -1;
#endif
const std::byte* m_base = nullptr;
std::size_t m_size = 0;
// UUID -> on-disk entry index, built once here (see Sasquatch::Sasquatch) so Find() is a hash lookup
// instead of a scan.
ankerl::unordered_dense::segmented_map<UUID, const SasquatchEntry*> m_index;
~Impl()
{
#if defined BIGFOOT_WINDOWS
if (m_base != nullptr)
{
UnmapViewOfFile(m_base);
}
if (m_mapping != nullptr)
{
CloseHandle(m_mapping);
}
if (m_fileHandle != INVALID_HANDLE_VALUE)
{
CloseHandle(m_fileHandle);
}
#else
if (m_base != nullptr && m_base != MAP_FAILED)
{
munmap(const_cast<std::byte*>(m_base), m_size);
}
if (m_fileDescriptor != -1)
{
close(m_fileDescriptor);
}
#endif
}
};
Sasquatch::Sasquatch(const File& p_file):
m_impl(eastl::make_unique<Impl>())
{
const File absoluteFile = p_file.ToAbsolute();
const eastl::string_view path = absoluteFile.GetPath();
#if defined BIGFOOT_WINDOWS
// FILE_SHARE_DELETE matches POSIX unlink-while-open semantics (Sasquatch::WriteFrom/a packaging tool
// replacing this file, or a test cleaning up, shouldn't have to wait on every reader to close first) -
// Windows otherwise refuses to delete a file that's still open here, even just for reading.
m_impl->m_fileHandle = CreateFileW(ToWidePath(path).c_str(), GENERIC_READ,
FILE_SHARE_READ | FILE_SHARE_DELETE, nullptr, OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL, nullptr);
CRITICAL_ASSERT(EngineAssertHandler, m_impl->m_fileHandle != INVALID_HANDLE_VALUE,
"Failed to open Sasquatch file {}", path);
LARGE_INTEGER size {};
GetFileSizeEx(m_impl->m_fileHandle, &size);
m_impl->m_size = static_cast<std::size_t>(size.QuadPart);
m_impl->m_mapping = CreateFileMappingW(m_impl->m_fileHandle, nullptr, PAGE_READONLY, 0, 0, nullptr);
CRITICAL_ASSERT(EngineAssertHandler, m_impl->m_mapping != nullptr, "Failed to map Sasquatch file {}", path);
m_impl->m_base = static_cast<const std::byte*>(MapViewOfFile(m_impl->m_mapping, FILE_MAP_READ, 0, 0, 0));
CRITICAL_ASSERT(EngineAssertHandler, m_impl->m_base != nullptr, "Failed to map Sasquatch file {}", path);
#else
const std::string nullTerminatedPath {path.begin(), path.end()};
m_impl->m_fileDescriptor = open(nullTerminatedPath.c_str(), O_RDONLY);
CRITICAL_ASSERT(EngineAssertHandler, m_impl->m_fileDescriptor != -1, "Failed to open Sasquatch file {}", path);
struct stat fileStat
{
};
fstat(m_impl->m_fileDescriptor, &fileStat);
m_impl->m_size = static_cast<std::size_t>(fileStat.st_size);
m_impl->m_base = static_cast<const std::byte*>(
mmap(nullptr, m_impl->m_size, PROT_READ, MAP_PRIVATE, m_impl->m_fileDescriptor, 0));
CRITICAL_ASSERT(EngineAssertHandler, m_impl->m_base != MAP_FAILED, "Failed to map Sasquatch file {}", path);
#endif
const auto* header = reinterpret_cast<const SasquatchHeader*>(m_impl->m_base);
CRITICAL_ASSERT(EngineAssertHandler, header->m_magic == SASQUATCH_MAGIC, "{} is not a Sasquatch file", path);
const auto* entries = reinterpret_cast<const SasquatchEntry*>(m_impl->m_base + sizeof(SasquatchHeader));
m_impl->m_index.reserve(header->m_assetCount);
for (std::uint32_t i = 0; i < header->m_assetCount; ++i)
{
const SasquatchEntry& entry = entries[i];
const UUID uuid {std::span<const std::byte, UUID::UUID_BYTE_SIZE> {entry.m_uuid, UUID::UUID_BYTE_SIZE}};
m_impl->m_index.emplace(uuid, &entry);
}
}
/****************************************************************************************/
eastl::vector<eastl::vector<std::byte>> Sasquatch::GetAssetFlatbuffers(const eastl::span<const UUID> p_uuids) const
{
eastl::vector<eastl::vector<std::byte>> flatbuffers;
flatbuffers.reserve(p_uuids.size());
Sasquatch::~Sasquatch() = default;
for (const UUID& uuid: p_uuids)
/****************************************************************************************/
std::optional<Sasquatch::Entry> Sasquatch::Find(const UUID& p_uuid) const
{
const auto it = m_impl->m_index.find(p_uuid);
if (it == m_impl->m_index.end())
{
flatbuffers.push_back(GetAssetFlatbuffer(uuid));
return std::nullopt;
}
return flatbuffers;
const SasquatchEntry& entry = *it->second;
return Entry {entry.m_typeID,
eastl::span<const std::byte> {m_impl->m_base + entry.m_blobOffset, entry.m_blobSize},
eastl::span<const std::byte> {m_impl->m_base + entry.m_hardReferenceOffset,
entry.m_hardReferenceCount * UUID::UUID_BYTE_SIZE}};
}
/****************************************************************************************/
eastl::vector<UUID> Sasquatch::GetAssetHardReferences(const UUID& p_uuid) const
#if defined BIGFOOT_TOOL
void Sasquatch::WriteFrom(BigFile& p_bigFile, const File& p_outFile)
{
eastl::vector<UUID> hardReferences;
const auto entryIt = m_entries.find(p_uuid);
if (entryIt == m_entries.end())
struct Row
{
return hardReferences;
std::byte m_uuid[UUID::UUID_BYTE_SIZE];
std::uint64_t m_typeID;
eastl::vector<std::byte> m_bytes;
eastl::vector<std::byte> m_hardReferenceUUIDs;
};
eastl::vector<Row> rows;
ankerl::unordered_dense::segmented_map<UUID, std::size_t> uuidToRowIndex;
{
BigFile::Request request {p_bigFile,
"SELECT AssetHeader.UUID, AssetHeader.TypeID, Asset.Asset FROM Asset "
"JOIN AssetHeader ON AssetHeader.UUID = Asset.UUID;"};
while (request.Step())
{
const eastl::span<const std::byte> uuidBytes = request.Get(0);
const AssetTypeID typeID = request.Get(1);
const eastl::span<const std::byte> assetBytes = request.Get(2);
Row row;
std::memcpy(row.m_uuid, uuidBytes.data(), UUID::UUID_BYTE_SIZE);
row.m_typeID = typeID;
row.m_bytes.assign(assetBytes.begin(), assetBytes.end());
const UUID uuid {std::span<const std::byte, UUID::UUID_BYTE_SIZE> {row.m_uuid, UUID::UUID_BYTE_SIZE}};
uuidToRowIndex.emplace(uuid, rows.size());
rows.push_back(std::move(row));
}
}
const eastl::span<const std::byte> data = m_file.GetData();
const eastl::span<const std::byte> bytes {data.data() + entryIt->second.m_hardReferencesOffset,
static_cast<std::size_t>(entryIt->second.m_hardReferencesSize)};
const std::size_t hardReferenceCount = bytes.size() / UUID::UUID_BYTE_SIZE;
hardReferences.reserve(hardReferenceCount);
for (std::size_t i = 0; i < hardReferenceCount; ++i)
{
std::array<std::byte, UUID::UUID_BYTE_SIZE> rawUUID;
std::memcpy(rawUUID.data(), bytes.data() + (i * UUID::UUID_BYTE_SIZE), UUID::UUID_BYTE_SIZE);
hardReferences.emplace_back(rawUUID);
BigFile::Request request {p_bigFile, "SELECT UUID, ReferencedUUID FROM AssetHardReference;"};
while (request.Step())
{
const eastl::span<const std::byte> uuidBytes = request.Get(0);
const eastl::span<const std::byte> referencedUUIDBytes = request.Get(1);
const UUID uuid {std::span<const std::byte, UUID::UUID_BYTE_SIZE> {uuidBytes.data(), UUID::UUID_BYTE_SIZE}};
if (const auto it = uuidToRowIndex.find(uuid); it != uuidToRowIndex.end())
{
Row& row = rows[it->second];
row.m_hardReferenceUUIDs.insert(row.m_hardReferenceUUIDs.end(), referencedUUIDBytes.begin(),
referencedUUIDBytes.end());
}
}
}
return hardReferences;
}
SasquatchHeader header;
header.m_assetCount = static_cast<std::uint32_t>(rows.size());
/****************************************************************************************/
std::uint64_t cursor = sizeof(SasquatchHeader) + rows.size() * sizeof(SasquatchEntry);
eastl::vector<eastl::vector<UUID>> Sasquatch::GetAssetsHardReferences(eastl::span<const UUID> p_uuids) const
{
eastl::vector<eastl::vector<UUID>> hardReferences;
hardReferences.reserve(p_uuids.size());
for (const UUID& uuid: p_uuids)
eastl::vector<SasquatchEntry> entries;
entries.reserve(rows.size());
for (const Row& row: rows)
{
hardReferences.push_back(GetAssetHardReferences(uuid));
SasquatchEntry entry;
std::memcpy(entry.m_uuid, row.m_uuid, UUID::UUID_BYTE_SIZE);
entry.m_typeID = row.m_typeID;
entry.m_hardReferenceOffset = cursor;
entry.m_hardReferenceCount = row.m_hardReferenceUUIDs.size() / UUID::UUID_BYTE_SIZE;
cursor += row.m_hardReferenceUUIDs.size();
entry.m_blobOffset = cursor;
entry.m_blobSize = row.m_bytes.size();
cursor += row.m_bytes.size();
entries.push_back(entry);
}
return hardReferences;
}
#if defined BIGFOOT_WINDOWS
std::ofstream out {ToWidePath(p_outFile.ToAbsolute().GetPath()), std::ios::binary | std::ios::trunc};
#else
const eastl::string_view outPath = p_outFile.ToAbsolute().GetPath();
std::ofstream out {std::string {outPath.begin(), outPath.end()}, std::ios::binary | std::ios::trunc};
#endif
CRITICAL_ASSERT(EngineAssertHandler, out.is_open(), "Failed to create Sasquatch file {}",
p_outFile.ToAbsolute().GetPath());
/****************************************************************************************/
void Sasquatch::SaveAssetFlatbuffer([[maybe_unused]] const UUID& p_uuid,
[[maybe_unused]] eastl::span<const std::byte> p_flatbuffer,
[[maybe_unused]] eastl::span<const UUID> p_hardReferences)
{
ASSERT(EngineAssertHandler, false, "Sasquatch is read-only, it cannot save assets");
}
/****************************************************************************************/
void Sasquatch::SaveAssetFlatbuffers([[maybe_unused]] const eastl::span<const UUID> p_uuids,
[[maybe_unused]] eastl::span<const eastl::vector<std::byte>> p_flatbuffers,
[[maybe_unused]] eastl::span<const eastl::vector<UUID>> p_hardReferences)
{
ASSERT(EngineAssertHandler, false, "Sasquatch is read-only, it cannot save assets");
}
/****************************************************************************************/
bool Sasquatch::Has(const UUID& p_uuid) const
{
return m_entries.contains(p_uuid);
out.write(reinterpret_cast<const char*>(&header), sizeof(header));
out.write(reinterpret_cast<const char*>(entries.data()),
static_cast<std::streamsize>(entries.size() * sizeof(SasquatchEntry)));
for (const Row& row: rows)
{
if (!row.m_hardReferenceUUIDs.empty())
{
out.write(reinterpret_cast<const char*>(row.m_hardReferenceUUIDs.data()),
static_cast<std::streamsize>(row.m_hardReferenceUUIDs.size()));
}
if (!row.m_bytes.empty())
{
out.write(reinterpret_cast<const char*>(row.m_bytes.data()),
static_cast<std::streamsize>(row.m_bytes.size()));
}
}
}
#endif
} // namespace Bigfoot
@@ -1,249 +0,0 @@
/*********************************************************************
* \file SerializationManager.cpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#include <Engine/SerializationManager/SerializationManager.hpp>
#include <Engine/Asset/AssetLibrary.hpp>
#include <Engine/DataAccessor/DataAccessor.hpp>
#include <Utils/Algorithm/Graph.hpp>
#include <ankerl/unordered_dense.h>
namespace Bigfoot::SerializationManager
{
/*
* Build the transitive HardReference graph reachable from p_roots
*
* \param p_roots The assets to start the discovery from
* \param p_discovered Every discovered asset that isn't already loaded, including p_roots
* \return Every asset's HardReferences
*/
[[nodiscard]]
ankerl::unordered_dense::map<UUID, eastl::vector<UUID>> BuildDependencyGraph(
const eastl::span<const UUID> p_roots,
ankerl::unordered_dense::set<UUID>& p_discovered)
{
ankerl::unordered_dense::map<UUID, eastl::vector<UUID>> graph;
eastl::vector<UUID> toQuery;
toQuery.reserve(p_roots.size());
for (const UUID& uuid: p_roots)
{
if (!g_activeAssetLibrary->IsLoaded(uuid) && p_discovered.insert(uuid).second)
{
toQuery.push_back(uuid);
}
}
while (!toQuery.empty())
{
eastl::vector<eastl::vector<UUID>> hardReferencesBatch = g_dataAccessor->GetAssetsHardReferences(toQuery);
eastl::vector<UUID> nextToQuery;
for (std::size_t i = 0; i < toQuery.size(); ++i)
{
eastl::vector<UUID>& hardReferences = graph[toQuery[i]];
hardReferences = std::move(hardReferencesBatch[i]);
for (const UUID& hardReference: hardReferences)
{
if (!g_activeAssetLibrary->IsLoaded(hardReference) && p_discovered.insert(hardReference).second)
{
nextToQuery.push_back(hardReference);
}
}
}
toQuery = std::move(nextToQuery);
}
return graph;
}
/****************************************************************************************/
void LoadAssets(const eastl::span<const UUID> p_uuids)
{
ankerl::unordered_dense::set<UUID> discovered;
const ankerl::unordered_dense::map<UUID, eastl::vector<UUID>> graph = BuildDependencyGraph(p_uuids, discovered);
// Assets that are part of a cyclic HardReference dependency are left out of waves and simply never loaded.
eastl::vector<eastl::vector<UUID>> waves;
eastl::vector<UUID> cyclicUuids;
std::ignore = LayeredTopologicalSort(graph, discovered, waves, cyclicUuids);
BIGFOOT_LOG_ERROR_IF(!cyclicUuids.empty(),
ENGINE_LOGGER,
"Cyclic HardReference dependency detected while loading assets, involved UUIDs: {}",
cyclicUuids);
// Waves are loaded one after another since a wave's assets may HardReference the previous wave's; within a
// wave, every asset is independent of the others, so this is where loading could later be parallelized.
for (const eastl::vector<UUID>& wave: waves)
{
const eastl::vector<eastl::vector<std::byte>> assetFlatbuffers = g_dataAccessor->GetAssetFlatbuffers(wave);
for (std::size_t i = 0; i < wave.size(); ++i)
{
const eastl::vector<std::byte>& flatbuffer = assetFlatbuffers[i];
if (flatbuffer.empty())
{
BIGFOOT_LOG_ERROR(ENGINE_LOGGER, "Asset {} could not be found while loading, skipping it", wave[i]);
continue;
}
#ifdef BIGFOOT_NOT_OPTIMIZED
flatbuffers::Verifier verifier {reinterpret_cast<const std::uint8_t*>(flatbuffer.data()),
flatbuffer.size()};
const bool assetVerified = ::Flat::Bigfoot::VerifyAssetBuffer(verifier);
CRITICAL_ASSERT(EngineAssertHandler, assetVerified, "Given asset could not be verified!");
#endif
const ::Flat::Bigfoot::Asset* asset = flatbuffers::GetRoot<::Flat::Bigfoot::Asset>(flatbuffer.data());
AssetContainerBase* container = g_activeAssetLibrary->GetAssetContainer(
flatbuffers::UnPack(*asset->header()->type_id()));
CRITICAL_ASSERT(EngineAssertHandler,
container != nullptr,
"No AssetContainer registered for asset {}",
wave[i]);
std::ignore = container->Insert(flatbuffer);
}
}
}
/****************************************************************************************/
void LoadAsset(const UUID& p_uuid)
{
LoadAssets(eastl::span<const UUID> {&p_uuid, 1});
}
/****************************************************************************************/
#ifdef BIGFOOT_TOOLMODE
/*
* Finds the cyclic dependencies that may exist in Assets
*
* \param p_uuids The assets we want to explore
* \param p_hardReferences The HardReferences of p_uuids, matching by index
* \return Every UUID caught in a cyclic dependency
*/
[[nodiscard]]
eastl::vector<UUID> FindCyclicDependencies(const eastl::span<const UUID> p_uuids,
const eastl::span<const eastl::vector<UUID>> p_hardReferences)
{
ankerl::unordered_dense::map<UUID, eastl::vector<UUID>> graph;
ankerl::unordered_dense::set<UUID> discovered;
eastl::vector<UUID> toQuery;
for (std::size_t i = 0; i < p_uuids.size(); ++i)
{
discovered.insert(p_uuids[i]);
graph[p_uuids[i]] = p_hardReferences[i];
}
for (const eastl::vector<UUID>& hardReferences: p_hardReferences)
{
for (const UUID& hardReference: hardReferences)
{
if (discovered.insert(hardReference).second)
{
toQuery.push_back(hardReference);
}
}
}
while (!toQuery.empty())
{
eastl::vector<eastl::vector<UUID>> hardReferencesBatch = g_dataAccessor->GetAssetsHardReferences(toQuery);
eastl::vector<UUID> nextToQuery;
for (std::size_t i = 0; i < toQuery.size(); ++i)
{
eastl::vector<UUID>& hardReferences = graph[toQuery[i]];
hardReferences = std::move(hardReferencesBatch[i]);
for (const UUID& hardReference: hardReferences)
{
if (discovered.insert(hardReference).second)
{
nextToQuery.push_back(hardReference);
}
}
}
toQuery = std::move(nextToQuery);
}
eastl::vector<eastl::vector<UUID>> waves;
eastl::vector<UUID> cyclicUuids;
std::ignore = LayeredTopologicalSort(graph, discovered, waves, cyclicUuids);
return cyclicUuids;
}
/****************************************************************************************/
bool SaveAsset(const UUID& p_uuid)
{
AssetContainerBase* container = g_activeAssetLibrary->GetAssetContainer(p_uuid);
CRITICAL_ASSERT(EngineAssertHandler, container != nullptr, "No AssetContainer registered for asset {}", p_uuid);
const AssetBase* asset = container->AcquireContainerHandle(p_uuid)->GetAsset();
const eastl::vector<std::byte> flatbuffer = asset->Pack();
const eastl::vector<UUID> hardReferences = asset->GetHardReferences(flatbuffer);
const eastl::vector<UUID> cyclicUuids = FindCyclicDependencies(
eastl::span<const UUID> {&p_uuid, 1},
eastl::span<const eastl::vector<UUID>> {&hardReferences, 1});
BIGFOOT_LOG_ERROR_IF(!cyclicUuids.empty(),
ENGINE_LOGGER,
"Cyclic HardReference dependency detected while saving asset {}, involved UUIDs: {}",
p_uuid,
cyclicUuids);
if (!cyclicUuids.empty())
{
return false;
}
g_dataAccessor->SaveAssetFlatbuffer(p_uuid, flatbuffer, hardReferences);
return true;
}
/****************************************************************************************/
bool SaveAssets(const eastl::span<const UUID> p_uuids)
{
eastl::vector<eastl::vector<std::byte>> flatbuffers;
eastl::vector<eastl::vector<UUID>> hardReferences;
flatbuffers.reserve(p_uuids.size());
hardReferences.reserve(p_uuids.size());
for (const UUID& uuid: p_uuids)
{
AssetContainerBase* container = g_activeAssetLibrary->GetAssetContainer(uuid);
CRITICAL_ASSERT(EngineAssertHandler, container != nullptr, "No AssetContainer registered for asset {}", uuid);
const AssetBase* asset = container->AcquireContainerHandle(uuid)->GetAsset();
const eastl::vector<std::byte> flatbuffer = asset->Pack();
flatbuffers.push_back(flatbuffer);
hardReferences.push_back(asset->GetHardReferences(flatbuffer));
}
const eastl::vector<UUID> cyclicUuids = FindCyclicDependencies(p_uuids, hardReferences);
BIGFOOT_LOG_ERROR_IF(!cyclicUuids.empty(),
ENGINE_LOGGER,
"Cyclic HardReference dependency detected while saving assets, involved UUIDs: {}",
cyclicUuids);
if (!cyclicUuids.empty())
{
return false;
}
g_dataAccessor->SaveAssetFlatbuffers(p_uuids, flatbuffers, hardReferences);
return true;
}
#endif
} // namespace Bigfoot::SerializationManager
+66
View File
@@ -0,0 +1,66 @@
/*********************************************************************
* \file File.cpp
*
* \author Romain BOULLARD
* \date December 2025
*********************************************************************/
#include <System/File.hpp>
namespace Bigfoot
{
File::File(const eastl::string_view p_path):
m_path(p_path.data()),
m_pathString(p_path)
{
}
/****************************************************************************************/
bool File::IsAbsolute() const
{
return m_path.is_absolute();
}
/****************************************************************************************/
bool File::IsRelative() const
{
return m_path.is_relative();
}
/****************************************************************************************/
bool File::Exists() const
{
return std::filesystem::exists(m_path);
}
/****************************************************************************************/
eastl::string_view File::GetPath() const
{
return m_pathString;
}
/****************************************************************************************/
File File::ToAbsolute() const
{
return File {std::filesystem::absolute(m_path)};
}
/****************************************************************************************/
File File::ToRelative() const
{
return File {std::filesystem::relative(m_path)};
}
/****************************************************************************************/
File::File(const std::filesystem::path& p_path):
m_path(p_path),
m_pathString(m_path.string().c_str())
{
}
} // namespace Bigfoot
-119
View File
@@ -1,119 +0,0 @@
/*********************************************************************
* \file File.cpp
*
* \author Romain BOULLARD
* \date December 2025
*********************************************************************/
#include <System/File/File.hpp>
#include <System/SystemAssertHandler.hpp>
#include <fstream>
#include <string>
namespace Bigfoot
{
File::File(const eastl::string_view p_path):
m_path(p_path.data()),
m_pathString(p_path)
{
}
/****************************************************************************************/
bool File::IsAbsolute() const
{
return m_path.is_absolute();
}
/****************************************************************************************/
bool File::IsRelative() const
{
return m_path.is_relative();
}
/****************************************************************************************/
bool File::Exists() const
{
return std::filesystem::exists(m_path);
}
/****************************************************************************************/
eastl::string_view File::GetPath() const
{
return m_pathString;
}
/****************************************************************************************/
File File::ToAbsolute() const
{
return File {std::filesystem::absolute(m_path)};
}
/****************************************************************************************/
File File::ToRelative() const
{
return File {std::filesystem::relative(m_path)};
}
/****************************************************************************************/
std::size_t File::GetSize() const
{
return static_cast<std::size_t>(std::filesystem::file_size(m_path));
}
/****************************************************************************************/
eastl::vector<std::byte> File::Read(const std::size_t p_offset, const std::size_t p_size) const
{
const File absoluteFile = ToAbsolute();
const eastl::string_view path = absoluteFile.GetPath();
std::ifstream file {path.data(), std::ios::binary};
CRITICAL_ASSERT(SystemAssertHandler, file.is_open(), "Failed to open file {} for reading", path);
file.seekg(static_cast<std::streamoff>(p_offset), std::ios::beg);
eastl::vector<std::byte> data;
data.resize(p_size);
file.read(reinterpret_cast<char*>(data.data()), static_cast<std::streamsize>(data.size()));
return data;
}
/****************************************************************************************/
void File::Write(const eastl::span<const std::byte> p_data, const std::size_t p_offset) const
{
const File absoluteFile = ToAbsolute();
const eastl::string_view path = absoluteFile.GetPath();
std::ofstream {path.data(), std::ios::binary | std::ios::app};
std::fstream file {path.data(), std::ios::binary | std::ios::in | std::ios::out};
CRITICAL_ASSERT(SystemAssertHandler, file.is_open(), "Failed to open file {} for writing", path);
file.seekp(static_cast<std::streamoff>(p_offset), std::ios::beg);
file.write(reinterpret_cast<const char*>(p_data.data()), static_cast<std::streamsize>(p_data.size()));
}
/****************************************************************************************/
void File::Delete() const
{
std::filesystem::remove(m_path);
}
/****************************************************************************************/
File::File(std::filesystem::path p_path):
m_path(std::move(p_path)),
m_pathString(m_path.string().c_str())
{
}
} // namespace Bigfoot
@@ -1,234 +0,0 @@
/*********************************************************************
* \file MemoryMappedFile.cpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#include <System/File/MemoryMappedFile.hpp>
#include <System/SystemAssertHandler.hpp>
#include <string>
#ifdef BIGFOOT_WINDOWS
#include <Windows.h>
#elif defined BIGFOOT_LINUX
#include <fcntl.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <unistd.h>
#endif
namespace Bigfoot
{
MemoryMappedFile::MemoryMappedFile(const File& p_file)
{
const File absoluteFile = p_file.ToAbsolute();
const eastl::string_view path = absoluteFile.GetPath();
#ifdef BIGFOOT_WINDOWS
m_fileHandle = CreateFileA(path.data(),
GENERIC_READ,
FILE_SHARE_READ,
nullptr,
OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL | FILE_FLAG_RANDOM_ACCESS,
nullptr);
CRITICAL_ASSERT(SystemAssertHandler, m_fileHandle != INVALID_HANDLE_VALUE, "Failed to open file {}", path);
LARGE_INTEGER fileSize {};
[[maybe_unused]]
const BOOL gotFileSize = GetFileSizeEx(m_fileHandle, &fileSize);
CRITICAL_ASSERT(SystemAssertHandler, gotFileSize, "Failed to get the size of file {}", path);
m_size = static_cast<std::size_t>(fileSize.QuadPart);
if (m_size != 0)
{
m_mappingHandle = CreateFileMappingA(m_fileHandle, nullptr, PAGE_READONLY, 0, 0, nullptr);
CRITICAL_ASSERT(SystemAssertHandler, m_mappingHandle != nullptr, "Failed create the file mapping {}", path);
m_data = static_cast<std::byte*>(MapViewOfFile(m_mappingHandle, FILE_MAP_READ, 0, 0, 0));
CRITICAL_ASSERT(SystemAssertHandler, m_data != nullptr, "Failed to map the file {}", path);
}
#elif defined BIGFOOT_LINUX
m_fileDescriptor = open(path.data(), O_RDONLY);
CRITICAL_ASSERT(SystemAssertHandler, m_fileDescriptor != -1, "Failed to open file {}", path);
struct stat fileStat {};
[[maybe_unused]]
const int statResult = fstat(m_fileDescriptor, &fileStat);
CRITICAL_ASSERT(SystemAssertHandler, statResult == 0, "Failed to get the size of file {}", path);
m_size = static_cast<std::size_t>(fileStat.st_size);
if (m_size != 0)
{
void* const mapped = mmap(nullptr, m_size, PROT_READ, MAP_PRIVATE, m_fileDescriptor, 0);
CRITICAL_ASSERT(SystemAssertHandler, mapped != MAP_FAILED, "Failed to map file {}", path);
m_data = static_cast<std::byte*>(mapped);
[[maybe_unused]]
const int adviseResult = madvise(mapped, m_size, MADV_RANDOM);
}
#else
CRITICAL_ASSERT(SystemAssertHandler, false, "MemoryMappedFileBase is not implemented on this platform");
#endif
}
/****************************************************************************************/
MemoryMappedFile::~MemoryMappedFile()
{
#ifdef BIGFOOT_WINDOWS
if (m_data != nullptr)
{
[[maybe_unused]]
const BOOL unmapped = UnmapViewOfFile(m_data);
CRITICAL_ASSERT(SystemAssertHandler, unmapped, "Failed to unmap the file");
}
if (m_mappingHandle != nullptr)
{
[[maybe_unused]]
const BOOL closedMapping = CloseHandle(m_mappingHandle);
CRITICAL_ASSERT(SystemAssertHandler, closedMapping, "Failed to close the file mapping");
}
if (m_fileHandle != INVALID_HANDLE_VALUE)
{
[[maybe_unused]]
const BOOL closedFile = CloseHandle(m_fileHandle);
CRITICAL_ASSERT(SystemAssertHandler, closedFile, "Failed to close the file");
}
#elif defined BIGFOOT_LINUX
if (m_data != nullptr)
{
[[maybe_unused]]
const int unmapped = munmap(m_data, m_size);
CRITICAL_ASSERT(SystemAssertHandler, unmapped == 0, "Failed to unmap the file");
}
if (m_fileDescriptor != -1)
{
[[maybe_unused]]
const int closedFile = close(m_fileDescriptor);
CRITICAL_ASSERT(SystemAssertHandler, closedFile == 0, "Failed to close the file");
}
#else
CRITICAL_ASSERT(SystemAssertHandler, false, "MemoryMappedFileBase is not implemented on this platform");
#endif
}
/****************************************************************************************/
eastl::span<const std::byte> MemoryMappedFile::GetData() const
{
return eastl::span<const std::byte> {m_data, m_size};
}
/****************************************************************************************/
WritableMemoryMappedFile::WritableMemoryMappedFile(const File& p_file, const std::size_t p_size):
m_size(p_size)
{
const File absoluteFile = p_file.ToAbsolute();
const eastl::string_view path = absoluteFile.GetPath();
#ifdef BIGFOOT_WINDOWS
m_fileHandle =
CreateFileA(path.data(), GENERIC_READ | GENERIC_WRITE, 0, nullptr, OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, nullptr);
CRITICAL_ASSERT(SystemAssertHandler, m_fileHandle != INVALID_HANDLE_VALUE, "Failed to create file {}", path);
const LARGE_INTEGER size {.QuadPart = static_cast<LONGLONG>(p_size)};
m_mappingHandle = CreateFileMappingA(m_fileHandle, nullptr, PAGE_READWRITE, size.HighPart, size.LowPart, nullptr);
CRITICAL_ASSERT(SystemAssertHandler, m_mappingHandle != nullptr, "Failed create the file mapping {}", path);
m_data = static_cast<std::byte*>(MapViewOfFile(m_mappingHandle, FILE_MAP_WRITE, 0, 0, 0));
CRITICAL_ASSERT(SystemAssertHandler, m_data != nullptr, "Failed to map the file {}", path);
#elif defined BIGFOOT_LINUX
m_fileDescriptor = open(path.data(), O_RDWR | O_CREAT, S_IRUSR | S_IWUSR);
CRITICAL_ASSERT(SystemAssertHandler, m_fileDescriptor != -1, "Failed to create file {}", path);
struct stat fileStat {};
fstat(m_fileDescriptor, &fileStat);
if (static_cast<std::size_t>(fileStat.st_size) < p_size)
{
[[maybe_unused]]
const int truncateResult = ftruncate(m_fileDescriptor, static_cast<off_t>(p_size));
CRITICAL_ASSERT(SystemAssertHandler, truncateResult == 0, "Failed to size file {}", path);
}
void* const mapped = mmap(nullptr, m_size, PROT_READ | PROT_WRITE, MAP_SHARED, m_fileDescriptor, 0);
CRITICAL_ASSERT(SystemAssertHandler, mapped != MAP_FAILED, "Failed to map file {}", path);
m_data = static_cast<std::byte*>(mapped);
#else
CRITICAL_ASSERT(SystemAssertHandler, false, "MemoryMappedFileBase is not implemented on this platform");
#endif
}
/****************************************************************************************/
WritableMemoryMappedFile::~WritableMemoryMappedFile()
{
#ifdef BIGFOOT_WINDOWS
if (m_data != nullptr)
{
[[maybe_unused]]
const BOOL unmapped = UnmapViewOfFile(m_data);
CRITICAL_ASSERT(SystemAssertHandler, unmapped, "Failed to unmap the file");
}
if (m_mappingHandle != nullptr)
{
[[maybe_unused]]
const BOOL closedMapping = CloseHandle(m_mappingHandle);
CRITICAL_ASSERT(SystemAssertHandler, closedMapping, "Failed to close the file mapping");
}
if (m_fileHandle != INVALID_HANDLE_VALUE)
{
[[maybe_unused]]
const BOOL closedFile = CloseHandle(m_fileHandle);
CRITICAL_ASSERT(SystemAssertHandler, closedFile, "Failed to close the file");
}
#elif defined BIGFOOT_LINUX
if (m_data != nullptr)
{
[[maybe_unused]]
const int unmapped = munmap(m_data, m_size);
CRITICAL_ASSERT(SystemAssertHandler, unmapped == 0, "Failed to unmap the file");
}
if (m_fileDescriptor != -1)
{
[[maybe_unused]]
const int closedFile = close(m_fileDescriptor);
CRITICAL_ASSERT(SystemAssertHandler, closedFile == 0, "Failed to close the file");
}
#else
CRITICAL_ASSERT(SystemAssertHandler, false, "MemoryMappedFileBase is not implemented on this platform");
#endif
}
/****************************************************************************************/
eastl::span<std::byte> WritableMemoryMappedFile::GetData() const
{
return eastl::span<std::byte> {m_data, m_size};
}
/****************************************************************************************/
void WritableMemoryMappedFile::Prefetch(const std::size_t p_offset, const std::size_t p_size) const
{
#ifdef BIGFOOT_WINDOWS
WIN32_MEMORY_RANGE_ENTRY range {.VirtualAddress = m_data + p_offset, .NumberOfBytes = p_size};
[[maybe_unused]]
const BOOL prefetched = PrefetchVirtualMemory(GetCurrentProcess(), 1, &range, 0);
#elif defined BIGFOOT_LINUX
[[maybe_unused]]
const int adviseResult = madvise(m_data + p_offset, p_size, MADV_WILLNEED);
#else
CRITICAL_ASSERT(SystemAssertHandler, false, "MemoryMappedFileBase is not implemented on this platform");
#endif
}
} // namespace Bigfoot
@@ -7,10 +7,8 @@
#ifndef BIGFOOT_SYSTEM_FILE_HPP
#define BIGFOOT_SYSTEM_FILE_HPP
#include <EASTL/span.h>
#include <EASTL/string.h>
#include <EASTL/string_view.h>
#include <EASTL/vector.h>
#include <filesystem>
@@ -26,6 +24,9 @@ class File
*/
File(const eastl::string_view p_path);
File(const File& p_path) = default;
File(File&& p_path) = default;
/**
* Check if the file is relative to the current executable
*
@@ -74,37 +75,10 @@ class File
[[nodiscard]]
File ToAbsolute() const;
/**
* Get the size of the file
*
* \return The size in bytes
*/
[[nodiscard]]
std::size_t GetSize() const;
~File() = default;
/**
* Read bytes from the file
*
* \param p_offset The offset in bytes
* \param p_size The number of bytes to read
* \return The bytes read
*/
[[nodiscard]]
eastl::vector<std::byte> Read(std::size_t p_offset, std::size_t p_size) const;
/**
* Write bytes to the file at a given offset, creating the file if needed. Content before
* p_offset, or from past p_offset + p_data.size(), is left untouched.
*
* \param p_data The bytes to write
* \param p_offset The offset, in bytes, to start writing at
*/
void Write(eastl::span<const std::byte> p_data, std::size_t p_offset) const;
/**
* Delete the file
*/
void Delete() const;
File& operator=(const File& p_path) = default;
File& operator=(File&& p_path) = default;
private:
/**
@@ -112,7 +86,7 @@ class File
*
* \param p_path The path
*/
File(std::filesystem::path p_path);
File(const std::filesystem::path& p_path);
/**
* The path
@@ -1,150 +0,0 @@
/*********************************************************************
* \file MemoryMappedFile.hpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#ifndef BIGFOOT_SYSTEM_MEMORYMAPPEDFILE_HPP
#define BIGFOOT_SYSTEM_MEMORYMAPPEDFILE_HPP
#include <System/File/File.hpp>
#include <EASTL/span.h>
#include <cstddef>
#include <cstdint>
namespace Bigfoot
{
class MemoryMappedFile
{
public:
/**
* Constructor
* Opens and maps the whole file
*
* \param p_file The file to map
*/
explicit MemoryMappedFile(const File& p_file);
MemoryMappedFile(const MemoryMappedFile& p_file) = delete;
MemoryMappedFile(MemoryMappedFile&& p_file) = delete;
/**
* Destructor
*
*/
~MemoryMappedFile();
/**
* Get the mapped content of the file
*
* \return The mapped bytes
*/
[[nodiscard]]
eastl::span<const std::byte> GetData() const;
MemoryMappedFile& operator=(const MemoryMappedFile& p_file) = delete;
MemoryMappedFile& operator=(MemoryMappedFile&& p_file) = delete;
private:
#ifdef BIGFOOT_WINDOWS
/**
* Handle to the file
*/
void* m_fileHandle = nullptr;
/**
* Handle to the file mapping
*/
void* m_mappingHandle = nullptr;
#elif defined BIGFOOT_LINUX
/**
* Handle to the file
*/
int m_fileDescriptor = -1;
#endif
/**
* The mapped size
*/
std::size_t m_size = 0;
/**
* The mapped data
*/
std::byte* m_data = nullptr;
};
/****************************************************************************************/
class WritableMemoryMappedFile
{
public:
/**
* Constructor
* Creates and maps the file
*
* \param p_file The file to create and map
* \param p_size The size to map
*/
WritableMemoryMappedFile(const File& p_file, std::size_t p_size);
WritableMemoryMappedFile(const WritableMemoryMappedFile& p_file) = delete;
WritableMemoryMappedFile(WritableMemoryMappedFile&& p_file) = delete;
/**
* Destructor
*
*/
~WritableMemoryMappedFile();
/**
* Get the mapped content of the file
*
* \return The mapped bytes
*/
[[nodiscard]]
eastl::span<std::byte> GetData() const;
/**
* Hint the OS to prefetch a range of data, so writes into that range do not each pay a page fault
*
* \param p_offset The offset, in bytes, to start prefetching from
* \param p_size The number of bytes to prefetch
*/
void Prefetch(std::size_t p_offset, std::size_t p_size) const;
WritableMemoryMappedFile& operator=(const WritableMemoryMappedFile& p_file) = delete;
WritableMemoryMappedFile& operator=(WritableMemoryMappedFile&& p_file) = delete;
private:
#ifdef BIGFOOT_WINDOWS
/**
* Handle to the file
*/
void* m_fileHandle = nullptr;
/**
* Handle to the file mapping
*/
void* m_mappingHandle = nullptr;
#elif defined BIGFOOT_LINUX
/**
* Handle to the file
*/
int m_fileDescriptor = -1;
#endif
/**
* The mapped size
*/
std::size_t m_size = 0;
/**
* The mapped data
*/
std::byte* m_data = nullptr;
};
} // namespace Bigfoot
#endif
@@ -10,7 +10,7 @@
#include <Utils/Assert/AssertHandler.hpp>
#ifdef BIGFOOT_NOT_OPTIMIZED
#if defined BIGFOOT_NOT_OPTIMIZED
namespace Bigfoot
{
@@ -15,7 +15,7 @@ namespace Bigfoot
/*
* Logger
*/
inline Log::LoggerInfo SYSTEM_LOGGER { .m_name = "SYSTEM_LOGGER", .m_level = Flat::LogLevel::Trace};
inline Log::LoggerInfo SYSTEM_LOGGER {"SYSTEM_LOGGER", Flat::LogLevel::Trace};
} // namespace Bigfoot
#endif
#endif
@@ -17,6 +17,9 @@ class Time
public:
Time(const std::uint64_t p_epoch);
Time(const Time& p_time) = default;
Time(Time&& p_time) = default;
[[nodiscard]]
std::uint32_t GetYear() const;
@@ -41,11 +44,10 @@ class Time
[[nodiscard]]
std::uint64_t GetEpoch() const;
[[nodiscard]]
bool operator==(const Time& p_other) const
{
return m_epoch == p_other.m_epoch;
}
~Time() = default;
Time& operator=(const Time& p_time) = default;
Time& operator=(Time&& p_time) = default;
[[nodiscard]]
auto operator<=>(const Time& p_other) const
@@ -65,10 +67,13 @@ class Time
/****************************************************************************************/
namespace Flat::Bigfoot
namespace Flat
{
namespace Bigfoot
{
struct Time;
} // namespace Flat::Bigfoot
}
} // namespace Flat
/****************************************************************************************/
@@ -6,9 +6,6 @@
*********************************************************************/
#ifndef BIGFOOT_SYSTEM_UUID_UUID_HPP
#define BIGFOOT_SYSTEM_UUID_UUID_HPP
#include <EASTL/string.h>
#include <ankerl/unordered_dense.h>
#include <fmt/format.h>
#include <uuid.h>
namespace Bigfoot
@@ -18,11 +15,6 @@ class UUID
public:
static constexpr std::uint32_t UUID_BYTE_SIZE = 16;
/**
* Constructor
* Will default initialize an UUID to a valid value
*
*/
UUID();
/**
@@ -39,6 +31,9 @@ class UUID
*/
explicit UUID(const std::span<const std::byte, UUID_BYTE_SIZE> p_raw);
UUID(const UUID& p_uuid) = default;
UUID(UUID&& p_uuid) noexcept = default;
/**
* Gets the raw bytes of a UUID
*
@@ -53,10 +48,15 @@ class UUID
* \return A string representing the UUID
*/
[[nodiscard]]
eastl::string ToString() const;
std::string ToString() const;
~UUID() = default;
operator bool() const;
UUID& operator=(const UUID& p_uuid) = default;
UUID& operator=(UUID&& p_uuid) noexcept = default;
[[nodiscard]]
bool operator==(const UUID& p_uuid) const;
[[nodiscard]]
@@ -84,6 +84,8 @@ class UUID
static UUID NULL_UUID;
private:
friend struct std::hash<UUID>;
/**
* The uuid.
*/
@@ -106,48 +108,43 @@ class UUID
template<>
struct std::hash<Bigfoot::UUID>
{
using is_avalanching = void;
[[nodiscard]]
std::size_t operator()(const Bigfoot::UUID& p_uuid) const noexcept
{
const auto bytes = p_uuid.ToBytes();
return ankerl::unordered_dense::detail::wyhash::hash(bytes.data(), bytes.size());
return std::hash<uuids::uuid>()(p_uuid.m_uuid);
}
};
/****************************************************************************************/
template<>
struct fmt::formatter<Bigfoot::UUID>
struct std::formatter<Bigfoot::UUID, char>
{
static constexpr auto parse(format_parse_context& ctx)
constexpr auto parse(std::format_parse_context& ctx)
{
return ctx.begin();
}
static auto format(const Bigfoot::UUID& p_uuid, format_context& ctx)
template<typename FormatContext>
auto format(const Bigfoot::UUID& p_uuid, FormatContext& ctx) const
{
return fmt::format_to(ctx.out(), "{}", p_uuid.ToString().c_str());
return std::format_to(ctx.out(), "{}", p_uuid.ToString());
}
};
/****************************************************************************************/
#ifdef BIGFOOT_NOT_OPTIMIZED
#if defined BIGFOOT_NOT_OPTIMIZED
#include <quill/DeferredFormatCodec.h>
template<>
struct fmtquill::formatter<Bigfoot::UUID>
{
static constexpr auto parse(format_parse_context& ctx)
constexpr auto parse(format_parse_context& ctx)
{
return ctx.begin();
}
static auto format(const Bigfoot::UUID& p_uuid, format_context& ctx)
auto format(const Bigfoot::UUID& p_uuid, format_context& ctx) const
{
return fmtquill::format_to(ctx.out(), "{}", p_uuid.ToString().c_str());
return fmtquill::format_to(ctx.out(), "{}", p_uuid.ToString());
}
};
@@ -159,10 +156,13 @@ struct quill::Codec<Bigfoot::UUID>: quill::DeferredFormatCodec<Bigfoot::UUID>
/****************************************************************************************/
namespace Flat::Bigfoot
namespace Flat
{
namespace Bigfoot
{
struct UUID;
} // namespace Flat::Bigfoot
}
} // namespace Flat
/****************************************************************************************/
+1 -2
View File
@@ -57,8 +57,7 @@ Time::Time(const std::uint64_t p_epoch):
std::uint32_t Time::GetYear() const
{
constexpr std::uint32_t TM_YEAR_EPOCH = 1900;
return m_timeInfo.tm_year + TM_YEAR_EPOCH;
return m_timeInfo.tm_year + 1900;
}
/****************************************************************************************/
+3 -4
View File
@@ -35,7 +35,7 @@ UUID::UUID(const std::string_view p_string):
UUID::UUID(const std::span<const std::byte, UUID_BYTE_SIZE> p_raw)
{
const std::span<std::uint8_t, UUID_BYTE_SIZE> raw {
const_cast<std::uint8_t*>(reinterpret_cast<const std::uint8_t*>(p_raw.data())),
const_cast<std::uint8_t*>(std::bit_cast<const std::uint8_t*>(p_raw.data())),
UUID_BYTE_SIZE};
m_uuid = uuids::uuid {raw};
@@ -50,10 +50,9 @@ std::span<const std::byte, UUID::UUID_BYTE_SIZE> UUID::ToBytes() const
/****************************************************************************************/
eastl::string UUID::ToString() const
std::string UUID::ToString() const
{
const std::string uuid = uuids::to_string(m_uuid);
return eastl::string {uuid.data(), uuid.size()};
return uuids::to_string(m_uuid);
}
/****************************************************************************************/
+1 -2
View File
@@ -1,2 +1 @@
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/SasquatchBaker)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/Executables/SasquatchBaker)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/SasquatchPacker)
@@ -1,50 +0,0 @@
/*********************************************************************
* \file main.cpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#include <System/File/File.hpp>
#include <Utils/Log/LogMacros.hpp>
#include <Utils/TargetMacros.h>
#include <SasquatchBaker/SasquatchBaker.hpp>
#include <EASTL/string_view.h>
#include <lyra/lyra.hpp>
#include <iostream>
#include <string>
int main(int argc, char** argv)
{
BIGFOOT_NOT_OPTIMIZED_ONLY(::Bigfoot::Singleton<::Bigfoot::Log>::Lifetime logLifetime);
bool showHelp = false;
std::string bigFilePath;
std::string outputPath;
const auto& cli = lyra::help(showHelp) |
lyra::opt(bigFilePath, "bigfile")["-b"]["--bigfile"]("The BigFile to bake (must exist)")
.required() |
lyra::opt(outputPath, "output")["-o"]["--output"]("The Sasquatch file to write").required();
const lyra::parse_result result = cli.parse({argc, argv});
if (!result)
{
std::cerr << result.message() << "\n\n" << cli << '\n';
return 1;
}
if (showHelp)
{
std::cout << cli << '\n';
return 0;
}
const Bigfoot::BigFile bigFile {Bigfoot::File {eastl::string_view {bigFilePath.c_str()}}};
Bigfoot::Bake(bigFile, Bigfoot::File {eastl::string_view {outputPath.c_str()}});
return 0;
}
@@ -1,18 +0,0 @@
get_filename_component(PackageName ${CMAKE_CURRENT_SOURCE_DIR} NAME)
project(${PackageName})
set(PublicDependencies)
set(PrivateDependencies)
set(BigfootPublicDependencies
Engine)
set(BigfootPrivateDependencies
Utils)
bigfoot_create_package_lib(
"${PublicDependencies}"
"${PrivateDependencies}"
"${BigfootPublicDependencies}"
"${BigfootPrivateDependencies}"
"Tools")
bigfoot_create_logger()
@@ -1,24 +0,0 @@
/*********************************************************************
* \file SasquatchBaker.hpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#ifndef BIGFOOT_TOOLS_SASQUATCHBAKER_SASQUATCHBAKER_HPP
#define BIGFOOT_TOOLS_SASQUATCHBAKER_SASQUATCHBAKER_HPP
#include <Engine/BigFile/BigFile.hpp>
#include <System/File/File.hpp>
namespace Bigfoot
{
/**
* Bake every Asset of a BigFile into a Sasquatch file
*
* \param p_bigFile The BigFile to bake
* \param p_outputFile The file to write to
*/
void Bake(const BigFile& p_bigFile, const File& p_outputFile);
} // namespace Bigfoot
#endif
@@ -1,20 +0,0 @@
/*********************************************************************
* \file SasquatchBakerAssertHandler.hpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#ifndef BIGFOOT_TOOLS_SASQUATCHBAKER_SASQUATCHBAKERASSERTHANDLER_HPP
#define BIGFOOT_TOOLS_SASQUATCHBAKER_SASQUATCHBAKERASSERTHANDLER_HPP
#include <Utils/Assert/AssertHandler.hpp>
#include <SasquatchBaker/SasquatchBakerLogger_generated.hpp>
#ifdef BIGFOOT_NOT_OPTIMIZED
namespace Bigfoot
{
using SasquatchBakerAssertHandler = AssertHandler<SASQUATCHBAKER_LOGGER>;
} // namespace Bigfoot
#endif
#endif
@@ -1,21 +0,0 @@
// AUTO-GENERATED DO NOT TOUCH
/*********************************************************************
* \file SasquatchBakerLogger.generated.hpp
*
*********************************************************************/
#ifndef BIGFOOT_SASQUATCHBAKERLOGGER_GENERATED_HPP
#define BIGFOOT_SASQUATCHBAKERLOGGER_GENERATED_HPP
#include <Utils/Log/LogMacros.hpp>
#if defined BIGFOOT_NOT_OPTIMIZED
namespace Bigfoot
{
/*
* Logger
*/
inline Log::LoggerInfo SASQUATCHBAKER_LOGGER { .m_name = "SASQUATCHBAKER_LOGGER", .m_level = Flat::LogLevel::Trace};
} // namespace Bigfoot
#endif
#endif
@@ -1,65 +0,0 @@
/*********************************************************************
* \file SasquatchBaker.cpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#include <SasquatchBaker/SasquatchBaker.hpp>
#include <Engine/Sasquatch/Sasquatch.hpp>
#include <SasquatchBaker/SasquatchBakerAssertHandler.hpp>
namespace Bigfoot
{
void Bake(const BigFile& p_bigFile, const File& p_outputFile)
{
std::uint64_t assetCount = 0;
{
BigFile::Request countRequest {p_bigFile, "SELECT COUNT(*) FROM Asset;"};
[[maybe_unused]]
const bool hasCount = countRequest.Step();
ASSERT(SasquatchBakerAssertHandler, hasCount, "Failed to count the assets of the BigFile");
assetCount = countRequest.Get(0);
}
std::uint64_t totalFlatbufferBytes = 0;
{
BigFile::Request sizeRequest {p_bigFile, "SELECT COALESCE(SUM(LENGTH(Asset)), 0) FROM Asset;"};
[[maybe_unused]]
const bool hasSize = sizeRequest.Step();
ASSERT(SasquatchBakerAssertHandler, hasSize, "Failed to sum the asset sizes of the BigFile");
totalFlatbufferBytes = sizeRequest.Get(0);
}
std::uint64_t totalHardReferencesBytes = 0;
{
BigFile::Request sizeRequest {p_bigFile,
"SELECT COALESCE(SUM(LENGTH(HardReference)), 0) FROM AssetHardReference;"};
[[maybe_unused]]
const bool hasSize = sizeRequest.Step();
ASSERT(SasquatchBakerAssertHandler, hasSize, "Failed to sum the hardreferences sizes of the BigFile");
totalHardReferencesBytes = sizeRequest.Get(0);
}
SasquatchWriter writer {p_outputFile, assetCount, totalFlatbufferBytes, totalHardReferencesBytes};
BigFile::Request request {p_bigFile, "SELECT UUID FROM Asset;"};
while (request.Step())
{
const std::span<const std::byte> rawUUID = request.Get(0);
ASSERT(SasquatchBakerAssertHandler,
rawUUID.size() == UUID::UUID_BYTE_SIZE,
"Unexpected UUID blob size in BigFile (got {} / expected {})",
rawUUID.size(),
UUID::UUID_BYTE_SIZE);
const UUID uuid {std::span<const std::byte, UUID::UUID_BYTE_SIZE> {rawUUID.data(), UUID::UUID_BYTE_SIZE}};
const eastl::vector<std::byte> flatbuffer = p_bigFile.GetAssetFlatbuffer(uuid);
const eastl::vector<UUID> hardReferences = p_bigFile.GetAssetHardReferences(uuid);
writer.WriteAsset(uuid, flatbuffer, hardReferences);
}
}
} // namespace Bigfoot
@@ -1,4 +1,5 @@
project(SasquatchBakerCLI)
get_filename_component(PackageName ${CMAKE_CURRENT_SOURCE_DIR} NAME)
project(${PackageName})
add_executable(${PROJECT_NAME})
@@ -7,27 +8,18 @@ target_compile_features(${PROJECT_NAME} PRIVATE cxx_std_20)
target_link_libraries(${PROJECT_NAME}
PRIVATE
BigfootCompileAndLinkFlags
bfg::lyra
$<LINK_LIBRARY:WHOLE_ARCHIVE,SasquatchBaker>)
$<LINK_LIBRARY:WHOLE_ARCHIVE,Engine>)
file(GLOB_RECURSE _SOURCES
CONFIGURE_DEPENDS
${CMAKE_CURRENT_SOURCE_DIR}/*.cpp
)
file(GLOB_RECURSE _HEADERS
CONFIGURE_DEPENDS
${CMAKE_CURRENT_SOURCE_DIR}/*.h
${CMAKE_CURRENT_SOURCE_DIR}/*.hpp
)
target_sources(${PROJECT_NAME}
PRIVATE
${_SOURCES}
${_HEADERS}
)
source_group(TREE ${CMAKE_CURRENT_SOURCE_DIR} PREFIX Src FILES ${_SOURCES} ${_HEADERS})
source_group(TREE ${CMAKE_CURRENT_SOURCE_DIR} PREFIX Src FILES ${_SOURCES})
set_target_properties(${PROJECT_NAME} PROPERTIES FOLDER Bigfoot/Tools/Executables)
set_target_properties(${PROJECT_NAME} PROPERTIES OUTPUT_NAME SasquatchBaker)
set_target_properties(${PROJECT_NAME} PROPERTIES FOLDER Bigfoot/Tools)
@@ -0,0 +1,30 @@
/*********************************************************************
* \file main.cpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#include <Engine/BigFile/BigFile.hpp>
#include <Engine/Sasquatch/Sasquatch.hpp>
#include <System/File.hpp>
#include <EASTL/string_view.h>
#include <cstdio>
// Packages a BigFile into a .sasquatch file for Game-mode builds to load. Tool-mode-only build (see
// Sasquatch::WriteFrom): run once as a ship/packaging step, after BigFile content is finalized.
int main(const int argc, char** argv)
{
if (argc != 3)
{
std::fprintf(stderr, "Usage: %s <BigFile.db> <Output.sasquatch>\n", argv[0]);
return 1;
}
Bigfoot::BigFile bigFile {Bigfoot::File {eastl::string_view {argv[1]}}};
Bigfoot::Sasquatch::WriteFrom(bigFile, Bigfoot::File {eastl::string_view {argv[2]}});
return 0;
}
+1 -5
View File
@@ -6,11 +6,8 @@ set(PublicDependencies
$<IF:$<BOOL:${ASAN}>,mimalloc-asan,mimalloc-static>
unordered_dense::unordered_dense
EASTL::EASTL
EAStdC::EAStdC
fmt::fmt
flatbuffers::flatbuffers
rapidhash::rapidhash
effolkronium_random)
rapidhash::rapidhash)
set(PrivateDependencies)
set(BigfootPublicDependencies)
set(BigfootPrivateDependencies)
@@ -22,7 +19,6 @@ bigfoot_create_package_lib(
"${BigfootPrivateDependencies}"
"")
set_source_files_properties(../Utils/MimallocImpl.cpp PROPERTIES SKIP_UNITY_BUILD_INCLUSION ON)
set_source_files_properties(../Utils/EASTLVsnprintfImpl.cpp PROPERTIES SKIP_UNITY_BUILD_INCLUSION ON)
bigfoot_create_logger()
@@ -1,154 +0,0 @@
/*********************************************************************
* \file Graph.hpp
*
* \author Romain BOULLARD
* \date August 2026
*********************************************************************/
#ifndef BIGFOOT_UTILS_ALGORITHM_GRAPH_HPP
#define BIGFOOT_UTILS_ALGORITHM_GRAPH_HPP
#include <Utils/UtilsAssertHandler.hpp>
#include <EASTL/vector.h>
#include <ankerl/unordered_dense.h>
#include <cstdint>
#include <limits>
namespace Bigfoot
{
/*
* Kahn's algorithm
*
* \param p_graph Every discovered node's outgoing edges (eg: its dependencies)
* \param p_nodes The set of nodes to sort
* \param p_waves The sorted nodes
* \param p_cyclicNodes Nodes that could not be sorted (ie: part of a cycle)
* \return True if a cycle was found
*/
template<class NODE_TYPE>
[[nodiscard]]
bool LayeredTopologicalSort(const ankerl::unordered_dense::map<NODE_TYPE, eastl::vector<NODE_TYPE>>& p_graph,
const ankerl::unordered_dense::set<NODE_TYPE>& p_nodes,
eastl::vector<eastl::vector<NODE_TYPE>>& p_waves,
eastl::vector<NODE_TYPE>& p_cyclicNodes)
{
ASSERT(UtilsAssertHandler, p_waves.empty(), "p_waves must be empty");
ASSERT(UtilsAssertHandler, p_cyclicNodes.empty(), "p_cyclicNodes must be empty");
ASSERT(UtilsAssertHandler,
p_nodes.size() <= std::numeric_limits<std::uint32_t>::max(),
"Can't sort more than {} nodes",
std::numeric_limits<std::uint32_t>::max());
const auto nodeCount = static_cast<std::uint32_t>(p_nodes.size());
ankerl::unordered_dense::map<NODE_TYPE, std::uint32_t> nodeToIndex;
nodeToIndex.reserve(nodeCount);
eastl::vector<NODE_TYPE> indexToNode;
indexToNode.reserve(nodeCount);
for (const NODE_TYPE& node: p_nodes)
{
nodeToIndex.emplace(node, static_cast<std::uint32_t>(indexToNode.size()));
indexToNode.push_back(node);
}
// Step 1: Compute in-degree for each node
eastl::vector<std::uint32_t> inDegree(nodeCount, 0);
// Reverse adjacency (which nodes depend on each node) stored as CSR: adjacencyOffsets[i]..adjacencyOffsets[i + 1]
// is node i's slice of adjacency. Built in two passes (count listeners per node, then fill) so the whole
// structure is a couple of flat allocations instead of one per node.
eastl::vector<std::uint32_t> adjacencyOffsets(nodeCount + 1, 0);
// Initialize queue with nodes having in-degree 0
eastl::vector<std::uint32_t> queue;
queue.reserve(nodeCount);
for (std::uint32_t index = 0; index < nodeCount; ++index)
{
std::uint32_t degree = 0;
for (const NODE_TYPE& edge: p_graph.at(indexToNode[index]))
{
const auto nodeIt = nodeToIndex.find(edge);
if (nodeIt != nodeToIndex.end())
{
++degree;
++adjacencyOffsets[nodeIt->second + 1];
}
}
inDegree[index] = degree;
if (degree == 0)
{
queue.push_back(index);
}
}
for (std::uint32_t index = 0; index < nodeCount; ++index)
{
adjacencyOffsets[index + 1] += adjacencyOffsets[index];
}
eastl::vector<std::uint32_t> adjacency(adjacencyOffsets.back());
eastl::vector<std::uint32_t> cursor(adjacencyOffsets.begin(), adjacencyOffsets.end() - 1);
for (std::uint32_t index = 0; index < nodeCount; ++index)
{
for (const NODE_TYPE& edge: p_graph.at(indexToNode[index]))
{
const auto nodeIt = nodeToIndex.find(edge);
if (nodeIt != nodeToIndex.end())
{
adjacency[cursor[nodeIt->second]++] = index;
}
}
}
// Step 2: Process nodes
std::size_t resolvedCount = 0;
eastl::vector<std::uint32_t> nextQueue;
nextQueue.reserve(nodeCount);
while (!queue.empty())
{
resolvedCount += queue.size();
// Reduce in-degree for neighbors
nextQueue.clear();
for (const std::uint32_t index: queue)
{
for (std::uint32_t i = adjacencyOffsets[index]; i < adjacencyOffsets[index + 1]; ++i)
{
const std::uint32_t next = adjacency[i];
if (--inDegree[next] == 0)
{
nextQueue.push_back(next);
}
}
}
eastl::vector<NODE_TYPE> waveNodes;
waveNodes.reserve(queue.size());
for (const std::uint32_t index: queue)
{
waveNodes.push_back(indexToNode[index]);
}
p_waves.push_back(std::move(waveNodes));
queue.swap(nextQueue);
}
// Step 3 & 4: Check for valid topological order
if (resolvedCount != nodeCount)
{
p_cyclicNodes.reserve(nodeCount - resolvedCount);
for (std::uint32_t index = 0; index < nodeCount; ++index)
{
if (inDegree[index] != 0)
{
p_cyclicNodes.push_back(indexToNode[index]);
}
}
}
return resolvedCount != nodeCount;
}
} // namespace Bigfoot
#endif
@@ -7,23 +7,26 @@
#ifndef BIGFOOT_UTILS_ASSERT_HPP
#define BIGFOOT_UTILS_ASSERT_HPP
#include <Utils/Log/LogMacros.hpp>
#include <Utils/TargetMacros.h>
#ifdef BIGFOOT_NOT_OPTIMIZED
#if defined BIGFOOT_NOT_OPTIMIZED
#include <Utils/Format.hpp>
#include <EASTL/fixed_function.h>
#include <EASTL/string.h>
#include <cstdlib>
#include <source_location>
#include <string>
#ifdef BIGFOOT_LINUX
#if defined BIGFOOT_LINUX
#include <csignal>
#endif
#ifdef BIGFOOT_WINDOWS
#define BIGFOOT_FLUSH_LOG \
do \
{ \
if (Bigfoot::Singleton<Bigfoot::Log>::HasInstance()) \
{ \
Bigfoot::Singleton<Bigfoot::Log>::GetInstance().Flush(); \
} \
} while (false)
#if defined BIGFOOT_WINDOWS
#define BREAK \
do \
{ \
@@ -39,68 +42,39 @@
} while (false)
#endif
namespace Bigfoot
{
/**
* Assert then break
*
* \param p_condition The condition to check
* \param p_message A function that will return the formatting string describing the issue
* \param p_location The location the assert originates from
*/
template<typename HANDLER>
BIGFOOT_ALWAYS_INLINE void AssertCheck(const bool p_condition,
const eastl::fixed_function<32, eastl::string()>& p_messageFn,
const std::source_location p_location = std::source_location::current())
{
if (!p_condition) [[unlikely]]
{
HANDLER::Handle(p_location, p_messageFn());
BREAK;
}
}
/**
* Assert then break then kill
*
* \param p_condition The condition to check
* \param p_message A function that will return the formatting string describing the issue
* \param p_location The location the assert originates from
*/
template<typename HANDLER>
BIGFOOT_ALWAYS_INLINE void CriticalAssertCheck(const bool p_condition,
const eastl::fixed_function<32, eastl::string()>& p_messageFn,
const std::source_location p_location = std::source_location::current())
{
if (!p_condition) [[unlikely]]
{
HANDLER::Handle(p_location, p_messageFn());
BREAK;
std::abort();
}
}
} // namespace Bigfoot
#define ASSERT(HANDLER, p_assert, p_message, ...) \
::Bigfoot::AssertCheck<HANDLER>((p_assert), \
[&] \
{ \
return ::Bigfoot::Format(p_message __VA_OPT__(, ) __VA_ARGS__); \
})
do \
{ \
constexpr std::source_location location = std::source_location::current(); \
if (!(p_assert)) [[unlikely]] \
{ \
HANDLER::Handle(location, p_message __VA_OPT__(, ) __VA_ARGS__); \
BREAK; \
} \
} while (false)
#define SOFT_ASSERT(HANDLER, p_assert, p_message, ...) \
::Bigfoot::AssertCheck<HANDLER>((p_assert), \
[&] \
{ \
return ::Bigfoot::Format(p_message __VA_OPT__(, ) __VA_ARGS__); \
})
do \
{ \
constexpr std::source_location location = std::source_location::current(); \
if (!(p_assert)) [[unlikely]] \
{ \
HANDLER::Handle(location, p_message __VA_OPT__(, ) __VA_ARGS__); \
BREAK; \
} \
} while (false)
#define CRITICAL_ASSERT(HANDLER, p_assert, p_message, ...) \
::Bigfoot::CriticalAssertCheck<HANDLER>((p_assert), \
[&] \
{ \
return ::Bigfoot::Format(p_message __VA_OPT__(, ) __VA_ARGS__); \
})
do \
{ \
constexpr std::source_location location = std::source_location::current(); \
if (!(p_assert)) [[unlikely]] \
{ \
HANDLER::Handle(location, p_message __VA_OPT__(, ) __VA_ARGS__); \
BREAK; \
std::abort(); \
} \
} while (false)
#else
#define ASSERT(HANDLER, p_assert, p_message, ...)
@@ -9,10 +9,9 @@
#include <Utils/Assert/Assert.hpp>
#include <Utils/Log/Log.hpp>
#ifdef BIGFOOT_NOT_OPTIMIZED
#include <EASTL/string.h>
#if defined BIGFOOT_NOT_OPTIMIZED
#include <format>
#include <source_location>
namespace Bigfoot
@@ -32,13 +31,15 @@ class AssertHandler
* Handle an assertion.
*
* \param p_location Location of the assertion.
* \param p_message Message describing the assertion failure.
* \param p_format Format string for the assertion message.
* \param p_args Arguments for the format string.
*/
static void Handle(const std::source_location& p_location, const eastl::string& p_message)
template<typename... ARGS>
static void Handle(const std::source_location& p_location, std::format_string<ARGS...> p_format, ARGS&&... p_args)
{
BIGFOOT_LOG_FATAL(LOGGER,
"Assert: {} (File:{}, Line:{}, Function:{}\n",
p_message,
std::format(p_format, std::forward<ARGS>(p_args)...),
p_location.file_name(),
p_location.line(),
p_location.function_name());
+13 -9
View File
@@ -6,33 +6,37 @@
*********************************************************************/
#ifndef BIGFOOT_UTILS_CASTER_HPP
#define BIGFOOT_UTILS_CASTER_HPP
#include <type_traits>
#include <EASTL/type_traits.h>
namespace Bigfoot
{
template<class TO, class FROM>
requires(std::is_base_of_v<FROM, TO> || std::is_base_of_v<TO, FROM>)
template<class TO,
class FROM,
eastl::enable_if_t<eastl::is_base_of_v<FROM, TO> || eastl::is_base_of_v<TO, FROM>, bool> = true>
constexpr TO* Cast(FROM* p_object)
{
return static_cast<TO*>(p_object);
}
template<class TO, class FROM>
requires(std::is_base_of_v<FROM, TO> || std::is_base_of_v<TO, FROM>)
template<class TO,
class FROM,
eastl::enable_if_t<eastl::is_base_of_v<FROM, TO> || eastl::is_base_of_v<TO, FROM>, bool> = true>
constexpr const TO* Cast(const FROM* p_object)
{
return static_cast<const TO*>(p_object);
}
template<class TO, class FROM>
requires(std::is_base_of_v<FROM, TO> || std::is_base_of_v<TO, FROM>)
template<class TO,
class FROM,
eastl::enable_if_t<eastl::is_base_of_v<FROM, TO> || eastl::is_base_of_v<TO, FROM>, bool> = true>
constexpr TO& Cast(FROM& p_object)
{
return static_cast<TO&>(p_object);
}
template<class TO, class FROM>
requires(std::is_base_of_v<FROM, TO> || std::is_base_of_v<TO, FROM>)
template<class TO,
class FROM,
eastl::enable_if_t<eastl::is_base_of_v<FROM, TO> || eastl::is_base_of_v<TO, FROM>, bool> = true>
constexpr const TO& Cast(const FROM& p_object)
{
return static_cast<const TO&>(p_object);
@@ -28,7 +28,7 @@ class SlotMapKey
private:
static constexpr std::uint32_t VERSION_BIT_COUNT = sizeof(VersionType) * std::numeric_limits<unsigned char>::digits;
static constexpr std::uint32_t INDEX_BIT_COUNT = sizeof(IndexType) * std::numeric_limits<unsigned char>::digits;
static_assert(VERSION_BIT_COUNT + INDEX_BIT_COUNT <= std::numeric_limits<std::uint64_t>::digits,
static_assert(VERSION_BIT_COUNT + INDEX_BIT_COUNT <= 64,
"We cant construct a 64 bit key from the given Version and Index types!");
static constexpr std::uint64_t INDEX_MASK = (static_cast<std::uint64_t>(1) << INDEX_BIT_COUNT) - 1;
@@ -39,93 +39,68 @@ class SlotMapKey
static constexpr IndexType MAX_INDEX = std::numeric_limits<IndexType>::max();
static constexpr VersionType MAX_VERSION = std::numeric_limits<VersionType>::max();
/**
* Constructor
*
* \param p_version The version
* \param p_index The index
*/
constexpr SlotMapKey(const VersionType p_version, const IndexType p_index):
m_key((static_cast<std::uint64_t>(p_version) << INDEX_BIT_COUNT) |
(static_cast<std::uint64_t>(p_index) & INDEX_MASK))
{
}
/**
* Constructor
*
* Constructs an invalid key
*/
constexpr SlotMapKey():
SlotMapKey(INVALID_VERSION, 0)
{
}
/**
* Check if the key is valid (Version different than INVALID_VERSION)
*
* \return True if valid, false otherwise
*/
constexpr SlotMapKey(const SlotMapKey& p_slotKey) = default;
constexpr SlotMapKey(SlotMapKey&& p_slotKey) = default;
constexpr ~SlotMapKey() = default;
[[nodiscard]]
constexpr bool Valid() const
{
return GetVersion() != INVALID_VERSION;
}
/**
* Get the version
*
* \return The version
*/
[[nodiscard]]
constexpr VersionType GetVersion() const
{
return static_cast<VersionType>(m_key >> INDEX_BIT_COUNT);
}
/**
* Get the index
*
* \return The index
*/
[[nodiscard]]
constexpr IndexType GetIndex() const
{
return static_cast<IndexType>(m_key & INDEX_MASK);
}
constexpr SlotMapKey& operator=(const SlotMapKey& p_slotKey) = default;
constexpr SlotMapKey& operator=(SlotMapKey&& p_slotKey) = default;
[[nodiscard]]
constexpr bool operator==(const SlotMapKey& p_other) const = default;
private:
/**
* The key => (VERSION | INDEX)
*/
std::uint64_t m_key;
};
/**
* Storage is dense
* Pointer stability not guarantee
* Fast iteration
*/
template<class T, class SLOT_MAP_KEY_TYPE = SlotMapKey<>>
class DenseSlotMap
{
public:
using SlotMapKey = SLOT_MAP_KEY_TYPE;
using iterator = eastl::vector<T>::iterator;
using const_iterator = eastl::vector<T>::const_iterator;
using reverse_iterator = eastl::vector<T>::reverse_iterator;
using const_reverse_iterator = eastl::vector<T>::const_reverse_iterator;
using iterator = typename eastl::vector<T>::iterator;
using const_iterator = typename eastl::vector<T>::const_iterator;
using reverse_iterator = typename eastl::vector<T>::reverse_iterator;
using const_reverse_iterator = typename eastl::vector<T>::const_reverse_iterator;
DenseSlotMap() = default;
DenseSlotMap(const DenseSlotMap&) = default;
DenseSlotMap(DenseSlotMap&&) = default;
~DenseSlotMap() = default;
/**
* Check if the map has the key
*
* \param p_key The key to check
* \return True if it has the key, false otherwise
*/
[[nodiscard]]
bool Has(const SlotMapKey p_key) const
{
@@ -133,46 +108,24 @@ class DenseSlotMap
(p_key.GetIndex() < m_keys.size() && p_key.GetVersion() == m_keys[p_key.GetIndex()].GetVersion());
}
/**
* Get the size of the map
*
* \return The size of the map
*/
[[nodiscard]]
std::size_t Size() const
{
return m_data.size();
}
/**
* Get the capacity of the map
*
* \return The capacity of the map
*/
[[nodiscard]]
std::size_t Capacity() const
{
return m_data.capacity();
}
/**
* Check if the map is empty
*
* \return True if empty, false otherwise
*/
[[nodiscard]]
bool Empty() const
{
return m_data.empty();
}
/**
* Insert to the slotmap
*
* \tparam Templated arguments
* \param p_args The thing to insert, arguments will be forwarded
* \return A key to the inserted element
*/
template<class... ARGS>
[[nodiscard]]
SlotMapKey Insert(ARGS&&... p_args)
@@ -204,11 +157,6 @@ class DenseSlotMap
return SlotMapKey {1, static_cast<SlotMapKey::IndexType>(m_keys.size() - 1)};
}
/**
* Remove an element from the map
*
* \param p_key The element to remove
*/
void Remove(const SlotMapKey p_key)
{
if (!Has(p_key))
@@ -230,12 +178,6 @@ class DenseSlotMap
RecycleKey(p_key);
}
/**
* Reset the slotmap
* Everything will be cleared
* Old keys are not stale
*
*/
void Reset()
{
m_data.clear();
@@ -244,12 +186,6 @@ class DenseSlotMap
m_freeListHead = SlotMapKey::MAX_INDEX;
}
/**
* Clear the slotmap
* Data is lost
* Old keys are stale
*
*/
void Clear()
{
for (const std::size_t keyIndex: m_dataToKey)
@@ -261,24 +197,12 @@ class DenseSlotMap
m_dataToKey.clear();
}
/**
* Get the element at p_slotKey
*
* \param p_slotKey The key to get
* \return Pointer to the element, nullptr if p_slotKey is not in the map
*/
[[nodiscard]]
T* Get(const SlotMapKey p_slotKey)
{
return Has(p_slotKey) ? &m_data[GetDataIndex(p_slotKey)] : nullptr;
}
/**
* Get the element at p_slotKey
*
* \param p_slotKey The key to get
* \return Pointer to the element, nullptr if p_slotKey is not in the map
*/
[[nodiscard]]
const T* Get(const SlotMapKey p_slotKey) const
{
@@ -357,35 +281,22 @@ class DenseSlotMap
return m_data.crend();
}
DenseSlotMap& operator=(const DenseSlotMap&) = default;
DenseSlotMap& operator=(DenseSlotMap&&) = default;
private:
/**
* Does the map have free slots (ie: indices that can be recyled)
*
* \return True if indices can be recycled, false otherwise
*/
[[nodiscard]]
bool HasFreeSlots() const
{
return m_freeListHead != SlotMapKey::MAX_INDEX;
}
/**
* The the index of the data in the contiguous array
*
* \param p_key The key of the element to get the index of
* \return The index of the element in the contiguous array
*/
[[nodiscard]]
std::size_t GetDataIndex(const SlotMapKey p_key) const
{
return m_keys[p_key.GetIndex()].GetIndex();
}
/**
* Recycle a key
*
* \param p_key The key to recycle
*/
void RecycleKey(const SlotMapKey p_key)
{
const typename SlotMapKey::VersionType nextVersion = static_cast<SlotMapKey::VersionType>(p_key.GetVersion() +
@@ -402,17 +313,13 @@ class DenseSlotMap
m_freeListHead = p_key.GetIndex();
}
/**
* The actual storage for our data
*/
private:
eastl::vector<T> m_data;
/**
* The keys containing the indices to our data
*/
eastl::vector<SlotMapKey> m_keys;
/**
* Allows us to match m_data and m_keys.
* dataIndex = m_keys[m_dataToKey[dataIndex]].GetIndex()
*/
eastl::vector<std::size_t> m_dataToKey;
@@ -422,21 +329,15 @@ class DenseSlotMap
std::size_t m_freeListHead = SlotMapKey::MAX_INDEX;
};
/**
* Storage is not dense
* Pointer stability is guarantee
* Slower iteration, fast insert/remove/random access
*/
template<class T, class SLOT_MAP_KEY_TYPE = SlotMapKey<>>
class StableSlotMap
{
public:
private:
static constexpr std::size_t PAGE_SIZE = 64;
private:
struct AlignedSlot
{
alignas(T) eastl::array<std::byte, sizeof(T)> m_bytes;
alignas(T) std::byte m_bytes[sizeof(T)];
};
struct Page
@@ -458,50 +359,25 @@ class StableSlotMap
Page& operator=(const Page& p_page) = delete;
Page& operator=(Page&& p_page) = delete;
/**
* Get the data at the specified offset
*
* \return Pointer to the data
*/
[[nodiscard]]
T* Get(const std::size_t p_offset)
{
ASSERT(UtilsAssertHandler, m_occupied[p_offset], "Requesting a dead data!");
void* const bytes = m_data[p_offset].m_bytes.data();
return std::launder(static_cast<T*>(bytes));
return std::launder(reinterpret_cast<T*>(m_data[p_offset].m_bytes));
}
/**
* Get the data at the specified offset
*
* \return Pointer to the data
*/
[[nodiscard]]
const T* Get(const std::size_t p_offset) const
{
ASSERT(UtilsAssertHandler, m_occupied[p_offset], "Requesting a dead data!");
const void* const bytes = m_data[p_offset].m_bytes.data();
return std::launder(static_cast<const T*>(bytes));
return std::launder(reinterpret_cast<const T*>(m_data[p_offset].m_bytes));
}
/**
* Get the size of the page
*
* \return The size of the page
*/
[[nodiscard]]
std::size_t Size() const
{
return m_occupied.count();
}
/**
* The data
*/
eastl::array<AlignedSlot, PAGE_SIZE> m_data;
/**
* Bitset representing which indices are alive or not
*/
eastl::bitset<PAGE_SIZE> m_occupied;
};
@@ -529,8 +405,7 @@ class StableSlotMap
IteratorImpl() = default;
// Allows an iterator to be implicitly used as a const_iterator.
template<class OTHER_VALUE_TYPE>
requires std::is_same_v<VALUE_TYPE, const OTHER_VALUE_TYPE>
template<class OTHER_VALUE_TYPE, class = std::enable_if_t<std::is_same_v<VALUE_TYPE, const OTHER_VALUE_TYPE>>>
IteratorImpl(const IteratorImpl<OTHER_VALUE_TYPE>& p_other):
m_map(p_other.m_map),
m_pageIt(p_other.m_pageIt),
@@ -687,12 +562,13 @@ class StableSlotMap
using reverse_iterator = eastl::reverse_iterator<iterator>;
using const_reverse_iterator = eastl::reverse_iterator<const_iterator>;
/**
* Check if the map has the key
*
* \param p_key The key to check
* \return True if it has the key, false otherwise
*/
StableSlotMap() = default;
StableSlotMap(const StableSlotMap& p_map) = default;
StableSlotMap(StableSlotMap&& p_map) = default;
~StableSlotMap() = default;
[[nodiscard]]
bool Has(const SlotMapKey p_key) const
{
@@ -700,11 +576,6 @@ class StableSlotMap
(p_key.GetIndex() < m_keys.size() && p_key.GetVersion() == m_keys[p_key.GetIndex()].GetVersion());
}
/**
* Get the size of the map
*
* \return The size of the map
*/
[[nodiscard]]
std::size_t Size() const
{
@@ -717,35 +588,18 @@ class StableSlotMap
});
}
/**
* Get the capacity of the map
*
* \return The capacity of the map
*/
[[nodiscard]]
std::size_t Capacity() const
{
return m_pages.size() * PAGE_SIZE;
}
/**
* Check if the map is empty
*
* \return True if empty, false otherwise
*/
[[nodiscard]]
bool Empty() const
{
return Size() == 0;
}
/**
* Insert to the slotmap
*
* \tparam Templated arguments
* \param p_args The thing to insert, arguments will be forwarded
* \return A key to the inserted element
*/
template<class... ARGS>
[[nodiscard]]
SlotMapKey Insert(ARGS&&... p_args)
@@ -780,17 +634,12 @@ class StableSlotMap
Page* const page = GetPageAt(pageIndex);
const std::size_t offsetInPage = GetDataOffset(key);
new (page->m_data[offsetInPage].m_bytes.data()) T(std::forward<ARGS>(p_args)...);
new (page->m_data[offsetInPage].m_bytes) T(std::forward<ARGS>(p_args)...);
page->m_occupied.set(offsetInPage);
return key;
}
/**
* Remove an element from the map
*
* \param p_key The element to remove
*/
void Remove(const SlotMapKey p_key)
{
if (!Has(p_key))
@@ -806,12 +655,6 @@ class StableSlotMap
RecycleKey(p_key);
}
/**
* Reset the slotmap
* Everything will be cleared
* Old keys are not stale
*
*/
void Reset()
{
for (eastl::unique_ptr<Page>& page: m_pages)
@@ -822,12 +665,6 @@ class StableSlotMap
m_freeListHead = SlotMapKey::MAX_INDEX;
}
/**
* Clear the slotmap
* Data is lost
* Old keys are stale
*
*/
void Clear()
{
std::size_t pageIndex = 0;
@@ -838,7 +675,7 @@ class StableSlotMap
{
std::destroy_at(page->Get(offset));
const std::size_t keyIndex = (pageIndex * PAGE_SIZE) + offset;
const std::size_t keyIndex = pageIndex * PAGE_SIZE + offset;
RecycleKey(SlotMapKey {m_keys[keyIndex].GetVersion(), static_cast<SlotMapKey::IndexType>(keyIndex)});
}
page->m_occupied.reset();
@@ -846,24 +683,12 @@ class StableSlotMap
}
}
/**
* Get the element at p_slotKey
*
* \param p_slotKey The key to get
* \return Pointer to the element, nullptr if p_slotKey is not in the map
*/
[[nodiscard]]
const T* Get(const SlotMapKey p_key) const
{
return Has(p_key) ? GetPageAt(GetPageIndex(p_key))->Get(GetDataOffset(p_key)) : nullptr;
}
/**
* Get the element at p_slotKey
*
* \param p_slotKey The key to get
* \return Pointer to the element, nullptr if p_slotKey is not in the map
*/
[[nodiscard]]
T* Get(const SlotMapKey p_key)
{
@@ -873,9 +698,9 @@ class StableSlotMap
[[nodiscard]]
iterator begin()
{
iterator result {this};
result.SeekFirstFrom(m_pages.begin());
return result;
iterator it {this};
it.SeekFirstFrom(m_pages.begin());
return it;
}
[[nodiscard]]
@@ -887,9 +712,9 @@ class StableSlotMap
[[nodiscard]]
const_iterator begin() const
{
const_iterator result {this};
result.SeekFirstFrom(m_pages.begin());
return result;
const_iterator it {this};
it.SeekFirstFrom(m_pages.begin());
return it;
}
[[nodiscard]]
@@ -946,23 +771,16 @@ class StableSlotMap
return rend();
}
StableSlotMap& operator=(const StableSlotMap& p_map) = default;
StableSlotMap& operator=(StableSlotMap&& p_map) = default;
private:
/**
* Does the map have free slots (ie: indices that can be recyled)
*
* \return True if indices can be recycled, false otherwise
*/
[[nodiscard]]
bool HasFreeSlots() const
{
return m_freeListHead != SlotMapKey::MAX_INDEX;
}
/**
* Recycle a key
*
* \param p_key The key to recycle
*/
void RecycleKey(const SlotMapKey p_key)
{
const typename SlotMapKey::VersionType nextVersion = static_cast<SlotMapKey::VersionType>(p_key.GetVersion() +
@@ -979,61 +797,32 @@ class StableSlotMap
m_freeListHead = p_key.GetIndex();
}
/**
* Given a key, get the index of its page
*
* \param p_key The key
* \return The index of the page
*/
[[nodiscard]]
static std::size_t GetPageIndex(const SlotMapKey p_key)
{
return p_key.GetIndex() / PAGE_SIZE;
}
/**
* Given a key, get the offset of the data inside its page
*
* \param p_key The key
* \return The offset the of data
*/
[[nodiscard]]
static std::size_t GetDataOffset(const SlotMapKey p_key)
{
return p_key.GetIndex() % PAGE_SIZE;
}
/**
* Get the page
*
* \param p_pageIndex The idnex to get
* \return The page
*/
[[nodiscard]]
const Page* GetPageAt(const std::size_t p_pageIndex) const
{
return m_pages[p_pageIndex].get();
}
/**
* Get the page
*
* \param p_pageIndex The idnex to get
* \return The page
*/
[[nodiscard]]
Page* GetPageAt(const std::size_t p_pageIndex)
{
return m_pages[p_pageIndex].get();
}
/**
* The pages
*/
private:
eastl::vector<eastl::unique_ptr<Page>> m_pages;
/**
* The keys containing the indices to our data
*/
eastl::vector<SlotMapKey> m_keys;
/**
@@ -1,31 +0,0 @@
/*********************************************************************
* \file Format.hpp
*
* \author Romain BOULLARD
* \date September 2026
*********************************************************************/
#ifndef BIGFOOT_UTILS_FORMAT_HPP
#define BIGFOOT_UTILS_FORMAT_HPP
#include <EASTL/string.h>
#include <fmt/format.h>
#include <iterator>
namespace Bigfoot
{
/**
* Format a string, like fmt::format, but returning an eastl::string instead of a std::string.
*
* \param p_format The format string
* \param p_args The arguments for the format string
* \return The formatted eastl::string
*/
template<typename... ARGS>
eastl::string Format(fmt::format_string<ARGS...> p_format, ARGS&&... p_args)
{
fmt::memory_buffer buffer;
fmt::format_to(std::back_inserter(buffer), p_format, std::forward<ARGS>(p_args)...);
return eastl::string {buffer.data(), buffer.size()};
}
} // namespace Bigfoot
#endif
@@ -8,167 +8,87 @@
#define BIGFOOT_UTILS_LOG_EASTLFORMATTERS_HPP
#include <Utils/TargetMacros.h>
#ifdef BIGFOOT_NOT_OPTIMIZED
#include <quill/core/Codec.h>
#if defined(BIGFOOT_NOT_OPTIMIZED)
#include <quill/DeferredFormatCodec.h>
#endif
#include <EASTL/string.h>
#include <EASTL/string_view.h>
#include <EASTL/vector.h>
#include <fmt/format.h>
#include <cstddef>
#include <format>
// STRING
template<>
struct fmt::formatter<eastl::string>
struct std::formatter<eastl::string>
{
static constexpr auto parse(format_parse_context& ctx)
constexpr auto parse(std::format_parse_context& ctx)
{
return ctx.begin();
}
static auto format(const eastl::string& p_string, format_context& ctx)
template<typename FormatContext>
auto format(const eastl::string& p_string, FormatContext& ctx) const
{
return fmt::format_to(ctx.out(), "{}", p_string.c_str());
return std::format_to(ctx.out(), "{}", p_string.c_str());
}
};
#ifdef BIGFOOT_NOT_OPTIMIZED
#if defined BIGFOOT_NOT_OPTIMIZED
template<>
struct fmtquill::formatter<eastl::string>
{
static constexpr auto parse(format_parse_context& ctx)
constexpr auto parse(format_parse_context& ctx)
{
return ctx.begin();
}
static auto format(const eastl::string& p_string, format_context& ctx)
auto format(const eastl::string& p_string, format_context& ctx) const
{
return fmtquill::format_to(ctx.out(), "{}", p_string.c_str());
}
};
template<>
struct quill::Codec<eastl::string>: quill::DeferredFormatCodec<eastl::string>
{
};
#endif
// STRING_VIEW
template<>
struct fmt::formatter<eastl::string_view>
struct std::formatter<eastl::string_view>
{
static constexpr auto parse(format_parse_context& ctx)
constexpr auto parse(std::format_parse_context& ctx)
{
return ctx.begin();
}
static auto format(const eastl::string_view& p_stringView, format_context& ctx)
template<typename FormatContext>
auto format(const eastl::string_view& p_stringView, FormatContext& ctx) const
{
return fmt::format_to(ctx.out(), "{}", p_stringView.data());
return std::format_to(ctx.out(), "{}", p_stringView.data());
}
};
#ifdef BIGFOOT_NOT_OPTIMIZED
#if defined BIGFOOT_NOT_OPTIMIZED
template<>
struct fmtquill::formatter<eastl::string_view>
{
static constexpr auto parse(format_parse_context& ctx)
constexpr auto parse(format_parse_context& ctx)
{
return ctx.begin();
}
static auto format(const eastl::string_view& p_stringView, format_context& ctx)
auto format(const eastl::string_view& p_stringView, format_context& ctx) const
{
return fmtquill::format_to(ctx.out(), "{}", p_stringView.data());
}
};
namespace Bigfoot::Codec
{
template<class STRING>
struct string_view: quill::Codec<std::string_view>
{
using Base = quill::Codec<std::string_view>;
[[nodiscard]]
static size_t compute_encoded_size(quill::detail::SizeCacheVector& p_cache, const STRING& p_arg) noexcept
{
return Base::compute_encoded_size(p_cache, std::string_view {p_arg.data(), p_arg.size()});
}
static void encode(std::byte*& p_buffer,
const quill::detail::SizeCacheVector& p_cache,
std::uint32_t& p_cacheIndex,
const STRING& p_arg) noexcept
{
Base::encode(p_buffer, p_cache, p_cacheIndex, std::string_view {p_arg.data(), p_arg.size()});
}
};
} // namespace Bigfoot::Codec
template<>
struct quill::Codec<eastl::string>: Bigfoot::Codec::string_view<eastl::string>
{
};
template<>
struct quill::Codec<eastl::string_view>: Bigfoot::Codec::string_view<eastl::string_view>
{
};
#endif
// VECTOR
template<class T>
struct fmt::formatter<eastl::vector<T>>
{
constexpr auto parse(format_parse_context& ctx)
{
return ctx.begin();
}
auto format(const eastl::vector<T>& p_vector, format_context& ctx) const
{
auto out = fmt::format_to(ctx.out(), "[");
for (std::size_t i = 0; i < p_vector.size(); ++i)
{
if (i > 0)
{
out = fmt::format_to(out, ", ");
}
out = fmt::format_to(out, "{}", p_vector[i]);
}
return fmt::format_to(out, "]");
}
};
#ifdef BIGFOOT_NOT_OPTIMIZED
#include <quill/DeferredFormatCodec.h>
template<class T>
struct fmtquill::formatter<eastl::vector<T>>
{
constexpr auto parse(format_parse_context& ctx)
{
return ctx.begin();
}
auto format(const eastl::vector<T>& p_vector, format_context& ctx) const
{
auto out = fmtquill::format_to(ctx.out(), "[");
for (std::size_t i = 0; i < p_vector.size(); ++i)
{
if (i > 0)
{
out = fmtquill::format_to(out, ", ");
}
out = fmtquill::format_to(out, "{}", p_vector[i]);
}
return fmtquill::format_to(out, "]");
}
};
template<class T>
struct quill::Codec<eastl::vector<T>>: quill::DeferredFormatCodec<eastl::vector<T>>
struct quill::Codec<eastl::string_view>: quill::DeferredFormatCodec<eastl::string_view>
{
};
#endif
@@ -2,8 +2,7 @@ namespace Bigfoot.Flat;
enum LogSinkType: byte
{
Console,
File
Console
}
enum LogLevel: byte
@@ -8,11 +8,10 @@
#define BIGFOOT_UTILS_LOG_LOG_HPP
#include <Utils/Log/EASTLFormatters.hpp>
#ifdef BIGFOOT_NOT_OPTIMIZED
#if defined BIGFOOT_NOT_OPTIMIZED
#include <Utils/Log/Log_generated.hpp>
#include <EASTL/array.h>
#include <EASTL/string_view.h>
#ifdef BIGFOOT_WINDOWS
#pragma warning(disable: 4702)
@@ -22,8 +21,7 @@
#include <quill/LogMacros.h>
#include <quill/Logger.h>
#include <quill/sinks/ConsoleSink.h>
#include <quill/sinks/FileSink.h>
#ifdef BIGFOOT_WINDOWS
#if defined BIGFOOT_WINDOWS
#pragma warning(default: 4702)
#endif
@@ -38,11 +36,6 @@ class Log
Flat::LogLevel m_level;
};
/**
* The path of the file the File sink writes to
*/
static constexpr eastl::string_view FILE_SINK_PATH = "Bigfoot.log";
Log();
Log(const Log& p_logger) = delete;
@@ -102,7 +95,7 @@ class Log
/*
* The sinks
*/
eastl::array<std::shared_ptr<quill::Sink>, 2> m_sinks;
eastl::array<std::shared_ptr<quill::Sink>, 1> m_sinks;
};
} // namespace Bigfoot
#endif
@@ -9,16 +9,7 @@
#include <Utils/Log/Log.hpp>
#include <Utils/Singleton.hpp>
#ifdef BIGFOOT_NOT_OPTIMIZED
#define BIGFOOT_FLUSH_LOG \
do \
{ \
if (Bigfoot::Singleton<Bigfoot::Log>::HasInstance()) \
{ \
Bigfoot::Singleton<Bigfoot::Log>::GetInstance().Flush(); \
} \
} while (false)
#if defined BIGFOOT_NOT_OPTIMIZED
#define BIGFOOT_LOG_DEBUG(loggerName, fmt, ...) \
do \
{ \
@@ -110,67 +101,4 @@
#define BIGFOOT_LOG_ERROR(loggerName, fmt, ...)
#define BIGFOOT_LOG_FATAL(loggerName, fmt, ...)
#endif
#ifdef BIGFOOT_NOT_OPTIMIZED
#define BIGFOOT_LOG_DEBUG_IF(condition, loggerName, fmt, ...) \
do \
{ \
if (condition) \
{ \
BIGFOOT_LOG_DEBUG(loggerName, fmt __VA_OPT__(, ) __VA_ARGS__); \
} \
} while (0)
#define BIGFOOT_LOG_TRACE_IF(condition, loggerName, fmt, ...) \
do \
{ \
if (condition) \
{ \
BIGFOOT_LOG_TRACE(loggerName, fmt __VA_OPT__(, ) __VA_ARGS__); \
} \
} while (0)
#define BIGFOOT_LOG_INFO_IF(condition, loggerName, fmt, ...) \
do \
{ \
if (condition) \
{ \
BIGFOOT_LOG_INFO(loggerName, fmt __VA_OPT__(, ) __VA_ARGS__); \
} \
} while (0)
#define BIGFOOT_LOG_WARN_IF(condition, loggerName, fmt, ...) \
do \
{ \
if (condition) \
{ \
BIGFOOT_LOG_WARN(loggerName, fmt __VA_OPT__(, ) __VA_ARGS__); \
} \
} while (0)
#define BIGFOOT_LOG_ERROR_IF(condition, loggerName, fmt, ...) \
do \
{ \
if (condition) \
{ \
BIGFOOT_LOG_ERROR(loggerName, fmt __VA_OPT__(, ) __VA_ARGS__); \
} \
} while (0)
#define BIGFOOT_LOG_FATAL_IF(condition, loggerName, fmt, ...) \
do \
{ \
if (condition) \
{ \
BIGFOOT_LOG_FATAL(loggerName, fmt __VA_OPT__(, ) __VA_ARGS__); \
} \
} while (0)
#else
#define BIGFOOT_LOG_DEBUG_IF(condition, loggerName, fmt, ...)
#define BIGFOOT_LOG_TRACE_IF(condition, loggerName, fmt, ...)
#define BIGFOOT_LOG_INFO_IF(condition, loggerName, fmt, ...)
#define BIGFOOT_LOG_WARN_IF(condition, loggerName, fmt, ...)
#define BIGFOOT_LOG_ERROR_IF(condition, loggerName, fmt, ...)
#define BIGFOOT_LOG_FATAL_IF(condition, loggerName, fmt, ...)
#endif
#endif
@@ -29,30 +29,27 @@ namespace Flat {
enum class LogSinkType : int8_t {
Console = 0,
File = 1,
MIN = Console,
MAX = File
MAX = Console
};
inline const LogSinkType (&EnumValuesLogSinkType())[2] {
inline const LogSinkType (&EnumValuesLogSinkType())[1] {
static const LogSinkType values[] = {
LogSinkType::Console,
LogSinkType::File
LogSinkType::Console
};
return values;
}
inline const char * const *EnumNamesLogSinkType() {
static const char * const names[3] = {
static const char * const names[2] = {
"Console",
"File",
nullptr
};
return names;
}
inline const char *EnumNameLogSinkType(LogSinkType e) {
if (::flatbuffers::IsOutRange(e, LogSinkType::Console, LogSinkType::File)) return "";
if (::flatbuffers::IsOutRange(e, LogSinkType::Console, LogSinkType::Console)) return "";
const size_t index = static_cast<size_t>(e);
return EnumNamesLogSinkType()[index];
}
@@ -101,14 +98,13 @@ inline const char *EnumNameLogLevel(LogLevel e) {
inline const ::flatbuffers::TypeTable *LogSinkTypeTypeTable() {
static const ::flatbuffers::TypeCode type_codes[] = {
{ ::flatbuffers::ET_CHAR, 0, 0 },
{ ::flatbuffers::ET_CHAR, 0, 0 }
};
static const ::flatbuffers::TypeFunction type_refs[] = {
Bigfoot::Flat::LogSinkTypeTypeTable
};
static const ::flatbuffers::TypeTable tt = {
::flatbuffers::ST_ENUM, 2, type_codes, type_refs, nullptr, nullptr, nullptr
::flatbuffers::ST_ENUM, 1, type_codes, type_refs, nullptr, nullptr, nullptr
};
return &tt;
}
@@ -15,7 +15,7 @@ namespace Bigfoot
/*
* Logger
*/
inline Log::LoggerInfo @LOGGER_NAME@ { .m_name = "@LOGGER_NAME@", .m_level = Flat::LogLevel::Trace};
inline Log::LoggerInfo @LOGGER_NAME@ {"@LOGGER_NAME@", Flat::LogLevel::Trace};
} // namespace Bigfoot
#endif
#endif

Some files were not shown because too many files have changed in this diff Show More