Skip to content

Codegen: SceneSerializer blocks for complex-field components (#380 follow-up) #451

Description

@drsnuggles8

Follow-up to #380 / #450. The fourth slice (#450) generates SceneSerializer.cpp serialize/deserialize blocks for the all-trivial components (every member a primitive / glm::vec* / std::string). The harder cases were deliberately left hand-written and are tracked here.

Shipped

  • First slice — AssetHandle / UUID fields (feat(codegen): SceneSerializer blocks for AssetHandle/UUID-field components (#451) #467): the codegen now classifies AssetHandle/UUID (a u64 wrapper) as a trivial-ish field type and round-trips it (static_cast<u64> on write, .as<u64> + the implicit UUID(u64) ctor on read — format-identical to the hand-written handle blocks). A component whose only non-primitive fields are asset handles / entity-reference UUIDs is now fully generated; UIWorldAnchorComponent migrated off its hand-written block. Four components that newly classify as trivial are kept hand-written via kComponentsCustomSerialize: DialogueComponent / IKTargetComponent / PerceptionComponent (Sanitize/clamp deserialize the round-trip would drop) and IDComponent (entity identity — serialized as the top-level Entity: line; auto-gen would emit a bogus sub-map + double-AddComponent on load, and neither coverage test catches that, so it's a hard exclusion).
  • Second slice — enum / enum class fields (feat(codegen): SceneSerializer enum-field support (#451 slice) #477): the classifier now recognises an enum field by matching the member type's leaf name against a scan of every enum/enum class definition under OloEngine/src (CollectEnumTypes in main.cpp) and round-trips it as an int (static_cast<int> on write, .as<int> on read, cast back to the member's own type via static_cast<decltype(comp.member)> — so a nested enum like AnimationStateComponent::State needs no qualified spelling). Regenerating flipped several components to all-trivial that then needed kComponentsCustomSerialize exclusions for omitted runtime fields / clamps / non-m_ keys: Rigidbody3DComponent (enum keyed BodyType, runtime m_RuntimeBodyToken + omitted fields), StreamingVolumeComponent (runtime IsLoaded + radius clamps), FogVolumeComponent (Shape/Priority clamps), UIButtonComponent (runtime m_State), UISliderComponent (runtime m_IsDragging).
  • Fourth slice — per-field OLO_SERIALIZE(Skip) (codegen: OLO_SERIALIZE(Skip) field annotation for SceneSerializer (#451) #500): a new marker macro (Scene/ComponentReflection.h, alongside OLO_PROPERTY) that drops a member from the generated (de)serialize and stops it marking the component non-trivial — so an otherwise-all-trivial component whose only obstacle is a runtime-only field is now fully generated (the field reloads at its ctor default) instead of a whole-component kComponentsCustomSerialize exclusion. The field parser peels a leading OLO_SERIALIZE(...) (Skip/Skip = true drops the field; Skip = false or a future Clamp/Min/Max peels the marker and serializes normally); it composes with OLO_PROPERTY on the same field (e.g. a runtime field still script-exposed), and the OLO_PROPERTY scanner skips an OLO_SERIALIZE( line so order doesn't matter. Migrated UIButtonComponent (m_State) and UISliderComponent (m_IsDragging) off kComponentsCustomSerialize with byte-identical on-disk format. This is the "field-level annotation → per-field control" direction below (the Skip half; Clamp/Min/Max remain). Motivation: the vector<struct> / nested-struct classes have no clean beneficiary today — every candidate must stay hand-written for an independent reason (private members ColliderMaterial, deserialize sanitize NavMeshBoundsComponent, runtime-only DialogueStateComponent, flattened format LODGroup) — so widening the classifier there would emit zero live blocks; the annotation direction does have beneficiaries.
  • Third slice — glm-math / small-int / std::vector<primitive> fields (feat(codegen): SceneSerializer glm-math / small-int / std::vector<primitive> support (#451 slice) #482): the classifier now also handles glm::quat/mat3/mat4/ivec2/ivec3/ivec4 (via new Encode/Decode + Emitter<< helpers in Core/YAMLConverters.h; float components finiteness-validated), the small ints u8/u16/i8/i16 (widened to u32/i32 on emit — yaml-cpp's convert<unsigned char>::encode writes a raw char, not a number — and read via .as<decltype(member)>), and std::vector<E> where E is any trivial type (round-tripped as a YAML sequence). The field parser now also tracks access level so a non-public member marks the component non-trivial (fail-safe: stays hand-written, no compile error / silent drop — e.g. TransformComponent's private Rotation/euler-sync now needs no exclusion). Migrated off hand-written blocks: InstancePortalComponent (u8), QuestGiverComponent (vector<string>), RelationshipComponent (UUID + vector<UUID>). Three components newly flipped to all-trivial but kept hand-written via kComponentsCustomSerialize: LightProbeVolumeComponent (runtime m_Dirty/m_ShowDebugProbes + clamps), NavAgentComponent (runtime path/target state), PhysicsJoint3DComponent (runtime m_RuntimeConstraintToken + dozens of clamps).
  • Fifth slice — Ref<T> asset-handle fields (this PR): the classifier now recognises a Ref<T> member as serializer-trivial when T transitively derives from Asset (CollectAssetTypes in main.cpp, walking class X : public Y inheritance chains — one hop of indirection matters, since Material/Model/EnvironmentMap derive from the intermediate RendererResource : public Asset rather than Asset directly). Round-trips as a "<Key>Handle" u64 written only when the Ref is non-null and actually asset-manager-registered (GetHandle() != 0), resolved back via AssetManager::GetAsset<T> on read — format-identical to the hand-written MeshComponent::m_MeshSource -> "MeshSourceHandle" idiom. Regenerating flipped 16 components to all-trivial; every one stayed hand-written via kComponentsCustomSerialize, for one of three reasons (see the exclusion set's comment in main.cpp for the full per-component breakdown): on-disk FORMAT INCOMPATIBILITY (DecalComponent, SpriteRendererComponent, TextComponent, UIImageComponent, UIPanelComponent, UITextComponent, UIInputFieldComponent, UIDropdownComponent persist textures/fonts by file PATH, not handle), DROPPED CUSTOM LOAD LOGIC (ModelComponent Reload(), MeshComponent primitive-range validation + mesh-from-primitive fallback, SubmeshComponent's m_Mesh reconstructed from its parent at runtime, ProceduralSkyComponent/StarNestSkyComponent/ReflectionProbeComponent positivity/range guards), or RUNTIME-ONLY FIELDS auto-gen would newly persist (CinematicComponent's playback state, ReflectionProbeComponent's baked cubemap). One component migrated cleanly: EnvironmentMapComponent (its Ref<EnvironmentMap> cache is always a no-op — never asset-manager-registered — so the round-trip is behaviorally identical to the hand-written block; bonus fix: m_EnvironmentMapAsset, an AssetHandle SaveGame already persists, was being silently dropped from scene YAML and is now round-tripped too).

Out of scope of #450 (still hand-written in SceneSerializer.cpp)

A component stays hand-written today if it has any non-trivial field, or is in kComponentsCustomSerialize. The field types / patterns the generator does not yet handle:

  • AssetHandle / UUID fields — serialized as static_cast<u64>(...), deserialized with re-resolution. Pervasive (materials, meshes, colliders, dialogue, streaming, …).Done in feat(codegen): SceneSerializer blocks for AssetHandle/UUID-field components (#451) #467 (see Shipped above).
  • Ref<T> runtime handles (textures, meshes, fonts, environment maps, players) — usually paired with an AssetHandle that is the persisted form; the Ref is rebuilt on load.Done (see Shipped above).
  • enums — currently static_cast<Enum>(node.as<int>(...)); the generator classifies any enum field as non-trivial. (Needs an enum-detection or annotation mechanism.)Done in feat(codegen): SceneSerializer enum-field support (#451 slice) #477 (see Shipped above).
  • std::vector of a non-trivial element / std::unordered_map / std::unordered_setstd::vector<primitive> is now done in feat(codegen): SceneSerializer glm-math / small-int / std::vector<primitive> support (#451 slice) #482 (sequence (de)serialization for vectors of scalars / small-ints / glm-math / strings / handles / enums); what remains is std::vector<struct> (inventory, quests, abilities, particles, terrain layers, off-mesh links, …) and map/set, which need recursive element serialization.
  • nested structsSceneCamera (setter-only API), ColliderMaterial, LODGroup, Inventory, BTBlackboard, … (glm::quat/mat/ivecdone in feat(codegen): SceneSerializer glm-math / small-int / std::vector<primitive> support (#451 slice) #482).
  • trivial-but-custom components in kComponentsCustomSerializeBuoyancyComponent, NoiseAnimationComponent (vec3 range clamps)Done — vec3-Clamp slice (this PR): OLO_SERIALIZE(Clamp, Min=…, Max=…) now also accepts glm::vec3 fields (clamped per-component via glm::clamp/glm::max/glm::min, mirroring the hand-written SanitizeVec3Clamped idiom), and both components migrated fully off kComponentsCustomSerialize — see the code comment on kComponentsCustomSerialize in tools/OloHeaderTool/main.cpp for detail. Other trivial-but-custom components remain (range clamps / Sanitize* / helper delegation), e.g. SphereAreaLightComponent, VehicleComponent (runtime field omitted), ScriptComponent (external C# field map), TagComponent/PhaseComponent/UIResolvedRectComponent (identity / runtime-only), plus the four added by the feat(codegen): SceneSerializer blocks for AssetHandle/UUID-field components (#451) #467 slice — DialogueComponent/IKTargetComponent/PerceptionComponent (Sanitize/clamp) and IDComponent (entity identity), plus the sixteen added by the Ref<T> slice (see Shipped above — format-incompatible path-based asset resolution, dropped custom load logic, or runtime-only fields). Each would need an opt-in for codegen with custom validation/skip hooks.

Possible directions

  • Field-level annotation (OLO_SERIALIZE(Skip=..., Clamp=..., Min=..., Max=...)) so the generator can emit validation and skip runtime fields — turning kComponentsCustomSerialize from "all-or-nothing per component" into "per-field control". Skip done in codegen: OLO_SERIALIZE(Skip) field annotation for SceneSerializer (#451) #500; scalar Clamp/Min/Max done in Add clamping functionality to component serialization and deserializa… #536; vec3 Clampdone — this PR (see Shipped above) — that unblocked BuoyancyComponent/NoiseAnimationComponent. Remaining clamp-excluded components need a cross-field invariant beyond a per-field range clamp (LightProbeVolumeComponent, NavMeshBoundsComponent, StreamingVolumeComponent, PhysicsJoint3DComponent). Also a candidate for migrating some of the Ref<T>-slice's runtime-only exclusions (CinematicComponent, ReflectionProbeComponent) off kComponentsCustomSerialize via OLO_SERIALIZE(Skip) on their runtime fields.
  • Enum detection (scan enum class definitions) or an explicit enum-field marker.Done in feat(codegen): SceneSerializer enum-field support (#451 slice) #477 (CollectEnumTypes leaf-name scan).
  • A registry of Encode/Decode helpers for known nested structs / Ref<T>+handle pairs, keyed by type name.Ref<T> half done (see Shipped above); nested-struct helpers remain (see "nested structs" above).
  • A follow-up slice could migrate the 8 format-incompatible Ref<T>-slice exclusions (DecalComponent, SpriteRendererComponent, TextComponent, UIImageComponent, UIPanelComponent, UITextComponent, UIInputFieldComponent, UIDropdownComponent) off kComponentsCustomSerialize by first migrating their on-disk format from a "*Path" string key to a handle-based key — a deliberate, separate on-disk-format-change slice, not a codegen-only one.

Also tracked separately

SaveGame SaveGameComponentSerializer::Serialize() / RegisterAll() body codegen (the lists are already generated by #405; the per-component bodies are still hand-written) — noted in #380.

Score

capability: 1
craft: 1
stability: 2
decay: 1
effort: 3
confidence: 0.5
learning: 3
fun: 1
kano: table-stakes
blocked_by: []
blocks: []

Rated per issue-scoring · score = confidence × (capability + craft + stability + decay) / effort, derived by the picker.

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureNew feature or requesttoolingMCP / dev-tooling / codegen — exempt from feature freeze

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions