# Code generation Generated trees are marked `linguist-generated` / `linguist-vendored` in `.gitattributes` and are **committed**. A fresh checkout therefore builds with no Python at all — the dev build scripts run codegen when `python3` is available and fall back to the committed files when it isn't. Every generator reads C++/CUDA sources. **None reads Python**, and none should be added: the training config used to be generated from the Python dataclasses and is now hand-written in `src/config/TrainConfig.h`. Generated locations: ``` src/generated/ device-math headers from Slang src/backend/api/ backend module forwarders src/instantiations/ kernel instantiation TUs ``` All generators run from the **repo root**: ```bash python3 tools/codegen/generate_headers.py python3 tools/codegen/generate_kernel_instantiation.py python3 tools/codegen/generate_backend_api.py # generate_vulkan_stubs.py takes arguments; see below ``` The one exception is `src/shaders/build.bash`, which runs from `src/` and needs `slangc` on PATH. **The dev build scripts do NOT run it.** Editing anything under `src/shaders/` and skipping it leaves the two backends computing different things: Vulkan compiles the Slang, CUDA compiles the committed `src/generated/*.cuh`. --- ## `generate_headers.py` — `.cu` → `.cuh` declarations Scans the `.cu` files listed in `HEADER_SOURCES` for functions preceded by the marker ```cpp /*[AutoHeaderGeneratorExport]*/ void my_kernel_launcher(...) { ... } ``` and rewrites the declaration section of the matching `.cuh`. **Invariants:** 1. Each `.cuh` has a splitter line: ``` /* == AUTO HEADER GENERATOR - DO NOT EDIT THIS LINE OR ANYTHING BELOW THIS LINE == */ ``` Everything **above** it is hand-written (includes, types, preamble) and preserved. Everything **below** is regenerated. Never edit below it. 2. `#if 0` blocks are stripped before scanning, so a disabled function is not exported. 3. **Sources are listed explicitly.** `HEADER_SOURCES` in the script maps each header stem to the ordered list of `.cu` files that declare into it. One header may be fed by many sources, and a source file's name is therefore free — name it after what it does. A listed file that does not exist raises `FileNotFoundError`, so a rename cannot silently drop declarations. 4. **A new kernel family needs an entry in `HEADER_SOURCES`.** 5. Declaration sections must stay CUDA-include-free — they must parse under `-DSS_BACKEND_VULKAN` with no CUDA toolkit present. Use `backend/api/BackendTypes.h` for vector types. ### Splitting a large `.cu` (the sanctioned workflow) Because of invariant 3, splitting is nearly free: ```bash # carve the sections out along the // ==== banners, naming each part after # what it does -- not after the file it came from: # PixelWise.cu -> ImageConvert.cu, ImageColorOps.cu, DepthGeometry.cu, # ImageDistort.cu, ImageWarp.cu, GtDepthNormalWarp.cu, Ppisp.cu # put the shared preamble in Common.cuh and shared device helpers in a # descriptively named header (BilinearSample.cuh), then: # - add the new files to HEADER_SOURCES['PixelWise'] python3 tools/codegen/generate_headers.py # PixelWise.cuh's contents are unchanged ``` No build file changes: both build systems glob the kernel directories through `cmake/sources.txt` (CMake with `CONFIGURE_DEPENDS`, so a new file is picked up without a manual reconfigure). No caller changes: the header keeps its name and its declaration set. Naming rules when splitting: - Name each part after its **function**, not its origin — `ImageWarp.cu`, not `PixelWise.Warp.cu`. - `_kernel.cuh` is **reserved** for device bodies consumed by `generate_kernel_instantiation.py`. A plain shared-helper header gets a descriptive name instead. - `Common.cuh` for the shared preamble (slang namespace includes, host-side view helpers). --- ## `generate_kernel_instantiation.py` — explicit template instantiations Reads kernel declarations out of `src/kernels/**/*.cuh` and emits one small TU per (primitive × camera model × SH degree × …) combination into `src/instantiations/` (currently 111 files). This keeps each nvcc edge small and parallelizable rather than instantiating everything in one TU. --- ## The training config — *not* generated `src/config/TrainConfig.h` is hand-written and is the training config's single source of truth. It holds: - `SS_CONFIG_FIELDS(X)` — one X-macro row per flag, `(type, member, default, section, tier, choices)`. The flag's NAME and help sentence are translated and live in `src/i18n/catalog/TrainFields.h`, keyed by member and resolved at compile time; - `struct TrainConfig` — **expanded from that same table**, so a field cannot exist in one and not the other; - `kTrainPresets` + `train_apply_preset()` — one branch per preset. The table is expanded by the CLI's generic parser and `--help` printer, the GUI's "All Options" editor, `TrainerCore`'s `config.json` dump and the resume path's `config.json` reader. Add a row and the flag appears in all of them. The CLI flag is `member` stringified (`--sh-degree` sets `sh_degree`; `-` and `_` are interchangeable), so a flag name cannot drift from its member. The `config.json` key is the flag, spelled the same way, at the top level: the file is flat. It used to nest under the `group` column, which made a presentational choice part of an on-disk format — moving a flag to another heading moved its key, and the reader (`spirula mesh`, `--resume`) silently fell back to the default when it could not find it. `section` and `tier` are display metadata only and can be reshuffled freely. It is hand-written: nothing generates it, and nothing is generated from it except the struct and the tables `SS_CONFIG_FIELDS` expands into. --- ## `generate_backend_api.py` — backend module headers Emits `src/backend/api/*.h` (`Projection.h`, `Rasterization.h`, `PixelWise.h`, …) as **forwarders** that `#include` the relevant per-kernel `.cuh` headers, grouped by subsystem. Forwarding rather than copying declarations is deliberate: it keeps one definition of every preamble type, so a TU may include both a module header and an individual `.cuh` without ODR violations. --- ## `generate_vulkan_stubs.py` — link-probe for unported kernels ```bash python3 tools/codegen/generate_vulkan_stubs.py ``` Link-probes `csrc_portable`'s objects against `libss_backend_vulkan.a`, collects undefined C++ symbols, finds each function's declaration in the per-kernel `.cuh` headers, and emits one **throwing** definition per function (`backend/vulkan/kernels/TrainingStubs.gen.cpp`). This lets the portable engine link today and lose stubs one module at a time as ports land. Symbols with no header declaration (class methods, templates) are listed in the generated file's header comment for manual stubbing in `TrainingStubsManual.cpp`. Rerun it whenever the engine gains a new kernel call, **or when a port lands** — a ported symbol would otherwise be defined twice. --- ## Non-script generation done by CMake - `viewer.html` → `app_generated/viewer_html.h` (byte array), so `spirula train` and the GUI serve the viewer from a self-contained binary. Regenerated at configure time when the HTML changes. `Viewer.cpp` has a dev override so HTML edits don't require a rebuild. - Slang → SPIR-V blobs → `vk_shaders_embedded.cpp` (`backend/vulkan/shaders/SpirvShaders.cmake` + `backend/vulkan/shaders/spirv_tool.cpp`). SPIR-V is **never committed**; one `slangc` edge per blob, with capability variants (`.atomicadd`, `.noint64`, `.int8`) compiled per entry point.