Closing a checker process checked for its exit once and then slept 10 ms
before checking again, although the process exits a fraction of a
millisecond after its input closes. The checks now start 0.1 ms apart and
double, so the process is reaped as soon as it exits. The total wait before
the kill stays at about 50 ms (51.1 ms).
With SCRIPTC_CHECKER_SERVER=1 on Linux and macOS, the native compiler's
checker connections go to a shared `tsc-ts --api --listen` server, which
keeps the files it parsed for the next build. When the variable is unset,
nothing changes.
checkerServerName names the server by a digest of the tsc-ts executable's
identity, the working directory, the file system callbacks, the timing
flag and the TSC_TS_* environment. scriptc_ts7_attach takes the server's
lock file with a non-blocking flock, connects to its socket in this
user's private server directory ($XDG_RUNTIME_DIR/scriptc or
/tmp/scriptc-<uid>), and starts the server when none answers: in its own
process group, with its standard streams on /dev/null and no other
descriptor of the build, with --idle-exit 300 and TSC_TS_LISTEN_MEMORY=2048
unless the environment sets it. A build that finds the server in use,
cannot start or reach it, or gets no answer to a ping checks with its own
tsc-ts process, as without the variable; so does a tsc-ts without
--listen, which exits as it starts. The syntax-only source parser, which
reads no libraries, always uses its own process and leaves the server to
the program's checker.
scriptc_ts7_json_value_end searched for the next quote again after each
backslash escape in a string, so a string with many escapes was scanned
once per escape. The quote it found now stays the candidate end until an
escape skips it, and each byte of a string is scanned once by each
search.
A test scans strings of 2,000,000 newline, backslash and quote escapes,
alone and inside an object, and compares the ends with jsonValueEnd's;
before the change it does not finish within 60 s.
The committed profile was collected before about 200 commits that changed
12,800 lines of the compiler. Functions whose control flow changed since are
optimized as without a profile, so its gain had mostly gone: a cold tsc-ts
speed build was 2.7% faster with it than with no profile, against 5% to 6%
with a fresh one.
Regenerated with scripts/refresh-cli-profile.mjs on this commit.
The LLVM emitter's constructor runs many analyses that each walk every
function body to collect facts from a few node kinds. IrNodeIndex
(ir/node-index.ts) walks the bodies once, before those analyses, and
records each body's nodes in everyStmtList's order with a kind code per
node and the kinds present per function. An analysis names the kinds it
reads and receives exactly those nodes, in the same order, without
recursing through the tree; a function the index does not hold (a
rewritten copy) is walked as before. The index also lists the distinct
type objects of the module, which the boxed-union and runtime-feature
scans read instead of walking every expression's type tree.
These read the index: walk borrows, appended array literals, constant
callbacks and the rebinding census, int32 slots (dynamic field stores,
call facts, the box census, candidate writes and the range walks),
initializer bindings, stack observations, the may-throw call-graph
census, published types and construction facts, final fields, stable
globals, reference writes (private arrays, call targets, definitions,
fresh constructors), boxed unions and runtime features. Analyses whose
walks track their position (statement lists, nesting, ordered
dataflow) keep their own walks.
Callers outside the emitter pass no index and walk as before. The
emitted LLVM IR is unchanged.
packages/compiler/native/cli.profdata.gz is refreshed with
scripts/refresh-cli-profile.mjs on this change, so the native
compiler's optimization profile covers the rewritten analyses.
host.c defines _POSIX_C_SOURCE, under which Darwin's <sys/file.h> hides
flock and LOCK_EX/LOCK_UN, so the native compiler build failed on macOS
with an implicit declaration error. Define _DARWIN_C_SOURCE on Apple
targets before the system headers. Linux preprocessing is unchanged.
The cycle collector widens the growth fraction that schedules full passes
up to 16 times the heap left by the last pass while passes find little.
Lowering a large program leaves hundreds of megabytes of cyclic garbage,
and whether a scheduled pass reclaims it before the build's peak depends
on where the growth rule fires: small changes to the compiler moved the
peak of a tsc-ts build between 1.66 and 2.05 GB.
The native compiler now sets the growth cap to 2 at startup, as
SCR_CYCLE_GROWTH_CAP=2 does, so a full pass runs once the heap has grown
to three times its size after the last one. scr_cyc_set_growth_cap sets
the cap for a program from its own code; the environment variable still
takes precedence, and child processes are not affected.
On hosts that can fork, the native compiler now emits a large module's
function bodies on worker processes (SCRIPTC_EMIT_WORKERS, 4 by default).
The function list splits into contiguous ranges, each emitted by a forked
child of the emitter, and the ranges merge in function order into the text
one emitter gives. A range whose worker fails is emitted in the parent.
Modules under 2,000 functions, Node and Windows keep the serial path.
A range's emitter records an ordered journal (emit-journal.ts):
- every helper it numbers from a counter (the dyn conversions and
matchers, message listeners and readers, JSON writers, publication
walkers, stream adapters and resolve thunks), recorded when its number
is taken, with the entry point and arguments that create it;
- the first request of each key of a module-level table (declarations,
literals, C strings, unit and absent instances, retype guards, regex and
template-strings instances, map descriptors, alias tags) made outside a
helper's definition;
- unkeyed sites (inline caches, speculative loop sites) taken outside a
definition. A speculative loop site's global is now `@sc_spec_bails_N`,
named like the other numbered sites.
The merging emitter replays each range's journal in order. Replay calls
the same entry points, so every helper gets the number and definition the
serial emission gives it, and table entries land in the serial order. The
range's text is then renamed to the merged numbers in one pass. Sets and
flags the ranges set (await callees, function values, RC helper requests)
merge after the replay. LlvmTargetOptions.functionChunks runs the same
merge in one process, for tests.
The fork, exit, wait and lock primitives come from the native host
(host.c, process-fork.ts); the native compiler passes them to the emitter
as LlvmTargetOptions.forkPrimitives.
The frontend's syntax trees and program state refer to each other, so
reference counting frees almost none of them when a build disposes the
frontend. Whether a cycle pass reclaimed them before the build ended
depended on where the collector's schedule happened to be. Passes during
the frontend found its data live and widened the growth fractions, so the
next pass waited until the heap had grown well past its size at disposal,
and the dead frontend and the emitter's garbage stayed until the build
ended.
The native compiler now runs scr_collect_cycles_phase right after it
disposes the frontend, through a host function. The runtime entry point
sweeps, then restarts the growth fractions and the death rule's backoff
from the heap the sweep leaves, as after a productive pass. Under Node
the hook is unset.
* feat: ship the self-hosted native scriptc CLI
- Run the shared compiler and CLI natively, including library builds and compile-time evaluation.
- Install native platform packages and ship relocatable standalone distributions.
- Preserve program semantics and reuse validated build artifacts for fast rebuilds.
* fix: complete native CLI integration and validation
- Preserve library diagnostics and run recursive compiler work on the main stack.
- Restore packaged helper permissions and exercise native npm and Wasm library builds.
- Recover interrupted Sandbox status probes within the original command deadline.
- Launch and manage TypeScript server processes through a portable native FFI boundary.
- Share the production session API between Node and statically compiled clients.
- Preserve Unicode command-line arguments and release server resources on every exit path.
Chris Tate