mirror of
https://github.com/MG1937/ASC.git
synced 2026-10-02 08:34:55 +08:00
Memory: record that the profile is at its floor and what was left on the table
The peak is the resident result set (the sorted, deterministic output contract requires holding it), plus workers x in-flight DEX images, plus allocator slack. Everything cheaper was measured and rejected: the sort is proportional to the result (0.01 ms for 53 rows, 7.1 ms for 170,923, 8.9 ms for 567,192 classes), file pages are reclaimed by the kernel on its own, payload pre-reservation and MADV_DONTNEED both showed no gain, and the allocator already recycles the per-entry buffers. Three options were assessed and deliberately not taken, recorded with their numbers: piping through an external sort is worse (2.3x slower at LC_ALL=C on a 58 MB index and 14.5x slower under a UTF-8 locale, and its order depends on the environment); a byte-budgeted scheduler would save an estimated 50-75 MiB at eight threads for some throughput; and true streaming output behind --no-sort (a bounded reorder queue, still deterministic) is the only structural lever - classes 230 -> ~60-90 MiB, findrefs 375 -> ~330 MiB - at the cost of central-directory order.
This commit is contained in:
+12
-3
@@ -242,9 +242,18 @@ libdeflate 的一次性 API 在输出缓冲不足时**不发布已写出的字
|
||||
的高水位特性,不是可回收的内存泄漏 ✓。
|
||||
|
||||
**下沉空间有多大?** 峰值 ≈ 结果集(170,923 行 / 567,192 个类,受"排序 + 确定性输出"契约约束,
|
||||
必须全部驻留 ✓)+ worker 数 × 在飞 DEX。所以只有两条路:`--threads`(现成的旋钮 ✓),或者改输出契约
|
||||
(流式输出 / 外部归并排序 ✗,工程量与语义代价都大),以及给"在飞字节数"上限(信号量 ✓ 预计 8 线程下
|
||||
省 ~50–75 MiB、吞吐略降 ✗,尚未做 ✓)。
|
||||
必须全部驻留 ✓)+ worker 数 × 在飞 DEX(每个 ≤ ~11 MiB ✓)+ 分配器余量。**到这里基本就是地板** ✓:
|
||||
自适应排序不是瓶颈(53 行的查询只花 0.01 ms,17 万行 7.1 ms,567k 类 8.9 ms —— 成本与结果规模成正比 ✓),
|
||||
文件页由内核回收 ✓,分配器已回收复用(试过容量预留与 `madvise`,都无可测收益 ✓)。
|
||||
|
||||
实测中曾评估、**未采纳**的三个点,供将来判断:
|
||||
- **外部 `| sort`** 不是替代品:`classes` 58 MB 时 `LC_ALL=C sort` 慢 2.3×(默认 locale 慢 **14.5×** ✗),
|
||||
且顺序受运行环境影响 ✗(`LC_ALL=C` 才等于 rasc 的字节序 ✓);
|
||||
- **在飞字节预算**(信号量限制同时解压的 DEX 总量)✓ 预计 8 线程下省 ~50–75 MiB、吞吐略降 ✗;
|
||||
- **流式输出**(`--no-sort` + 按条目序号的有界重排队列 ✓ 顺序仍是确定的 ✓)是唯一的结构性杠杆:
|
||||
`classes` 峰值 230 → ~60–90 MiB ✓(省掉结果集,仅留已去重集合)、`findrefs` 375 → ~330 MiB ✓,
|
||||
代价是输出顺序改为中央目录序(依赖打包布局 ✗)与 ~100–150 行实现 ✓。默认保持有序输出,不做。
|
||||
|
||||
|
||||
## 复现
|
||||
|
||||
|
||||
Reference in New Issue
Block a user