feat: 完成 OpenNexus 第三阶段核心功能与生产化基础 #45
@@ -635,3 +635,12 @@ Core 的独立数据目录目前不等于已授权 Vault。Python 旧笔记写
|
||||
- 新增 100 MiB Markdown 与 100 MiB 附件测试,分别走首次合并与空远端绑定:仅改末字节即可使旧预览失效,绑定后重复发现不增加事件,UI scan 后的外部修改仍被 discover 识别一次,重开不重复;切换远端 Vault/账号重新绑定时两类文件均进入新队列且原 file_id 保持。测试通过,耗时 10.36 秒,日志 `.build/sync-discovery-stream-large.log`。先前同步回归 17 项通过,日志 `.build/sync-discovery-stream-tests.log`。
|
||||
- 测试补充阶段修正了 Binding 不实现 Debug 时不能使用 unwrap_err 的测试写法,没有为调试输出扩大生产类型的可见字段。上述测试不是真实进程峰值内存测量、10000 文件基准或完整首次绑定故障矩阵;这些验收仍需继续。
|
||||
- 最终 desktop 全目标累计 136 通过、12 ignored(库 121、Host 8、其余集成 7),包含真实 Sync 服务与故障恢复集成,日志 `.build/sync-discovery-stream-full.log`;desktop 全目标 Clippy -D warnings 通过,日志 `.build/sync-discovery-stream-clippy.log`。仍需独立进程峰值内存和完整基准/部署/发布验收,完整生产化目标未完成。
|
||||
|
||||
|
||||
## 增量:独立进程大附件内存观测
|
||||
|
||||
- 新增 Windows 标准库脚本 scripts/measure-sync-memory.py,无额外 Python 依赖。先由 Cargo 构建当前 desktop library test executable,再以 exact 测试名称为每个工作负载单独启动进程;记录可执行文件/Rust 源码/Cargo.lock 摘要、HEAD、编译器与 CPU/OS 信息。每次必须确认恰好 1 项测试通过、0 失败/ignored 且有内存样本。
|
||||
- 通过 GetProcessMemoryInfo 每约 10 ms 读取操作系统累计峰值工作集和峰值提交内存,分别报告,不将二者相加。先用保留 64 MiB bytearray 的独立 Python 进程进行校验,本次峰值工作集约 74.70 MiB、峰值提交约 69.29 MiB,校验通过。15 次工作负载内存查询无错误,最大末次观测至退出间隔约 10.61 ms,原始数据保留该覆盖限制。
|
||||
- 对照(生成式 100 MiB 流)、真实文件写入/恢复、重命名/删除、三种冲突解决、发现/重绑定各运行 3 次,全部通过。四个实际文件工作负载三次最大工作集分别约 12.70、12.63、12.72、12.91 MiB;最大提交内存分别约 2.75、2.85、2.82、2.92 MiB。没有改造前实测基线,不报告下降百分比。
|
||||
- 可审阅报告 [Windows Sync 组件内存观测](benchmarks/sync-memory-windows-5d3c214.md) 及 [原始 JSON](benchmarks/sync-memory-windows-5d3c214.json)。原始逐次测试日志留在 `.build/sync-memory-5d3c214/`。脚本 py_compile 通过,测量器和实际工作负载均已运行;本轮未改变 Rust 生产代码,未重复无关全量测试。
|
||||
- scope 仅为单个 Rust 组件测试进程,不包括 WebView/Core/服务端/系统文件缓存;没有测量规划基准的磁盘/网络/10000 笔记/双账号双 Vault,也不是 S-09 四并发上传服务总 RSS 的证明。报告明确 NOT_ASSESSED,完整基准、部署和生产化目标保持未完成。
|
||||
|
||||
@@ -0,0 +1,393 @@
|
||||
{
|
||||
"schema": 1,
|
||||
"created_at": "2026-09-08T22:26:52.331121+00:00",
|
||||
"scope": "Single Rust component test process; excludes WebView, Core, service and system file cache",
|
||||
"acceptance": "NOT_ASSESSED",
|
||||
"polling_interval_seconds": 0.01,
|
||||
"runs": [
|
||||
{
|
||||
"exit_code": 0,
|
||||
"elapsed_seconds": 0.3068778000306338,
|
||||
"samples": 23,
|
||||
"read_errors": 0,
|
||||
"first_sample_seconds": 0.06473810004536062,
|
||||
"last_sample_seconds": 0.2963416000129655,
|
||||
"unobserved_tail_seconds": 0.010536200017668307,
|
||||
"peaks": {
|
||||
"working_set_bytes": 9543680,
|
||||
"commit_bytes": 2260992,
|
||||
"sampled_private_bytes": 2260992
|
||||
},
|
||||
"log": "stream_verify-1.log",
|
||||
"workload": "stream_verify",
|
||||
"test": "payloads::tests::hundred_mib_verification_is_bounded_and_rejects_growth_or_truncation",
|
||||
"iteration": 1,
|
||||
"passed": true
|
||||
},
|
||||
{
|
||||
"exit_code": 0,
|
||||
"elapsed_seconds": 0.18750940001336858,
|
||||
"samples": 15,
|
||||
"read_errors": 0,
|
||||
"first_sample_seconds": 0.030444499978329986,
|
||||
"last_sample_seconds": 0.17701089999172837,
|
||||
"unobserved_tail_seconds": 0.010498500021640211,
|
||||
"peaks": {
|
||||
"working_set_bytes": 8744960,
|
||||
"commit_bytes": 2260992,
|
||||
"sampled_private_bytes": 2260992
|
||||
},
|
||||
"log": "stream_verify-2.log",
|
||||
"workload": "stream_verify",
|
||||
"test": "payloads::tests::hundred_mib_verification_is_bounded_and_rejects_growth_or_truncation",
|
||||
"iteration": 2,
|
||||
"passed": true
|
||||
},
|
||||
{
|
||||
"exit_code": 0,
|
||||
"elapsed_seconds": 0.21081650000996888,
|
||||
"samples": 17,
|
||||
"read_errors": 0,
|
||||
"first_sample_seconds": 0.03400759998476133,
|
||||
"last_sample_seconds": 0.20036680001066998,
|
||||
"unobserved_tail_seconds": 0.0104496999992989,
|
||||
"peaks": {
|
||||
"working_set_bytes": 8851456,
|
||||
"commit_bytes": 2269184,
|
||||
"sampled_private_bytes": 2269184
|
||||
},
|
||||
"log": "stream_verify-3.log",
|
||||
"workload": "stream_verify",
|
||||
"test": "payloads::tests::hundred_mib_verification_is_bounded_and_rejects_growth_or_truncation",
|
||||
"iteration": 3,
|
||||
"passed": true
|
||||
},
|
||||
{
|
||||
"exit_code": 0,
|
||||
"elapsed_seconds": 1.9924072999856435,
|
||||
"samples": 187,
|
||||
"read_errors": 0,
|
||||
"first_sample_seconds": 0.03317010001046583,
|
||||
"last_sample_seconds": 1.9821281000040472,
|
||||
"unobserved_tail_seconds": 0.010279199981596321,
|
||||
"peaks": {
|
||||
"working_set_bytes": 12771328,
|
||||
"commit_bytes": 2887680,
|
||||
"sampled_private_bytes": 2842624
|
||||
},
|
||||
"log": "write_recovery-1.log",
|
||||
"workload": "write_recovery",
|
||||
"test": "workspace::tests::hundred_mib_spooled_writes_and_journal_recovery_keep_receipts_exactly_once",
|
||||
"iteration": 1,
|
||||
"passed": true
|
||||
},
|
||||
{
|
||||
"exit_code": 0,
|
||||
"elapsed_seconds": 2.0112279999884777,
|
||||
"samples": 189,
|
||||
"read_errors": 0,
|
||||
"first_sample_seconds": 0.030658300034701824,
|
||||
"last_sample_seconds": 2.0006705000414513,
|
||||
"unobserved_tail_seconds": 0.010557499947026372,
|
||||
"peaks": {
|
||||
"working_set_bytes": 13320192,
|
||||
"commit_bytes": 2830336,
|
||||
"sampled_private_bytes": 2830336
|
||||
},
|
||||
"log": "write_recovery-2.log",
|
||||
"workload": "write_recovery",
|
||||
"test": "workspace::tests::hundred_mib_spooled_writes_and_journal_recovery_keep_receipts_exactly_once",
|
||||
"iteration": 2,
|
||||
"passed": true
|
||||
},
|
||||
{
|
||||
"exit_code": 0,
|
||||
"elapsed_seconds": 1.8806199999526143,
|
||||
"samples": 176,
|
||||
"read_errors": 0,
|
||||
"first_sample_seconds": 0.036153299966827035,
|
||||
"last_sample_seconds": 1.8701112999697216,
|
||||
"unobserved_tail_seconds": 0.010508699982892722,
|
||||
"peaks": {
|
||||
"working_set_bytes": 12763136,
|
||||
"commit_bytes": 2830336,
|
||||
"sampled_private_bytes": 2830336
|
||||
},
|
||||
"log": "write_recovery-3.log",
|
||||
"workload": "write_recovery",
|
||||
"test": "workspace::tests::hundred_mib_spooled_writes_and_journal_recovery_keep_receipts_exactly_once",
|
||||
"iteration": 3,
|
||||
"passed": true
|
||||
},
|
||||
{
|
||||
"exit_code": 0,
|
||||
"elapsed_seconds": 4.99780670000473,
|
||||
"samples": 474,
|
||||
"read_errors": 0,
|
||||
"first_sample_seconds": 0.030193000042345375,
|
||||
"last_sample_seconds": 4.987429900036659,
|
||||
"unobserved_tail_seconds": 0.010376799968071282,
|
||||
"peaks": {
|
||||
"working_set_bytes": 12873728,
|
||||
"commit_bytes": 2928640,
|
||||
"sampled_private_bytes": 2830336
|
||||
},
|
||||
"log": "rename_delete-1.log",
|
||||
"workload": "rename_delete",
|
||||
"test": "workspace::tests::hundred_mib_file_operations_recover_each_copy_stage_without_duplicate_events",
|
||||
"iteration": 1,
|
||||
"passed": true
|
||||
},
|
||||
{
|
||||
"exit_code": 0,
|
||||
"elapsed_seconds": 5.227037100004964,
|
||||
"samples": 496,
|
||||
"read_errors": 0,
|
||||
"first_sample_seconds": 0.03234159998828545,
|
||||
"last_sample_seconds": 5.21647149999626,
|
||||
"unobserved_tail_seconds": 0.010565600008703768,
|
||||
"peaks": {
|
||||
"working_set_bytes": 12673024,
|
||||
"commit_bytes": 2859008,
|
||||
"sampled_private_bytes": 2826240
|
||||
},
|
||||
"log": "rename_delete-2.log",
|
||||
"workload": "rename_delete",
|
||||
"test": "workspace::tests::hundred_mib_file_operations_recover_each_copy_stage_without_duplicate_events",
|
||||
"iteration": 2,
|
||||
"passed": true
|
||||
},
|
||||
{
|
||||
"exit_code": 0,
|
||||
"elapsed_seconds": 4.705287000047974,
|
||||
"samples": 446,
|
||||
"read_errors": 0,
|
||||
"first_sample_seconds": 0.03485880000516772,
|
||||
"last_sample_seconds": 4.69486870005494,
|
||||
"unobserved_tail_seconds": 0.010418299993034452,
|
||||
"peaks": {
|
||||
"working_set_bytes": 13242368,
|
||||
"commit_bytes": 2990080,
|
||||
"sampled_private_bytes": 2891776
|
||||
},
|
||||
"log": "rename_delete-3.log",
|
||||
"workload": "rename_delete",
|
||||
"test": "workspace::tests::hundred_mib_file_operations_recover_each_copy_stage_without_duplicate_events",
|
||||
"iteration": 3,
|
||||
"passed": true
|
||||
},
|
||||
{
|
||||
"exit_code": 0,
|
||||
"elapsed_seconds": 12.0912092999788,
|
||||
"samples": 1150,
|
||||
"read_errors": 0,
|
||||
"first_sample_seconds": 0.033383999951183796,
|
||||
"last_sample_seconds": 12.08060469996417,
|
||||
"unobserved_tail_seconds": 0.010604600014630705,
|
||||
"peaks": {
|
||||
"working_set_bytes": 13340672,
|
||||
"commit_bytes": 2957312,
|
||||
"sampled_private_bytes": 2883584
|
||||
},
|
||||
"log": "conflicts-1.log",
|
||||
"workload": "conflicts",
|
||||
"test": "sync_resolution::tests::hundred_mib_conflicts_resolve_all_choices_and_reopen_without_duplicate_jobs",
|
||||
"iteration": 1,
|
||||
"passed": true
|
||||
},
|
||||
{
|
||||
"exit_code": 0,
|
||||
"elapsed_seconds": 12.346776099991985,
|
||||
"samples": 1176,
|
||||
"read_errors": 0,
|
||||
"first_sample_seconds": 0.03363140003057197,
|
||||
"last_sample_seconds": 12.336225600040052,
|
||||
"unobserved_tail_seconds": 0.01055049995193258,
|
||||
"peaks": {
|
||||
"working_set_bytes": 12689408,
|
||||
"commit_bytes": 2904064,
|
||||
"sampled_private_bytes": 2805760
|
||||
},
|
||||
"log": "conflicts-2.log",
|
||||
"workload": "conflicts",
|
||||
"test": "sync_resolution::tests::hundred_mib_conflicts_resolve_all_choices_and_reopen_without_duplicate_jobs",
|
||||
"iteration": 2,
|
||||
"passed": true
|
||||
},
|
||||
{
|
||||
"exit_code": 0,
|
||||
"elapsed_seconds": 11.75401180004701,
|
||||
"samples": 1120,
|
||||
"read_errors": 0,
|
||||
"first_sample_seconds": 0.03740810003364459,
|
||||
"last_sample_seconds": 11.74377210001694,
|
||||
"unobserved_tail_seconds": 0.010239700030069798,
|
||||
"peaks": {
|
||||
"working_set_bytes": 12697600,
|
||||
"commit_bytes": 2920448,
|
||||
"sampled_private_bytes": 2863104
|
||||
},
|
||||
"log": "conflicts-3.log",
|
||||
"workload": "conflicts",
|
||||
"test": "sync_resolution::tests::hundred_mib_conflicts_resolve_all_choices_and_reopen_without_duplicate_jobs",
|
||||
"iteration": 3,
|
||||
"passed": true
|
||||
},
|
||||
{
|
||||
"exit_code": 0,
|
||||
"elapsed_seconds": 10.770926599972881,
|
||||
"samples": 1026,
|
||||
"read_errors": 0,
|
||||
"first_sample_seconds": 0.0303565000067465,
|
||||
"last_sample_seconds": 10.760348799987696,
|
||||
"unobserved_tail_seconds": 0.010577799985185266,
|
||||
"peaks": {
|
||||
"working_set_bytes": 13529088,
|
||||
"commit_bytes": 3035136,
|
||||
"sampled_private_bytes": 2969600
|
||||
},
|
||||
"log": "discovery_rebind-1.log",
|
||||
"workload": "discovery_rebind",
|
||||
"test": "sync_initial::tests::hundred_mib_discovery_preview_and_rebinding_preserve_all_current_files",
|
||||
"iteration": 1,
|
||||
"passed": true
|
||||
},
|
||||
{
|
||||
"exit_code": 0,
|
||||
"elapsed_seconds": 10.228322899958584,
|
||||
"samples": 974,
|
||||
"read_errors": 0,
|
||||
"first_sample_seconds": 0.037193399970419705,
|
||||
"last_sample_seconds": 10.217878499999642,
|
||||
"unobserved_tail_seconds": 0.010444399958942086,
|
||||
"peaks": {
|
||||
"working_set_bytes": 12881920,
|
||||
"commit_bytes": 2969600,
|
||||
"sampled_private_bytes": 2879488
|
||||
},
|
||||
"log": "discovery_rebind-2.log",
|
||||
"workload": "discovery_rebind",
|
||||
"test": "sync_initial::tests::hundred_mib_discovery_preview_and_rebinding_preserve_all_current_files",
|
||||
"iteration": 2,
|
||||
"passed": true
|
||||
},
|
||||
{
|
||||
"exit_code": 0,
|
||||
"elapsed_seconds": 13.704265699954703,
|
||||
"samples": 1297,
|
||||
"read_errors": 0,
|
||||
"first_sample_seconds": 0.03508279996458441,
|
||||
"last_sample_seconds": 13.693803099973593,
|
||||
"unobserved_tail_seconds": 0.010462599981110543,
|
||||
"peaks": {
|
||||
"working_set_bytes": 13533184,
|
||||
"commit_bytes": 3059712,
|
||||
"sampled_private_bytes": 2969600
|
||||
},
|
||||
"log": "discovery_rebind-3.log",
|
||||
"workload": "discovery_rebind",
|
||||
"test": "sync_initial::tests::hundred_mib_discovery_preview_and_rebinding_preserve_all_current_files",
|
||||
"iteration": 3,
|
||||
"passed": true
|
||||
}
|
||||
],
|
||||
"limitations": [
|
||||
"OS cumulative peaks observed before exit; final sampling gap is reported",
|
||||
"Patterns and local hardware differ from the complete release benchmark",
|
||||
"No network/load/RSS acceptance threshold is inferred from these measurements"
|
||||
],
|
||||
"build": {
|
||||
"executable": "notesagent_host-af6ed12d4c03aedc.exe",
|
||||
"executable_sha256": "df59b97fca293d91a3958edcee94a01c8838fe096b0e43bc0283176742e0d268",
|
||||
"sources_sha256": "9ff9851ce36763889a1c64640a055e78bd77de8ab2acc587cf921bb9bb720119",
|
||||
"cargo_lock_sha256": "e626516edab41831c8585974822486e5f91796429aee56c92fbffe1d964b30d9",
|
||||
"git_head": "5d3c214ab916e677e3513705a37eb1102b4e8c5f",
|
||||
"rustc": "rustc 1.98.1 (48a229cea 2026-09-01)"
|
||||
},
|
||||
"machine": {
|
||||
"os": "Windows-11-10.0.26100-SP0",
|
||||
"cpu": "13th Gen Intel(R) Core(TM) i7-13620H",
|
||||
"logical_processors": 16,
|
||||
"python": "3.13.9",
|
||||
"disk": "not measured",
|
||||
"network": "not exercised"
|
||||
},
|
||||
"calibration": {
|
||||
"exit_code": 0,
|
||||
"elapsed_seconds": 0.37081300001591444,
|
||||
"samples": 35,
|
||||
"read_errors": 0,
|
||||
"first_sample_seconds": 0.004643600026611239,
|
||||
"last_sample_seconds": 0.3603207999840379,
|
||||
"unobserved_tail_seconds": 0.010492200031876564,
|
||||
"peaks": {
|
||||
"working_set_bytes": 78327808,
|
||||
"commit_bytes": 72654848,
|
||||
"sampled_private_bytes": 72634368
|
||||
},
|
||||
"log": "calibration.log"
|
||||
},
|
||||
"summary": {
|
||||
"stream_verify": {
|
||||
"working_set_bytes": {
|
||||
"max": 9543680,
|
||||
"median": 8851456,
|
||||
"min": 8744960
|
||||
},
|
||||
"commit_bytes": {
|
||||
"max": 2269184,
|
||||
"median": 2260992,
|
||||
"min": 2260992
|
||||
}
|
||||
},
|
||||
"write_recovery": {
|
||||
"working_set_bytes": {
|
||||
"max": 13320192,
|
||||
"median": 12771328,
|
||||
"min": 12763136
|
||||
},
|
||||
"commit_bytes": {
|
||||
"max": 2887680,
|
||||
"median": 2830336,
|
||||
"min": 2830336
|
||||
}
|
||||
},
|
||||
"rename_delete": {
|
||||
"working_set_bytes": {
|
||||
"max": 13242368,
|
||||
"median": 12873728,
|
||||
"min": 12673024
|
||||
},
|
||||
"commit_bytes": {
|
||||
"max": 2990080,
|
||||
"median": 2928640,
|
||||
"min": 2859008
|
||||
}
|
||||
},
|
||||
"conflicts": {
|
||||
"working_set_bytes": {
|
||||
"max": 13340672,
|
||||
"median": 12697600,
|
||||
"min": 12689408
|
||||
},
|
||||
"commit_bytes": {
|
||||
"max": 2957312,
|
||||
"median": 2920448,
|
||||
"min": 2904064
|
||||
}
|
||||
},
|
||||
"discovery_rebind": {
|
||||
"working_set_bytes": {
|
||||
"max": 13533184,
|
||||
"median": 13529088,
|
||||
"min": 12881920
|
||||
},
|
||||
"commit_bytes": {
|
||||
"max": 3059712,
|
||||
"median": 3035136,
|
||||
"min": 2969600
|
||||
}
|
||||
}
|
||||
},
|
||||
"measurement_completed": true
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
# Windows Sync 组件内存观测(5d3c214)
|
||||
|
||||
这是单独 Rust 测试进程的组件观测,完整生产化验收状态为 **NOT_ASSESSED**。每个场景启动独立进程,串行运行 3 次。四个文件场景使用已有真实 100 MiB 数据测试;对照场景使用生成式 100 MiB 流。全部 15 次测试均确认恰好 1 项通过、0 失败、0 ignored。
|
||||
|
||||
| 场景 | 三次最大峰值工作集 MiB | 三次最大峰值提交内存 MiB |
|
||||
| --- | ---: | ---: |
|
||||
| 生成流校验(对照) | 9.10 | 2.16 |
|
||||
| 写入与日志恢复 | 12.70 | 2.75 |
|
||||
| 重命名与删除 | 12.63 | 2.85 |
|
||||
| 三种冲突解决 | 12.72 | 2.82 |
|
||||
| 扫描与重新绑定 | 12.91 | 2.92 |
|
||||
|
||||
峰值工作集与峰值提交内存来自 [GetProcessMemoryInfo](https://learn.microsoft.com/en-us/windows/win32/api/psapi/nf-psapi-getprocessmemoryinfo) 的操作系统累计计数;工作集表示物理映射内存,提交内存不是实际分页文件读写量,二者不能相加。计数定义参考 [Memory Performance Information](https://learn.microsoft.com/en-us/windows/win32/memory/memory-performance-information)。父进程约每 10 ms 读取一次,JSON 保留样本数、读取错误和最后一次观测到退出的间隔,不能声称覆盖退出前每个瞬间。
|
||||
|
||||
环境:Windows 11 build 26100,Intel Core i7-13620H,16 逻辑处理器,Python 3.13.9。Rust 编译器、可执行文件摘要、Rust 源码摘要及 Cargo.lock 摘要记录在 [原始报告](sync-memory-windows-5d3c214.json)。测量器先用分配并保留 64 MiB 的独立 Python 子进程校验,工作集与提交峰值都必须至少达到该分配量,否则不会继续报告成功。
|
||||
|
||||
范围不含 WebView、Core、Sync 服务端、其他后代进程和系统文件缓存。磁盘与网络未测量;该机器和数据组合不是规划的 4 核/16 GiB、10000 笔记、双账号双 Vault 和规定网络环境。未建立改造前的实测基线,因此不报告降低百分比。不能将本结果作为 S-09 四并发上传服务总 RSS≤2 GiB、吞吐/延迟或完整客户端端到端验收通过。
|
||||
|
||||
复现命令:`python scripts/measure-sync-memory.py --runs 3 --output .build/sync-memory`。脚本先构建当前 Rust library test executable,再使用冻结的 exact 测试名称逐个启动,拒绝空匹配、ignored、测试失败或无内存样本。退出 0 表示这些组件测量完成,不表示生产化发布门槛全部满足。
|
||||
@@ -0,0 +1,173 @@
|
||||
"""Windows component measurements, not the S-09 service/release benchmark.
|
||||
|
||||
Build the current Rust library test executable, then run each fixed workload in
|
||||
its own process. Reports observed OS cumulative peaks, with sampling coverage.
|
||||
No third-party Python packages required. Usage: python scripts/measure-sync-memory.py
|
||||
"""
|
||||
from __future__ import annotations
|
||||
import argparse
|
||||
import ctypes as c
|
||||
from ctypes import wintypes as w
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import platform
|
||||
import re
|
||||
import statistics
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
from datetime import datetime, timezone
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
WORKLOADS = {
|
||||
"stream_verify": "payloads::tests::hundred_mib_verification_is_bounded_and_rejects_growth_or_truncation",
|
||||
"write_recovery": "workspace::tests::hundred_mib_spooled_writes_and_journal_recovery_keep_receipts_exactly_once",
|
||||
"rename_delete": "workspace::tests::hundred_mib_file_operations_recover_each_copy_stage_without_duplicate_events",
|
||||
"conflicts": "sync_resolution::tests::hundred_mib_conflicts_resolve_all_choices_and_reopen_without_duplicate_jobs",
|
||||
"discovery_rebind": "sync_initial::tests::hundred_mib_discovery_preview_and_rebinding_preserve_all_current_files",
|
||||
}
|
||||
|
||||
class Counters(c.Structure):
|
||||
_fields_ = [("cb", w.DWORD), ("PageFaultCount", w.DWORD)] + [
|
||||
(name, c.c_size_t) for name in (
|
||||
"PeakWorkingSetSize", "WorkingSetSize", "QuotaPeakPagedPoolUsage",
|
||||
"QuotaPagedPoolUsage", "QuotaPeakNonPagedPoolUsage", "QuotaNonPagedPoolUsage",
|
||||
"PagefileUsage", "PeakPagefileUsage", "PrivateUsage")]
|
||||
|
||||
class Monitor:
|
||||
def __init__(self):
|
||||
self.kernel = c.WinDLL("kernel32", use_last_error=True)
|
||||
self.psapi = c.WinDLL("psapi", use_last_error=True)
|
||||
self.kernel.OpenProcess.argtypes = [w.DWORD, w.BOOL, w.DWORD]
|
||||
self.kernel.OpenProcess.restype = w.HANDLE
|
||||
self.kernel.CloseHandle.argtypes = [w.HANDLE]
|
||||
self.kernel.CloseHandle.restype = w.BOOL
|
||||
self.psapi.GetProcessMemoryInfo.argtypes = [w.HANDLE, c.POINTER(Counters), w.DWORD]
|
||||
self.psapi.GetProcessMemoryInfo.restype = w.BOOL
|
||||
|
||||
def run(self, command, log: Path, timeout=120):
|
||||
env = {key: value for key, value in os.environ.items() if key.upper() in {
|
||||
"SYSTEMROOT", "WINDIR", "PATH", "TEMP", "TMP", "USERPROFILE", "LOCALAPPDATA"}}
|
||||
env["PYTHONUTF8"] = "1"
|
||||
start = time.perf_counter()
|
||||
samples = errors = 0
|
||||
first = last = None
|
||||
peaks = {"working_set_bytes": 0, "commit_bytes": 0, "sampled_private_bytes": 0}
|
||||
with log.open("wb") as output:
|
||||
process = subprocess.Popen(command, cwd=ROOT, stdin=subprocess.DEVNULL,
|
||||
stdout=output, stderr=subprocess.STDOUT, env=env,
|
||||
creationflags=subprocess.CREATE_NO_WINDOW)
|
||||
handle = self.kernel.OpenProcess(0x1000 | 0x0010, False, process.pid)
|
||||
try:
|
||||
if not handle:
|
||||
raise OSError("MEMORY_PROCESS_OPEN_FAILED")
|
||||
while process.poll() is None:
|
||||
if time.perf_counter() - start > timeout:
|
||||
raise TimeoutError("MEMORY_WORKLOAD_TIMEOUT")
|
||||
counters = Counters()
|
||||
counters.cb = c.sizeof(counters)
|
||||
if self.psapi.GetProcessMemoryInfo(handle, c.byref(counters), counters.cb):
|
||||
samples += 1
|
||||
last = time.perf_counter() - start
|
||||
if first is None:
|
||||
first = last
|
||||
peaks["working_set_bytes"] = max(peaks["working_set_bytes"], counters.PeakWorkingSetSize)
|
||||
peaks["commit_bytes"] = max(peaks["commit_bytes"], counters.PeakPagefileUsage)
|
||||
peaks["sampled_private_bytes"] = max(peaks["sampled_private_bytes"], counters.PrivateUsage)
|
||||
else:
|
||||
errors += 1
|
||||
time.sleep(0.01)
|
||||
elapsed = time.perf_counter() - start
|
||||
return dict(exit_code=process.returncode, elapsed_seconds=elapsed,
|
||||
samples=samples, read_errors=errors, first_sample_seconds=first,
|
||||
last_sample_seconds=last, unobserved_tail_seconds=None if last is None else elapsed-last,
|
||||
peaks=peaks, log=log.name)
|
||||
finally:
|
||||
if process.poll() is None:
|
||||
process.kill()
|
||||
process.wait(timeout=10)
|
||||
if handle:
|
||||
self.kernel.CloseHandle(handle)
|
||||
|
||||
def digest(path):
|
||||
with path.open("rb") as file:
|
||||
return hashlib.file_digest(file, "sha256").hexdigest()
|
||||
|
||||
def build(directory):
|
||||
command = ["cargo", "test", "--manifest-path", "frontend/src-tauri/Cargo.toml",
|
||||
"--features", "desktop", "--lib", "--no-run", "--message-format=json"]
|
||||
with (directory / "build.log").open("wb") as error:
|
||||
result = subprocess.run(command, cwd=ROOT, stdout=subprocess.PIPE, stderr=error, check=True)
|
||||
executables = []
|
||||
for line in result.stdout.decode("utf-8").splitlines():
|
||||
try:
|
||||
message = json.loads(line)
|
||||
except ValueError:
|
||||
continue
|
||||
if message.get("reason") == "compiler-artifact" and message.get("profile", {}).get("test") and message.get("target", {}).get("name") == "notesagent_host" and message.get("executable"):
|
||||
executables.append(Path(message["executable"]))
|
||||
if len(executables) != 1:
|
||||
raise RuntimeError("MEMORY_TEST_EXECUTABLE_AMBIGUOUS")
|
||||
return executables[0]
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--runs", type=int, default=3)
|
||||
parser.add_argument("--output", type=Path, default=ROOT / ".build/sync-memory")
|
||||
args = parser.parse_args()
|
||||
if os.name != "nt" or not 1 <= args.runs <= 10:
|
||||
parser.error("Windows required; runs must be 1..10")
|
||||
directory = args.output.resolve()
|
||||
directory.mkdir(parents=True, exist_ok=True)
|
||||
report = dict(schema=1, created_at=datetime.now(timezone.utc).isoformat(),
|
||||
scope="Single Rust component test process; excludes WebView, Core, service and system file cache",
|
||||
acceptance="NOT_ASSESSED", polling_interval_seconds=0.01, runs=[],
|
||||
limitations=["OS cumulative peaks observed before exit; final sampling gap is reported",
|
||||
"Patterns and local hardware differ from the complete release benchmark",
|
||||
"No network/load/RSS acceptance threshold is inferred from these measurements"])
|
||||
try:
|
||||
exe = build(directory)
|
||||
source = hashlib.sha256()
|
||||
paths = sorted((ROOT / "frontend/src-tauri/src").rglob("*.rs")) + sorted((ROOT / "frontend/src-tauri/tests").rglob("*.rs"))
|
||||
for path in paths:
|
||||
source.update(str(path.relative_to(ROOT)).replace(chr(92), "/").encode())
|
||||
source.update(bytes.fromhex(digest(path)))
|
||||
report["build"] = dict(executable=exe.name, executable_sha256=digest(exe),
|
||||
sources_sha256=source.hexdigest(), cargo_lock_sha256=digest(ROOT / "frontend/src-tauri/Cargo.lock"),
|
||||
git_head=subprocess.check_output(["git", "rev-parse", "HEAD"], cwd=ROOT, text=True).strip(),
|
||||
rustc=subprocess.check_output(["rustc", "--version"], text=True).strip())
|
||||
import winreg
|
||||
with winreg.OpenKey(winreg.HKEY_LOCAL_MACHINE, r"HARDWARE\DESCRIPTION\System\CentralProcessor\0") as key:
|
||||
cpu = winreg.QueryValueEx(key, "ProcessorNameString")[0].strip()
|
||||
report["machine"] = dict(os=platform.platform(), cpu=cpu, logical_processors=os.cpu_count(),
|
||||
python=platform.python_version(), disk="not measured", network="not exercised")
|
||||
monitor = Monitor()
|
||||
calibration = monitor.run([sys.executable, "-c",
|
||||
"import time; data=bytearray(64*1024*1024); time.sleep(0.3)"], directory / "calibration.log")
|
||||
report["calibration"] = calibration
|
||||
if calibration["exit_code"] != 0 or calibration["peaks"]["working_set_bytes"] < 64*1024*1024 or calibration["peaks"]["commit_bytes"] < 64*1024*1024:
|
||||
raise RuntimeError("MEMORY_CALIBRATION_FAILED")
|
||||
for name, test in WORKLOADS.items():
|
||||
for iteration in range(1, args.runs + 1):
|
||||
log = directory / f"{name}-{iteration}.log"
|
||||
value = monitor.run([str(exe), "--exact", test, "--nocapture", "--test-threads=1"], log)
|
||||
value.update(workload=name, test=test, iteration=iteration)
|
||||
output = log.read_text(encoding="utf-8", errors="replace")
|
||||
value["passed"] = value["exit_code"] == 0 and value["samples"] > 0 and bool(re.search(r"test result: ok\. 1 passed; 0 failed; 0 ignored;", output))
|
||||
report["runs"].append(value)
|
||||
print(json.dumps(dict(workload=name, iteration=iteration, passed=value["passed"], peaks=value["peaks"])), flush=True)
|
||||
if not value["passed"]:
|
||||
raise RuntimeError("MEMORY_WORKLOAD_FAILED")
|
||||
report["summary"] = {name: {
|
||||
metric: dict(max=max(values), median=statistics.median(values), min=min(values))
|
||||
for metric in ("working_set_bytes", "commit_bytes")
|
||||
for values in [[item["peaks"][metric] for item in report["runs"] if item["workload"] == name]]
|
||||
} for name in WORKLOADS}
|
||||
report["measurement_completed"] = True
|
||||
finally:
|
||||
(directory / "report.json").write_text(json.dumps(report, ensure_ascii=False, indent=2)+"\n", encoding="utf-8")
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user