fix(backend): 落实 PR #11 第二轮评审意见

- Benchmark 容量淘汰只删终态 run,满容量且全活动时返回 BENCHMARK_CAPACITY_EXCEEDED
- 创建 run 前校验索引兼容性(BENCHMARK_INDEX_INCOMPATIBLE)
- 取消 run 补发 RunCancelled 终止事件;失败分支脱敏(BENCHMARK_RUN_FAILED)
- 失败样本计入汇总分母,报告输出 total/successful/failed/failure_rate
- load_dataset 按文件名隔离无关损坏文件,顶层非对象拒绝
- FTS score_threshold 先于计数/分页,total 与 items 一致
- Benchmark SSE 支持 Last-Event-ID 游标
- 移除 Agent Benchmark 501 占位接口
- 同步第二阶段接口契约与开发说明文档

Co-Authored-By: Claude Code <noreply@anthropic.com>
This commit is contained in:
yxx
2026-09-03 22:20:11 +08:00
co-authored by Claude Code
parent c6cde2500b
commit fcc601fcf3
11 changed files with 378 additions and 69 deletions
+9
View File
@@ -720,6 +720,11 @@ class RAGMetrics(Contract):
citation_hit_rate: float = 0.0
p50_latency_ms: float = 0.0
p95_latency_ms: float = 0.0
# 样本构成:失败样本按零分计入质量指标,汇总不虚高;报告据此可知实际分母
total_cases: int = 0
successful_cases: int = 0
failed_cases: int = 0
failure_rate: float = 0.0
class BenchmarkDatasetInfo(Contract):
@@ -745,6 +750,7 @@ class BenchmarkRun(Contract):
metrics: dict[str, Any] | None = None
config_snapshot: dict[str, Any] = Field(default_factory=dict)
error: str | None = None
error_code: str | None = None
created_at: datetime
started_at: datetime | None = None
completed_at: datetime | None = None
@@ -760,6 +766,7 @@ class BenchmarkEventType(str, Enum):
case_completed = "CaseCompleted"
run_completed = "RunCompleted"
run_failed = "RunFailed"
run_cancelled = "RunCancelled"
class BenchmarkEvent(Contract):
@@ -785,6 +792,7 @@ class RAGCaseResult(Contract):
# 该 Case 是否声明了 expected_block_ids(决定是否计入 citation_hit_rate 分母)
citation_applicable: bool = False
error: str | None = None
error_code: str | None = None
class BenchmarkReport(Contract):
@@ -797,3 +805,4 @@ class BenchmarkReport(Contract):
metrics: dict[str, Any] = Field(default_factory=dict)
cases: list[RAGCaseResult] = Field(default_factory=list)
error: str | None = None
error_code: str | None = None