fix(backend): 落实 PR #11 第二轮评审意见
- Benchmark 容量淘汰只删终态 run,满容量且全活动时返回 BENCHMARK_CAPACITY_EXCEEDED - 创建 run 前校验索引兼容性(BENCHMARK_INDEX_INCOMPATIBLE) - 取消 run 补发 RunCancelled 终止事件;失败分支脱敏(BENCHMARK_RUN_FAILED) - 失败样本计入汇总分母,报告输出 total/successful/failed/failure_rate - load_dataset 按文件名隔离无关损坏文件,顶层非对象拒绝 - FTS score_threshold 先于计数/分页,total 与 items 一致 - Benchmark SSE 支持 Last-Event-ID 游标 - 移除 Agent Benchmark 501 占位接口 - 同步第二阶段接口契约与开发说明文档 Co-Authored-By: Claude Code <noreply@anthropic.com>
This commit is contained in:
@@ -55,9 +55,9 @@
|
||||
| Provider | 现有路径 | `/api/providers/*`、`POST /api/chat` | 扩展 | 补齐协议能力和统一行为 |
|
||||
| Retrieval | GET/POST | `/api/index/status`、`/api/index/rebuild` | 扩展 | 暴露 Embedding 兼容状态并安全重建向量 |
|
||||
| Benchmark | GET | `/api/benchmarks/datasets` | 计划新增 | 枚举受控 Dataset |
|
||||
| Benchmark | POST | `/api/benchmarks/rag/runs` | 计划新增 | 创建 RAG Benchmark |
|
||||
| Benchmark | POST | `/api/benchmarks/agent/runs` | 计划新增 | 创建 Agent Benchmark |
|
||||
| Benchmark | GET | `/api/benchmarks/runs` | 计划新增 | 分页获取 Benchmark Run |
|
||||
| Benchmark | POST | `/api/benchmarks/rag/runs` | 已实现 | 创建 RAG Benchmark |
|
||||
| Benchmark | POST | `/api/benchmarks/agent/runs` | 暂缓 | 创建 Agent Benchmark(依赖 Agent Runtime 完成后交付) |
|
||||
| Benchmark | GET | `/api/benchmarks/runs` | 已实现 | 分页获取 Benchmark Run |
|
||||
| Benchmark | GET/POST | `/api/benchmarks/runs/{run_id}/*` | 计划新增 | 查询、订阅、取消和读取报告 |
|
||||
| Export | POST | `/api/exports` | 计划新增 | 创建 HTML/PDF/DOCX 导出任务 |
|
||||
| Export | GET | `/api/exports` | 计划新增 | 分页获取导出任务 |
|
||||
@@ -817,7 +817,7 @@ Dataset 从仓库或受控导入目录注册。API 不接受调用方提交任
|
||||
|
||||
配置快照必须记录 Embedding model ID/version/dimension、Reranker、索引版本、Dataset Hash 和运行环境。
|
||||
|
||||
### 9.5 创建 Agent Benchmark
|
||||
### 9.5 创建 Agent Benchmark(暂缓,未暴露接口)
|
||||
|
||||
`POST /api/benchmarks/agent/runs`
|
||||
|
||||
@@ -851,12 +851,16 @@ RAG 和 Agent 创建接口均返回 `202 BenchmarkRun`:
|
||||
"metrics": null,
|
||||
"config_snapshot": {},
|
||||
"error": null,
|
||||
"error_code": null,
|
||||
"created_at": "2026-08-31T10:30:00Z",
|
||||
"started_at": null,
|
||||
"completed_at": null
|
||||
}
|
||||
```
|
||||
|
||||
`status` 取值:`queued` → `running` → `completed` | `failed` | `cancelled`。失败/取消时 `error` 与
|
||||
`error_code` 只返回项目错误码与安全消息,不暴露第三方堆栈。
|
||||
|
||||
公共接口:
|
||||
|
||||
| 方法 | 路径 | 用途 |
|
||||
@@ -867,6 +871,10 @@ RAG 和 Agent 创建接口均返回 `202 BenchmarkRun`:
|
||||
| POST | `/api/benchmarks/runs/{run_id}/cancel` | 取消运行 |
|
||||
| GET | `/api/benchmarks/runs/{run_id}/report` | 获取结构化完整报告 |
|
||||
|
||||
SSE 事件流(`RunStarted` → `CaseCompleted`* → `RunCompleted` | `RunFailed` | `RunCancelled`):
|
||||
`GET /api/benchmarks/runs/{run_id}/events` 支持 `Last-Event-ID` 与 `?after_sequence=` 游标恢复,
|
||||
`RunCompleted` / `RunFailed` / `RunCancelled` 为终止事件,收到后即断流。
|
||||
|
||||
### 9.7 指标 Contract
|
||||
|
||||
RAG:
|
||||
@@ -879,10 +887,17 @@ RAG:
|
||||
"mrr": 0.81,
|
||||
"citation_hit_rate": 0.89,
|
||||
"p50_latency_ms": 24.5,
|
||||
"p95_latency_ms": 67.3
|
||||
"p95_latency_ms": 67.3,
|
||||
"total_cases": 50,
|
||||
"successful_cases": 48,
|
||||
"failed_cases": 2,
|
||||
"failure_rate": 0.04
|
||||
}
|
||||
```
|
||||
|
||||
失败样本按零分计入质量指标分母,`total_cases` / `successful_cases` / `failed_cases` /
|
||||
`failure_rate` 让报告明确实际分母;延迟仅统计成功样本。
|
||||
|
||||
Agent:
|
||||
|
||||
```json
|
||||
@@ -906,8 +921,10 @@ BENCHMARK_DATASET_NOT_FOUND
|
||||
BENCHMARK_DATASET_INVALID
|
||||
BENCHMARK_CONFIG_INVALID
|
||||
BENCHMARK_INDEX_INCOMPATIBLE
|
||||
BENCHMARK_CAPACITY_EXCEEDED
|
||||
BENCHMARK_RUN_NOT_FOUND
|
||||
BENCHMARK_RUN_FAILED
|
||||
BENCHMARK_CASE_EVALUATION_FAILED
|
||||
```
|
||||
|
||||
### 9.9 Retrieval Profile 与索引兼容
|
||||
|
||||
Reference in New Issue
Block a user