Author SHA1 Message Date
Kronecker 5df3c370aa docs(frontend): document managed language submission 2026-10-10 04:09:41 +08:00
Kronecker 029585739c fix(worker): bound configurable control request deadlines 2026-10-10 02:15:52 +08:00
Kronecker 47f535ea79 docs(frontend): clarify connected judging services 2026-10-10 00:24:08 +08:00
Kronecker 67c3bc55ba chore(judge): document explicit managed runtime rollout 2026-10-09 23:29:23 +08:00
Kronecker d20721313e test(judge): verify managed worker faults and sealed programs 2026-10-09 22:49:57 +08:00
Kronecker 9d960c2a9b feat(frontend): enable problem-scoped Python and Java submissions 2026-10-09 22:32:38 +08:00
Kronecker 507b15b64a test(coordinator): fence stale managed reports and bound retries 2026-10-09 21:57:40 +08:00
Kronecker 643a8d342e fix(judge): bind managed inputs and confirm artifact cleanup 2026-10-09 21:45:21 +08:00
Kronecker 802e019699 feat(worker): judge versioned sealed runtime programs 2026-10-09 20:15:17 +08:00
Kronecker 2f1166954a feat(sandbox): route pinned runtimes through one bounded factory 2026-10-09 18:50:10 +08:00
Kronecker 38ee332a3e test(sandbox): verify managed file quotas and live wall deadlines 2026-10-09 18:12:37 +08:00
Kronecker 9394a914f8 feat(sandbox): bind managed requests to pinned runtime identities 2026-10-09 17:49:30 +08:00
Kronecker 1dd6d1f971 feat(sandbox): add immutable Python and Java execution profiles 2026-10-09 17:02:27 +08:00
Kronecker 53caafc157 feat(coordinator): dispatch versioned runtime leases and reports 2026-10-09 15:18:08 +08:00
Kronecker 4b8f255180 feat(judging): persist immutable per-language problem configurations 2026-10-09 14:53:09 +08:00
Kronecker a5a7e40308 feat(contracts): add versioned multi-runtime judging contracts 2026-10-09 14:15:24 +08:00
Kronecker 81f8dc3204 docs(platform): describe controlled judging startup and recovery 2026-10-09 10:06:17 +08:00
Kronecker 3e44779578 fix(frontend): ignore expired responses from prior sessions 2026-10-09 09:55:11 +08:00
Kronecker fb1b161c0d feat(platform): connect owned submissions and real browser flows 2026-10-09 09:30:12 +08:00
Kronecker 19a071573b feat(worker): consume durable jobs with fenced execution and recovery 2026-10-09 08:38:20 +08:00
Kronecker 76b565cce8 fix(contracts): reject unfulfillable output capture budgets 2026-10-09 08:25:39 +08:00
Kronecker b7e9111254 fix(messaging): suppress credential-bearing transport logs 2026-10-09 07:55:15 +08:00
Kronecker f838008c71 feat(coordinator): persist fenced execution authority and atomic results 2026-10-09 07:15:24 +08:00
Kronecker 38738122b3 feat(messaging): deliver durable outbox events with publisher confirms 2026-10-09 06:59:13 +08:00
Kronecker 75d74908e0 feat(submissions): create durable queued evaluations idempotently 2026-10-09 06:31:15 +08:00
Kronecker 8a01081cc9 feat(judge): compile once and run isolated points through helper factory 2026-10-09 04:12:22 +08:00
Kronecker 93fb41c608 feat(engine): judge immutable submissions through bounded execution ports 2026-10-09 03:34:04 +08:00
Kronecker 996dbc5144 feat(problems): publish immutable versions and fence artifact access 2026-10-09 01:51:13 +08:00
Kronecker ff60136f8d feat(identity): add bounded login and revocable sessions 2026-10-09 01:20:37 +08:00
Kronecker f4bf218081 feat(database): add judging schema and separate write roles 2026-10-09 01:09:34 +08:00
Kronecker df52446dac feat(contracts): validate fenced judging jobs and bounded results 2026-10-09 00:43:17 +08:00
Kronecker ea0bc2b578 feat(worker): inject engine port for authenticated sandbox execution 2026-10-08 17:28:26 +08:00
Kronecker ee6371f269 feat(sandbox): enable verified ordinary execution entry 2026-10-08 17:00:13 +08:00
Kronecker fc61e9d053 test(sandbox): verify repeated restart and failed capture lifecycle 2026-10-08 13:52:01 +08:00
Kronecker 32d94f3a66 feat(sandbox): recover service-owned tasks after supervisor restart 2026-10-08 13:46:02 +08:00
Kronecker 22cc427da9 test(sandbox): validate full isolation against live peer tasks 2026-10-08 10:27:32 +08:00
Kronecker ddf1e0ca68 test(sandbox): verify complete user filesystem and descriptor boundaries 2026-10-08 10:12:49 +08:00
Kronecker b5db44ba16 feat(sandbox): enforce trusted process event budgets and verify resource limits 2026-10-08 07:19:01 +08:00
Kronecker 057c02f39d test(sandbox): verify complete native and compiler task workflows 2026-10-08 07:12:11 +08:00
Kronecker f9789ea4c4 fix(sandbox): unwind every prepared owner before releasing resources 2026-10-08 04:49:02 +08:00
Kronecker 2e4d9d4ea2 feat(sandbox): supervise bounded root execution and final teardown 2026-10-08 04:38:08 +08:00
Kronecker 62b08fea5b feat(sandbox): apply finite inherited resource limits before identity 2026-10-08 01:37:06 +08:00
Kronecker 4c7adda719 feat(sandbox): supervise aggregate CPU and monotonic wall budgets 2026-10-08 01:24:56 +08:00
Kronecker cc6ee1bfa7 feat(sandbox): transport bounded stdio and close inherited user descriptors 2026-10-07 22:49:25 +08:00
Kronecker 89da9ffd9f feat(sandbox): reap namespace descendants and cancel via init lifecycle 2026-10-07 22:28:13 +08:00
Kronecker 03eaab719e feat(sandbox): restrict dangerous syscalls and verify GNU language profiles 2026-10-07 19:31:24 +08:00
Kronecker 9bb27f2e02 feat(sandbox): install profile seccomp with ABI and inheritance checks 2026-10-07 19:16:20 +08:00
Kronecker 46714bd9ea feat(sandbox): lease task identities and drop initialization privileges 2026-10-07 18:22:58 +08:00
Kronecker 329c9222a3 feat(sandbox): bind readonly inputs through authenticated capabilities 2026-10-07 17:57:10 +08:00
Kronecker 9a2b1c071f feat(sandbox): snapshot artifacts through bounded file capabilities 2026-10-07 16:19:05 +08:00
Kronecker 2b5e79dd39 feat(sandbox): restrict proc and devices in private roots 2026-10-07 13:21:37 +08:00
Kronecker 0f68eb5fb0 feat(sandbox): build private read-only roots with pivot_root 2026-10-07 10:02:21 +08:00
Kronecker f22308080f feat(sandbox): initialize isolated namespaces through gated helpers 2026-10-07 04:16:11 +08:00
Kronecker 1a91c2e0f6 feat(sandbox): gate target startup with verified task cgroups 2026-10-07 01:40:32 +08:00
Kronecker 605785edee test(sandbox): allow launcher checks in independent VMs 2026-10-07 01:09:50 +08:00
Kronecker 08db00b56e feat(sandbox): add bounded standalone launcher control 2026-10-06 22:04:29 +08:00
Kronecker 957135a112 feat(sandbox): enforce launcher lifecycle ordering 2026-10-06 11:02:55 +08:00
Kronecker 1f45266517 feat(sandbox): define runtime policy and preflight refusals 2026-10-06 03:58:19 +08:00
Kronecker ee1f43a4cb feat(sandbox): model execution outcomes and limit attribution 2026-10-06 01:00:04 +08:00
Kronecker 7ef77dfb34 feat(sandbox): validate bounded local execution specifications 2026-10-05 22:04:59 +08:00
Kronecker 8b8828d136 chore(merge): integrate workspace and frontend prototype
合并审阅 note(2026-10-05)

合并范围
- chore/workspace-bootstrap:b5b286c44f6a1456bef1db4ab904cecb92ff15a3。
- feat/frontend-pages:f108e88c47e6d604184d8a34dfd39be06762e159。
- 前端分支已包含工程骨架提交;一次合并保留两个分支的原有历史。

审阅结论
- 工程骨架与前端原型可以合入 main。
- 平台后端按业务域、Runtime、基础设施和后台进程划分;Rust 判题工程按契约、协调器、Worker、引擎和沙箱划分。
- 前端包含练习、比赛、训练、提交、账号和社区页面原型,以及独立入口的管理端;预览数据与未实现能力在界面中明确标记。
- 已修复编辑后立即刷新导致最后一段草稿丢失的问题(f108e88):页面隐藏、离开和 Vue scope 销毁时保存待写入内容,同时保留输入防抖和存储异常处理。

验证结果
- 本次完整 make check 通过:架构边界、Python/Rust 契约、Ruff、Rust fmt/clippy、前端 OpenAPI 类型、ESLint、Prettier、vue-tsc、生产构建及 Docker Compose 配置。
- Python 测试 7 项、Rust 集成测试 2 项、前端草稿回归测试 7 项全部通过。
- 前序浏览器验证覆盖桌面及移动布局、独立管理端入口、真实健康接口、编辑器 Worker;修复后已验证立即刷新可恢复最新草稿。

后续工作
- 编辑器页面生产 JS chunk 约 3,005.41 kB(gzip 785.90 kB),构建仍有包体告警,后续优化拆包和加载。
- 业务写入、认证和实际判题能力仍处于工程骨架阶段;沙箱继续拒绝未经验证隔离的执行。
2026-10-05 18:29:40 +08:00
Kronecker f108e88c47 fix(frontend): save pending code drafts before page exit 2026-10-05 15:26:41 +08:00
Kronecker 8c3cca6fde feat(frontend): isolate admin app with sidebar navigation
Build a separate admin entrypoint with its own router and sidebar layout.
Add preview sections for problems, contests, users, judge nodes and site settings.
Configure Vite to emit both public and admin HTML entries.
2026-10-05 01:50:07 +08:00
Kronecker 1443f00495 feat(frontend): add account and community page previews
Add profiles, discussions, team creation, authentication forms and the about page.
Validate preview forms and display the available platform health endpoint.
2026-10-05 01:50:02 +08:00
Kronecker 2ae3bfb6fb feat(frontend): build homepage dashboard and interactive tag sphere
Present announcements, submission trends, recent problems, contests and solver rankings.
Arrange blue pill labels in a TagCanvas sphere with adaptive sizing, mouse-follow and drag modes.
Allow tag selection to filter the problem list and provide keyboard-accessible links.
2026-10-05 01:49:58 +08:00
Kronecker 5c7a45fe41 feat(frontend): add practice page previews and code workbench
Add problem filtering, trainings, contests, submission details and rankings.
Provide a split problem/editor layout with multilingual local drafts and self-test panels.
Run and submit actions show pending service integration.
2026-10-05 01:49:55 +08:00
Kronecker 9fa25189a2 feat(frontend): establish shared components and blue page layouts
Add reusable headings, panels, tables, charts, pagination and explicit preview fixtures.
Apply the shared blue theme, responsive top navigation and Chinese UI locale.
2026-10-05 01:48:53 +08:00
Kronecker 720c69a198 chore(frontend): configure UI dependencies and Vue linting
Pin icon components, Monaco Editor and TagCanvas modules in the lockfile.
Let TypeScript validate identifiers in Vue script setup.
2026-10-05 01:48:51 +08:00
450 changed files with 69233 additions and 241 deletions
+2 -1
View File
@@ -43,6 +43,7 @@ check-frontend:
cd frontend && $(PNPM) lint
cd frontend && $(PNPM) format:check
cd frontend && $(PNPM) typecheck
cd frontend && $(PNPM) test
cd frontend && $(PNPM) build
contracts:
@@ -56,7 +57,7 @@ format:
cd frontend && $(PNPM) format
api:
cd backend && $(UV) run --locked uvicorn kf_oj.api.app:create_app --factory --reload --host 127.0.0.1 --port 8000
cd backend && $(UV) run --locked uvicorn kf_oj.api.app:create_app --factory --reload --host 127.0.0.1 --port 8000 --no-proxy-headers --limit-concurrency 32 --backlog 64
frontend:
cd frontend && $(PNPM) dev
+15 -5
View File
@@ -3,8 +3,10 @@
以 Rust 自研判题与 Linux 沙箱、以 Python/FastAPI 提供平台服务的在线评测系统。
前端使用 Vue 3、TypeScript、Element Plus 和 Vite。
当前版本为 **0.0.1 工程骨架**:平台 API 和协调器提供存活接口,前端可检查实际 API 连接。
沙箱隔离执行、判题、账号、题库、提交与学习业务尚未实现。
当前版本为 **0.0.1**:沙箱已有受控隔离、C/C++ 编译及 native 执行;平台已实现真实身份、
不可变题目、工件与幂等提交创建。事务 outbox、持久化 Coordinator、普通 Worker 和受控
helper 工厂已接通 C/C++ 编译一次、多测试点评测;登录、题库、提交和个人成绩页面
使用真实接口。训练、比赛、社区等扩展页面仍为开发预览。
## 工程划分
@@ -25,7 +27,8 @@ Python 业务域分为 `identity`、`problems`、`submissions`、`community`、`
Rust 分为 `kf-contracts`、`kf-sandbox`、`kf-engine`、`kf-worker`、`kf-coordinator` 五个 crate。
协调器拥有执行状态写入权,判题 Worker 不直接写评测数据库。
沙箱尚未验证隔离能力,当前执行入口明确拒绝运行程序。
沙箱执行入口每次核验隔离与限额,任何未验证配置继续拒绝执行。
用户程序及隔离/超限验收只在独立、整体限额的无业务凭据测试 VM 内运行。
## 安装与检查
@@ -62,11 +65,18 @@ make sandbox-probe # 只读观察 Linux 环境,不创建隔离资源
```
本地数据库与队列的启动步骤见 [部署说明](deploy/README.md)。
业务表和数据库 revision 尚未建立;迁移统一由后端 Alembic 管理。
业务表和权限分工由后端 Alembic 唯一管理,见 [迁移与权限说明](backend/migrations/README.md)。
模型服务与第三方插件尚未接入。
开发中的规划文档仅在本地维护,不包含在远端仓库中。
后续逐步实现沙箱、单机判题、分布式恢复、并行优化和学习功能。
单机提交服务的配置、最小权限、停止恢复与独立 VM 验收要求见
[判题说明](judge/README.md#单机持久化协调与常驻-worker)和[部署说明](deploy/README.md#单机提交服务的启动与停止)。
当前普通题执行支持固定 GNU C17/C++20、CPython 3.13.16 和 OpenJDK 21.0.12.1+1-LTS,
使用精确字节比较。Python 固定单文件 main.py 与标准库;Java 固定默认包 Main.java/Main,
允许有界辅助/内部/匿名类。题目必须显式声明对应语言能力,旧题目不会自动获得新语言。
新版源码与普通 Worker 的沙箱执行已在独立 VM 验收;完整平台/浏览器及故障验收按工程阶段
分别记录,生产部署需另行进行。自助注册、自测、SPJ/交互题、Rust 用户程序、分布式调度
和学习能力仍待后续实现。
## 许可证
+11 -1
View File
@@ -1,5 +1,15 @@
KF_OJ_ENVIRONMENT=development
KF_OJ_LOG_LEVEL=INFO
# Optional until database-backed use cases are implemented.
# Dedicated platform login inherits only kf_oj_platform; replace development credentials.
KF_OJ_DATABASE_URL=postgresql+asyncpg://kf_oj:local-dev-only@127.0.0.1:5432/kf_oj
KF_OJ_AMQP_URL=amqp://kf_oj:local-dev-only@127.0.0.1:5672/kf_oj
KF_OJ_BROWSER_ORIGINS='["http://localhost:5173","http://localhost:8000","http://127.0.0.1:5173","http://127.0.0.1:8000"]'
KF_OJ_SESSION_LIFETIME_SECONDS=43200
# Required for submissions: private absolute artifact root and trusted JSON admission policy.
# KF_OJ_ARTIFACT_ROOT=/srv/kf-platform/artifacts
KF_OJ_ARTIFACT_MAX_BYTES=536870912
KF_OJ_ARTIFACT_MAX_FILES=4096
# KF_OJ_JUDGE_ADMISSION=<JSON matching the Coordinator and Worker policy>
# Cancellation requires both; token equals Coordinator platform_token, never a Worker token.
# KF_OJ_COORDINATOR_URL=http://127.0.0.1:8100
# KF_OJ_COORDINATOR_TOKEN=<privately generated 64 lowercase hexadecimal characters>
+26 -4
View File
@@ -15,11 +15,33 @@ make api
make worker-check
```
API 当前只有 `/api/v1/health/live`,不连接数据库或消息队列。
Worker 目前仅支持 `--check`,默认拒绝进入尚未实现的任务循环。
日志、配置和迁移入口已建立;业务表、登录、判题投递和模型接入尚未实现。
API 提供存活、真实登录/退出/当前用户、公开题目与有效 Worker 租约的清单/工件读取。
配置数据库后在 lifespan 内组合模块,配置缺失时业务端口明确不可用,存活接口仍可用。
学习 Worker 目前仅支持 `--check`;Rust 判题 Worker 使用单独的常驻消费入口。
提交创建 API 在持久化源码后,同一事务写入初始评测和 outbox;按用户与幂等键
防止重复创建,待评测总量和个人待评测数有固定上限。
待投递事件使用独立进程 `uv run --locked python -m kf_oj.infra.messaging.publisher`;
配置 `KF_OJ_DATABASE_URL` 为专用平台角色,`KF_OJ_AMQP_URL` 为内部 broker。
发布器一次只锁定一个到期事件;使用持久队列、消息及 mandatory publisher confirm,
确认后才提交投递标记。锁过期允许重投,消息不是执行权,Coordinator claim 才是。
队列最多 64 条、32 KiB,拒绝溢出,消息有效期 10 分钟;broker 不可用采用有界退避,
不阻塞提交事务。日志仅固定错误代码,不打印源码、答案或连接口令。
后台每次投递最多 12 秒,关闭连接最多 2 秒,SIGTERM 在当前有界操作结束后退出。
账号/题目/工件命令及限制见对应模块 README;模型能力尚未接入。
个人提交查询 `/api/v1/submissions` 使用 `(created_at,id)` 降序游标,最多100条;
`/{id}`、`/{id}/source` 仅允许提交本人读取,统一拒绝外来/不存在记录。
成绩投影不包含租约、工件摘要、隐藏用例 ID/输入/答案或程序输出;编译诊断至多64KiB,
缺失计量保留 null,未完成报告不展示成绩。身份/提交/内部响应均为 no-store。
`POST /{id}/cancel` 验证本人、Origin、会话和 CSRF,再调用 Coordinator 接受取消;
202仅表示已持久化意图,实际停止与资源回收由执行服务确认。
配置 `KF_OJ_COORDINATOR_URL` 为数值 loopback HTTP origin,
`KF_OJ_COORDINATOR_TOKEN` 与私有 Coordinator 的 `platform_token` 一致;
缺少配置或通信失败返回503,不直接修改执行状态。新增运行依赖 `httpx` 仅负责这个
3秒有界控制调用,禁用代理/重定向,不缓存或打印远端响应及内部凭据。
Python 虚拟环境位于 `backend/.venv`,可在 `backend/` 直接使用 `uv sync --locked` 和 `uv run --locked`。
数据库由 Alembic 唯一管理;业务模型落地后在 `migrations/env.py` 中显式登记元数据。
数据库由 Alembic 唯一管理;业务模型在 `migrations/env.py` 中显式登记元数据。
不要用 API 启动时的 `create_all` 代替迁移,也不要让 Rust 维护另一套迁移历史。
配置示例在 `.env.example`;执行迁移前复制为 `.env` 并设置实际数据库连接。
+23
View File
@@ -0,0 +1,23 @@
# 数据库所有权
Alembic 是唯一的 schema 入口。先对专用 PostgreSQL 18 数据库运行 `uv run --locked alembic
upgrade head`,API 不在启动时创建表。迁移账号拥有该数据库对象并需要 `CREATEROLE`;它不作为
任何运行服务账号。生产配置及登录角色由部署方提供,不在源码里设置默认数据库口令。
初始迁移建立两个受限的 `NOLOGIN` 权限组:`kf_oj_platform` 和 `kf_oj_coordinator`。
平台组创建 submission / queued evaluation / outbox,不能指定或更新执行状态、attempt 或成绩。
协调器组领取和更新评测、插入 attempt / 用例成绩及恢复通知,不能修改账号、题库或提交身份。
Worker 没有数据库角色,也不需要数据库连接配置。
服务使用各自独立的普通 `LOGIN` 角色,分别加入对应权限组;登录角色不能拥有数据库/表、
不能加入迁移账号或其他高权限组,且必须为 `NOSUPERUSER NOCREATEDB NOCREATEROLE NOBYPASSRLS`。
初始迁移拒绝已有的高权限同名组和继承其他角色的同名组。应用启动时不运行 migration。
降级删除本项目业务表与触发器,保留权限组及操作方配置的登录账号,避免删除集群级身份。
发布是一次单向转换:先在事务内创建版本和连续测试点,再设置 `published_at`。
数据库触发器拒绝再次发布、修改已发布版本和追加/改写已发布测试点;评测通过复合外键
永久绑定 submission 的版本,当前 attempt 和用例成绩也由复合外键绑定同一评测及版本。
工件引用具有数据库生成的 `ready` 状态,引用存在时不能转为 `deleting` 或删除元数据。
真实数据库验收使用显式 `KF_OJ_TEST_POSTGRES` 管理连接。每个测试只创建和删除随机命名的
`kf_oj_test_*` 数据库,不清空已有业务库。例行无数据库测试会明确跳过这些验收;跳过不算通过。
+6 -1
View File
@@ -5,14 +5,19 @@ from alembic import context
from sqlalchemy import Connection
from kf_oj.core.settings import Settings
from kf_oj.infra.artifacts import models as artifact_models # noqa: F401
from kf_oj.infra.db.base import Base
from kf_oj.infra.db.session import build_engine
from kf_oj.infra.messaging import models as messaging_models # noqa: F401
from kf_oj.modules.identity import models as identity_models # noqa: F401
from kf_oj.modules.problems import models as problem_models # noqa: F401
from kf_oj.modules.submissions import models as submission_models # noqa: F401
config = context.config
if config.config_file_name is not None:
fileConfig(config.config_file_name)
# Register each business module's ORM models here when its schema is implemented.
# Explicit composition root. Infra and domains do not import foreign ORM models.
target_metadata = Base.metadata
@@ -0,0 +1,588 @@
"""judging foundation and role boundaries
Revision ID: 0001_judging_foundation
Revises:
Create Date: 2026-10-09 00:57:34.004943
"""
from collections.abc import Sequence
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
revision: str = "0001_judging_foundation"
down_revision: str | None = None
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def install_guards_and_permissions() -> None:
# Serialize case insertion against the one-way publication transition.
op.execute("""
CREATE FUNCTION public.kf_immutable_case() RETURNS trigger LANGUAGE plpgsql
SET search_path = pg_catalog, public AS $$
DECLARE published timestamptz;
BEGIN
IF TG_OP = 'UPDATE' THEN
RAISE EXCEPTION 'test cases are immutable' USING ERRCODE = '23514';
END IF;
SELECT published_at INTO published FROM public.problem_versions
WHERE id = CASE WHEN TG_OP = 'DELETE' THEN OLD.version_id ELSE NEW.version_id END
FOR UPDATE;
IF published IS NOT NULL THEN
RAISE EXCEPTION 'published test cases are immutable' USING ERRCODE = '23514';
END IF;
IF TG_OP = 'DELETE' THEN RETURN OLD; END IF;
RETURN NEW;
END $$
""")
op.execute("""
CREATE TRIGGER kf_immutable_case BEFORE INSERT OR UPDATE OR DELETE ON public.problem_cases
FOR EACH ROW EXECUTE FUNCTION public.kf_immutable_case()
""")
op.execute("""
CREATE FUNCTION public.kf_immutable_version() RETURNS trigger LANGUAGE plpgsql
SET search_path = pg_catalog, public AS $$
DECLARE count_cases integer; last_position integer;
BEGIN
IF TG_OP = 'INSERT' THEN
IF NEW.published_at IS NOT NULL THEN
RAISE EXCEPTION 'publish after inserting cases' USING ERRCODE = '23514';
END IF;
RETURN NEW;
END IF;
IF TG_OP = 'DELETE' THEN
IF OLD.published_at IS NOT NULL THEN
RAISE EXCEPTION 'published version is immutable' USING ERRCODE = '23514';
END IF;
RETURN OLD;
END IF;
IF OLD.published_at IS NOT NULL OR NEW.published_at IS NULL
OR (to_jsonb(NEW) - 'published_at') != (to_jsonb(OLD) - 'published_at') THEN
RAISE EXCEPTION 'version may only be published once' USING ERRCODE = '23514';
END IF;
SELECT count(*), max(position) INTO count_cases, last_position
FROM public.problem_cases WHERE version_id = OLD.id;
IF count_cases NOT BETWEEN 1 AND 256 OR last_position != count_cases - 1 THEN
RAISE EXCEPTION 'publication requires contiguous ordered cases'
USING ERRCODE = '23514';
END IF;
RETURN NEW;
END $$
""")
op.execute("""
CREATE TRIGGER kf_immutable_version BEFORE INSERT OR UPDATE OR DELETE
ON public.problem_versions
FOR EACH ROW EXECUTE FUNCTION public.kf_immutable_version()
""")
op.execute("REVOKE ALL ON FUNCTION public.kf_immutable_case() FROM PUBLIC")
op.execute("REVOKE ALL ON FUNCTION public.kf_immutable_version() FROM PUBLIC")
# Migration owner needs CREATEROLE; runtime logins are separate, unprivileged group members.
for role in ("kf_oj_platform", "kf_oj_coordinator"):
op.execute(f"""
DO $$ BEGIN
IF NOT EXISTS (SELECT 1 FROM pg_roles WHERE rolname = '{role}') THEN
CREATE ROLE {role} NOLOGIN NOSUPERUSER NOCREATEDB NOCREATEROLE NOBYPASSRLS;
ELSIF EXISTS (SELECT 1 FROM pg_roles WHERE rolname = '{role}' AND
(rolcanlogin OR rolsuper OR rolcreatedb OR rolcreaterole OR rolbypassrls))
OR EXISTS (SELECT 1 FROM pg_auth_members m JOIN pg_roles r ON r.oid=m.member
WHERE r.rolname='{role}') THEN
RAISE EXCEPTION 'unsafe existing runtime role';
END IF;
END $$
""")
tables = (
"accounts,sessions,login_throttles,artifacts,problems,problem_versions,problem_cases,"
"submissions,evaluations,judge_attempts,case_results,outbox"
)
op.execute(f"REVOKE ALL ON {tables} FROM PUBLIC, kf_oj_platform, kf_oj_coordinator")
op.execute("REVOKE CREATE ON SCHEMA public FROM PUBLIC")
op.execute("GRANT USAGE ON SCHEMA public TO kf_oj_platform, kf_oj_coordinator")
op.execute(f"GRANT SELECT ON {tables} TO kf_oj_platform")
op.execute("GRANT INSERT,UPDATE,DELETE ON accounts,sessions,login_throttles TO kf_oj_platform")
op.execute("GRANT INSERT,SELECT,DELETE ON artifacts TO kf_oj_platform")
op.execute("GRANT UPDATE(state) ON artifacts TO kf_oj_platform")
op.execute(
"GRANT INSERT ON problems,problem_versions,problem_cases,submissions,outbox "
"TO kf_oj_platform"
)
op.execute("GRANT UPDATE(active_version_id,enabled) ON problems TO kf_oj_platform")
op.execute("GRANT UPDATE(published_at) ON problem_versions TO kf_oj_platform")
op.execute("GRANT INSERT(id,submission_id,problem_version_id) ON evaluations TO kf_oj_platform")
op.execute(
"GRANT UPDATE(failures,next_attempt_at,locked_until,lock_owner,delivered_at,"
"last_error_code) ON outbox TO kf_oj_platform"
)
op.execute(
"GRANT SELECT ON problems,problem_versions,problem_cases,artifacts,submissions,"
"evaluations,judge_attempts,case_results,outbox TO kf_oj_coordinator"
)
op.execute(
"GRANT UPDATE(state,fencing_version,attempts_count,current_attempt_id,not_before,"
"cancel_requested_at,updated_at,verdict,failure_code,final_report,final_report_digest) "
"ON evaluations TO kf_oj_coordinator"
)
op.execute("GRANT INSERT ON judge_attempts,case_results TO kf_oj_coordinator")
op.execute(
"GRANT UPDATE(expires_at,revoked_at,sequence,stage,case_index,report,report_digest,"
"cleanup_complete) ON judge_attempts TO kf_oj_coordinator"
)
# Recovery may enqueue the same immutable notification, but cannot change identity fields.
op.execute("GRANT INSERT ON outbox TO kf_oj_coordinator")
op.execute("GRANT UPDATE(next_attempt_at,delivered_at) ON outbox TO kf_oj_coordinator")
def upgrade() -> None:
op.create_table(
"accounts",
sa.Column("id", sa.Uuid(), nullable=False),
sa.Column("username", sa.String(length=24), nullable=False),
sa.Column("password_hash", sa.Text(), nullable=False),
sa.Column("enabled", sa.Boolean(), server_default="true", nullable=False),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.CheckConstraint(
"username ~ '^[a-z0-9_]{3,24}$'", name=op.f("ck_accounts_canonical_username")
),
sa.CheckConstraint(
"char_length(password_hash) BETWEEN 40 AND 256",
name=op.f("ck_accounts_password_hash_size"),
),
sa.PrimaryKeyConstraint("id", name=op.f("pk_accounts")),
sa.UniqueConstraint("username", name=op.f("uq_accounts_username")),
)
op.create_table(
"artifacts",
sa.Column("digest", sa.String(length=64), nullable=False),
sa.Column("bytes", sa.BigInteger(), nullable=False),
sa.Column("state", sa.String(length=8), server_default="ready", nullable=False),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.CheckConstraint("digest ~ '^[0-9a-f]{64}$'", name=op.f("ck_artifacts_digest")),
sa.CheckConstraint("state IN ('ready', 'deleting')", name=op.f("ck_artifacts_state")),
sa.CheckConstraint("bytes BETWEEN 0 AND 67108864", name=op.f("ck_artifacts_bytes")),
sa.PrimaryKeyConstraint("digest", name=op.f("pk_artifacts")),
sa.UniqueConstraint("digest", "state", name=op.f("uq_artifacts_digest")),
)
op.create_table(
"login_throttles",
sa.Column("bucket_hash", sa.String(length=64), nullable=False),
sa.Column("attempts", sa.Integer(), nullable=False),
sa.Column("reset_at", sa.DateTime(timezone=True), nullable=False),
sa.CheckConstraint(
"bucket_hash ~ '^[0-9a-f]{64}$'", name=op.f("ck_login_throttles_bucket_digest")
),
sa.CheckConstraint(
"attempts BETWEEN 1 AND 10", name=op.f("ck_login_throttles_attempt_budget")
),
sa.PrimaryKeyConstraint("bucket_hash", name=op.f("pk_login_throttles")),
)
op.create_index("ix_login_throttles_reset", "login_throttles", ["reset_at"], unique=False)
op.create_table(
"problems",
sa.Column("id", sa.Uuid(), nullable=False),
sa.Column("slug", sa.String(length=64), nullable=False),
sa.Column("active_version_id", sa.Uuid(), nullable=True),
sa.Column("enabled", sa.Boolean(), server_default="true", nullable=False),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.CheckConstraint("slug ~ '^[a-z0-9][a-z0-9_-]{0,63}$'", name=op.f("ck_problems_slug")),
sa.PrimaryKeyConstraint("id", name=op.f("pk_problems")),
sa.UniqueConstraint("slug", name=op.f("uq_problems_slug")),
)
op.create_table(
"problem_versions",
sa.Column("id", sa.Uuid(), nullable=False),
sa.Column("problem_id", sa.Uuid(), nullable=False),
sa.Column("revision", sa.Integer(), nullable=False),
sa.Column("title", sa.String(length=160), nullable=False),
sa.Column("statement", sa.Text(), nullable=False),
sa.Column("languages", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("toolchain", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("compile_limits", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("run_limits", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("job_wall_time_us", sa.BigInteger(), nullable=False),
sa.Column("comparison", sa.String(length=16), server_default="exact_bytes", nullable=False),
sa.Column("public_samples", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("published_at", sa.DateTime(timezone=True), nullable=True),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.CheckConstraint(
"comparison = 'exact_bytes'", name=op.f("ck_problem_versions_comparison")
),
sa.CheckConstraint(
"char_length(title) BETWEEN 1 AND 160", name=op.f("ck_problem_versions_title_size")
),
sa.CheckConstraint(
"job_wall_time_us BETWEEN 1000 AND 3600000000",
name=op.f("ck_problem_versions_job_wall"),
),
sa.CheckConstraint(
"octet_length(statement) BETWEEN 1 AND 262144",
name=op.f("ck_problem_versions_statement_bytes"),
),
sa.CheckConstraint("revision > 0", name=op.f("ck_problem_versions_revision")),
sa.ForeignKeyConstraint(
["problem_id"], ["problems.id"], name=op.f("fk_problem_versions_problem_id_problems")
),
sa.PrimaryKeyConstraint("id", name=op.f("pk_problem_versions")),
sa.UniqueConstraint("id", "problem_id", name=op.f("uq_problem_versions_id")),
sa.UniqueConstraint("problem_id", "revision", name=op.f("uq_problem_versions_problem_id")),
)
op.create_index(
"ix_problem_versions_public",
"problem_versions",
["problem_id", "published_at"],
unique=False,
)
op.create_table(
"sessions",
sa.Column("token_hash", sa.String(length=64), nullable=False),
sa.Column("account_id", sa.Uuid(), nullable=False),
sa.Column("csrf_hash", sa.String(length=64), nullable=False),
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("revoked_at", sa.DateTime(timezone=True), nullable=True),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.CheckConstraint("csrf_hash ~ '^[0-9a-f]{64}$'", name=op.f("ck_sessions_csrf_digest")),
sa.CheckConstraint("token_hash ~ '^[0-9a-f]{64}$'", name=op.f("ck_sessions_token_digest")),
sa.CheckConstraint("expires_at > created_at", name=op.f("ck_sessions_expiry")),
sa.ForeignKeyConstraint(
["account_id"], ["accounts.id"], name=op.f("fk_sessions_account_id_accounts")
),
sa.PrimaryKeyConstraint("token_hash", name=op.f("pk_sessions")),
)
op.create_index(
"ix_sessions_account_expiry", "sessions", ["account_id", "expires_at"], unique=False
)
op.create_table(
"problem_cases",
sa.Column("id", sa.Uuid(), nullable=False),
sa.Column("version_id", sa.Uuid(), nullable=False),
sa.Column("position", sa.Integer(), nullable=False),
sa.Column("input_digest", sa.String(length=64), nullable=False),
sa.Column("expected_digest", sa.String(length=64), nullable=False),
sa.Column(
"artifact_state",
sa.String(length=8),
sa.Computed("'ready'", persisted=True),
nullable=False,
),
sa.CheckConstraint("position BETWEEN 0 AND 255", name=op.f("ck_problem_cases_position")),
sa.ForeignKeyConstraint(
["expected_digest", "artifact_state"],
["artifacts.digest", "artifacts.state"],
name="fk_problem_cases_expected_digest_artifacts",
),
sa.ForeignKeyConstraint(
["input_digest", "artifact_state"],
["artifacts.digest", "artifacts.state"],
name=op.f("fk_problem_cases_input_digest_artifacts"),
),
sa.ForeignKeyConstraint(
["version_id"],
["problem_versions.id"],
name=op.f("fk_problem_cases_version_id_problem_versions"),
),
sa.PrimaryKeyConstraint("id", name=op.f("pk_problem_cases")),
sa.UniqueConstraint("id", "version_id", name=op.f("uq_problem_cases_id")),
sa.UniqueConstraint("version_id", "position", name=op.f("uq_problem_cases_version_id")),
)
op.create_table(
"submissions",
sa.Column("id", sa.Uuid(), nullable=False),
sa.Column("account_id", sa.Uuid(), nullable=False),
sa.Column("problem_id", sa.Uuid(), nullable=False),
sa.Column("problem_version_id", sa.Uuid(), nullable=False),
sa.Column("language", sa.String(length=8), nullable=False),
sa.Column("source_digest", sa.String(length=64), nullable=False),
sa.Column(
"artifact_state",
sa.String(length=8),
sa.Computed("'ready'", persisted=True),
nullable=False,
),
sa.Column("idempotency_key", sa.Uuid(), nullable=False),
sa.Column("request_digest", sa.String(length=64), nullable=False),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.CheckConstraint("language IN ('c17', 'cpp20')", name=op.f("ck_submissions_language")),
sa.CheckConstraint(
"request_digest ~ '^[0-9a-f]{64}$'", name=op.f("ck_submissions_request_digest")
),
sa.ForeignKeyConstraint(
["account_id"], ["accounts.id"], name=op.f("fk_submissions_account_id_accounts")
),
sa.ForeignKeyConstraint(
["problem_version_id", "problem_id"],
["problem_versions.id", "problem_versions.problem_id"],
name=op.f("fk_submissions_problem_version_id_problem_versions"),
),
sa.ForeignKeyConstraint(
["source_digest", "artifact_state"],
["artifacts.digest", "artifacts.state"],
name=op.f("fk_submissions_source_digest_artifacts"),
),
sa.PrimaryKeyConstraint("id", name=op.f("pk_submissions")),
sa.UniqueConstraint(
"account_id", "idempotency_key", name=op.f("uq_submissions_account_id")
),
sa.UniqueConstraint("id", "problem_version_id", name=op.f("uq_submissions_id")),
)
op.create_index(
"ix_submissions_account_history",
"submissions",
["account_id", "created_at", "id"],
unique=False,
)
op.create_table(
"evaluations",
sa.Column("id", sa.Uuid(), nullable=False),
sa.Column("submission_id", sa.Uuid(), nullable=False),
sa.Column("problem_version_id", sa.Uuid(), nullable=False),
sa.Column("state", sa.String(length=16), server_default="queued", nullable=False),
sa.Column("fencing_version", sa.BigInteger(), server_default="0", nullable=False),
sa.Column("attempts_count", sa.Integer(), server_default="0", nullable=False),
sa.Column("max_attempts", sa.Integer(), server_default="3", nullable=False),
sa.Column("current_attempt_id", sa.Uuid(), nullable=True),
sa.Column(
"not_before",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.Column("cancel_requested_at", sa.DateTime(timezone=True), nullable=True),
sa.Column(
"updated_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.Column("verdict", sa.String(length=32), nullable=True),
sa.Column("failure_code", sa.String(length=32), nullable=True),
sa.Column("final_report", postgresql.JSONB(astext_type=sa.Text()), nullable=True),
sa.Column("final_report_digest", sa.String(length=64), nullable=True),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.CheckConstraint(
"state IN ('queued','claimed','compiling','running','cancelling',"
"'finished','cancelled','failed')",
name=op.f("ck_evaluations_state"),
),
sa.CheckConstraint(
"state NOT IN ('claimed','compiling','running','cancelling') "
"OR current_attempt_id IS NOT NULL",
name=op.f("ck_evaluations_active_attempt"),
),
sa.CheckConstraint(
"attempts_count BETWEEN 0 AND max_attempts", name=op.f("ck_evaluations_attempts_count")
),
sa.CheckConstraint("fencing_version >= 0", name=op.f("ck_evaluations_fence")),
sa.CheckConstraint(
"max_attempts BETWEEN 1 AND 5", name=op.f("ck_evaluations_max_attempts")
),
sa.ForeignKeyConstraint(
["submission_id", "problem_version_id"],
["submissions.id", "submissions.problem_version_id"],
name=op.f("fk_evaluations_submission_id_submissions"),
),
sa.PrimaryKeyConstraint("id", name=op.f("pk_evaluations")),
sa.UniqueConstraint("id", "problem_version_id", name=op.f("uq_evaluations_id")),
sa.UniqueConstraint("submission_id", name=op.f("uq_evaluations_submission_id")),
)
op.create_index(
"ix_evaluations_recovery", "evaluations", ["state", "not_before", "id"], unique=False
)
op.create_table(
"judge_attempts",
sa.Column("id", sa.Uuid(), nullable=False),
sa.Column("evaluation_id", sa.Uuid(), nullable=False),
sa.Column("worker_id", sa.Uuid(), nullable=False),
sa.Column("number", sa.Integer(), nullable=False),
sa.Column("fencing_version", sa.BigInteger(), nullable=False),
sa.Column("lease_token_hash", sa.String(length=64), nullable=False),
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("revoked_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("manifest", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("manifest_digest", sa.String(length=64), nullable=False),
sa.Column("sequence", sa.BigInteger(), server_default="0", nullable=False),
sa.Column("stage", sa.String(length=16), nullable=True),
sa.Column("case_index", sa.Integer(), nullable=True),
sa.Column("report", postgresql.JSONB(astext_type=sa.Text()), nullable=True),
sa.Column("report_digest", sa.String(length=64), nullable=True),
sa.Column("cleanup_complete", sa.Boolean(), nullable=True),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.CheckConstraint(
"lease_token_hash ~ '^[0-9a-f]{64}$'", name=op.f("ck_judge_attempts_token_digest")
),
sa.CheckConstraint(
"manifest_digest ~ '^[0-9a-f]{64}$'", name=op.f("ck_judge_attempts_manifest_digest")
),
sa.CheckConstraint("fencing_version > 0", name=op.f("ck_judge_attempts_fence")),
sa.CheckConstraint("number BETWEEN 1 AND 5", name=op.f("ck_judge_attempts_number")),
sa.CheckConstraint("sequence >= 0", name=op.f("ck_judge_attempts_sequence")),
sa.ForeignKeyConstraint(
["evaluation_id"],
["evaluations.id"],
name=op.f("fk_judge_attempts_evaluation_id_evaluations"),
),
sa.PrimaryKeyConstraint("id", name=op.f("pk_judge_attempts")),
sa.UniqueConstraint(
"evaluation_id", "fencing_version", name="uq_judge_attempts_evaluation_fence"
),
sa.UniqueConstraint("evaluation_id", "number", name="uq_judge_attempts_evaluation_number"),
sa.UniqueConstraint("id", "evaluation_id", name=op.f("uq_judge_attempts_id")),
)
op.create_index(
"ix_judge_attempts_expiry", "judge_attempts", ["expires_at", "evaluation_id"], unique=False
)
op.create_table(
"outbox",
sa.Column("id", sa.Uuid(), nullable=False),
sa.Column("evaluation_id", sa.Uuid(), nullable=False),
sa.Column("kind", sa.String(length=32), server_default="judge.evaluation", nullable=False),
sa.Column("failures", sa.Integer(), server_default="0", nullable=False),
sa.Column(
"next_attempt_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.Column("locked_until", sa.DateTime(timezone=True), nullable=True),
sa.Column("lock_owner", sa.Uuid(), nullable=True),
sa.Column("delivered_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("last_error_code", sa.String(length=32), nullable=True),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.CheckConstraint("kind = 'judge.evaluation'", name=op.f("ck_outbox_kind")),
sa.CheckConstraint("failures BETWEEN 0 AND 100", name=op.f("ck_outbox_failure_budget")),
sa.ForeignKeyConstraint(
["evaluation_id"], ["evaluations.id"], name=op.f("fk_outbox_evaluation_id_evaluations")
),
sa.PrimaryKeyConstraint("id", name=op.f("pk_outbox")),
sa.UniqueConstraint("evaluation_id", name=op.f("uq_outbox_evaluation_id")),
)
op.create_index(
"ix_outbox_delivery", "outbox", ["delivered_at", "next_attempt_at", "id"], unique=False
)
op.create_table(
"case_results",
sa.Column("attempt_id", sa.Uuid(), nullable=False),
sa.Column("case_id", sa.Uuid(), nullable=False),
sa.Column("evaluation_id", sa.Uuid(), nullable=False),
sa.Column("problem_version_id", sa.Uuid(), nullable=False),
sa.Column("position", sa.Integer(), nullable=False),
sa.Column("verdict", sa.String(length=32), nullable=False),
sa.Column("cpu_time_us", sa.BigInteger(), nullable=True),
sa.Column("memory_peak_bytes", sa.BigInteger(), nullable=True),
sa.CheckConstraint(
"cpu_time_us IS NULL OR cpu_time_us >= 0", name=op.f("ck_case_results_cpu")
),
sa.CheckConstraint(
"memory_peak_bytes IS NULL OR memory_peak_bytes >= 0",
name=op.f("ck_case_results_memory"),
),
sa.CheckConstraint("position BETWEEN 0 AND 255", name=op.f("ck_case_results_position")),
sa.ForeignKeyConstraint(
["attempt_id", "evaluation_id"],
["judge_attempts.id", "judge_attempts.evaluation_id"],
name=op.f("fk_case_results_attempt_id_judge_attempts"),
),
sa.ForeignKeyConstraint(
["case_id", "problem_version_id"],
["problem_cases.id", "problem_cases.version_id"],
name=op.f("fk_case_results_case_id_problem_cases"),
),
sa.ForeignKeyConstraint(
["evaluation_id", "problem_version_id"],
["evaluations.id", "evaluations.problem_version_id"],
name=op.f("fk_case_results_evaluation_id_evaluations"),
),
sa.PrimaryKeyConstraint("attempt_id", "case_id", name=op.f("pk_case_results")),
sa.UniqueConstraint("attempt_id", "position", name=op.f("uq_case_results_attempt_id")),
)
op.create_foreign_key(
"fk_problems_active_version",
"problems",
"problem_versions",
["active_version_id", "id"],
["id", "problem_id"],
deferrable=True,
initially="DEFERRED",
)
op.create_foreign_key(
"fk_evaluations_current_attempt",
"evaluations",
"judge_attempts",
["current_attempt_id", "id"],
["id", "evaluation_id"],
deferrable=True,
initially="DEFERRED",
)
install_guards_and_permissions()
def downgrade() -> None:
op.drop_constraint("fk_evaluations_current_attempt", "evaluations", type_="foreignkey")
op.drop_constraint("fk_problems_active_version", "problems", type_="foreignkey")
op.drop_table("case_results")
op.drop_index("ix_outbox_delivery", table_name="outbox")
op.drop_table("outbox")
op.drop_index("ix_judge_attempts_expiry", table_name="judge_attempts")
op.drop_table("judge_attempts")
op.drop_index("ix_evaluations_recovery", table_name="evaluations")
op.drop_table("evaluations")
op.drop_index("ix_submissions_account_history", table_name="submissions")
op.drop_table("submissions")
op.drop_table("problem_cases")
op.drop_index("ix_sessions_account_expiry", table_name="sessions")
op.drop_table("sessions")
op.drop_index("ix_problem_versions_public", table_name="problem_versions")
op.drop_table("problem_versions")
op.drop_table("problems")
op.drop_index("ix_login_throttles_reset", table_name="login_throttles")
op.drop_table("login_throttles")
op.drop_table("artifacts")
op.drop_table("accounts")
op.execute("DROP FUNCTION public.kf_immutable_version()")
op.execute("DROP FUNCTION public.kf_immutable_case()")
# NOLOGIN groups are intentionally retained: a downgrade never deletes operator roles.
@@ -0,0 +1,247 @@
"""Immutable per-language judging configuration and extended submission language IDs.
Revision ID: 0002_multi_runtime
Revises: 0001_judging_foundation
"""
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects.postgresql import JSONB
revision = "0002_multi_runtime"
down_revision = "0001_judging_foundation"
branch_labels = None
depends_on = None
CONFIGURATION_CHECK = """
(judging_schema_version = 1 AND language_configurations IS NULL
AND toolchain IS NOT NULL AND compile_limits IS NOT NULL
AND run_limits IS NOT NULL AND job_wall_time_us IS NOT NULL)
OR
(judging_schema_version = 2 AND language_configurations IS NOT NULL
AND toolchain IS NULL AND compile_limits IS NULL
AND run_limits IS NULL AND job_wall_time_us IS NULL
AND public.kf_valid_language_configurations(languages, language_configurations))
"""
def upgrade() -> None:
# Metadata-only additions/widening; do not UPDATE any immutable published rows.
op.add_column(
"problem_versions",
sa.Column("judging_schema_version", sa.SmallInteger(), nullable=False, server_default="1"),
)
op.add_column(
"problem_versions",
sa.Column("language_configurations", JSONB(none_as_null=True), nullable=True),
)
for column, kind in (
("toolchain", JSONB()),
("compile_limits", JSONB()),
("run_limits", JSONB()),
("job_wall_time_us", sa.BigInteger()),
):
op.alter_column("problem_versions", column, existing_type=kind, nullable=True)
op.alter_column(
"submissions",
"language",
existing_type=sa.String(8),
type_=sa.String(16),
existing_nullable=False,
)
op.drop_constraint(op.f("ck_submissions_language"), "submissions", type_="check")
op.create_check_constraint(
op.f("ck_submissions_language"),
"submissions",
"language IN ('c17','cpp20','python313','java21')",
)
op.execute("""
CREATE FUNCTION public.kf_valid_runtime_limits(limits jsonb)
RETURNS boolean LANGUAGE plpgsql IMMUTABLE STRICT
SET search_path = pg_catalog, public AS $$
DECLARE entry record;
BEGIN
IF jsonb_typeof(limits) != 'object' OR
(SELECT count(*) FROM jsonb_object_keys(limits)) != 10 THEN RETURN false; END IF;
FOR entry IN SELECT * FROM (VALUES
('cpu_time_us',1000::numeric,60000000::numeric),
('wall_time_us',1000,120000000), ('memory_bytes',1,2147483648),
('stack_bytes',1,268435456), ('processes',1,128), ('open_files',3,1024),
('stdout_bytes',1,16777216), ('stderr_bytes',1,16777216),
('file_bytes',1,67108864), ('workdir_bytes',1,268435456)
) AS bounds(name,lo,hi) LOOP
IF NOT limits ? entry.name OR jsonb_typeof(limits->entry.name) != 'number'
OR (limits->>entry.name) !~ '^(0|[1-9][0-9]*)$'
OR length(limits->>entry.name) > 19 THEN RETURN false; END IF;
IF (limits->>entry.name)::numeric NOT BETWEEN entry.lo AND entry.hi
THEN RETURN false; END IF;
END LOOP;
RETURN (limits->>'stack_bytes')::numeric <= (limits->>'memory_bytes')::numeric
AND (limits->>'file_bytes')::numeric <= (limits->>'workdir_bytes')::numeric
AND (limits->>'stdout_bytes')::numeric <= (limits->>'file_bytes')::numeric
AND (limits->>'stderr_bytes')::numeric <= (limits->>'file_bytes')::numeric
AND (limits->>'stdout_bytes')::numeric + (limits->>'stderr_bytes')::numeric
<= (limits->>'workdir_bytes')::numeric;
END $$
""")
op.execute("""
CREATE FUNCTION public.kf_valid_language_configurations(languages jsonb, configs jsonb)
RETURNS boolean LANGUAGE plpgsql IMMUTABLE STRICT
SET search_path = pg_catalog, public AS $$
DECLARE item record; cfg jsonb; runtime jsonb; lang text; seen text[] := ARRAY[]::text[];
phase jsonb; preparing boolean;
BEGIN
IF jsonb_typeof(languages) != 'array' OR jsonb_typeof(configs) != 'array'
THEN RETURN false; END IF;
IF jsonb_array_length(configs) NOT BETWEEN 1 AND 4 OR
jsonb_array_length(languages) != jsonb_array_length(configs)
THEN RETURN false; END IF;
FOR item IN SELECT value, ordinality FROM jsonb_array_elements(configs)
WITH ORDINALITY LOOP
cfg := item.value;
IF jsonb_typeof(cfg) != 'object' OR
(SELECT count(*) FROM jsonb_object_keys(cfg)) != 5 OR NOT cfg ?&
ARRAY['language','toolchain','compile_limits','run_limits','job_wall_time_us']
THEN RETURN false; END IF;
IF jsonb_typeof(cfg->'language') != 'string' THEN RETURN false; END IF;
lang := cfg->>'language';
IF lang NOT IN ('c17','cpp20','python313','java21') OR lang = ANY(seen)
OR languages->(item.ordinality::int-1) != cfg->'language'
THEN RETURN false; END IF;
seen := array_append(seen,lang); runtime := cfg->'toolchain';
IF jsonb_typeof(runtime) != 'object' OR
(SELECT count(*) FROM jsonb_object_keys(runtime)) != 5 OR NOT runtime ?&
ARRAY['toolchain_id','image_kind','image_revision','image_digest','runtime_version']
THEN RETURN false; END IF;
IF jsonb_typeof(runtime->'toolchain_id') != 'string' OR
jsonb_typeof(runtime->'image_kind') != 'string' OR
jsonb_typeof(runtime->'runtime_version') != 'string' OR
jsonb_typeof(runtime->'image_digest') != 'string' OR
(runtime->>'image_digest') !~ '^[0-9a-f]{64}$' OR
(runtime->>'runtime_version') !~ '^[A-Za-z0-9][A-Za-z0-9_.+-]{0,63}$' OR
jsonb_typeof(runtime->'image_revision') != 'number' OR
(runtime->>'image_revision') !~ '^[1-9][0-9]{0,9}$'
THEN RETURN false; END IF;
IF (runtime->>'image_revision')::numeric > 4294967295 THEN RETURN false; END IF;
IF lang IN ('c17','cpp20') AND (runtime->>'toolchain_id' != 'gnu-cxx-v1' OR
runtime->>'image_kind' != 'linux_gnu_cxx_x86_64_v1') THEN RETURN false; END IF;
IF lang = 'python313' AND (runtime->>'toolchain_id' != 'cpython313-v1' OR
runtime->>'image_kind' != 'linux_cpython313_x86_64_v1' OR
runtime->>'runtime_version' != '3.13.16') THEN RETURN false; END IF;
IF lang = 'java21' AND (runtime->>'toolchain_id' != 'openjdk21-v1' OR
runtime->>'image_kind' != 'linux_openjdk21_x86_64_v1' OR
(runtime->>'runtime_version') !~
'^21[.][0-9]+[.][0-9]+([.][0-9]+)?[+][1-9][0-9]*(-LTS)?$')
THEN RETURN false; END IF;
IF NOT public.kf_valid_runtime_limits(cfg->'compile_limits') OR
NOT public.kf_valid_runtime_limits(cfg->'run_limits') THEN RETURN false; END IF;
IF jsonb_typeof(cfg->'job_wall_time_us') != 'number' OR
(cfg->>'job_wall_time_us') !~ '^[1-9][0-9]{0,9}$' THEN RETURN false; END IF;
IF (cfg->>'job_wall_time_us')::numeric NOT BETWEEN 1000 AND 3600000000 OR
(cfg->'compile_limits'->>'wall_time_us')::numeric +
(cfg->'run_limits'->>'wall_time_us')::numeric >
(cfg->>'job_wall_time_us')::numeric
THEN RETURN false; END IF;
FOR preparing, phase IN SELECT true,cfg->'compile_limits' UNION ALL
SELECT false,cfg->'run_limits' LOOP
IF lang = 'python313' AND ((phase->>'cpu_time_us')::numeric < 10000 OR
(phase->>'wall_time_us')::numeric < 50000 OR
(phase->>'memory_bytes')::numeric < 33554432 OR
(phase->>'open_files')::numeric < 16) THEN RETURN false; END IF;
IF lang = 'java21' AND ((phase->>'cpu_time_us')::numeric <
CASE WHEN preparing THEN 1000000 ELSE 200000 END OR
(phase->>'wall_time_us')::numeric <
CASE WHEN preparing THEN 3000000 ELSE 1000000 END OR
(phase->>'memory_bytes')::numeric <
CASE WHEN preparing THEN 402653184 ELSE 268435456 END OR
(phase->>'processes')::numeric < 16 OR
(phase->>'open_files')::numeric < 32 OR
(phase->>'stack_bytes')::numeric < 262144 OR
(phase->>'memory_bytes')::numeric - 167772160 -
(phase->>'processes')::numeric * 262144 < 33554432)
THEN RETURN false; END IF;
END LOOP;
END LOOP;
RETURN true;
END $$
""")
op.execute("REVOKE ALL ON FUNCTION public.kf_valid_runtime_limits(jsonb) FROM PUBLIC")
op.execute(
"REVOKE ALL ON FUNCTION public.kf_valid_language_configurations(jsonb,jsonb) FROM PUBLIC"
)
op.execute("GRANT EXECUTE ON FUNCTION public.kf_valid_runtime_limits(jsonb) TO kf_oj_platform")
op.execute(
"GRANT EXECUTE ON FUNCTION public.kf_valid_language_configurations(jsonb,jsonb) "
"TO kf_oj_platform"
)
op.create_check_constraint(
op.f("ck_problem_versions_judging_schema"),
"problem_versions",
"judging_schema_version IN (1,2)",
)
op.create_check_constraint(
op.f("ck_problem_versions_language_configurations"), "problem_versions", CONFIGURATION_CHECK
)
op.execute("""
CREATE FUNCTION public.kf_submission_language() RETURNS trigger LANGUAGE plpgsql
SET search_path = pg_catalog, public AS $$
DECLARE supported jsonb;
BEGIN
SELECT languages INTO supported FROM public.problem_versions
WHERE id=NEW.problem_version_id;
IF FOUND AND NOT supported @> jsonb_build_array(NEW.language) THEN
RAISE EXCEPTION 'submission language not supported by version'
USING ERRCODE = '23514';
END IF;
RETURN NEW;
END $$
""")
op.execute("REVOKE ALL ON FUNCTION public.kf_submission_language() FROM PUBLIC")
op.execute("""
CREATE TRIGGER kf_submission_language BEFORE INSERT OR UPDATE OF
language,problem_version_id ON public.submissions
FOR EACH ROW EXECUTE FUNCTION public.kf_submission_language()
""")
def downgrade() -> None:
# Losing a runtime configuration is not a rollback strategy. Preserve all user data.
op.execute("""
DO $$ BEGIN
IF EXISTS (SELECT 1 FROM problem_versions WHERE judging_schema_version = 2) OR
EXISTS (SELECT 1 FROM submissions WHERE language NOT IN ('c17','cpp20')) THEN
RAISE EXCEPTION 'cannot downgrade while multi-runtime data exists'
USING ERRCODE = '23514';
END IF;
END $$
""")
op.drop_constraint(
op.f("ck_problem_versions_language_configurations"), "problem_versions", type_="check"
)
op.drop_constraint(
op.f("ck_problem_versions_judging_schema"), "problem_versions", type_="check"
)
op.execute("DROP TRIGGER kf_submission_language ON public.submissions")
op.execute("DROP FUNCTION public.kf_submission_language()")
op.execute("DROP FUNCTION public.kf_valid_language_configurations(jsonb,jsonb)")
op.execute("DROP FUNCTION public.kf_valid_runtime_limits(jsonb)")
op.drop_column("problem_versions", "language_configurations")
op.drop_column("problem_versions", "judging_schema_version")
for column, kind in (
("toolchain", JSONB()),
("compile_limits", JSONB()),
("run_limits", JSONB()),
("job_wall_time_us", sa.BigInteger()),
):
op.alter_column("problem_versions", column, existing_type=kind, nullable=False)
op.drop_constraint(op.f("ck_submissions_language"), "submissions", type_="check")
op.create_check_constraint(
op.f("ck_submissions_language"), "submissions", "language IN ('c17','cpp20')"
)
op.alter_column(
"submissions",
"language",
existing_type=sa.String(16),
type_=sa.String(8),
existing_nullable=False,
)
+5 -1
View File
@@ -17,9 +17,14 @@ dependencies = [
"asyncpg>=0.30,<1",
"alembic>=1.15,<2",
"aio-pika>=9.5,<10",
"argon2-cffi>=25.1,<26",
"httpx>=0.28,<1",
]
[project.scripts]
kf-accounts = "kf_oj.modules.identity.cli:main"
kf-problems = "kf_oj.modules.problems.cli:main"
kf-artifacts = "kf_oj.infra.artifacts.cli:main"
kf-learning-worker = "kf_oj.workers.learning:main"
kf-outbox-publisher = "kf_oj.workers.outbox:main"
@@ -27,7 +32,6 @@ kf-outbox-publisher = "kf_oj.workers.outbox:main"
dev = [
"ruff>=0.11,<1",
"pytest>=8.3,<10",
"httpx>=0.28,<1",
"jsonschema>=4.23,<5",
]
+79 -1
View File
@@ -2,11 +2,27 @@ import logging
from contextlib import asynccontextmanager
from fastapi import FastAPI
from fastapi.exceptions import RequestValidationError
from starlette.responses import JSONResponse
from kf_oj import __version__
from kf_oj.api.body_limits import BodyLimitMiddleware
from kf_oj.api.v1.identity import router as identity_router
from kf_oj.api.v1.problems import router as problems_router
from kf_oj.api.v1.router import router
from kf_oj.api.v1.submissions import router as submission_router
from kf_oj.api.v1.worker_artifacts import router as worker_router
from kf_oj.core.logging import configure_logging
from kf_oj.core.settings import Settings
from kf_oj.infra.artifacts.repository import ArtifactRepository
from kf_oj.infra.artifacts.store import ArtifactStore
from kf_oj.infra.coordinator import CoordinatorControl
from kf_oj.infra.db.session import build_engine, build_session_factory
from kf_oj.modules.identity.service import IdentityService
from kf_oj.modules.problems.service import ProblemService
from kf_oj.modules.submissions.queries import SubmissionQueries
from kf_oj.modules.submissions.service import SubmissionService
from kf_oj.modules.submissions.worker_access import WorkerAccess
def create_app(settings: Settings | None = None) -> FastAPI:
@@ -17,20 +33,82 @@ def create_app(settings: Settings | None = None) -> FastAPI:
configure_logging(settings.log_level)
logger = logging.getLogger("kf_oj.api")
logger.info("Platform API started")
engine = None
store = None
try:
if settings.database_url is not None:
engine = build_engine(settings)
app.state.sessions = build_session_factory(engine)
app.state.identity = IdentityService(
app.state.sessions, lifetime_seconds=settings.session_lifetime_seconds
)
artifacts = None
if settings.artifact_root is not None:
store = ArtifactStore(
settings.artifact_root,
max_bytes=settings.artifact_max_bytes,
max_files=settings.artifact_max_files,
)
artifacts = ArtifactRepository(store)
app.state.problems = ProblemService(
app.state.sessions, artifacts, settings.judge_admission
)
if artifacts is not None:
app.state.submissions = SubmissionService(
app.state.sessions, app.state.problems, artifacts, settings.judge_admission
)
app.state.worker_access = WorkerAccess(
app.state.sessions, app.state.problems, artifacts
)
cancellation = (
CoordinatorControl(
settings.coordinator_url, settings.coordinator_token.get_secret_value()
)
if settings.coordinator_url and settings.coordinator_token
else None
)
app.state.submission_queries = SubmissionQueries(
app.state.sessions, artifacts, cancellation
)
yield
finally:
app.state.identity = None
app.state.problems = None
app.state.worker_access = None
app.state.submissions = None
app.state.submission_queries = None
if engine is not None:
await engine.dispose()
if store is not None:
store.close()
logger.info("Platform API stopped")
app = FastAPI(
title="KF-OnlineJudge Platform API",
version=__version__,
description="工程骨架。当前只提供进程存活检查,业务接口逐项实现。",
description="平台 API。身份使用真实可撤销会话,业务接口按模块逐项接入。",
docs_url="/api/docs" if settings.environment != "production" else None,
redoc_url=None,
openapi_url="/api/openapi.json" if settings.environment != "production" else None,
lifespan=lifespan,
)
app.state.settings = settings
app.add_middleware(BodyLimitMiddleware)
@app.middleware("http")
async def private_responses(request, call_next):
response = await call_next(request)
if request.url.path.startswith(("/api/v1/auth", "/api/v1/submissions", "/api/v1/internal")):
response.headers["Cache-Control"] = "no-store"
return response
@app.exception_handler(RequestValidationError)
async def invalid_request(_, __):
return JSONResponse({"detail": "invalid request"}, status_code=422)
app.include_router(router, prefix="/api/v1")
app.include_router(identity_router, prefix="/api/v1")
app.include_router(problems_router, prefix="/api/v1")
app.include_router(worker_router, prefix="/api/v1")
app.include_router(submission_router, prefix="/api/v1")
return app
+66
View File
@@ -0,0 +1,66 @@
"""Bound request bytes and upload time before schema parsing can allocate large inputs."""
import asyncio
from starlette.responses import JSONResponse
class BodyLimitMiddleware:
def __init__(self, app):
self.app = app
async def __call__(self, scope, receive, send):
if scope["type"] != "http" or scope["method"] in {"GET", "HEAD", "OPTIONS"}:
await self.app(scope, receive, send)
return
limit = 8192 if scope["path"] == "/api/v1/auth/login" else 1024 * 1024
if scope["path"] == "/api/v1/submissions":
# Max256KiB UTF-8 source, worst-case JSON escaping plus bounded metadata.
limit = 6 * 256 * 1024 + 8192
headers = dict(scope.get("headers", []))
length = headers.get(b"content-length")
if length is not None:
try:
size = int(length)
if size < 0 or size > limit:
raise ValueError
except ValueError:
await JSONResponse({"detail": "request body outside byte policy"}, 413)(
scope, receive, send
)
return
# Buffer a finite body so errors can be returned before response headers are sent.
chunks, size, received_chunks = [], 0, 0
try:
async with asyncio.timeout(10):
while True:
message = await receive()
if message["type"] == "http.disconnect":
return
body = message.get("body", b"")
size += len(body)
received_chunks += 1
if size > limit or received_chunks > 1024:
await JSONResponse({"detail": "request body outside byte policy"}, 413)(
scope, receive, send
)
return
chunks.append(body)
if not message.get("more_body", False):
break
except TimeoutError:
await JSONResponse({"detail": "request body deadline exceeded"}, 408)(
scope, receive, send
)
return
body = b"".join(chunks)
delivered = False
async def bounded_receive():
nonlocal delivered
if not delivered:
delivered = True
return {"type": "http.request", "body": body, "more_body": False}
return await receive()
await self.app(scope, bounded_receive, send)
+50
View File
@@ -0,0 +1,50 @@
from typing import Annotated
from fastapi import Depends, HTTPException, Request
from kf_oj.core.settings import Settings
from kf_oj.modules.identity.public import AuthenticationFailed, Principal
from kf_oj.modules.identity.service import IdentityService
SESSION_COOKIE = "kf_session"
def identity_service(request: Request) -> IdentityService:
service = getattr(request.app.state, "identity", None)
if service is None:
raise HTTPException(503, "identity service unavailable")
return service
def trusted_origin(request: Request) -> None:
settings: Settings = request.app.state.settings
if request.headers.get("origin") not in settings.browser_origins:
raise HTTPException(403, "request origin rejected")
if request.headers.get("sec-fetch-site") == "cross-site":
raise HTTPException(403, "cross-site request rejected")
async def current_principal(
request: Request, identity: Annotated[IdentityService, Depends(identity_service)]
) -> Principal:
try:
return await identity.authenticate(request.cookies.get(SESSION_COOKIE))
except AuthenticationFailed:
raise HTTPException(401, "authentication required") from None
async def unsafe_principal(
request: Request, identity: Annotated[IdentityService, Depends(identity_service)]
) -> Principal:
trusted_origin(request)
try:
await identity.authenticate(request.cookies.get(SESSION_COOKIE))
except AuthenticationFailed:
raise HTTPException(401, "authentication required") from None
csrf = request.headers.get("x-csrf-token")
if csrf is None:
raise HTTPException(403, "request token required")
try:
return await identity.authenticate(request.cookies.get(SESSION_COOKIE), csrf_token=csrf)
except AuthenticationFailed:
raise HTTPException(403, "request authentication rejected") from None
+95
View File
@@ -0,0 +1,95 @@
from typing import Annotated
from fastapi import APIRouter, Depends, HTTPException, Request, Response
from kf_oj.api.dependencies import (
SESSION_COOKIE,
current_principal,
identity_service,
trusted_origin,
unsafe_principal,
)
from kf_oj.contracts.identity import CurrentUser, LoginRequest, LoginResponse
from kf_oj.modules.identity.public import AuthenticationFailed, LoginThrottled, Principal
from kf_oj.modules.identity.service import IdentityService
router = APIRouter(prefix="/auth", tags=["identity"])
@router.post("/login", response_model=LoginResponse, dependencies=[Depends(trusted_origin)])
async def login(
body: LoginRequest,
request: Request,
response: Response,
identity: Annotated[IdentityService, Depends(identity_service)],
) -> LoginResponse:
if request.headers.get("content-type", "").split(";", 1)[0] != "application/json":
raise HTTPException(415, "JSON request required")
try:
issued = await identity.login(
body.username,
body.password.get_secret_value(),
peer=request.client.host if request.client else "",
)
except LoginThrottled:
raise HTTPException(
429, "login temporarily limited", headers={"Retry-After": "900"}
) from None
except (AuthenticationFailed, ValueError):
raise HTTPException(401, "invalid credentials") from None
settings = request.app.state.settings
response.set_cookie(
SESSION_COOKIE,
issued.token,
max_age=settings.session_lifetime_seconds,
httponly=True,
secure=settings.environment == "production",
samesite="strict",
path="/api/v1",
)
response.set_cookie(
"kf_csrf",
issued.csrf_token,
max_age=settings.session_lifetime_seconds,
httponly=False,
secure=settings.environment == "production",
samesite="strict",
path="/",
)
response.headers["Cache-Control"] = "no-store"
return LoginResponse(
user=CurrentUser(id=issued.principal.account_id, username=issued.principal.username),
expires_at=issued.expires_at,
csrf_token=issued.csrf_token,
)
@router.get("/me", response_model=CurrentUser)
async def me(response: Response, principal: Annotated[Principal, Depends(current_principal)]):
response.headers["Cache-Control"] = "no-store"
return CurrentUser(id=principal.account_id, username=principal.username)
@router.post("/logout", status_code=204, dependencies=[Depends(unsafe_principal)])
async def logout(
request: Request, identity: Annotated[IdentityService, Depends(identity_service)]
) -> Response:
try:
await identity.logout(request.cookies[SESSION_COOKIE], request.headers["x-csrf-token"])
except AuthenticationFailed:
raise HTTPException(401, "authentication required") from None
response = Response(status_code=204, headers={"Cache-Control": "no-store"})
response.delete_cookie(
SESSION_COOKIE,
path="/api/v1",
httponly=True,
secure=request.app.state.settings.environment == "production",
samesite="strict",
)
response.delete_cookie(
"kf_csrf",
path="/",
secure=request.app.state.settings.environment == "production",
samesite="strict",
)
return response
+34
View File
@@ -0,0 +1,34 @@
from typing import Annotated
from uuid import UUID
from fastapi import APIRouter, HTTPException, Query, Request
from kf_oj.contracts.problems import ProblemDetail, ProblemPage
from kf_oj.modules.problems.public import ProblemUnavailable
from kf_oj.modules.problems.service import ProblemService
router = APIRouter(prefix="/problems", tags=["problems"])
def service(request: Request) -> ProblemService:
component = getattr(request.app.state, "problems", None)
if component is None:
raise HTTPException(503, "problem service unavailable")
return component
@router.get("", response_model=ProblemPage)
async def list_problems(
request: Request,
after: Annotated[str | None, Query(pattern=r"^[a-z0-9][a-z0-9_-]{0,63}$")] = None,
limit: Annotated[int, Query(ge=1, le=100)] = 20,
) -> ProblemPage:
return await service(request).list(after=after, limit=limit)
@router.get("/versions/{version_id}", response_model=ProblemDetail)
async def problem_version(version_id: UUID, request: Request) -> ProblemDetail:
try:
return await service(request).detail(version_id)
except ProblemUnavailable:
raise HTTPException(404, "problem version unavailable") from None
+119
View File
@@ -0,0 +1,119 @@
from typing import Annotated
from uuid import UUID
from fastapi import APIRouter, Depends, HTTPException, Query, Request, Response
from kf_oj.api.dependencies import current_principal, unsafe_principal
from kf_oj.contracts.submissions import (
CancellationAccepted,
CreateSubmission,
SubmissionCreated,
SubmissionDetail,
SubmissionPage,
SubmissionSource,
)
from kf_oj.infra.artifacts.store import ArtifactError
from kf_oj.modules.identity.public import Principal
from kf_oj.modules.submissions.public import (
CancelUnavailable,
IdempotencyConflict,
SubmissionBusy,
SubmissionUnavailable,
)
router = APIRouter(prefix="/submissions", tags=["submissions"])
@router.post("", status_code=201, response_model=SubmissionCreated)
async def create(
body: CreateSubmission,
request: Request,
response: Response,
principal: Annotated[Principal, Depends(unsafe_principal)],
):
if request.headers.get("content-type", "").split(";", 1)[0] != "application/json":
raise HTTPException(415, "JSON request required")
service = getattr(request.app.state, "submissions", None)
if service is None:
raise HTTPException(503, "submission service unavailable")
try:
result = await service.create(principal, body)
except IdempotencyConflict:
raise HTTPException(409, "idempotency request conflict") from None
except SubmissionBusy:
raise HTTPException(429, "pending submission capacity exhausted") from None
except SubmissionUnavailable:
raise HTTPException(404, "submission capability unavailable") from None
except ArtifactError:
raise HTTPException(503, "source storage unavailable") from None
except ValueError:
raise HTTPException(422, "submission outside admission policy") from None
response.headers["Cache-Control"] = "no-store"
return result
def queries(request: Request):
service = getattr(request.app.state, "submission_queries", None)
if service is None:
raise HTTPException(503, "submission service unavailable")
return service
@router.get("", response_model=SubmissionPage)
async def history(
request: Request,
principal: Annotated[Principal, Depends(current_principal)],
after: Annotated[str | None, Query(max_length=128)] = None,
limit: Annotated[int, Query(ge=1, le=100)] = 20,
):
try:
return await queries(request).page(principal, after, limit)
except ValueError:
raise HTTPException(422, "invalid history request") from None
@router.get("/{submission_id}", response_model=SubmissionDetail)
async def detail(
submission_id: UUID,
request: Request,
principal: Annotated[Principal, Depends(current_principal)],
):
try:
return await queries(request).detail(principal, submission_id)
except SubmissionUnavailable:
raise HTTPException(404, "submission unavailable") from None
except ValueError:
raise HTTPException(503, "stored result unavailable") from None
@router.get("/{submission_id}/source", response_model=SubmissionSource)
async def source(
submission_id: UUID,
request: Request,
principal: Annotated[Principal, Depends(current_principal)],
):
service = queries(request)
try:
result = await service.detail(principal, submission_id)
data = await service.owned_source(principal, submission_id)
return SubmissionSource(
submission_id=submission_id, language=result.language, source=data.decode("utf-8")
)
except SubmissionUnavailable:
raise HTTPException(404, "submission unavailable") from None
except ValueError:
raise HTTPException(503, "stored source unavailable") from None
@router.post("/{submission_id}/cancel", status_code=202, response_model=CancellationAccepted)
async def cancel(
submission_id: UUID,
request: Request,
principal: Annotated[Principal, Depends(unsafe_principal)],
):
try:
return await queries(request).cancel(principal, submission_id)
except SubmissionUnavailable:
raise HTTPException(404, "submission unavailable") from None
except CancelUnavailable:
raise HTTPException(503, "cancellation service unavailable") from None
@@ -0,0 +1,96 @@
from typing import Annotated
from uuid import UUID
from fastapi import APIRouter, Header, HTTPException, Request, Response, Security
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
from kf_oj.contracts.judging import JobManifest, LeaseBinding
from kf_oj.modules.submissions.public import WorkerAccessDenied, WorkerReadBusy
from kf_oj.modules.submissions.worker_access import WorkerAccess
router = APIRouter(prefix="/internal/evaluations", tags=["worker artifacts"])
bearer = HTTPBearer(auto_error=False, scheme_name="WorkerLease")
def grant(
evaluation_id: UUID,
attempt_id: UUID,
worker: UUID,
fence: int,
credentials: HTTPAuthorizationCredentials | None,
) -> tuple[LeaseBinding, str]:
if (
credentials is None
or credentials.scheme.lower() != "bearer"
or len(credentials.credentials) != 64
):
raise HTTPException(401, "worker lease required")
try:
return LeaseBinding(
evaluation_id=evaluation_id,
attempt_id=attempt_id,
worker_id=worker,
fencing_version=fence,
), credentials.credentials
except ValueError:
raise HTTPException(404, "worker grant rejected") from None
def component(request: Request) -> WorkerAccess:
access = getattr(request.app.state, "worker_access", None)
if access is None:
raise HTTPException(503, "worker artifact service unavailable")
return access
@router.get("/{evaluation_id}/attempts/{attempt_id}/manifest", response_model=JobManifest)
async def manifest(
evaluation_id: UUID,
attempt_id: UUID,
request: Request,
worker: Annotated[UUID, Header(alias="X-Worker-ID")],
fence: Annotated[int, Header(alias="X-Fencing-Version", ge=1)],
credentials: Annotated[HTTPAuthorizationCredentials | None, Security(bearer)],
):
binding, token = grant(evaluation_id, attempt_id, worker, fence, credentials)
try:
payload = await component(request).manifest(binding, token)
except WorkerAccessDenied:
raise HTTPException(404, "worker grant rejected") from None
except WorkerReadBusy:
raise HTTPException(429, "worker reads temporarily limited") from None
return Response(payload, media_type="application/json", headers={"Cache-Control": "no-store"})
@router.get(
"/{evaluation_id}/attempts/{attempt_id}/artifacts/{digest}",
response_class=Response,
responses={
200: {
"content": {
"application/octet-stream": {"schema": {"type": "string", "format": "binary"}}
}
}
},
)
async def artifact(
evaluation_id: UUID,
attempt_id: UUID,
digest: str,
request: Request,
worker: Annotated[UUID, Header(alias="X-Worker-ID")],
fence: Annotated[int, Header(alias="X-Fencing-Version", ge=1)],
credentials: Annotated[HTTPAuthorizationCredentials | None, Security(bearer)],
):
binding, token = grant(evaluation_id, attempt_id, worker, fence, credentials)
try:
payload = await component(request).artifact(binding, token, digest)
except WorkerAccessDenied:
raise HTTPException(404, "worker grant rejected") from None
except WorkerReadBusy:
raise HTTPException(429, "worker reads temporarily limited") from None
return Response(
payload,
media_type="application/octet-stream",
headers={"Cache-Control": "no-store", "X-Content-Type-Options": "nosniff"},
)
+20
View File
@@ -6,6 +6,8 @@ from pathlib import Path
from kf_oj.api.app import create_app
from kf_oj.contracts.judge import JudgeTaskNotification
from kf_oj.contracts.judging import WIRE_MODELS
from kf_oj.contracts.judging_v2 import WIRE_MODELS as WIRE_MODELS_V2
from kf_oj.core.settings import Settings
@@ -23,6 +25,24 @@ def main() -> None:
Settings(_env_file=None, environment="test")
).openapi(),
}
for name, model in WIRE_MODELS.items():
document = model.model_json_schema()
document["$schema"] = "https://json-schema.org/draft/2020-12/schema"
document["$id"] = f"urn:kf-oj:{name}:v1"
document["$comment"] = (
"Structural schema; also run versioned model validation for cross-field "
"budgets, result consistency, UUID canonical form and UTF-8 byte limits."
)
outputs[root / f"contracts/schemas/{name}.v1.schema.json"] = document
for name, model in WIRE_MODELS_V2.items():
document = model.model_json_schema()
document["$schema"] = "https://json-schema.org/draft/2020-12/schema"
document["$id"] = f"urn:kf-oj:{name}:v2"
document["$comment"] = (
"Multi-runtime protocol. Also validate runtime/source/attempt bindings, startup "
"and node budgets, canonical bounded class sets and result consistency."
)
outputs[root / f"contracts/schemas/{name}.v2.schema.json"] = document
stale = []
for path, document in outputs.items():
content = json.dumps(document, indent=2, sort_keys=True, ensure_ascii=False) + "\n"
+22
View File
@@ -0,0 +1,22 @@
from datetime import datetime
from typing import Annotated
from pydantic import Field, SecretStr
from kf_oj.contracts.judging import Digest, Identifier, WireModel
class LoginRequest(WireModel):
username: Annotated[str, Field(strict=True, pattern=r"^[a-z0-9_]{3,24}$")]
password: Annotated[SecretStr, Field(min_length=12, max_length=128)]
class CurrentUser(WireModel):
id: Identifier
username: str
class LoginResponse(WireModel):
user: CurrentUser
expires_at: datetime
csrf_token: Digest = Field(repr=False)
+5 -3
View File
@@ -1,7 +1,9 @@
from typing import Literal
from typing import Annotated, Literal
from uuid import UUID
from pydantic import BaseModel, ConfigDict
from pydantic import BaseModel, BeforeValidator, ConfigDict
from kf_oj.contracts.judging import integer_version
class JudgeTaskNotification(BaseModel):
@@ -9,6 +11,6 @@ class JudgeTaskNotification(BaseModel):
model_config = ConfigDict(extra="forbid", frozen=True)
schema_version: Literal[1]
schema_version: Annotated[Literal[1], BeforeValidator(integer_version)]
kind: Literal["judge.evaluation"]
evaluation_id: UUID
@@ -0,0 +1,48 @@
"""Pure transition/fencing rules; callers still need a locked database transaction."""
from enum import StrEnum
from kf_oj.contracts.judging import LeaseBinding
class EvaluationState(StrEnum):
QUEUED = "queued"
CLAIMED = "claimed"
COMPILING = "compiling"
RUNNING = "running"
CANCELLING = "cancelling"
FINISHED = "finished"
CANCELLED = "cancelled"
FAILED = "failed"
def allows(self, target: "EvaluationState") -> bool:
transitions = {
self.QUEUED: {self.CLAIMED, self.CANCELLED},
self.CLAIMED: {self.COMPILING, self.CANCELLING, self.FAILED, self.QUEUED},
self.COMPILING: {
self.RUNNING,
self.FINISHED,
self.CANCELLING,
self.FAILED,
self.QUEUED,
},
self.RUNNING: {self.FINISHED, self.CANCELLING, self.FAILED, self.QUEUED},
self.CANCELLING: {self.CANCELLED, self.FAILED, self.QUEUED},
}
return target in transitions.get(self, set())
def require_current_lease(
expected: LeaseBinding,
supplied: LeaseBinding,
expires_at_unix_ms: int,
now_unix_ms: int,
) -> None:
"""Identity/version equality and a strict deadline; never trust client time."""
if expected != supplied or now_unix_ms < 0 or now_unix_ms >= expires_at_unix_ms:
raise ValueError("stale or foreign execution authority")
def require_transition(current: EvaluationState, target: EvaluationState) -> None:
if not current.allows(target):
raise ValueError("invalid evaluation transition")
+403
View File
@@ -0,0 +1,403 @@
"""Versioned judge wire models. Authorization and persistence belong to services."""
import json
from enum import StrEnum
from typing import Annotated, Literal, Self
from uuid import UUID
from pydantic import (
BaseModel,
BeforeValidator,
ConfigDict,
Field,
StrictBool,
model_validator,
)
MIB = 1024 * 1024
I64_MAX = (1 << 63) - 1
MAX_SOURCE_BYTES = 256 * 1024
MAX_PACKAGE_BYTES = 64 * MIB
MAX_CASES = 256
MAX_JOB_WALL_US = 3_600_000_000
MAX_WIRE_BYTES = 256 * 1024
def integer_version(value: object) -> object:
if type(value) is not int:
raise ValueError("schema version must be an integer")
return value
def canonical_id(value: object) -> UUID:
try:
parsed = UUID(str(value))
except (ValueError, TypeError, AttributeError) as error:
raise ValueError("invalid identifier") from error
if parsed.int == 0 or str(value) != str(parsed):
raise ValueError("identifier must be non-nil and canonical")
return parsed
Version = Annotated[Literal[1], BeforeValidator(integer_version)]
Identifier = Annotated[UUID, BeforeValidator(canonical_id)]
Digest = Annotated[str, Field(strict=True, min_length=64, max_length=64, pattern=r"^[0-9a-f]{64}$")]
Label = Annotated[str, Field(strict=True, pattern=r"^[a-z0-9][a-z0-9_-]{0,63}$")]
Counter = Annotated[int, Field(strict=True, ge=1, le=I64_MAX)]
Measurement = Annotated[int, Field(strict=True, ge=0, le=I64_MAX)]
class WireModel(BaseModel):
model_config = ConfigDict(extra="forbid", frozen=True)
class Language(StrEnum):
C17 = "c17"
CPP20 = "cpp20"
class Verdict(StrEnum):
ACCEPTED = "accepted"
WRONG_ANSWER = "wrong_answer"
COMPILATION_ERROR = "compilation_error"
RUNTIME_ERROR = "runtime_error"
TIME_LIMIT = "time_limit_exceeded"
MEMORY_LIMIT = "memory_limit_exceeded"
OUTPUT_LIMIT = "output_limit_exceeded"
CANCELLED = "cancelled"
SYSTEM_ERROR = "system_error"
class CompilationStatus(StrEnum):
SUCCEEDED = "succeeded"
COMPILATION_ERROR = "compilation_error"
COMPILATION_LIMIT = "compilation_limit"
CANCELLED = "cancelled"
SYSTEM_ERROR = "system_error"
class FailureCode(StrEnum):
ARTIFACT_UNAVAILABLE = "artifact_unavailable"
SANDBOX_UNAVAILABLE = "sandbox_unavailable"
CLEANUP_FAILED = "cleanup_failed"
LEASE_LOST = "lease_lost"
JOB_DEADLINE = "job_deadline"
class ArtifactReference(WireModel):
digest: Digest
bytes: Annotated[int, Field(strict=True, ge=0, le=MAX_PACKAGE_BYTES)]
class Toolchain(WireModel):
toolchain_id: Label
image_kind: Literal["linux_gnu_cxx_x86_64_v1"]
image_revision: Annotated[int, Field(strict=True, ge=1, le=(1 << 32) - 1)]
class RunLimits(WireModel):
cpu_time_us: Annotated[int, Field(strict=True, ge=1000, le=60_000_000)]
wall_time_us: Annotated[int, Field(strict=True, ge=1000, le=120_000_000)]
memory_bytes: Annotated[int, Field(strict=True, ge=1, le=2048 * MIB)]
stack_bytes: Annotated[int, Field(strict=True, ge=1, le=256 * MIB)]
processes: Annotated[int, Field(strict=True, ge=1, le=128)]
open_files: Annotated[int, Field(strict=True, ge=3, le=1024)]
stdout_bytes: Annotated[int, Field(strict=True, ge=1, le=16 * MIB)]
stderr_bytes: Annotated[int, Field(strict=True, ge=1, le=16 * MIB)]
file_bytes: Annotated[int, Field(strict=True, ge=1, le=64 * MIB)]
workdir_bytes: Annotated[int, Field(strict=True, ge=1, le=256 * MIB)]
@model_validator(mode="after")
def consistent(self) -> Self:
if (
self.stack_bytes > self.memory_bytes
or self.file_bytes > self.workdir_bytes
or self.stdout_bytes > self.file_bytes
or self.stderr_bytes > self.file_bytes
or self.stdout_bytes + self.stderr_bytes > self.workdir_bytes
):
raise ValueError("inconsistent resource limits")
return self
def fits(self, ceiling: "RunLimits") -> bool:
return all(
getattr(self, name) <= getattr(ceiling, name) for name in type(self).model_fields
)
class TestCase(WireModel):
case_id: Identifier
input: ArtifactReference
expected: ArtifactReference
class JobManifest(WireModel):
schema_version: Version
kind: Literal["judge.manifest"]
evaluation_id: Identifier
submission_id: Identifier
problem_version_id: Identifier
language: Language
toolchain: Toolchain
source: ArtifactReference
cases: Annotated[tuple[TestCase, ...], Field(min_length=1, max_length=MAX_CASES)]
comparison: Literal["exact_bytes"]
compile_limits: RunLimits
run_limits: RunLimits
job_wall_time_us: Annotated[int, Field(strict=True, ge=1000, le=MAX_JOB_WALL_US)]
def package_bytes(self) -> int:
# Count references, including duplicates: this bounds actual staged/read bytes.
return self.source.bytes + sum(c.input.bytes + c.expected.bytes for c in self.cases)
@model_validator(mode="after")
def consistent(self) -> Self:
if not 1 <= self.source.bytes <= MAX_SOURCE_BYTES:
raise ValueError("source size outside policy")
if len({c.case_id for c in self.cases}) != len(self.cases):
raise ValueError("duplicate testcase identifiers")
if self.package_bytes() > MAX_PACKAGE_BYTES:
raise ValueError("test package outside policy")
if any(c.expected.bytes > self.run_limits.stdout_bytes for c in self.cases):
raise ValueError("expected output exceeds output budget")
required_wall = (
self.compile_limits.wall_time_us + len(self.cases) * self.run_limits.wall_time_us
)
if required_wall > self.job_wall_time_us:
raise ValueError("job deadline cannot cover execution budgets")
return self
def manifest_bytes(manifest: JobManifest) -> bytes:
"""Stable UTF-8 bytes for persisted snapshots and their SHA-256 identity."""
return json.dumps(
manifest.model_dump(mode="json"),
sort_keys=True,
separators=(",", ":"),
ensure_ascii=False,
allow_nan=False,
).encode("utf-8")
class LeaseBinding(WireModel):
evaluation_id: Identifier
attempt_id: Identifier
worker_id: Identifier
fencing_version: Counter
class LeaseGrant(WireModel):
schema_version: Version
kind: Literal["judge.lease"]
binding: LeaseBinding
lease_token: Digest = Field(repr=False)
attempt_number: Annotated[int, Field(strict=True, ge=1, le=5)]
expires_at_unix_ms: Counter
manifest_digest: Digest
class StageReport(WireModel):
schema_version: Version
kind: Literal["judge.stage"]
binding: LeaseBinding
sequence: Counter
stage: Literal["compiling", "running"]
case_index: Annotated[int, Field(strict=True, ge=0, lt=MAX_CASES)] | None
@model_validator(mode="after")
def consistent(self) -> Self:
if (self.stage == "running") != (self.case_index is not None):
raise ValueError("testcase index only belongs to running phase")
return self
class CompilationReport(WireModel):
status: CompilationStatus
program: ArtifactReference | None
diagnostics: Annotated[str, Field(strict=True, max_length=65536)] = Field(repr=False)
diagnostics_truncated: StrictBool
cpu_time_us: Measurement | None
memory_peak_bytes: Measurement | None
@model_validator(mode="after")
def consistent(self) -> Self:
if len(self.diagnostics.encode("utf-8")) > 65536:
raise ValueError("compiler diagnostics exceed byte budget")
if (self.status == CompilationStatus.SUCCEEDED) != (self.program is not None):
raise ValueError("compiled artifact only belongs to successful compilation")
if self.program is not None and self.program.bytes == 0:
raise ValueError("compiled artifact cannot be empty")
return self
class CaseReport(WireModel):
case_id: Identifier
verdict: Verdict
cpu_time_us: Measurement | None
memory_peak_bytes: Measurement | None
@model_validator(mode="after")
def consistent(self) -> Self:
if self.verdict == Verdict.COMPILATION_ERROR:
raise ValueError("compilation failure is not a testcase outcome")
return self
class JudgementReport(WireModel):
schema_version: Version
kind: Literal["judge.result"]
binding: LeaseBinding
manifest_digest: Digest
compilation: CompilationReport
cases: Annotated[tuple[CaseReport, ...], Field(max_length=MAX_CASES)]
verdict: Verdict
cleanup_complete: StrictBool
failure_code: FailureCode | None
run_cpu_time_us: Measurement | None
run_memory_peak_bytes: Measurement | None
@model_validator(mode="after")
def consistent(self) -> Self:
if len({c.case_id for c in self.cases}) != len(self.cases):
raise ValueError("duplicate testcase results")
if not self.cleanup_complete and self.failure_code != FailureCode.CLEANUP_FAILED:
raise ValueError("incomplete cleanup requires its own failure evidence")
if self.cleanup_complete and self.failure_code == FailureCode.CLEANUP_FAILED:
raise ValueError("cleanup failure cannot claim completed cleanup")
if (self.verdict == Verdict.SYSTEM_ERROR) != (self.failure_code is not None):
raise ValueError("infrastructure failure and verdict must agree")
terminal = next(
(
i
for i, c in enumerate(self.cases)
if c.verdict in (Verdict.SYSTEM_ERROR, Verdict.CANCELLED)
),
None,
)
if terminal is not None and terminal != len(self.cases) - 1:
raise ValueError("results after fatal testcase outcome")
if self.compilation.status != CompilationStatus.SUCCEEDED:
mapped = {
CompilationStatus.COMPILATION_ERROR: Verdict.COMPILATION_ERROR,
CompilationStatus.COMPILATION_LIMIT: Verdict.COMPILATION_ERROR,
CompilationStatus.CANCELLED: Verdict.CANCELLED,
CompilationStatus.SYSTEM_ERROR: Verdict.SYSTEM_ERROR,
}[self.compilation.status]
if self.cases or self.verdict != mapped:
raise ValueError("failed compilation cannot have runtime results")
elif self.failure_code is None:
if any(c.verdict == Verdict.SYSTEM_ERROR for c in self.cases):
raise ValueError("infrastructure failure requires failure code")
if not self.cases and self.verdict != Verdict.CANCELLED:
raise ValueError("missing testcase results cannot be a completed judgement")
if self.cases and self.verdict != Verdict.CANCELLED:
failures = (Verdict.SYSTEM_ERROR, Verdict.CANCELLED)
summary = next(
(c.verdict for c in self.cases if c.verdict in failures),
next(
(c.verdict for c in self.cases if c.verdict != Verdict.ACCEPTED),
Verdict.ACCEPTED,
),
)
if self.verdict != summary:
raise ValueError("testcase summary disagrees")
cpu = (
sum(c.cpu_time_us for c in self.cases if c.cpu_time_us is not None)
if self.cases and all(c.cpu_time_us is not None for c in self.cases)
else None
)
memory = (
max(c.memory_peak_bytes for c in self.cases if c.memory_peak_bytes is not None)
if self.cases and all(c.memory_peak_bytes is not None for c in self.cases)
else None
)
if self.run_cpu_time_us != cpu or self.run_memory_peak_bytes != memory:
raise ValueError("runtime measurement summary disagrees")
return self
def validate_for_manifest(self, manifest: JobManifest, digest: str) -> None:
if self.binding.evaluation_id != manifest.evaluation_id or self.manifest_digest != digest:
raise ValueError("report does not belong to authorized manifest")
if [c.case_id for c in self.cases] != [
c.case_id for c in manifest.cases[: len(self.cases)]
]:
raise ValueError("testcase results differ from authorized order")
if self.verdict not in (Verdict.COMPILATION_ERROR, Verdict.CANCELLED, Verdict.SYSTEM_ERROR):
if len(self.cases) != len(manifest.cases):
raise ValueError("incomplete ordinary judgement")
class AdmissionPolicy(WireModel):
"""Trusted node policy. A manifest never supplies or replaces this policy."""
max_concurrent_jobs: Annotated[int, Field(strict=True, ge=1, le=64)] = 1
prefetch: Annotated[int, Field(strict=True, ge=1, le=64)] = 1
max_retries: Annotated[int, Field(strict=True, ge=0, le=4)] = 2
max_job_wall_time_us: Annotated[int, Field(strict=True, ge=1000, le=MAX_JOB_WALL_US)]
max_package_bytes: Annotated[int, Field(strict=True, ge=1, le=MAX_PACKAGE_BYTES)]
toolchain: Toolchain
compile_ceiling: RunLimits
run_ceiling: RunLimits
@model_validator(mode="after")
def consistent(self) -> Self:
if self.prefetch > self.max_concurrent_jobs:
raise ValueError("prefetch exceeds available job slots")
return self
def admit(self, manifest: JobManifest, attempt_number: int) -> None:
if type(attempt_number) is not int or not 1 <= attempt_number <= self.max_retries + 1:
raise ValueError("attempt retry budget exhausted")
if (
manifest.toolchain != self.toolchain
or not manifest.compile_limits.fits(self.compile_ceiling)
or not manifest.run_limits.fits(self.run_ceiling)
or manifest.job_wall_time_us > self.max_job_wall_time_us
or manifest.package_bytes() > self.max_package_bytes
):
raise ValueError("job exceeds trusted node budget")
class StoppedReport(WireModel):
schema_version: Version
kind: Literal["judge.stopped"]
binding: LeaseBinding
cleanup_complete: StrictBool
WIRE_MODELS = {
"judge-manifest": JobManifest,
"judge-lease": LeaseGrant,
"judge-stage": StageReport,
"judge-result": JudgementReport,
"judge-admission": AdmissionPolicy,
"judge-stopped": StoppedReport,
}
def decode_wire[T: BaseModel](model: type[T], payload: bytes) -> T:
"""UTF-8, bounded size and no ambiguous duplicate members before model validation."""
if len(payload) > MAX_WIRE_BYTES:
raise ValueError("wire payload exceeds byte budget")
def unique_members(pairs: list[tuple[str, object]]) -> dict[str, object]:
result = {}
for key, value in pairs:
if key in result:
raise ValueError("duplicate JSON member")
result[key] = value
return result
def invalid_constant(_: str) -> None:
raise ValueError("invalid JSON constant")
try:
value = json.loads(
payload.decode("utf-8"),
object_pairs_hook=unique_members,
parse_constant=invalid_constant,
)
return model.model_validate(value)
except (ValueError, TypeError):
# Errors at the transport boundary must not echo credentials or hidden inputs.
raise ValueError("invalid wire payload") from None
+358
View File
@@ -0,0 +1,358 @@
"""Explicit multi-runtime protocol; existing v1 snapshots remain byte-for-byte stable."""
import re
from enum import StrEnum
from typing import Annotated, Literal, Self
from pydantic import BeforeValidator, Field, model_validator
from kf_oj.contracts import judging as v1
Version = Annotated[Literal[2], BeforeValidator(v1.integer_version)]
MAX_CLASS_FILES = 256
JVM_NATIVE_RESERVE_BYTES = 160 * v1.MIB
JVM_THREAD_STACK_BYTES = 256 * 1024
class Language(StrEnum):
C17 = "c17"
CPP20 = "cpp20"
PYTHON313 = "python313"
JAVA21 = "java21"
class PreparationKind(StrEnum):
COMPILE = "compile"
SYNTAX_CHECK = "syntax_check"
class ProgramKind(StrEnum):
NATIVE = "native_executable"
PYTHON = "python_source"
JAVA = "java_classes"
class RuntimeIdentity(v1.WireModel):
"""An exact identity matched against trusted configuration, never a host path."""
toolchain_id: v1.Label
image_kind: Literal[
"linux_gnu_cxx_x86_64_v1", "linux_cpython313_x86_64_v1", "linux_openjdk21_x86_64_v1"
]
image_revision: Annotated[int, Field(strict=True, ge=1, le=(1 << 32) - 1)]
image_digest: v1.Digest
runtime_version: Annotated[
str, Field(strict=True, pattern=r"^[A-Za-z0-9][A-Za-z0-9_.+-]{0,63}$")
]
def runtime_contract(language: Language) -> tuple[str, str, str, ProgramKind, PreparationKind]:
"""Fixed source/profile names. Runtime callers cannot introduce argv or paths."""
return {
Language.C17: (
"main.c",
"compile_c",
"native",
ProgramKind.NATIVE,
PreparationKind.COMPILE,
),
Language.CPP20: (
"main.cpp",
"compile_cpp",
"native",
ProgramKind.NATIVE,
PreparationKind.COMPILE,
),
Language.PYTHON313: (
"main.py",
"python_check",
"python_run",
ProgramKind.PYTHON,
PreparationKind.SYNTAX_CHECK,
),
Language.JAVA21: (
"Main.java",
"java_compile",
"java_run",
ProgramKind.JAVA,
PreparationKind.COMPILE,
),
}[language]
def validate_runtime(language: Language, identity: RuntimeIdentity) -> None:
expected = {
Language.C17: ("gnu-cxx-v1", "linux_gnu_cxx_x86_64_v1"),
Language.CPP20: ("gnu-cxx-v1", "linux_gnu_cxx_x86_64_v1"),
Language.PYTHON313: ("cpython313-v1", "linux_cpython313_x86_64_v1"),
Language.JAVA21: ("openjdk21-v1", "linux_openjdk21_x86_64_v1"),
}[language]
if (identity.toolchain_id, identity.image_kind) != expected:
raise ValueError("runtime family differs from language contract")
if language == Language.PYTHON313 and identity.runtime_version != "3.13.16":
raise ValueError("unsupported CPython version")
if language == Language.JAVA21 and not re.fullmatch(
r"21\.[0-9]+\.[0-9]+(?:\.[0-9]+)?\+[1-9][0-9]*(?:-LTS)?", identity.runtime_version
):
raise ValueError("JDK version must identify a fixed Java 21 build")
def validate_startup_budget(language: Language, limits: v1.RunLimits, *, preparing: bool) -> None:
"""Admission floors, not multipliers on problem limits; real execution still required."""
if language == Language.PYTHON313:
floor = (10_000, 50_000, 32 * v1.MIB, 1, 16)
elif language == Language.JAVA21:
floor = (
(1_000_000, 3_000_000, 384 * v1.MIB, 16, 32)
if preparing
else (200_000, 1_000_000, 256 * v1.MIB, 16, 32)
)
else:
return
actual = (
limits.cpu_time_us,
limits.wall_time_us,
limits.memory_bytes,
limits.processes,
limits.open_files,
)
if any(value < minimum for value, minimum in zip(actual, floor, strict=True)):
raise ValueError("budget below runtime startup policy")
if language == Language.JAVA21:
jvm_heap_bytes(limits)
def jvm_heap_bytes(limits: v1.RunLimits) -> int:
"""Reserve capped metaspace/code/native and every allowed thread stack inside cgroup."""
reserve = JVM_NATIVE_RESERVE_BYTES + limits.processes * JVM_THREAD_STACK_BYTES
heap = (limits.memory_bytes - reserve) // v1.MIB * v1.MIB
if heap < 32 * v1.MIB or limits.stack_bytes < JVM_THREAD_STACK_BYTES:
raise ValueError("budget cannot cover JVM heap and explicit non-heap reserve")
return heap
class LanguageConfiguration(v1.WireModel):
language: Language
toolchain: RuntimeIdentity
compile_limits: v1.RunLimits
run_limits: v1.RunLimits
job_wall_time_us: Annotated[int, Field(strict=True, ge=1000, le=v1.MAX_JOB_WALL_US)]
@model_validator(mode="after")
def runtime_and_budget(self) -> Self:
validate_runtime(self.language, self.toolchain)
validate_startup_budget(self.language, self.compile_limits, preparing=True)
validate_startup_budget(self.language, self.run_limits, preparing=False)
if self.compile_limits.wall_time_us + self.run_limits.wall_time_us > self.job_wall_time_us:
raise ValueError("job deadline cannot cover preparation and execution")
return self
class JobManifest(v1.JobManifest):
schema_version: Version
language: Language
toolchain: RuntimeIdentity
@model_validator(mode="after")
def runtime_and_budget(self) -> Self:
validate_runtime(self.language, self.toolchain)
validate_startup_budget(self.language, self.compile_limits, preparing=True)
validate_startup_budget(self.language, self.run_limits, preparing=False)
return self
class ProgramFile(v1.WireModel):
path: Annotated[str, Field(strict=True, min_length=1, max_length=128)]
artifact: v1.ArtifactReference
class CompiledProgram(v1.WireModel):
"""Sealed file identities bound to a particular source, runtime and lease attempt."""
kind: ProgramKind
binding: v1.LeaseBinding
source: v1.ArtifactReference
toolchain: RuntimeIdentity
files: Annotated[tuple[ProgramFile, ...], Field(min_length=1, max_length=MAX_CLASS_FILES)]
@property
def bytes(self) -> int:
return sum(file.artifact.bytes for file in self.files)
@model_validator(mode="after")
def bounded_files(self) -> Self:
names = [file.path for file in self.files]
if names != sorted(set(names)):
raise ValueError("program files must have unique canonical order")
if not 1 <= self.source.bytes <= v1.MAX_SOURCE_BYTES:
raise ValueError("program source outside policy")
if self.bytes > v1.MAX_PACKAGE_BYTES or any(
file.artifact.bytes == 0 for file in self.files
):
raise ValueError("program files empty or outside byte policy")
if self.kind == ProgramKind.NATIVE:
validate_runtime(Language.C17, self.toolchain)
if names != ["main"] or self.toolchain.image_kind != "linux_gnu_cxx_x86_64_v1":
raise ValueError("native program requires its fixed executable and runtime")
elif self.kind == ProgramKind.PYTHON:
validate_runtime(Language.PYTHON313, self.toolchain)
if names != ["main.py"] or self.files[0].artifact != self.source:
raise ValueError("Python program must be the exact checked source")
else:
validate_runtime(Language.JAVA21, self.toolchain)
if "Main.class" not in names or any(
not re.fullmatch(r"[A-Za-z_$][A-Za-z0-9_$]*\.class", name) for name in names
):
raise ValueError("Java requires a bounded default-package Main class set")
return self
class CompilationReport(v1.CompilationReport):
program: CompiledProgram | None
class JudgementReport(v1.JudgementReport):
schema_version: Version
compilation: CompilationReport
@model_validator(mode="after")
def program_attempt(self) -> Self:
if (
self.compilation.program is not None
and self.compilation.program.binding != self.binding
):
raise ValueError("program belongs to another attempt")
return self
def validate_for_manifest(self, manifest: JobManifest, digest: str) -> None:
super().validate_for_manifest(manifest, digest)
program = self.compilation.program
if program is not None and (
program.source != manifest.source
or program.toolchain != manifest.toolchain
or program.kind != runtime_contract(manifest.language)[3]
):
raise ValueError("program differs from authorized source or runtime")
class LeaseGrant(v1.LeaseGrant):
schema_version: Version
class StageReport(v1.StageReport):
schema_version: Version
preparation_kind: PreparationKind
def validate_for_manifest(self, manifest: JobManifest) -> None:
if self.binding.evaluation_id != manifest.evaluation_id or (
self.preparation_kind != runtime_contract(manifest.language)[4]
):
raise ValueError("stage differs from authorized language preparation")
class StoppedReport(v1.StoppedReport):
schema_version: Version
class RuntimeCapability(v1.WireModel):
language: Language
toolchain: RuntimeIdentity
compile_ceiling: v1.RunLimits
run_ceiling: v1.RunLimits
@model_validator(mode="after")
def runtime_and_budget(self) -> Self:
validate_runtime(self.language, self.toolchain)
validate_startup_budget(self.language, self.compile_ceiling, preparing=True)
validate_startup_budget(self.language, self.run_ceiling, preparing=False)
return self
class AdmissionPolicy(v1.WireModel):
schema_version: Version
max_concurrent_jobs: Annotated[int, Field(strict=True, ge=1, le=64)] = 1
prefetch: Annotated[int, Field(strict=True, ge=1, le=64)] = 1
max_retries: Annotated[int, Field(strict=True, ge=0, le=4)] = 2
max_job_wall_time_us: Annotated[int, Field(strict=True, ge=1000, le=v1.MAX_JOB_WALL_US)]
max_package_bytes: Annotated[int, Field(strict=True, ge=1, le=v1.MAX_PACKAGE_BYTES)]
capabilities: Annotated[tuple[RuntimeCapability, ...], Field(min_length=1, max_length=4)]
@model_validator(mode="after")
def consistent(self) -> Self:
if self.prefetch > self.max_concurrent_jobs:
raise ValueError("prefetch exceeds available job slots")
if len({cap.language for cap in self.capabilities}) != len(self.capabilities):
raise ValueError("duplicate language capability")
return self
def admit(self, manifest: JobManifest, attempt_number: int) -> None:
# Revalidate constructed/copied values as well as transport-decoded models.
manifest = JobManifest.model_validate(manifest.model_dump(mode="json"))
AdmissionPolicy.model_validate(self.model_dump(mode="json"))
if type(attempt_number) is not int or not 1 <= attempt_number <= self.max_retries + 1:
raise ValueError("attempt retry budget exhausted")
cap = next((cap for cap in self.capabilities if cap.language == manifest.language), None)
if cap is None or (
cap.toolchain != manifest.toolchain
or not manifest.compile_limits.fits(cap.compile_ceiling)
or not manifest.run_limits.fits(cap.run_ceiling)
or manifest.job_wall_time_us > self.max_job_wall_time_us
or manifest.package_bytes() > self.max_package_bytes
):
raise ValueError("job exceeds trusted runtime capability or budget")
class AdmissionRegistry(v1.WireModel):
"""Explicit operator opt-in for both protocol generations; no silent conversion."""
schema_version: Version
legacy: v1.AdmissionPolicy | None
multi_runtime: AdmissionPolicy | None
@model_validator(mode="after")
def at_least_one_protocol(self) -> Self:
if self.legacy is None and self.multi_runtime is None:
raise ValueError("node must advertise at least one explicit protocol capability")
if self.legacy is not None and self.multi_runtime is not None:
for field in ("max_concurrent_jobs", "prefetch", "max_retries"):
if getattr(self.legacy, field) != getattr(self.multi_runtime, field):
raise ValueError("protocol policies disagree on node capacity or retries")
return self
def admit(self, manifest: v1.JobManifest | JobManifest, attempt_number: int) -> None:
AdmissionRegistry.model_validate(self.model_dump(mode="json"))
policy = self.legacy if manifest.schema_version == 1 else self.multi_runtime
admit_manifest(policy, manifest, attempt_number)
NodeAdmission = v1.AdmissionPolicy | AdmissionPolicy | AdmissionRegistry
def admit_manifest(
policy: NodeAdmission | None, manifest: v1.JobManifest | JobManifest, attempt_number: int
) -> None:
if type(manifest) not in (v1.JobManifest, JobManifest):
raise ValueError("unsupported manifest model")
# Copied/constructed values must pass the original strict protocol validators too.
manifest = type(manifest).model_validate(manifest.model_dump(mode="json"))
if isinstance(policy, AdmissionRegistry):
policy.admit(manifest, attempt_number)
elif manifest.schema_version == 1 and type(policy) is v1.AdmissionPolicy:
v1.AdmissionPolicy.model_validate(policy.model_dump(mode="json")).admit(
manifest, attempt_number
)
elif manifest.schema_version == 2 and type(policy) is AdmissionPolicy:
policy.admit(manifest, attempt_number)
else:
raise ValueError("node does not advertise this manifest protocol")
WIRE_MODELS = {
"judge-manifest": JobManifest,
"judge-lease": LeaseGrant,
"judge-stage": StageReport,
"judge-result": JudgementReport,
"judge-admission": AdmissionPolicy,
"judge-stopped": StoppedReport,
"judge-registry": AdmissionRegistry,
}
@@ -0,0 +1,35 @@
"""Version dispatch at storage/transport boundaries; never rewrite legacy snapshots."""
from pydantic import ConfigDict, RootModel
from kf_oj.contracts import judging as v1
from kf_oj.contracts import judging_v2 as v2
Manifest = v1.JobManifest | v2.JobManifest
Report = v1.JudgementReport | v2.JudgementReport
class ManifestEnvelope(RootModel[Manifest]):
model_config = ConfigDict(frozen=True)
class ReportEnvelope(RootModel[Report]):
model_config = ConfigDict(frozen=True)
def decode_manifest(payload: bytes) -> Manifest:
return v1.decode_wire(ManifestEnvelope, payload).root
def decode_report(payload: bytes) -> Report:
return v1.decode_wire(ReportEnvelope, payload).root
def manifest_bytes(manifest: Manifest) -> bytes:
return v1.manifest_bytes(manifest)
def validate_report(report: Report, manifest: Manifest, digest: str) -> None:
if report.schema_version != manifest.schema_version:
raise ValueError("report protocol differs from authorized manifest")
report.validate_for_manifest(manifest, digest)
+58
View File
@@ -0,0 +1,58 @@
from typing import Annotated, Literal
from pydantic import Field
from kf_oj.contracts.judging import Identifier, Label, RunLimits, WireModel
from kf_oj.contracts.judging_v2 import Language, PreparationKind, runtime_contract
class PublicSample(WireModel):
encoding: Literal["base64"] = "base64"
input: Annotated[str, Field(max_length=87384)]
output: Annotated[str, Field(max_length=87384)]
class ProblemSummary(WireModel):
id: Identifier
slug: Label
version_id: Identifier
revision: int
title: str
languages: tuple[Language, ...]
class PublicLanguageConfiguration(WireModel):
language: Language
runtime_version: str
source_name: str
entrypoint: str
preparation_kind: PreparationKind
compile_limits: RunLimits
run_limits: RunLimits
job_wall_time_us: int
@classmethod
def from_configuration(cls, configuration):
source, _, _, _, preparation = runtime_contract(configuration.language)
return cls(
language=configuration.language,
runtime_version=configuration.toolchain.runtime_version,
source_name=source,
entrypoint="Main" if configuration.language == Language.JAVA21 else source,
preparation_kind=preparation,
compile_limits=configuration.compile_limits,
run_limits=configuration.run_limits,
job_wall_time_us=configuration.job_wall_time_us,
)
class ProblemDetail(ProblemSummary):
statement: str
samples: tuple[PublicSample, ...]
run_limits: RunLimits | None
language_configurations: tuple[PublicLanguageConfiguration, ...] = ()
class ProblemPage(WireModel):
items: tuple[ProblemSummary, ...]
next_after: Label | None
@@ -0,0 +1,92 @@
from datetime import datetime
from typing import Annotated, Literal
from pydantic import Field, model_validator
from kf_oj.contracts.judging import (
MAX_SOURCE_BYTES,
CompilationStatus,
FailureCode,
Identifier,
Measurement,
Verdict,
WireModel,
)
from kf_oj.contracts.judging_v2 import Language
EvaluationState = Literal[
"queued", "claimed", "compiling", "running", "cancelling", "finished", "cancelled", "failed"
]
class CreateSubmission(WireModel):
problem_version_id: Identifier
language: Language
idempotency_key: Identifier
source: Annotated[
str, Field(strict=True, min_length=1, max_length=MAX_SOURCE_BYTES, repr=False)
]
@model_validator(mode="after")
def bounded_source(self):
try:
if len(self.source.encode("utf-8")) > MAX_SOURCE_BYTES:
raise ValueError("source exceeds byte policy")
except UnicodeError:
raise ValueError("source must be UTF-8") from None
return self
class SubmissionCreated(WireModel):
submission_id: Identifier
evaluation_id: Identifier
problem_version_id: Identifier
language: Language
state: EvaluationState
created_at: datetime
class SubmissionSummary(SubmissionCreated):
updated_at: datetime
attempts: int
verdict: Verdict | None
failure_code: FailureCode | None
class PublicCaseResult(WireModel):
position: int
verdict: Verdict
cpu_time_us: Measurement | None
memory_peak_bytes: Measurement | None
class PublicJudgement(WireModel):
compilation_status: CompilationStatus
diagnostics: Annotated[str, Field(max_length=65536)]
diagnostics_truncated: bool
cpu_time_us: Measurement | None
memory_peak_bytes: Measurement | None
cleanup_complete: bool
cases: tuple[PublicCaseResult, ...]
class SubmissionDetail(SubmissionSummary):
result: PublicJudgement | None
class SubmissionPage(WireModel):
items: tuple[SubmissionSummary, ...]
next_after: str | None
class SubmissionSource(WireModel):
submission_id: Identifier
language: Language
source: Annotated[str, Field(max_length=MAX_SOURCE_BYTES, repr=False)]
class CancellationAccepted(WireModel):
submission_id: Identifier
evaluation_id: Identifier
# Acceptance means Coordinator durably accepted intent, not that processes have stopped.
accepted: Literal[True] = True
+66 -1
View File
@@ -1,8 +1,11 @@
from pathlib import Path
from typing import Literal
from pydantic import SecretStr
from pydantic import Field, SecretStr, model_validator
from pydantic_settings import BaseSettings, SettingsConfigDict
from kf_oj.contracts.judging_v2 import NodeAdmission
class Settings(BaseSettings):
model_config = SettingsConfigDict(env_prefix="KF_OJ_", env_file=".env", extra="ignore")
@@ -11,3 +14,65 @@ class Settings(BaseSettings):
log_level: Literal["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"] = "INFO"
database_url: SecretStr | None = None
amqp_url: SecretStr | None = None
browser_origins: tuple[str, ...] = (
"http://localhost:5173",
"http://localhost:8000",
"http://127.0.0.1:5173",
"http://127.0.0.1:8000",
)
session_lifetime_seconds: int = Field(default=43200, ge=300, le=86400)
artifact_root: Path | None = None
artifact_max_bytes: int = Field(default=536870912, ge=1, le=16 * 1024**3)
artifact_max_files: int = Field(default=4096, ge=1, le=65536)
judge_admission: NodeAdmission | None = None
coordinator_url: str | None = None
coordinator_token: SecretStr | None = None
@model_validator(mode="after")
def secure_origins(self):
from ipaddress import ip_address
from urllib.parse import urlsplit
if (self.coordinator_url is None) != (self.coordinator_token is None):
raise ValueError("both coordinator URL and token required")
if self.coordinator_url is not None:
parts = urlsplit(self.coordinator_url)
try:
valid = parts.hostname is not None and ip_address(parts.hostname).is_loopback
port = parts.port
except ValueError:
valid, port = False, None
if (
not valid
or parts.scheme != "http"
or port is None
or parts.path not in {"", "/"}
or parts.username
or parts.password
or parts.query
or parts.fragment
):
raise ValueError("coordinator must be an explicit numeric loopback HTTP origin")
import re
if not re.fullmatch(r"[0-9a-f]{64}", self.coordinator_token.get_secret_value()):
raise ValueError("invalid coordinator service credential")
if not self.browser_origins or len(self.browser_origins) > 8:
raise ValueError("one to eight browser origins required")
for origin in self.browser_origins:
parts = urlsplit(origin)
if (
parts.scheme not in {"http", "https"}
or not parts.hostname
or parts.username
or parts.password
or parts.path
or parts.query
or parts.fragment
or (self.environment == "production" and parts.scheme != "https")
):
raise ValueError(
"browser origins must be explicit origins; production requires HTTPS"
)
return self
@@ -0,0 +1,20 @@
# 不可变工件
存储根配置于 `KF_OJ_ARTIFACT_ROOT`,由服务账号持有,目录 0700、文件 0600。
单件至多 64 MiB;默认总字节 512 MiB / 文件 4096,可在有界配置中调整。
发布采用摘要名称、临时文件、文件/目录 fsync 和跨进程锁;已有工件必须通过长度/摘要核验,
拒绝静默覆盖/修复,普通读取不跟随链接。锁等待最多五秒。临时文件也计入容量。
适配器执行已经授权的操作,不自行识别用户、租约或题库。公开样例由 problems 的公开 DTO
提供;个人源码由 submissions 的已验证主体与归属端口读取;隐藏数据只经有效当前租约取得。
没有匿名工件下载或由文件路径选择的读取接口。
发布事务先取得 digest 锁,再持久化字节与元数据,业务引用最后提交;事务回滚最多留下
无元数据的耐久孤立文件,不留下数据库引用指向未发布字节。回收使用同一 digest 锁,
业务引用的数据库生成 ready 状态及复合外键阻止被引用元数据转为 deleting。
`uv run --locked kf-artifacts --limit 100 [--after <digest>]` 执行有限回收批次:恢复 deleting、
标记超过一天的未引用元数据、删除耐久字节/元数据及超过一天的无元数据文件/临时文件。
每次扫描至多 100 行;显示游标,后续调用接续,避免固定前页上的保留工件阻塞扫描。
引用工件不能被回收。文件删除后事务失败仍保持 deleting,下一批可恢复;新引用无法加入。
需要已迁移数据库和有限平台权限;回收不遍历业务域 ORM,也不删除不可变题目版本。
@@ -1 +1 @@
"""Reserved for authenticated artifact access and digest verification."""
"""Digest-addressed immutable storage; business ports enforce authorization."""
+48
View File
@@ -0,0 +1,48 @@
import argparse
import asyncio
from sqlalchemy.exc import SQLAlchemyError
from kf_oj.core.settings import Settings
from kf_oj.infra.artifacts.repository import ArtifactRepository
from kf_oj.infra.artifacts.store import ArtifactStore
from kf_oj.infra.db.session import build_engine, build_session_factory
async def run(limit: int, after: str | None) -> None:
settings = Settings()
if settings.artifact_root is None:
raise ValueError("artifact root required")
store = ArtifactStore(
settings.artifact_root,
max_bytes=settings.artifact_max_bytes,
max_files=settings.artifact_max_files,
)
engine = build_engine(settings)
try:
sessions = build_session_factory(engine)
repository = ArtifactRepository(store)
recovered = await repository.finish_deleting(sessions, limit=limit)
marked, cursor = await repository.collect_metadata(sessions, limit=limit, after=after)
removed = await repository.finish_deleting(sessions, limit=limit)
orphans = await repository.collect_orphans(sessions, limit=limit)
print(f"Recovered {recovered}; marked {marked}; removed {removed}; orphans {orphans}.")
print(f"Next metadata scan cursor: {cursor or 'complete'}")
finally:
await engine.dispose()
store.close()
def main() -> None:
parser = argparse.ArgumentParser(description="Collect bounded old unreferenced artifacts")
parser.add_argument("--limit", type=int, default=100, choices=range(1, 101))
parser.add_argument("--after")
args = parser.parse_args()
try:
asyncio.run(run(args.limit, args.after))
except (ValueError, OSError, SQLAlchemyError):
parser.exit(1, "Artifact collection rejected; check storage and database configuration.\n")
if __name__ == "__main__":
main()
@@ -0,0 +1,21 @@
"""Metadata ownership and lifecycle for immutable, digest-addressed bytes."""
from sqlalchemy import BigInteger, CheckConstraint, String, UniqueConstraint
from sqlalchemy.orm import Mapped, mapped_column
from kf_oj.infra.db.base import Base
from kf_oj.infra.db.columns import CreatedAt
class Artifact(CreatedAt, Base):
__tablename__ = "artifacts"
__table_args__ = (
CheckConstraint("digest ~ '^[0-9a-f]{64}$'", name="digest"),
CheckConstraint("bytes BETWEEN 0 AND 67108864", name="bytes"),
CheckConstraint("state IN ('ready', 'deleting')", name="state"),
UniqueConstraint("digest", "state"),
)
digest: Mapped[str] = mapped_column(String(64), primary_key=True)
bytes: Mapped[int] = mapped_column(BigInteger)
state: Mapped[str] = mapped_column(String(8), server_default="ready")
@@ -0,0 +1,144 @@
"""Byte durability precedes metadata. Digest locks span caller reference transactions."""
import asyncio
from datetime import timedelta
from sqlalchemy import func, select, text, update
from sqlalchemy.dialects.postgresql import insert
from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from kf_oj.contracts.judging import ArtifactReference
from kf_oj.infra.artifacts.models import Artifact
from kf_oj.infra.artifacts.store import ArtifactError, ArtifactStore
async def lock_digest(session: AsyncSession, digest: str) -> None:
await session.execute(
text("SELECT pg_advisory_xact_lock(:key)"),
{"key": int(digest[:16], 16) & ((1 << 63) - 1)},
)
class ArtifactRepository:
def __init__(self, store: ArtifactStore):
self.store = store
async def put(
self, session: AsyncSession, data: bytes, expected: ArtifactReference | None = None
) -> ArtifactReference:
reference = self.store.reference(data)
if expected is not None and reference != expected:
raise ArtifactError("artifact digest or length mismatch")
await lock_digest(session, reference.digest)
existing = await session.get(Artifact, reference.digest)
if existing is not None and (
existing.state != "ready" or existing.bytes != reference.bytes
):
raise ArtifactError("artifact metadata unavailable")
await asyncio.to_thread(self.store.put, data, reference)
await session.execute(
insert(Artifact)
.values(digest=reference.digest, bytes=reference.bytes)
.on_conflict_do_nothing(index_elements=[Artifact.digest])
)
return reference
async def read(self, session: AsyncSession, reference: ArtifactReference) -> bytes:
metadata = await session.get(Artifact, reference.digest)
if metadata is None or metadata.state != "ready" or metadata.bytes != reference.bytes:
raise ArtifactError("artifact metadata unavailable")
return await asyncio.to_thread(self.store.read, reference)
async def check_metadata(self, session: AsyncSession, reference: ArtifactReference) -> None:
metadata = await session.get(Artifact, reference.digest)
if metadata is None or metadata.state != "ready" or metadata.bytes != reference.bytes:
raise ArtifactError("artifact metadata unavailable")
async def collect_metadata(
self,
sessions: async_sessionmaker,
*,
min_age_seconds: int = 86400,
limit: int = 100,
after: str | None = None,
) -> tuple[int, str | None]:
if min_age_seconds < 3600 or not 1 <= limit <= 100:
raise ArtifactError("invalid collection budget")
if after is not None:
ArtifactReference(digest=after, bytes=0)
async with sessions() as session:
query = select(Artifact.digest).where(
Artifact.state == "ready",
Artifact.created_at < func.now() - timedelta(seconds=min_age_seconds),
)
if after is not None:
query = query.where(Artifact.digest > after)
names = list(await session.scalars(query.order_by(Artifact.digest).limit(limit)))
marked = 0
for digest in names:
async with sessions.begin() as session:
try:
async with session.begin_nested():
changed = await self.mark_deleting(session, digest)
marked += int(changed)
except IntegrityError:
# Referenced rows are protected by their generated ready-state foreign key.
# No domain ORM imports or reverse dependencies are needed for reclamation.
pass
return marked, names[-1] if len(names) == limit else None
async def collect_orphans(
self, sessions: async_sessionmaker, *, min_age_seconds: int = 86400, limit: int = 100
) -> int:
# Only filesystem orphans are collected here. Referenced metadata is never deleted.
# A durable row cannot become an orphan while another transaction publishes it:
# every writer and collector holds the same transaction-scoped digest lock.
names = await asyncio.to_thread(
self.store.old_digests, min_age_seconds=min_age_seconds, limit=limit
)
collected = 0
for digest in names:
async with sessions.begin() as session:
await lock_digest(session, digest)
if await session.get(Artifact, digest) is None:
await asyncio.to_thread(self.store.remove, digest)
collected += 1
await asyncio.to_thread(
self.store.clean_temporary, min_age_seconds=min_age_seconds, limit=limit
)
return collected
async def mark_deleting(self, session: AsyncSession, digest: str) -> bool:
# FK(digest, 'ready') on every business reference rejects this transition if referenced.
await lock_digest(session, digest)
result = await session.scalar(
update(Artifact)
.where(Artifact.digest == digest, Artifact.state == "ready")
.values(state="deleting")
.returning(Artifact.digest)
)
return result is not None
async def finish_deleting(self, sessions: async_sessionmaker, *, limit: int = 100) -> int:
if not 1 <= limit <= 100:
raise ArtifactError("invalid collection budget")
async with sessions() as session:
names = list(
await session.scalars(
select(Artifact.digest)
.where(Artifact.state == "deleting")
.order_by(Artifact.digest)
.limit(limit)
)
)
count = 0
for digest in names:
async with sessions.begin() as session:
await lock_digest(session, digest)
metadata = await session.get(Artifact, digest)
if metadata is not None and metadata.state == "deleting":
await asyncio.to_thread(self.store.remove, digest)
await session.delete(metadata)
count += 1
return count
+208
View File
@@ -0,0 +1,208 @@
"""Private immutable byte store. Authorization is applied before calling this adapter."""
import fcntl
import hashlib
import os
import re
import stat
import time
from collections.abc import Iterator
from contextlib import contextmanager
from pathlib import Path
from uuid import uuid4
from kf_oj.contracts.judging import MAX_PACKAGE_BYTES, ArtifactReference
DIGEST = re.compile(r"[0-9a-f]{64}\Z")
TEMP = re.compile(r"\.tmp-[0-9a-f]{32}\Z")
class ArtifactError(ValueError):
"""Sanitized artifact failure; never includes paths or private byte content."""
class ArtifactStore:
def __init__(self, root: Path, *, max_bytes: int = 512 * 1024 * 1024, max_files: int = 4096):
if not 1 <= max_bytes <= 16 * 1024**3 or not 1 <= max_files <= 65536:
raise ArtifactError("invalid storage capacity")
root = root.absolute()
root.mkdir(mode=0o700, parents=True, exist_ok=True)
if root.resolve() != root:
raise ArtifactError("storage root must not contain links")
self._directory = os.open(root, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW | os.O_CLOEXEC)
self.max_bytes = max_bytes
self.max_files = max_files
info = os.fstat(self._directory)
if info.st_uid != os.getuid() or stat.S_IMODE(info.st_mode) != 0o700:
os.close(self._directory)
raise ArtifactError("storage root must be private and owned")
self._closed = False
def close(self) -> None:
if not self._closed:
os.close(self._directory)
self._closed = True
@contextmanager
def exclusive(self) -> Iterator[None]:
# Open a different description for every operation: flock also serializes threads.
descriptor = os.open(
".lock",
os.O_RDWR | os.O_CREAT | os.O_NOFOLLOW | os.O_CLOEXEC,
0o600,
dir_fd=self._directory,
)
try:
self._regular(os.fstat(descriptor))
deadline = time.monotonic() + 5
while True:
try:
fcntl.flock(descriptor, fcntl.LOCK_EX | fcntl.LOCK_NB)
break
except BlockingIOError:
if time.monotonic() >= deadline:
raise ArtifactError("storage lock deadline exceeded") from None
time.sleep(0.01)
yield
finally:
os.close(descriptor)
@staticmethod
def _regular(info: os.stat_result) -> None:
if (
not stat.S_ISREG(info.st_mode)
or info.st_nlink != 1
or info.st_uid != os.getuid()
or stat.S_IMODE(info.st_mode) != 0o600
):
raise ArtifactError("invalid storage entry")
def _entries(self) -> list[tuple[str, os.stat_result]]:
entries = []
with os.scandir(self._directory) as directory:
for entry in directory:
if entry.name == ".lock":
continue
if not DIGEST.fullmatch(entry.name) and not TEMP.fullmatch(entry.name):
raise ArtifactError("unexpected storage entry")
info = entry.stat(follow_symlinks=False)
self._regular(info)
entries.append((entry.name, info))
if len(entries) > self.max_files:
raise ArtifactError("storage file capacity exceeded")
return entries
@staticmethod
def reference(data: bytes) -> ArtifactReference:
if type(data) is not bytes or len(data) > MAX_PACKAGE_BYTES:
raise ArtifactError("artifact size outside policy")
return ArtifactReference(digest=hashlib.sha256(data).hexdigest(), bytes=len(data))
def put(self, data: bytes, expected: ArtifactReference | None = None) -> ArtifactReference:
reference = self.reference(data)
if expected is not None and reference != expected:
raise ArtifactError("artifact digest or length mismatch")
with self.exclusive():
entries = self._entries()
if any(name == reference.digest for name, _ in entries):
self._read(reference)
return reference
if len(entries) + 1 > self.max_files or (
sum(info.st_size for _, info in entries) + len(data) > self.max_bytes
):
raise ArtifactError("storage capacity exhausted")
temporary = ".tmp-" + uuid4().hex
descriptor = os.open(
temporary,
os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW | os.O_CLOEXEC,
0o600,
dir_fd=self._directory,
)
try:
with os.fdopen(descriptor, "wb") as output:
output.write(data)
output.flush()
os.fsync(output.fileno())
# Locking covers every mutation. Never replace or repair an existing digest.
os.rename(
temporary,
reference.digest,
src_dir_fd=self._directory,
dst_dir_fd=self._directory,
)
os.fsync(self._directory)
finally:
try:
os.unlink(temporary, dir_fd=self._directory)
except FileNotFoundError:
pass
return reference
def _read(self, reference: ArtifactReference) -> bytes:
if (
type(reference.digest) is not str
or not DIGEST.fullmatch(reference.digest)
or type(reference.bytes) is not int
or not 0 <= reference.bytes <= MAX_PACKAGE_BYTES
):
raise ArtifactError("invalid artifact reference")
try:
descriptor = os.open(
reference.digest,
os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK | os.O_CLOEXEC,
dir_fd=self._directory,
)
with os.fdopen(descriptor, "rb") as source:
info = os.fstat(source.fileno())
self._regular(info)
if info.st_size != reference.bytes:
raise ArtifactError("artifact digest or length mismatch")
data = source.read(reference.bytes + 1)
if self.reference(data) != reference:
raise ArtifactError("artifact digest or length mismatch")
return data
except OSError:
raise ArtifactError("artifact unavailable") from None
def read(self, reference: ArtifactReference) -> bytes:
with self.exclusive():
return self._read(reference)
def remove(self, digest: str) -> None:
if not DIGEST.fullmatch(digest):
raise ArtifactError("invalid artifact identifier")
with self.exclusive():
try:
info = os.stat(digest, dir_fd=self._directory, follow_symlinks=False)
self._regular(info)
os.unlink(digest, dir_fd=self._directory)
os.fsync(self._directory)
except FileNotFoundError:
pass
def old_digests(self, *, min_age_seconds: int, limit: int) -> tuple[str, ...]:
if not 3600 <= min_age_seconds or not 1 <= limit <= 100:
raise ArtifactError("invalid collection budget")
with self.exclusive():
cutoff = time.time() - min_age_seconds
return tuple(
name
for name, info in self._entries()
if DIGEST.fullmatch(name) and info.st_mtime < cutoff
)[:limit]
def clean_temporary(self, *, min_age_seconds: int = 86400, limit: int = 100) -> int:
if min_age_seconds < 3600 or not 1 <= limit <= 100:
raise ArtifactError("invalid collection budget")
with self.exclusive():
cutoff = time.time() - min_age_seconds
names = [
name
for name, info in self._entries()
if TEMP.fullmatch(name) and info.st_mtime < cutoff
][:limit]
for name in names:
os.unlink(name, dir_fd=self._directory)
if names:
os.fsync(self._directory)
return len(names)
+30
View File
@@ -0,0 +1,30 @@
"""Bounded loopback control adapter; never owns evaluation database writes."""
from uuid import UUID
import httpx
class CoordinatorControl:
def __init__(self, url: str, token: str):
self.url, self.token = url.rstrip("/"), token
async def cancel(self, evaluation_id: UUID) -> bool:
try:
async with httpx.AsyncClient(
timeout=3, trust_env=False, follow_redirects=False
) as client:
# Response has no business payload. Do not buffer or log arbitrary remote bodies.
async with client.stream(
"POST",
self.url + "/internal/cancel",
headers={"Authorization": "Bearer " + self.token},
json={
"schema_version": 1,
"kind": "judge.evaluation",
"evaluation_id": str(evaluation_id),
},
) as response:
return response.status_code == 204
except httpx.HTTPError:
return False
+12
View File
@@ -0,0 +1,12 @@
"""Shared column types; business schemas are registered by composition roots."""
from datetime import datetime
from sqlalchemy import DateTime, func
from sqlalchemy.orm import Mapped, mapped_column
class CreatedAt:
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), nullable=False
)
+15 -1
View File
@@ -6,7 +6,21 @@ from kf_oj.core.settings import Settings
def build_engine(settings: Settings) -> AsyncEngine:
if settings.database_url is None:
raise ValueError("KF_OJ_DATABASE_URL is required for database-backed operations")
return create_async_engine(settings.database_url.get_secret_value(), pool_pre_ping=True)
return create_async_engine(
settings.database_url.get_secret_value(),
pool_pre_ping=True,
pool_size=5,
max_overflow=0,
pool_timeout=5,
connect_args={
"command_timeout": 10,
"server_settings": {
"statement_timeout": "10000",
"lock_timeout": "5000",
"idle_in_transaction_session_timeout": "30000",
},
},
)
def build_session_factory(engine: AsyncEngine) -> async_sessionmaker:
@@ -0,0 +1,31 @@
"""Transactional notifications; contains IDs, never source or hidden artifacts."""
from datetime import datetime
from uuid import UUID
from sqlalchemy import CheckConstraint, DateTime, ForeignKey, Index, String, func
from sqlalchemy.orm import Mapped, mapped_column
from kf_oj.infra.db.base import Base
from kf_oj.infra.db.columns import CreatedAt
class Outbox(CreatedAt, Base):
__tablename__ = "outbox"
__table_args__ = (
CheckConstraint("failures BETWEEN 0 AND 100", name="failure_budget"),
CheckConstraint("kind = 'judge.evaluation'", name="kind"),
Index("ix_outbox_delivery", "delivered_at", "next_attempt_at", "id"),
)
id: Mapped[UUID] = mapped_column(primary_key=True)
evaluation_id: Mapped[UUID] = mapped_column(ForeignKey("evaluations.id"), unique=True)
kind: Mapped[str] = mapped_column(String(32), server_default="judge.evaluation")
failures: Mapped[int] = mapped_column(server_default="0")
next_attempt_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now()
)
locked_until: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
lock_owner: Mapped[UUID | None]
delivered_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
last_error_code: Mapped[str | None] = mapped_column(String(32))
@@ -0,0 +1,183 @@
"""Finite transactional outbox delivery; confirmations are notifications, not grades."""
import asyncio
import logging
import signal
from uuid import uuid4
import aio_pika
from sqlalchemy import text
from kf_oj.contracts.judge import JudgeTaskNotification
from kf_oj.core.settings import Settings
from kf_oj.infra.db.session import build_engine, build_session_factory
QUEUE = "kf.judge.evaluations.v1"
QUEUE_ARGUMENTS = {
"x-max-length": 64,
"x-max-length-bytes": 32768,
"x-overflow": "reject-publish",
"x-message-ttl": 600000,
}
class OutboxRepository:
def __init__(self, sessions):
self.sessions = sessions
async def claim(self):
owner = uuid4()
async with self.sessions.begin() as s:
row = (
await s.execute(
text("""WITH candidate AS (
SELECT id FROM outbox
WHERE delivered_at IS NULL AND next_attempt_at<=clock_timestamp()
AND (locked_until IS NULL OR locked_until<=clock_timestamp())
ORDER BY next_attempt_at,id FOR UPDATE SKIP LOCKED LIMIT 1)
UPDATE outbox o SET lock_owner=:owner,
locked_until=clock_timestamp()+interval '15 seconds'
FROM candidate c WHERE o.id=c.id RETURNING o.id,o.evaluation_id"""),
{"owner": owner},
)
).first()
return None if row is None else (row.id, row.evaluation_id, owner)
async def confirmed(self, item):
async with self.sessions.begin() as s:
result = await s.execute(
text("""UPDATE outbox SET delivered_at=clock_timestamp(),
locked_until=NULL,lock_owner=NULL,last_error_code=NULL
WHERE id=:id AND lock_owner=:owner AND locked_until>clock_timestamp()"""),
{"id": item[0], "owner": item[2]},
)
return result.rowcount == 1
async def failed(self, item):
async with self.sessions.begin() as s:
await s.execute(
text("""UPDATE outbox SET failures=least(failures+1,100),
next_attempt_at=clock_timestamp()+
least(power(2,least(failures+1,6)),60)*interval '1 second',
locked_until=NULL,lock_owner=NULL,last_error_code='broker_unavailable'
WHERE id=:id AND lock_owner=:owner AND locked_until>clock_timestamp()"""),
{"id": item[0], "owner": item[2]},
)
class ConfirmedBroker:
def __init__(self, url):
# Parent disabled does not suppress descendant loggers; set their inherited level.
for name in ("aio_pika", "aiormq", "pamqp"):
logging.getLogger(name).setLevel(logging.CRITICAL)
self.url = url
self.connection = None
self.channel = None
async def open(self):
async with asyncio.timeout(10):
await self._open()
async def _open(self):
# Use a finite connect, not a reconnect loop holding an outbox lock.
self.connection = await aio_pika.connect(self.url, timeout=5, heartbeat=30)
self.channel = await self.connection.channel(publisher_confirms=True, on_return_raises=True)
await self.channel.declare_queue(QUEUE, durable=True, arguments=QUEUE_ARGUMENTS, timeout=5)
async def publish(self, item):
# The complete operation ends before the database claim expires.
async with asyncio.timeout(12):
await self._publish(item)
async def _publish(self, item):
if self.channel is None or self.channel.is_closed:
await self.open()
body = (
JudgeTaskNotification(schema_version=1, kind="judge.evaluation", evaluation_id=item[1])
.model_dump_json()
.encode()
)
assert len(body) <= 512
confirmation = await self.channel.default_exchange.publish(
aio_pika.Message(
body,
content_type="application/json",
delivery_mode=aio_pika.DeliveryMode.PERSISTENT,
message_id=str(item[0]),
),
routing_key=QUEUE,
mandatory=True,
timeout=5,
)
from pamqp.commands import Basic
if not isinstance(confirmation, Basic.Ack):
raise RuntimeError("notification not confirmed")
async def close(self):
if self.connection is not None and not self.connection.is_closed:
try:
await asyncio.wait_for(self.connection.close(), timeout=2)
except Exception:
pass
self.connection = self.channel = None
async def run(settings: Settings, stop: asyncio.Event):
if settings.amqp_url is None:
raise ValueError("AMQP configuration required")
engine = build_engine(settings)
repository = OutboxRepository(build_session_factory(engine))
broker = ConfirmedBroker(settings.amqp_url.get_secret_value())
logger = logging.getLogger("kf_oj.outbox")
# Client exceptions may include connection URIs; suppress transport-library logging.
for name in ("aio_pika", "aiormq", "pamqp"):
logging.getLogger(name).disabled = True
try:
async with engine.connect() as c:
allowed = await c.scalar(
text("""SELECT NOT r.rolsuper AND NOT r.rolbypassrls
AND pg_has_role(current_user,'kf_oj_platform','MEMBER')
AND NOT has_column_privilege(current_user,'evaluations','state','UPDATE')
FROM pg_roles r WHERE r.rolname=current_user""")
)
if not allowed:
raise ValueError("dedicated platform database role required")
while not stop.is_set():
try:
item = await repository.claim()
if item is not None:
try:
await broker.publish(item)
await repository.confirmed(item)
except Exception:
await repository.failed(item)
await broker.close()
logger.warning("outbox broker unavailable")
except Exception:
logger.warning("outbox database unavailable")
try:
await asyncio.wait_for(stop.wait(), timeout=1)
except TimeoutError:
pass
finally:
await broker.close()
await engine.dispose()
def main():
async def entry():
stop = asyncio.Event()
loop = asyncio.get_running_loop()
for sig in (signal.SIGTERM, signal.SIGINT):
loop.add_signal_handler(sig, stop.set)
await run(Settings(), stop)
try:
asyncio.run(entry())
except Exception:
raise SystemExit("outbox unavailable") from None
if __name__ == "__main__":
main()
+23 -2
View File
@@ -1,5 +1,26 @@
# identity
负责账号、登录凭证、会话与角色;向其他模块提供经过验证的用户身份和授权依据。
后续 `public.py` 暴露身份查询接口,`service.py` 编排用例,`repository.py` 封装持久化。
用户身份不能来自前端自报字段或模型参数。目前仅建立模块边界,未实现登录。
`public.py` 提供已验证主体,`service.py` 管理账号和可撤销会话,`repository.py` 封装持久化。
用户身份不能来自前端自报字段或模型参数。
本地管理命令为 `uv run --locked kf-accounts create <username>` / `disable <username>`。
创建口令由隐藏终端提示读取和确认,不接受命令行参数、环境中的默认口令或自助注册。
用户名为 3–24 个小写 ASCII 字母、数字或下划线;口令 12–128 字符,最多 512 UTF-8 字节。
密码摘要由新增的 `argon2-cffi` 依赖提供 Argon2id,固定 19 MiB/2 次/1 并行度。
参数依据 [OWASP](https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html)
和 [argon2-cffi 文档](https://argon2-cffi.readthedocs.io/en/stable/parameters.html)。
登录接口为 `/api/v1/auth/login`,退出 `/logout`,当前主体 `/me`。会话使用随机 256-bit token,
数据库只保存 token / CSRF 摘要;默认 12 小时有效,单账号保留至多 10 个有效会话。
账号禁用立即拒绝登录和旧会话。请求只采用 ASGI 连接 peer,不读取客户端转发地址。
受控启动使用 `--no-proxy-headers`,避免服务器中间件预先替换 peer;反向代理后的用户共享
该连接预算,引入可信代理的独立限流策略之前不能直接打开客户端转发头信任。
每 peer 至多 10 次登录/15 分钟,预算通过真实数据库串行更新,失败尝试也消耗预算。
摘要计算最多同时 2 次,超出直接 429;限流表至多 4096 桶且每次有界回收过期桶。
有副作用的浏览器请求必须匹配 `KF_OJ_BROWSER_ORIGINS` 的显式来源;生产只接受 HTTPS 来源,
必须提供生产来源配置。会话 Cookie 为 HttpOnly / SameSite=Strict,生产 Secure;CSRF Cookie
供前端读出请求 token,在 `X-CSRF-Token` 携带,服务端核对其与当前会话绑定的摘要。
登录、主体和退出响应不缓存;请求校验错误不回显密码或输入值。API 层限制请求字节、块数和
读取期限。跨服务 Worker 凭证不使用用户 Cookie。
+49
View File
@@ -0,0 +1,49 @@
"""Controlled local account management; passwords are read from the terminal only."""
import argparse
import asyncio
import getpass
from sqlalchemy.exc import IntegrityError
from kf_oj.core.settings import Settings
from kf_oj.infra.db.session import build_engine, build_session_factory
from kf_oj.modules.identity.service import IdentityService
async def run(command: str, username: str, password: str | None = None) -> None:
engine = build_engine(Settings())
try:
service = IdentityService(build_session_factory(engine))
if command == "create":
assert password is not None
await service.create_account(username, password)
print("Account created.")
else:
if not await service.disable_account(username):
raise ValueError("account not found")
print("Account disabled; sessions revoked.")
finally:
await engine.dispose()
def main() -> None:
parser = argparse.ArgumentParser(description="Manage local OJ accounts")
parser.add_argument("command", choices=("create", "disable"))
parser.add_argument("username")
args = parser.parse_args()
password = None
if args.command == "create":
password = getpass.getpass("Password (12–128 characters): ")
if password != getpass.getpass("Repeat password: "):
parser.error("passwords differ")
try:
asyncio.run(run(args.command, args.username, password))
except IntegrityError:
parser.exit(1, "Account operation rejected by database constraints.\n")
except ValueError as error:
parser.exit(1, f"Account operation rejected: {error}.\n")
if __name__ == "__main__":
main()
@@ -0,0 +1,52 @@
"""Identity-owned persistent account and revocable session records."""
from datetime import datetime
from uuid import UUID
from sqlalchemy import Boolean, CheckConstraint, DateTime, ForeignKey, Index, String, Text
from sqlalchemy.orm import Mapped, mapped_column
from kf_oj.infra.db.base import Base
from kf_oj.infra.db.columns import CreatedAt
class Account(CreatedAt, Base):
__tablename__ = "accounts"
__table_args__ = (
CheckConstraint("username ~ '^[a-z0-9_]{3,24}$'", name="canonical_username"),
CheckConstraint("char_length(password_hash) BETWEEN 40 AND 256", name="password_hash_size"),
)
id: Mapped[UUID] = mapped_column(primary_key=True)
username: Mapped[str] = mapped_column(String(24), unique=True)
password_hash: Mapped[str] = mapped_column(Text)
enabled: Mapped[bool] = mapped_column(Boolean, server_default="true")
class Session(CreatedAt, Base):
__tablename__ = "sessions"
__table_args__ = (
CheckConstraint("token_hash ~ '^[0-9a-f]{64}$'", name="token_digest"),
CheckConstraint("csrf_hash ~ '^[0-9a-f]{64}$'", name="csrf_digest"),
CheckConstraint("expires_at > created_at", name="expiry"),
Index("ix_sessions_account_expiry", "account_id", "expires_at"),
)
token_hash: Mapped[str] = mapped_column(String(64), primary_key=True)
account_id: Mapped[UUID] = mapped_column(ForeignKey("accounts.id"))
csrf_hash: Mapped[str] = mapped_column(String(64))
expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True))
revoked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
class LoginThrottle(Base):
__tablename__ = "login_throttles"
__table_args__ = (
CheckConstraint("bucket_hash ~ '^[0-9a-f]{64}$'", name="bucket_digest"),
CheckConstraint("attempts BETWEEN 1 AND 10", name="attempt_budget"),
Index("ix_login_throttles_reset", "reset_at"),
)
bucket_hash: Mapped[str] = mapped_column(String(64), primary_key=True)
attempts: Mapped[int]
reset_at: Mapped[datetime] = mapped_column(DateTime(timezone=True))
@@ -0,0 +1,32 @@
"""Argon2id password verification with explicit bounded parameters."""
import re
from argon2 import PasswordHasher, Type
from argon2.exceptions import InvalidHashError, VerificationError
USERNAME = re.compile(r"[a-z0-9_]{3,24}\Z")
HASHER = PasswordHasher(time_cost=2, memory_cost=19456, parallelism=1, type=Type.ID)
def validate_account(username: str, password: str) -> None:
if not USERNAME.fullmatch(username) or not 12 <= len(password) <= 128:
raise ValueError("username or password outside account policy")
if len(password.encode("utf-8")) > 512:
raise ValueError("password outside byte policy")
def hash_password(username: str, password: str) -> str:
validate_account(username, password)
return HASHER.hash(password)
def verify_password(stored: str, password: str) -> bool:
try:
# Stored hashes are owner-controlled, but don't allow corrupted parameters to allocate
# arbitrary memory. Account creation always uses these exact parameters.
if not stored.startswith("$argon2id$v=19$m=19456,t=2,p=1$"):
return False
return HASHER.verify(stored, password)
except (VerificationError, InvalidHashError):
return False
@@ -0,0 +1,18 @@
"""Only verified principals cross business module boundaries."""
from dataclasses import dataclass
from uuid import UUID
@dataclass(frozen=True)
class Principal:
account_id: UUID
username: str
class AuthenticationFailed(ValueError):
pass
class LoginThrottled(ValueError):
pass
@@ -0,0 +1,85 @@
from datetime import datetime
from sqlalchemy import delete, func, select, text, update
from sqlalchemy.dialects.postgresql import insert
from sqlalchemy.ext.asyncio import AsyncSession
from kf_oj.modules.identity.models import Account, LoginThrottle, Session
from kf_oj.modules.identity.public import LoginThrottled
async def consume_login_budget(
session: AsyncSession, bucket: str, now: datetime, reset_at: datetime
) -> None:
# An upsert establishes the key before FOR UPDATE; requests share one DB-backed budget.
inserted = await session.scalar(
insert(LoginThrottle)
.values(bucket_hash=bucket, attempts=1, reset_at=reset_at)
.on_conflict_do_nothing(index_elements=[LoginThrottle.bucket_hash])
.returning(LoginThrottle.bucket_hash)
)
row = await session.scalar(
select(LoginThrottle).where(LoginThrottle.bucket_hash == bucket).with_for_update()
)
assert row is not None
# The initial upsert consumes this request only if there was no prior row.
# A separate returning flag avoids granting an extra request during concurrency.
if inserted is not None:
return
if row.reset_at <= now:
row.attempts, row.reset_at = 1, reset_at
elif row.attempts >= 10:
raise LoginThrottled("login request budget exhausted")
else:
row.attempts += 1
async def prune_throttles(session: AsyncSession, now: datetime, *, limit: int = 100) -> None:
await session.execute(text("SELECT pg_advisory_xact_lock(-740164777)"))
keys = select(LoginThrottle.bucket_hash).where(LoginThrottle.reset_at <= now)
keys = keys.order_by(LoginThrottle.reset_at).limit(limit).with_for_update(skip_locked=True)
await session.execute(delete(LoginThrottle).where(LoginThrottle.bucket_hash.in_(keys)))
async def check_throttle_capacity(session: AsyncSession, bucket: str) -> None:
if await session.get(LoginThrottle, bucket) is None:
if await session.scalar(select(func.count()).select_from(LoginThrottle)) >= 4096:
raise LoginThrottled("login throttle capacity exhausted")
async def account_by_username(session: AsyncSession, username: str) -> Account | None:
return await session.scalar(select(Account).where(Account.username == username))
async def current_session(session: AsyncSession, token_hash: str):
return (
await session.execute(
select(Session, Account)
.join(Account)
.where(
Session.token_hash == token_hash,
Session.revoked_at.is_(None),
Session.expires_at > func.now(),
Account.enabled.is_(True),
)
)
).one_or_none()
async def prune_sessions(session: AsyncSession, account_id) -> None:
await session.execute(
delete(Session).where(
Session.account_id == account_id,
(Session.expires_at <= func.now()) | Session.revoked_at.is_not(None),
)
)
# Keep at most nine older valid sessions before adding the new one.
old = select(Session.token_hash).where(Session.account_id == account_id)
old = old.order_by(Session.created_at.desc(), Session.token_hash).offset(9)
await session.execute(delete(Session).where(Session.token_hash.in_(old)))
async def revoke(session: AsyncSession, token_hash: str) -> None:
await session.execute(
update(Session).where(Session.token_hash == token_hash).values(revoked_at=func.now())
)
@@ -0,0 +1,131 @@
import asyncio
import hashlib
import hmac
import re
import secrets
from dataclasses import dataclass, field
from datetime import datetime, timedelta
from uuid import uuid4
from sqlalchemy import func, select, update
from sqlalchemy.ext.asyncio import async_sessionmaker
from kf_oj.modules.identity import repository
from kf_oj.modules.identity.models import Account, Session
from kf_oj.modules.identity.passwords import (
HASHER,
hash_password,
validate_account,
verify_password,
)
from kf_oj.modules.identity.public import AuthenticationFailed, LoginThrottled, Principal
TOKEN = re.compile(r"[0-9a-f]{64}\Z")
def token_digest(value: str) -> str:
return hashlib.sha256(value.encode("ascii")).hexdigest()
@dataclass(frozen=True)
class IssuedSession:
principal: Principal
expires_at: datetime
token: str = field(repr=False)
csrf_token: str = field(repr=False)
class IdentityService:
def __init__(self, sessions: async_sessionmaker, *, lifetime_seconds: int = 43200):
if not 300 <= lifetime_seconds <= 86400:
raise ValueError("invalid session lifetime")
self.sessions = sessions
self.lifetime_seconds = lifetime_seconds
self._hash_slots = asyncio.Semaphore(2)
self._dummy_hash = HASHER.hash(secrets.token_hex(32))
async def create_account(self, username: str, password: str) -> Principal:
hashed = await asyncio.to_thread(hash_password, username, password)
async with self.sessions.begin() as session:
account = Account(id=uuid4(), username=username, password_hash=hashed)
session.add(account)
return Principal(account.id, username)
async def disable_account(self, username: str) -> bool:
async with self.sessions.begin() as session:
account = await repository.account_by_username(session, username)
if account is None:
return False
account.enabled = False
await session.execute(
update(Session)
.where(Session.account_id == account.id)
.values(revoked_at=func.now())
)
return True
async def login(self, username: str, password: str, *, peer: str) -> IssuedSession:
validate_account(username, password)
if not peer or len(peer) > 128:
raise AuthenticationFailed("invalid login context")
# Fail fast instead of allowing an unbounded password-hash queue.
if self._hash_slots.locked():
raise LoginThrottled("password verification capacity exhausted")
async with self._hash_slots:
async with self.sessions.begin() as session:
now = await session.scalar(select(func.now()))
assert now is not None
await repository.prune_throttles(session, now)
# Direct peer only; caller must never substitute forwarded client headers.
bucket = hashlib.sha256(peer.encode("utf-8")).hexdigest()
await repository.check_throttle_capacity(session, bucket)
await repository.consume_login_budget(
session, bucket, now, now + timedelta(minutes=15)
)
async with self.sessions() as session:
account = await repository.account_by_username(session, username)
hashed = account.password_hash if account is not None else self._dummy_hash
correct = await asyncio.to_thread(verify_password, hashed, password)
if not correct or account is None or not account.enabled:
raise AuthenticationFailed("invalid credentials")
token, csrf = secrets.token_hex(32), secrets.token_hex(32)
async with self.sessions.begin() as session:
# Serialize with disabling the account and concurrent logins before issuing.
account = await session.scalar(
select(Account).where(Account.id == account.id).with_for_update()
)
if account is None or not account.enabled:
raise AuthenticationFailed("invalid credentials")
now = await session.scalar(select(func.now()))
expires = now + timedelta(seconds=self.lifetime_seconds)
await repository.prune_sessions(session, account.id)
session.add(
Session(
token_hash=token_digest(token),
account_id=account.id,
csrf_hash=token_digest(csrf),
expires_at=expires,
)
)
principal = Principal(account.id, account.username)
return IssuedSession(principal, expires, token, csrf)
async def authenticate(self, token: str | None, *, csrf_token: str | None = None) -> Principal:
if token is None or not TOKEN.fullmatch(token):
raise AuthenticationFailed("invalid session")
async with self.sessions() as session:
row = await repository.current_session(session, token_digest(token))
if row is None:
raise AuthenticationFailed("invalid session")
record, account = row
if csrf_token is not None:
if not TOKEN.fullmatch(csrf_token) or not hmac.compare_digest(
record.csrf_hash, token_digest(csrf_token)
):
raise AuthenticationFailed("invalid request token")
return Principal(account.id, account.username)
async def logout(self, token: str, csrf_token: str) -> None:
await self.authenticate(token, csrf_token=csrf_token)
async with self.sessions.begin() as session:
await repository.revoke(session, token_digest(token))
+30 -2
View File
@@ -1,5 +1,33 @@
# problems
负责题目逻辑身份、不可变版本、导入、标签、限制、发布状态与测试工件元数据。
跨模块访问通过后续 `public.py`;大文件存取通过 `infra/artifacts` 的适配器。
隐藏数据与公开样例分别授权。当前未创建题库表或导入接口。
跨模块通过 `public.py` 的不可变判题投影;大文件存取通过 `infra/artifacts` 的适配器。
公开题面/显式样例与隐藏输入/答案分别授权。公开 API 不返回工件摘要或存储路径。
`uv run --locked kf-problems <directory>` 导入并发布一个新版本。目录必须属于当前操作账号,
权限 0700;`problem.json` 遵循 `import_format.py`,只引用该目录下的单层普通文件名。
不解压归档、不跟随链接、不读取任意路径、不执行导入内容。默认清单可由 `--manifest` 指定。
要求配置 `KF_OJ_ARTIFACT_ROOT` 和可信 `KF_OJ_JUDGE_ADMISSION` JSON;同一准入配置应交付给节点,
不能由题目放宽。结构示例来自共享 `judge-admission.valid.json`,不是当前宿主隔离验收证明。
v1 清单固定 C17/C++20、工具链/镜像版本、精确字节比较、编译/运行/整个作业预算,并声明按序
测试点 `{key,input_file,expected_file}` 和最多十对公开样例 `{input_file,output_file}`。
题面以 `statement_file` 读取 UTF-8。每份样例至多 64 KiB,响应采用 base64 保留二进制数据。
同一 slug 受事务锁保护,每次发布递增 revision;发布后题面、限额、样例、测试点不可修改。
旧评测继续绑定旧版本;当前列表指向新版本。工件先持久化,再在同一事务建立引用和发布。
`GET /api/v1/problems` 以 slug 游标分页(`after` / `limit`,最多 100 项),
`GET /api/v1/problems/versions/{uuid}` 读取已发布版本。禁用逻辑题目时其版本对公开读取不可用。
内部判题投影仅供授权应用编排使用;外部 Worker 读取由 submissions 的有效租约端口控制。
v2 使用 `schema_version: 2` 与 `language_configurations`,每个配置包含 `language`、
完整 `toolchain` 身份、`compile_limits`、`run_limits` 和 `job_wall_time_us`。
可分别声明 `c17`、`cpp20`、`python313`、`java21`;不再同时提供 v1 顶层工具链或预算。
所有语言按可信节点配置准入,缺失能力/错误补丁版本/镜像摘要/过宽预算在发布前拒绝。
`judge.admission.example.json` 的镜像摘要为占位值,使用前替换为管理员制作镜像的实际摘要。
公开题面仅显示固定语言、版本、入口与预算,不公布宿主镜像路径和隐藏工件。
导入同一 slug 的 v2 版本保持原 v1 版本与提交不变。已有提交按其保存的协议版本解析,
不能重新序列化旧 canonical 数据来补入新字段。Java 使用 Main.java 与默认包 Main;
Python 使用 main.py。每个新版本的语言集合独立,浏览器不得为旧版本补开新语言。
+49
View File
@@ -0,0 +1,49 @@
"""Local, controlled import of ordinary files; no archive extraction or program execution."""
import argparse
import asyncio
from pathlib import Path
from sqlalchemy.exc import SQLAlchemyError
from kf_oj.core.settings import Settings
from kf_oj.infra.artifacts.repository import ArtifactRepository
from kf_oj.infra.artifacts.store import ArtifactStore
from kf_oj.infra.db.session import build_engine, build_session_factory
from kf_oj.modules.problems.service import ProblemService
async def run(root: Path, manifest: str) -> None:
settings = Settings()
if settings.artifact_root is None or settings.judge_admission is None:
raise ValueError("artifact root and trusted judge admission required")
store = ArtifactStore(
settings.artifact_root,
max_bytes=settings.artifact_max_bytes,
max_files=settings.artifact_max_files,
)
engine = build_engine(settings)
try:
service = ProblemService(
build_session_factory(engine), ArtifactRepository(store), settings.judge_admission
)
version_id = await service.import_directory(root, manifest)
print(f"Problem version published: {version_id}")
finally:
await engine.dispose()
store.close()
def main() -> None:
parser = argparse.ArgumentParser(description="Import immutable problem version")
parser.add_argument("directory", type=Path)
parser.add_argument("--manifest", default="problem.json")
args = parser.parse_args()
try:
asyncio.run(run(args.directory, args.manifest))
except (ValueError, OSError, SQLAlchemyError):
parser.exit(1, "Problem import rejected; check controlled files, budgets and database.\n")
if __name__ == "__main__":
main()
@@ -0,0 +1,86 @@
"""Local operator format. Filenames never cross into the judge execution protocol."""
from typing import Annotated, Literal, Self
from pydantic import Field, RootModel, model_validator
from kf_oj.contracts import judging_v2 as v2
from kf_oj.contracts.judging import (
MAX_CASES,
MAX_JOB_WALL_US,
Label,
Language,
RunLimits,
Toolchain,
Version,
WireModel,
)
FileName = Annotated[str, Field(strict=True, pattern=r"^[a-zA-Z0-9][a-zA-Z0-9_.-]{0,127}$")]
class ImportCase(WireModel):
key: Label
input_file: FileName
expected_file: FileName
class ImportSample(WireModel):
input_file: FileName
output_file: FileName
class ProblemImport(WireModel):
schema_version: Version
slug: Label
title: Annotated[str, Field(strict=True, min_length=1, max_length=160)]
statement_file: FileName
languages: Annotated[tuple[Language, ...], Field(min_length=1, max_length=2)]
toolchain: Toolchain
comparison: Literal["exact_bytes"]
compile_limits: RunLimits
run_limits: RunLimits
job_wall_time_us: Annotated[int, Field(strict=True, ge=1000, le=MAX_JOB_WALL_US)]
cases: Annotated[tuple[ImportCase, ...], Field(min_length=1, max_length=MAX_CASES)]
samples: Annotated[tuple[ImportSample, ...], Field(max_length=10)]
@model_validator(mode="after")
def unique(self) -> Self:
if len(set(self.languages)) != len(self.languages):
raise ValueError("duplicate language")
if len({case.key for case in self.cases}) != len(self.cases):
raise ValueError("duplicate testcase key")
if len({(c.input_file, c.expected_file) for c in self.cases}) != len(self.cases):
raise ValueError("duplicate testcase reference")
return self
class ProblemImportV2(WireModel):
schema_version: v2.Version
slug: Label
title: Annotated[str, Field(strict=True, min_length=1, max_length=160)]
statement_file: FileName
language_configurations: Annotated[
tuple[v2.LanguageConfiguration, ...], Field(min_length=1, max_length=4)
]
comparison: Literal["exact_bytes"]
cases: Annotated[tuple[ImportCase, ...], Field(min_length=1, max_length=MAX_CASES)]
samples: Annotated[tuple[ImportSample, ...], Field(max_length=10)]
@property
def languages(self) -> tuple[v2.Language, ...]:
return tuple(config.language for config in self.language_configurations)
@model_validator(mode="after")
def unique(self) -> Self:
if len(set(self.languages)) != len(self.languages):
raise ValueError("duplicate language configuration")
if len({case.key for case in self.cases}) != len(self.cases):
raise ValueError("duplicate testcase key")
if len({(case.input_file, case.expected_file) for case in self.cases}) != len(self.cases):
raise ValueError("duplicate testcase reference")
return self
class ImportEnvelope(RootModel[ProblemImport | ProblemImportV2]):
pass
@@ -0,0 +1,48 @@
import os
import re
import stat
from pathlib import Path
class LocalImportFiles:
def __init__(self, root: Path):
root = root.absolute()
if root.resolve() != root:
raise ValueError("import directory must not contain links")
self.descriptor = os.open(root, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW)
info = os.fstat(self.descriptor)
if info.st_uid != os.getuid() or stat.S_IMODE(info.st_mode) != 0o700:
self.close()
raise ValueError("import directory must be private and owned")
def close(self):
os.close(self.descriptor)
def read(self, filename: str, maximum: int) -> bytes:
if not re.fullmatch(r"[a-zA-Z0-9][a-zA-Z0-9_.-]{0,127}", filename):
raise ValueError("invalid local filename")
try:
descriptor = os.open(
filename, os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK, dir_fd=self.descriptor
)
with os.fdopen(descriptor, "rb") as stream:
before = os.fstat(stream.fileno())
if (
not stat.S_ISREG(before.st_mode)
or before.st_nlink != 1
or before.st_uid != os.getuid()
or before.st_size > maximum
):
raise ValueError("invalid local input file")
data = stream.read(maximum + 1)
after = os.fstat(stream.fileno())
if (
len(data) > maximum
or before.st_size != len(data)
or before.st_mtime_ns != after.st_mtime_ns
or before.st_ctime_ns != after.st_ctime_ns
):
raise ValueError("local input changed during import")
return data
except OSError:
raise ValueError("local input unavailable") from None
@@ -0,0 +1,110 @@
"""Immutable published versions, ordered cases, and mutable public availability."""
from datetime import datetime
from uuid import UUID
from sqlalchemy import (
BigInteger,
Boolean,
CheckConstraint,
Computed,
DateTime,
ForeignKey,
ForeignKeyConstraint,
Index,
Integer,
SmallInteger,
String,
Text,
UniqueConstraint,
)
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Mapped, mapped_column
from kf_oj.infra.db.base import Base
from kf_oj.infra.db.columns import CreatedAt
class Problem(CreatedAt, Base):
__tablename__ = "problems"
__table_args__ = (
CheckConstraint("slug ~ '^[a-z0-9][a-z0-9_-]{0,63}$'", name="slug"),
ForeignKeyConstraint(
["active_version_id", "id"],
["problem_versions.id", "problem_versions.problem_id"],
name="fk_problems_active_version",
use_alter=True,
deferrable=True,
initially="DEFERRED",
),
)
id: Mapped[UUID] = mapped_column(primary_key=True)
slug: Mapped[str] = mapped_column(String(64), unique=True)
active_version_id: Mapped[UUID | None]
enabled: Mapped[bool] = mapped_column(Boolean, server_default="true")
class ProblemVersion(CreatedAt, Base):
__tablename__ = "problem_versions"
__table_args__ = (
UniqueConstraint("id", "problem_id"),
UniqueConstraint("problem_id", "revision"),
CheckConstraint("revision > 0", name="revision"),
CheckConstraint("octet_length(statement) BETWEEN 1 AND 262144", name="statement_bytes"),
CheckConstraint("char_length(title) BETWEEN 1 AND 160", name="title_size"),
CheckConstraint("comparison = 'exact_bytes'", name="comparison"),
CheckConstraint("job_wall_time_us BETWEEN 1000 AND 3600000000", name="job_wall"),
CheckConstraint("judging_schema_version IN (1,2)", name="judging_schema"),
CheckConstraint(
"(judging_schema_version = 1 AND language_configurations IS NULL "
"AND toolchain IS NOT NULL AND compile_limits IS NOT NULL "
"AND run_limits IS NOT NULL AND job_wall_time_us IS NOT NULL) OR "
"(judging_schema_version = 2 AND language_configurations IS NOT NULL "
"AND toolchain IS NULL AND compile_limits IS NULL "
"AND run_limits IS NULL AND job_wall_time_us IS NULL "
"AND public.kf_valid_language_configurations(languages, language_configurations))",
name="language_configurations",
),
Index("ix_problem_versions_public", "problem_id", "published_at"),
)
id: Mapped[UUID] = mapped_column(primary_key=True)
problem_id: Mapped[UUID] = mapped_column(ForeignKey("problems.id"))
revision: Mapped[int] = mapped_column(Integer)
title: Mapped[str] = mapped_column(String(160))
statement: Mapped[str] = mapped_column(Text)
languages: Mapped[list[str]] = mapped_column(JSONB)
judging_schema_version: Mapped[int] = mapped_column(SmallInteger, server_default="1")
language_configurations: Mapped[list[dict] | None] = mapped_column(JSONB(none_as_null=True))
toolchain: Mapped[dict | None] = mapped_column(JSONB(none_as_null=True))
compile_limits: Mapped[dict | None] = mapped_column(JSONB(none_as_null=True))
run_limits: Mapped[dict | None] = mapped_column(JSONB(none_as_null=True))
job_wall_time_us: Mapped[int | None] = mapped_column(BigInteger)
comparison: Mapped[str] = mapped_column(String(16), server_default="exact_bytes")
public_samples: Mapped[list[dict]] = mapped_column(JSONB)
published_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
class ProblemCase(Base):
__tablename__ = "problem_cases"
__table_args__ = (
UniqueConstraint("id", "version_id"),
UniqueConstraint("version_id", "position"),
CheckConstraint("position BETWEEN 0 AND 255", name="position"),
ForeignKeyConstraint(
["input_digest", "artifact_state"], ["artifacts.digest", "artifacts.state"]
),
ForeignKeyConstraint(
["expected_digest", "artifact_state"],
["artifacts.digest", "artifacts.state"],
name="fk_problem_cases_expected_digest_artifacts",
),
)
id: Mapped[UUID] = mapped_column(primary_key=True)
version_id: Mapped[UUID] = mapped_column(ForeignKey("problem_versions.id"))
position: Mapped[int]
input_digest: Mapped[str] = mapped_column(String(64))
expected_digest: Mapped[str] = mapped_column(String(64))
artifact_state: Mapped[str] = mapped_column(String(8), Computed("'ready'", persisted=True))
@@ -0,0 +1,80 @@
"""Authorized immutable version projection; hidden references stay out of public DTOs."""
from dataclasses import dataclass
from typing import Protocol
from uuid import UUID
from kf_oj.contracts import judging as v1
from kf_oj.contracts import judging_v2 as v2
from kf_oj.contracts.judging_versions import Manifest
class ProblemUnavailable(ValueError):
pass
@dataclass(frozen=True)
class JudgingVersion:
version_id: UUID
problem_id: UUID
languages: tuple[v1.Language | v2.Language, ...]
toolchain: v1.Toolchain | None
compile_limits: v1.RunLimits | None
run_limits: v1.RunLimits | None
job_wall_time_us: int | None
cases: tuple[v1.TestCase, ...]
judging_schema_version: int = 1
language_configurations: tuple[v2.LanguageConfiguration, ...] = ()
def manifest(
self,
evaluation_id: UUID,
submission_id: UUID,
language: v1.Language | v2.Language,
source: v1.ArtifactReference,
) -> Manifest:
"""Build only from the immutable authorized version, preserving v1 wire bytes."""
if language not in self.languages:
raise ProblemUnavailable("language unavailable")
fields = dict(
schema_version=self.judging_schema_version,
kind="judge.manifest",
evaluation_id=evaluation_id,
submission_id=submission_id,
problem_version_id=self.version_id,
language=language,
source=source,
cases=self.cases,
comparison="exact_bytes",
)
if self.judging_schema_version == 1:
if self.language_configurations:
raise ProblemUnavailable("inconsistent legacy version")
return v1.JobManifest(
**fields,
toolchain=self.toolchain,
compile_limits=self.compile_limits,
run_limits=self.run_limits,
job_wall_time_us=self.job_wall_time_us,
)
if self.judging_schema_version != 2 or any(
field is not None
for field in (
self.toolchain,
self.compile_limits,
self.run_limits,
self.job_wall_time_us,
)
):
raise ProblemUnavailable("unsupported judging configuration")
if tuple(c.language for c in self.language_configurations) != self.languages:
raise ProblemUnavailable("inconsistent language configurations")
configuration = next(c for c in self.language_configurations if c.language == language)
return v2.JobManifest(
**fields,
**configuration.model_dump(exclude={"language"}),
)
class ProblemPort(Protocol):
async def for_submission(self, session, version_id: UUID) -> JudgingVersion: ...
@@ -0,0 +1,59 @@
import hashlib
from uuid import UUID
from sqlalchemy import func, select, text
from sqlalchemy.ext.asyncio import AsyncSession
from kf_oj.modules.problems.models import Problem, ProblemCase, ProblemVersion
async def lock_slug(session: AsyncSession, slug: str) -> None:
# Two-key namespace is disjoint from the one-key artifact advisory locks.
key = int.from_bytes(hashlib.sha256(slug.encode()).digest()[:4], signed=True)
await session.execute(text("SELECT pg_advisory_xact_lock(1262907468,:key)"), {"key": key})
async def lock_problem(session: AsyncSession, slug: str) -> Problem | None:
return await session.scalar(select(Problem).where(Problem.slug == slug).with_for_update())
async def next_revision(session: AsyncSession, problem_id: UUID) -> int:
latest = await session.scalar(
select(func.max(ProblemVersion.revision)).where(ProblemVersion.problem_id == problem_id)
)
return (latest or 0) + 1
async def public_version(session: AsyncSession, version_id: UUID):
return (
await session.execute(
select(ProblemVersion, Problem)
.join(Problem, Problem.id == ProblemVersion.problem_id)
.where(
ProblemVersion.id == version_id,
ProblemVersion.published_at.is_not(None),
Problem.enabled.is_(True),
)
)
).one_or_none()
async def ordered_cases(session: AsyncSession, version_id: UUID):
return list(
await session.scalars(
select(ProblemCase)
.where(ProblemCase.version_id == version_id)
.order_by(ProblemCase.position)
)
)
async def list_public(session: AsyncSession, after: str | None, limit: int):
query = (
select(Problem, ProblemVersion)
.join(ProblemVersion, ProblemVersion.id == Problem.active_version_id)
.where(Problem.enabled.is_(True), ProblemVersion.published_at.is_not(None))
)
if after is not None:
query = query.where(Problem.slug > after)
return list(await session.execute(query.order_by(Problem.slug).limit(limit)))
@@ -0,0 +1,269 @@
import base64
from pathlib import Path
from uuid import UUID, uuid4, uuid5
from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import async_sessionmaker
from kf_oj.contracts import judging_v2 as v2
from kf_oj.contracts.judging import (
MAX_PACKAGE_BYTES,
MAX_SOURCE_BYTES,
ArtifactReference,
Language,
RunLimits,
TestCase,
Toolchain,
decode_wire,
)
from kf_oj.contracts.problems import (
ProblemDetail,
ProblemPage,
ProblemSummary,
PublicLanguageConfiguration,
PublicSample,
)
from kf_oj.infra.artifacts.models import Artifact
from kf_oj.infra.artifacts.repository import ArtifactRepository
from kf_oj.modules.problems import repository
from kf_oj.modules.problems.import_format import ImportEnvelope, ProblemImport
from kf_oj.modules.problems.local_files import LocalImportFiles
from kf_oj.modules.problems.models import Problem, ProblemCase, ProblemVersion
from kf_oj.modules.problems.public import JudgingVersion, ProblemUnavailable
class ProblemService:
def __init__(
self,
sessions: async_sessionmaker,
artifacts: ArtifactRepository | None = None,
policy: v2.NodeAdmission | None = None,
):
self.sessions, self.artifacts, self.policy = sessions, artifacts, policy
async def import_directory(self, root: Path, manifest_file: str = "problem.json") -> UUID:
if self.artifacts is None or self.policy is None:
raise ValueError("artifact store and trusted node policy required for import")
files = LocalImportFiles(root)
try:
manifest = decode_wire(ImportEnvelope, files.read(manifest_file, 256 * 1024)).root
stdout_limit = (
manifest.run_limits.stdout_bytes
if isinstance(manifest, ProblemImport)
else min(c.run_limits.stdout_bytes for c in manifest.language_configurations)
)
statement = files.read(manifest.statement_file, 256 * 1024).decode("utf-8")
if not statement or "\x00" in statement or "\x00" in manifest.title:
raise ValueError("invalid public text")
# Read the complete bounded package before writing anything. Public samples are
# separately declared, at most 64 KiB each, and never inferred from hidden tests.
remaining = MAX_PACKAGE_BYTES - MAX_SOURCE_BYTES
payloads = []
for case in manifest.cases:
data = files.read(case.input_file, remaining)
remaining -= len(data)
answer = files.read(case.expected_file, min(remaining, stdout_limit))
remaining -= len(answer)
payloads.append((data, answer))
samples = tuple(
PublicSample(
input=base64.b64encode(files.read(s.input_file, 65536)).decode("ascii"),
output=base64.b64encode(files.read(s.output_file, 65536)).decode("ascii"),
)
for s in manifest.samples
)
finally:
files.close()
version_id = uuid4()
cases = tuple(
TestCase(
case_id=uuid5(version_id, case.key),
input=self.artifacts.store.reference(data),
expected=self.artifacts.store.reference(answer),
)
for case, (data, answer) in zip(manifest.cases, payloads, strict=True)
)
legacy = isinstance(manifest, ProblemImport)
projection = JudgingVersion(
version_id,
uuid4(),
manifest.languages,
manifest.toolchain if legacy else None,
manifest.compile_limits if legacy else None,
manifest.run_limits if legacy else None,
manifest.job_wall_time_us if legacy else None,
cases,
manifest.schema_version,
() if legacy else manifest.language_configurations,
)
for language in manifest.languages:
check = projection.manifest(
uuid4(),
uuid4(),
language,
ArtifactReference(digest="0" * 64, bytes=MAX_SOURCE_BYTES),
)
v2.admit_manifest(self.policy, check, 1)
async with self.sessions.begin() as session:
# Lock existing and not-yet-existing logical slugs before assigning revisions.
await repository.lock_slug(session, manifest.slug)
problem = await repository.lock_problem(session, manifest.slug)
if problem is None:
problem = Problem(id=uuid4(), slug=manifest.slug)
session.add(problem)
await session.flush()
if not problem.enabled:
raise ProblemUnavailable("problem disabled")
revision = await repository.next_revision(session, problem.id)
session.add(
ProblemVersion(
id=version_id,
problem_id=problem.id,
revision=revision,
title=manifest.title,
statement=statement,
languages=[lang.value for lang in manifest.languages],
judging_schema_version=manifest.schema_version,
language_configurations=(
None
if legacy
else [c.model_dump(mode="json") for c in manifest.language_configurations]
),
toolchain=manifest.toolchain.model_dump(mode="json") if legacy else None,
compile_limits=manifest.compile_limits.model_dump(mode="json")
if legacy
else None,
run_limits=manifest.run_limits.model_dump(mode="json") if legacy else None,
job_wall_time_us=manifest.job_wall_time_us if legacy else None,
comparison=manifest.comparison,
public_samples=[s.model_dump() for s in samples],
)
)
await session.flush()
# Consistent digest lock order avoids deadlocks between overlapping imports/jobs.
unique = {
ref.digest: (data, ref)
for case, (input_bytes, answer_bytes) in zip(cases, payloads, strict=True)
for data, ref in [(input_bytes, case.input), (answer_bytes, case.expected)]
}
for digest in sorted(unique):
data, reference = unique[digest]
await self.artifacts.put(session, data, reference)
for position, case in enumerate(cases):
session.add(
ProblemCase(
id=case.case_id,
version_id=version_id,
position=position,
input_digest=case.input.digest,
expected_digest=case.expected.digest,
)
)
await session.flush()
version = await session.get(ProblemVersion, version_id)
version.published_at = await session.scalar(select(func.now()))
problem.active_version_id = version_id
return version_id
@staticmethod
def _summary(problem, version) -> ProblemSummary:
return ProblemSummary(
id=problem.id,
slug=problem.slug,
version_id=version.id,
revision=version.revision,
title=version.title,
languages=tuple(version.languages),
)
async def list(self, *, after: str | None = None, limit: int = 20) -> ProblemPage:
if not 1 <= limit <= 100:
raise ValueError("invalid page limit")
async with self.sessions() as session:
rows = await repository.list_public(session, after, limit + 1)
visible = rows[:limit]
return ProblemPage(
items=tuple(self._summary(*row) for row in visible),
next_after=visible[-1][0].slug if len(rows) > limit else None,
)
async def detail(self, version_id: UUID) -> ProblemDetail:
async with self.sessions() as session:
row = await repository.public_version(session, version_id)
if row is None:
raise ProblemUnavailable("problem version unavailable")
version, problem = row
return ProblemDetail(
**self._summary(problem, version).model_dump(),
statement=version.statement,
samples=tuple(PublicSample.model_validate(s) for s in version.public_samples),
run_limits=(
RunLimits.model_validate(version.run_limits)
if version.judging_schema_version == 1
else None
),
language_configurations=tuple(
PublicLanguageConfiguration.from_configuration(
v2.LanguageConfiguration.model_validate(c)
)
for c in version.language_configurations or ()
),
)
async def for_submission(self, session, version_id: UUID) -> JudgingVersion:
if self.artifacts is None:
raise ProblemUnavailable("artifact service unavailable")
row = await repository.public_version(session, version_id)
if row is None:
raise ProblemUnavailable("problem version unavailable")
version, problem = row
cases = await repository.ordered_cases(session, version.id)
refs = []
digests = {d for case in cases for d in (case.input_digest, case.expected_digest)}
records = {
a.digest: a
for a in await session.scalars(select(Artifact).where(Artifact.digest.in_(digests)))
}
for case in cases:
input_record = records.get(case.input_digest)
answer_record = records.get(case.expected_digest)
if (
not input_record
or not answer_record
or input_record.state != "ready"
or answer_record.state != "ready"
):
raise ProblemUnavailable("problem artifacts unavailable")
refs.append(
TestCase(
case_id=case.id,
input=ArtifactReference(digest=case.input_digest, bytes=input_record.bytes),
expected=ArtifactReference(
digest=case.expected_digest, bytes=answer_record.bytes
),
)
)
return JudgingVersion(
version.id,
problem.id,
tuple(
(Language if version.judging_schema_version == 1 else v2.Language)(v)
for v in version.languages
),
Toolchain.model_validate(version.toolchain)
if version.judging_schema_version == 1
else None,
RunLimits.model_validate(version.compile_limits)
if version.judging_schema_version == 1
else None,
RunLimits.model_validate(version.run_limits)
if version.judging_schema_version == 1
else None,
version.job_wall_time_us,
tuple(refs),
version.judging_schema_version,
tuple(
v2.LanguageConfiguration.model_validate(c)
for c in version.language_configurations or ()
),
)
@@ -3,4 +3,23 @@
负责提交幂等请求、不可变源码引用、创建初始评测、历史查询与学生可见结果。
创建提交、评测和待投递事件必须处于同一事务。
协调器拥有领取后的执行状态与最终结果写入权;本模块不能绕过协议覆盖终态。
后续 `public.py` 提供授权后的学习证据入口。当前未实现提交接口。
`queries.py` 提供已验证主体的提交历史、详情与私有源码读取,其他账号不可读取;
历史按创建时间/UUID降序游标翻页,一次至多100条。读取完整成绩时严格核对报告契约和
评测绑定,仅投影允许公开的诊断/计量/测试点序号。取消通过注入的 Coordinator 控制
端口接受意图,不更新执行状态;202与清理完成的终态不同。
`worker_access.py` 核对当前 evaluation / attempt / Worker / fencing、未撤销及未过期凭证、
固定 submission 与版本及任务清单摘要;只允许读取该清单声明的工件。
内部 API 使用 WorkerLease bearer 和 X-Worker-ID / X-Fencing-Version,不使用用户会话授权。
允许状态为 claimed / compiling / running;取消请求、终态和旧 attempt 均拒绝。
字节准备后再次核对当前租约;最多同时两个有限读取,超过直接 429,不排无界等待队列。
任务清单为 UTF-8 JSON、对象键递归排序、无额外空白,并以这些确切字节的 SHA-256 标识。
Python/Rust `manifest_bytes` 与共享字节样例保持一致;Coordinator 必须用同一序列化定义存快照。
读取凭证与客户端提交的身份分开:身份和归属从实际数据库绑定获得,不接受客户端自报 user ID。
`service.py` 按账号与幂等UUID绑定版本/语言/源码摘要,持久化源码后同一事务创建
submission、queued evaluation 和 outbox;同键同内容复用既有提交,不同内容409。
待评测固定总量64、个人8,取消清理中也占用容量;配置或容量不足明确拒绝。
Coordinator 通过正式 claim 创建短期领取权,Worker 不写业务数据库。
真实服务链与安全条件必须在独立 VM 验收;数据库 fixture 的租约或合成报告只能
验证控制规则、授权和投影,不能作为真实程序执行/停止证明。
@@ -0,0 +1,163 @@
"""Platform creates initial rows; Coordinator exclusively advances execution."""
from datetime import datetime
from uuid import UUID
from sqlalchemy import (
BigInteger,
Boolean,
CheckConstraint,
Computed,
DateTime,
ForeignKey,
ForeignKeyConstraint,
Index,
String,
UniqueConstraint,
func,
)
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Mapped, mapped_column
from kf_oj.infra.db.base import Base
from kf_oj.infra.db.columns import CreatedAt
class Submission(CreatedAt, Base):
__tablename__ = "submissions"
__table_args__ = (
UniqueConstraint("account_id", "idempotency_key"),
UniqueConstraint("id", "problem_version_id"),
CheckConstraint("language IN ('c17','cpp20','python313','java21')", name="language"),
CheckConstraint("request_digest ~ '^[0-9a-f]{64}$'", name="request_digest"),
ForeignKeyConstraint(
["problem_version_id", "problem_id"],
["problem_versions.id", "problem_versions.problem_id"],
),
ForeignKeyConstraint(
["source_digest", "artifact_state"], ["artifacts.digest", "artifacts.state"]
),
Index("ix_submissions_account_history", "account_id", "created_at", "id"),
)
id: Mapped[UUID] = mapped_column(primary_key=True)
account_id: Mapped[UUID] = mapped_column(ForeignKey("accounts.id"))
problem_id: Mapped[UUID]
problem_version_id: Mapped[UUID]
language: Mapped[str] = mapped_column(String(16))
source_digest: Mapped[str] = mapped_column(String(64))
artifact_state: Mapped[str] = mapped_column(String(8), Computed("'ready'", persisted=True))
idempotency_key: Mapped[UUID]
request_digest: Mapped[str] = mapped_column(String(64))
class Evaluation(CreatedAt, Base):
__tablename__ = "evaluations"
__table_args__ = (
UniqueConstraint("id", "problem_version_id"),
ForeignKeyConstraint(
["submission_id", "problem_version_id"],
["submissions.id", "submissions.problem_version_id"],
),
ForeignKeyConstraint(
["current_attempt_id", "id"],
["judge_attempts.id", "judge_attempts.evaluation_id"],
name="fk_evaluations_current_attempt",
use_alter=True,
deferrable=True,
initially="DEFERRED",
),
CheckConstraint(
"state IN ('queued','claimed','compiling','running','cancelling',"
"'finished','cancelled','failed')",
name="state",
),
CheckConstraint("fencing_version >= 0", name="fence"),
CheckConstraint("max_attempts BETWEEN 1 AND 5", name="max_attempts"),
CheckConstraint("attempts_count BETWEEN 0 AND max_attempts", name="attempts_count"),
CheckConstraint(
"state NOT IN ('claimed','compiling','running','cancelling') "
"OR current_attempt_id IS NOT NULL",
name="active_attempt",
),
Index("ix_evaluations_recovery", "state", "not_before", "id"),
)
id: Mapped[UUID] = mapped_column(primary_key=True)
submission_id: Mapped[UUID] = mapped_column(unique=True)
problem_version_id: Mapped[UUID]
state: Mapped[str] = mapped_column(String(16), server_default="queued")
fencing_version: Mapped[int] = mapped_column(BigInteger, server_default="0")
attempts_count: Mapped[int] = mapped_column(server_default="0")
max_attempts: Mapped[int] = mapped_column(server_default="3")
current_attempt_id: Mapped[UUID | None]
not_before: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
cancel_requested_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
verdict: Mapped[str | None] = mapped_column(String(32))
failure_code: Mapped[str | None] = mapped_column(String(32))
final_report: Mapped[dict | None] = mapped_column(JSONB)
final_report_digest: Mapped[str | None] = mapped_column(String(64))
class JudgeAttempt(CreatedAt, Base):
__tablename__ = "judge_attempts"
__table_args__ = (
UniqueConstraint("id", "evaluation_id"),
UniqueConstraint("evaluation_id", "number", name="uq_judge_attempts_evaluation_number"),
UniqueConstraint(
"evaluation_id", "fencing_version", name="uq_judge_attempts_evaluation_fence"
),
CheckConstraint("number BETWEEN 1 AND 5", name="number"),
CheckConstraint("fencing_version > 0", name="fence"),
CheckConstraint("lease_token_hash ~ '^[0-9a-f]{64}$'", name="token_digest"),
CheckConstraint("manifest_digest ~ '^[0-9a-f]{64}$'", name="manifest_digest"),
CheckConstraint("sequence >= 0", name="sequence"),
Index("ix_judge_attempts_expiry", "expires_at", "evaluation_id"),
)
id: Mapped[UUID] = mapped_column(primary_key=True)
evaluation_id: Mapped[UUID] = mapped_column(ForeignKey("evaluations.id"))
worker_id: Mapped[UUID]
number: Mapped[int]
fencing_version: Mapped[int] = mapped_column(BigInteger)
lease_token_hash: Mapped[str] = mapped_column(String(64))
expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True))
revoked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
manifest: Mapped[dict] = mapped_column(JSONB)
manifest_digest: Mapped[str] = mapped_column(String(64))
sequence: Mapped[int] = mapped_column(BigInteger, server_default="0")
stage: Mapped[str | None] = mapped_column(String(16))
case_index: Mapped[int | None]
report: Mapped[dict | None] = mapped_column(JSONB)
report_digest: Mapped[str | None] = mapped_column(String(64))
cleanup_complete: Mapped[bool | None] = mapped_column(Boolean)
class CaseResult(Base):
__tablename__ = "case_results"
__table_args__ = (
UniqueConstraint("attempt_id", "position"),
ForeignKeyConstraint(
["case_id", "problem_version_id"], ["problem_cases.id", "problem_cases.version_id"]
),
ForeignKeyConstraint(
["evaluation_id", "problem_version_id"],
["evaluations.id", "evaluations.problem_version_id"],
),
ForeignKeyConstraint(
["attempt_id", "evaluation_id"], ["judge_attempts.id", "judge_attempts.evaluation_id"]
),
CheckConstraint("position BETWEEN 0 AND 255", name="position"),
CheckConstraint("cpu_time_us IS NULL OR cpu_time_us >= 0", name="cpu"),
CheckConstraint("memory_peak_bytes IS NULL OR memory_peak_bytes >= 0", name="memory"),
)
attempt_id: Mapped[UUID] = mapped_column(primary_key=True)
case_id: Mapped[UUID] = mapped_column(primary_key=True)
evaluation_id: Mapped[UUID]
problem_version_id: Mapped[UUID]
position: Mapped[int]
verdict: Mapped[str] = mapped_column(String(32))
cpu_time_us: Mapped[int | None] = mapped_column(BigInteger)
memory_peak_bytes: Mapped[int | None] = mapped_column(BigInteger)
@@ -0,0 +1,30 @@
from typing import Protocol
from uuid import UUID
class CancelUnavailable(ValueError):
pass
class CancellationPort(Protocol):
async def cancel(self, evaluation_id: UUID) -> bool: ...
class SubmissionUnavailable(ValueError):
pass
class WorkerAccessDenied(ValueError):
"""Never reveal whether the evaluation, attempt or artifact exists."""
class WorkerReadBusy(ValueError):
pass
class IdempotencyConflict(ValueError):
pass
class SubmissionBusy(ValueError):
pass
@@ -0,0 +1,173 @@
import base64
import json
from datetime import datetime
from uuid import UUID
from sqlalchemy import select, tuple_
from sqlalchemy.ext.asyncio import async_sessionmaker
from kf_oj.contracts.judging import MAX_SOURCE_BYTES, ArtifactReference
from kf_oj.contracts.judging_versions import decode_report
from kf_oj.contracts.submissions import (
CancellationAccepted,
PublicCaseResult,
PublicJudgement,
SubmissionDetail,
SubmissionPage,
SubmissionSummary,
)
from kf_oj.infra.artifacts.models import Artifact
from kf_oj.infra.artifacts.repository import ArtifactRepository
from kf_oj.infra.artifacts.store import ArtifactError
from kf_oj.modules.identity.public import Principal
from kf_oj.modules.submissions.models import Evaluation, Submission
from kf_oj.modules.submissions.public import (
CancellationPort,
CancelUnavailable,
SubmissionUnavailable,
)
def cursor_encode(submission: Submission) -> str:
return (
base64.urlsafe_b64encode(
f"{submission.created_at.isoformat()}|{submission.id}".encode("ascii")
)
.decode("ascii")
.rstrip("=")
)
def cursor_decode(value: str) -> tuple[datetime, UUID]:
try:
if not 1 <= len(value) <= 128:
raise ValueError
date, identifier = (
base64.b64decode(value + "=" * (-len(value) % 4), altchars=b"-_", validate=True)
.decode("ascii")
.split("|")
)
timestamp = datetime.fromisoformat(date)
if timestamp.utcoffset() is None:
raise ValueError
return timestamp, UUID(identifier)
except (ValueError, UnicodeError):
raise ValueError("invalid history cursor") from None
def summary(submission: Submission, evaluation: Evaluation) -> SubmissionSummary:
terminal = evaluation.state in {"finished", "failed", "cancelled"}
return SubmissionSummary(
submission_id=submission.id,
evaluation_id=evaluation.id,
problem_version_id=submission.problem_version_id,
language=submission.language,
state=evaluation.state,
created_at=submission.created_at,
updated_at=evaluation.updated_at,
attempts=evaluation.attempts_count,
verdict=evaluation.verdict if terminal else None,
failure_code=evaluation.failure_code,
)
class SubmissionQueries:
def __init__(
self,
sessions: async_sessionmaker,
artifacts: ArtifactRepository,
cancellation: CancellationPort | None = None,
):
self.sessions, self.artifacts = sessions, artifacts
self.cancellation = cancellation
def owned(self, principal: Principal, submission_id: UUID):
return (
select(Submission, Evaluation)
.join(Evaluation, Evaluation.submission_id == Submission.id)
.where(Submission.id == submission_id, Submission.account_id == principal.account_id)
)
async def page(self, principal: Principal, after: str | None, limit: int) -> SubmissionPage:
if not 1 <= limit <= 100:
raise ValueError("invalid history limit")
query = (
select(Submission, Evaluation)
.join(Evaluation, Evaluation.submission_id == Submission.id)
.where(Submission.account_id == principal.account_id)
.order_by(Submission.created_at.desc(), Submission.id.desc())
.limit(limit + 1)
)
if after is not None:
query = query.where(tuple_(Submission.created_at, Submission.id) < cursor_decode(after))
async with self.sessions() as session:
rows = (await session.execute(query)).all()
return SubmissionPage(
items=tuple(summary(s, e) for s, e in rows[:limit]),
next_after=cursor_encode(rows[limit - 1][0]) if len(rows) > limit else None,
)
async def detail(self, principal: Principal, submission_id: UUID) -> SubmissionDetail:
async with self.sessions() as session:
row = (await session.execute(self.owned(principal, submission_id))).one_or_none()
if row is None:
raise SubmissionUnavailable("submission unavailable")
submission, evaluation = row
result = None
if evaluation.state in {"finished", "failed", "cancelled"} and evaluation.final_report:
# A single joined snapshot, strict validation and explicit projection prevent leaking
# internal bindings, artifacts, hidden case identifiers or partial attempt reports.
report = decode_report(
json.dumps(
evaluation.final_report, ensure_ascii=False, separators=(",", ":")
).encode()
)
if str(report.binding.evaluation_id) != str(evaluation.id):
raise ValueError("stored result binding invalid")
if report.verdict.value != evaluation.verdict:
raise ValueError("stored result verdict invalid")
result = PublicJudgement(
compilation_status=report.compilation.status,
diagnostics=report.compilation.diagnostics,
diagnostics_truncated=report.compilation.diagnostics_truncated,
cpu_time_us=report.run_cpu_time_us,
memory_peak_bytes=report.run_memory_peak_bytes,
cleanup_complete=report.cleanup_complete,
cases=tuple(
PublicCaseResult(position=i, **c.model_dump(exclude={"case_id"}))
for i, c in enumerate(report.cases)
),
)
return SubmissionDetail(**summary(submission, evaluation).model_dump(), result=result)
async def cancel(self, principal: Principal, submission_id: UUID) -> CancellationAccepted:
async with self.sessions() as session:
row = (await session.execute(self.owned(principal, submission_id))).one_or_none()
if row is None:
raise SubmissionUnavailable("submission unavailable")
submission, evaluation = row
if self.cancellation is None or not await self.cancellation.cancel(evaluation.id):
raise CancelUnavailable("cancellation service unavailable")
return CancellationAccepted(submission_id=submission.id, evaluation_id=evaluation.id)
async def owned_source(self, principal: Principal, submission_id: UUID) -> bytes:
async with self.sessions() as session:
submission = await session.scalar(
select(Submission).where(
Submission.id == submission_id, Submission.account_id == principal.account_id
)
)
if submission is None:
raise SubmissionUnavailable("submission unavailable")
metadata = await session.get(Artifact, submission.source_digest)
if (
metadata is None
or metadata.state != "ready"
or not 1 <= metadata.bytes <= MAX_SOURCE_BYTES
):
raise SubmissionUnavailable("submission source unavailable")
reference = ArtifactReference(digest=metadata.digest, bytes=metadata.bytes)
try:
return await self.artifacts.read(session, reference)
except ArtifactError:
raise SubmissionUnavailable("submission source unavailable") from None
@@ -0,0 +1,27 @@
from sqlalchemy import func, select
from kf_oj.contracts.judging import LeaseBinding
from kf_oj.modules.submissions.models import Evaluation, JudgeAttempt, Submission
async def leased_submission(session, binding: LeaseBinding, token_hash: str):
return (
await session.execute(
select(Evaluation, JudgeAttempt, Submission)
.join(JudgeAttempt, JudgeAttempt.id == Evaluation.current_attempt_id)
.join(Submission, Submission.id == Evaluation.submission_id)
.where(
Evaluation.id == binding.evaluation_id,
Evaluation.state.in_(("claimed", "compiling", "running")),
Evaluation.cancel_requested_at.is_(None),
Evaluation.fencing_version == binding.fencing_version,
JudgeAttempt.id == binding.attempt_id,
JudgeAttempt.evaluation_id == Evaluation.id,
JudgeAttempt.worker_id == binding.worker_id,
JudgeAttempt.fencing_version == binding.fencing_version,
JudgeAttempt.lease_token_hash == token_hash,
JudgeAttempt.expires_at > func.clock_timestamp(),
JudgeAttempt.revoked_at.is_(None),
)
)
).one_or_none()
@@ -0,0 +1,123 @@
"""Only initial queued evaluation/outbox writes; execution belongs to Coordinator."""
import hashlib
import json
from uuid import uuid4
from sqlalchemy import func, select, text
from kf_oj.contracts.judging_v2 import admit_manifest
from kf_oj.contracts.submissions import CreateSubmission, SubmissionCreated
from kf_oj.infra.artifacts.repository import ArtifactRepository
from kf_oj.infra.messaging.models import Outbox
from kf_oj.modules.identity.public import Principal
from kf_oj.modules.problems.public import ProblemPort, ProblemUnavailable
from kf_oj.modules.submissions.models import Evaluation, Submission
from kf_oj.modules.submissions.public import (
IdempotencyConflict,
SubmissionBusy,
SubmissionUnavailable,
)
class SubmissionService:
def __init__(self, sessions, problems: ProblemPort, artifacts: ArtifactRepository, admission):
self.sessions, self.problems = sessions, problems
self.artifacts, self.admission = artifacts, admission
@staticmethod
def _response(submission, evaluation):
return SubmissionCreated(
submission_id=submission.id,
evaluation_id=evaluation.id,
problem_version_id=submission.problem_version_id,
language=submission.language,
state=evaluation.state,
created_at=submission.created_at,
)
async def create(self, principal: Principal, request: CreateSubmission) -> SubmissionCreated:
# Validate even trusted non-HTTP callers. No path or argv is accepted.
request = CreateSubmission.model_validate(
request.model_dump(exclude={"language"}) | {"language": request.language}
)
source = request.source.encode("utf-8")
request_digest = hashlib.sha256(
json.dumps(
[
str(request.problem_version_id),
request.language.value,
hashlib.sha256(source).hexdigest(),
],
separators=(",", ":"),
).encode()
).hexdigest()
if self.admission is None:
raise SubmissionUnavailable("judge admission unavailable")
async with self.sessions.begin() as session:
# Bounded pending capacity and per-account idempotency serialize across API processes.
# Two-key namespace is separate from artifact/slug advisory locks.
await session.execute(text("SELECT pg_advisory_xact_lock(1262907470, 0)"))
existing = (
await session.execute(
select(Submission, Evaluation)
.join(Evaluation, Evaluation.submission_id == Submission.id)
.where(
Submission.account_id == principal.account_id,
Submission.idempotency_key == request.idempotency_key,
)
)
).one_or_none()
if existing is not None:
if existing[0].request_digest != request_digest:
raise IdempotencyConflict("idempotency key belongs to another request")
return self._response(*existing)
pending = ("queued", "claimed", "compiling", "running", "cancelling")
total = await session.scalar(
select(func.count()).select_from(Evaluation).where(Evaluation.state.in_(pending))
)
owned = await session.scalar(
select(func.count())
.select_from(Submission)
.join(Evaluation, Evaluation.submission_id == Submission.id)
.where(Submission.account_id == principal.account_id, Evaluation.state.in_(pending))
)
if total >= 64 or owned >= 8:
raise SubmissionBusy("pending submission capacity exhausted")
try:
version = await self.problems.for_submission(session, request.problem_version_id)
except ProblemUnavailable:
raise SubmissionUnavailable("problem version unavailable") from None
if request.language not in version.languages:
raise SubmissionUnavailable("language unavailable")
# Bytes fsync before metadata/ref transaction. Rollback may leave collectable bytes,
# but cannot leave an evaluation that refers to non-durable bytes.
reference = await self.artifacts.put(session, source)
submission_id, evaluation_id = uuid4(), uuid4()
manifest = version.manifest(evaluation_id, submission_id, request.language, reference)
admit_manifest(self.admission, manifest, 1)
submission = Submission(
id=submission_id,
account_id=principal.account_id,
problem_id=version.problem_id,
problem_version_id=version.version_id,
language=request.language.value,
source_digest=reference.digest,
idempotency_key=request.idempotency_key,
request_digest=request_digest,
)
session.add(submission)
await session.flush()
await session.execute(
Evaluation.__table__.insert().values(
id=evaluation_id,
submission_id=submission_id,
problem_version_id=version.version_id,
)
)
session.add(Outbox(id=uuid4(), evaluation_id=evaluation_id))
await session.flush()
evaluation = await session.get(Evaluation, evaluation_id)
response = self._response(submission, evaluation)
# The transaction commit must succeed before any success is returned.
return response
@@ -0,0 +1,108 @@
import asyncio
import hashlib
import hmac
import json
import re
from contextlib import asynccontextmanager
from sqlalchemy.ext.asyncio import async_sessionmaker
from kf_oj.contracts.judging import LeaseBinding
from kf_oj.contracts.judging_versions import (
Manifest,
decode_manifest,
manifest_bytes,
)
from kf_oj.infra.artifacts.repository import ArtifactRepository
from kf_oj.infra.artifacts.store import ArtifactError
from kf_oj.modules.problems.public import ProblemPort, ProblemUnavailable
from kf_oj.modules.submissions import repository
from kf_oj.modules.submissions.public import WorkerAccessDenied, WorkerReadBusy
class WorkerAccess:
def __init__(
self, sessions: async_sessionmaker, problems: ProblemPort, artifacts: ArtifactRepository
):
self.sessions, self.problems, self.artifacts = sessions, problems, artifacts
self._reads = asyncio.Semaphore(2)
@asynccontextmanager
async def _read_slot(self):
if self._reads.locked():
raise WorkerReadBusy("worker read capacity exhausted")
async with self._reads:
yield
@staticmethod
def _token_hash(token: str) -> str:
if not re.fullmatch(r"[0-9a-f]{64}", token):
raise WorkerAccessDenied("worker grant rejected")
return hashlib.sha256(token.encode("ascii")).hexdigest()
async def _manifest(self, session, binding: LeaseBinding, token: str) -> Manifest:
row = await repository.leased_submission(session, binding, self._token_hash(token))
if row is None:
raise WorkerAccessDenied("worker grant rejected")
evaluation, attempt, submission = row
# Use a sanitized decoder at this trust boundary. JSONB snapshots are immutable for
# runtime roles; use canonical bytes for both retrieval and digest verification.
try:
manifest = decode_manifest(json.dumps(attempt.manifest).encode())
if (
manifest.evaluation_id != evaluation.id
or manifest.submission_id != submission.id
or manifest.problem_version_id != evaluation.problem_version_id
or manifest.problem_version_id != submission.problem_version_id
or manifest.language.value != submission.language
or manifest.source.digest != submission.source_digest
or not hmac.compare_digest(
hashlib.sha256(manifest_bytes(manifest)).hexdigest(), attempt.manifest_digest
)
):
raise WorkerAccessDenied("worker snapshot rejected")
version = await self.problems.for_submission(session, manifest.problem_version_id)
await self.artifacts.check_metadata(session, manifest.source)
expected = version.manifest(
manifest.evaluation_id, manifest.submission_id, manifest.language, manifest.source
)
if version.problem_id != submission.problem_id or manifest_bytes(
manifest
) != manifest_bytes(expected):
raise WorkerAccessDenied("worker snapshot rejected")
return manifest
except (ValueError, ProblemUnavailable):
raise WorkerAccessDenied("worker snapshot rejected") from None
async def manifest(self, binding: LeaseBinding, token: str) -> bytes:
async with self._read_slot(), self.sessions() as session:
manifest = await self._manifest(session, binding, token)
payload = manifest_bytes(manifest)
# READ COMMITTED: recheck after all preparation and just before releasing bytes.
if (
await repository.leased_submission(session, binding, self._token_hash(token))
is None
):
raise WorkerAccessDenied("worker grant rejected")
return payload
async def artifact(self, binding: LeaseBinding, token: str, digest: str) -> bytes:
async with self._read_slot(), self.sessions() as session:
manifest = await self._manifest(session, binding, token)
references = (manifest.source,) + tuple(
reference for case in manifest.cases for reference in (case.input, case.expected)
)
reference = next((ref for ref in references if ref.digest == digest), None)
if reference is None:
raise WorkerAccessDenied("artifact grant rejected")
try:
payload = await self.artifacts.read(session, reference)
except ArtifactError:
raise WorkerAccessDenied("artifact unavailable") from None
if (
await repository.leased_submission(session, binding, self._token_hash(token))
is None
):
raise WorkerAccessDenied("worker grant rejected")
return payload
+53
View File
@@ -0,0 +1,53 @@
"""Optional real PostgreSQL acceptance; never reuse or reset an existing business DB."""
import asyncio
import os
import subprocess
from pathlib import Path
from uuid import uuid4
import asyncpg
import pytest
from sqlalchemy.engine import make_url
@pytest.fixture
def postgres_url():
admin = os.environ.get("KF_OJ_TEST_POSTGRES")
if admin is None:
pytest.skip("real PostgreSQL acceptance requires KF_OJ_TEST_POSTGRES")
url = make_url(admin)
database = "kf_oj_test_" + uuid4().hex
assert url.drivername == "postgresql+asyncpg"
test_url = url.set(database=database).render_as_string(hide_password=False)
async def manage(command):
connection = await asyncpg.connect(
url.set(drivername="postgresql").render_as_string(hide_password=False)
)
try:
await connection.execute(command)
finally:
await connection.close()
asyncio.run(manage(f'CREATE DATABASE "{database}"'))
try:
migrate(test_url, "upgrade", "head")
yield test_url
finally:
asyncio.run(manage(f'DROP DATABASE "{database}"'))
def migrate(url: str, command: str, target: str | None = None) -> None:
subprocess.run(
[str(Path(__file__).resolve().parents[1] / ".venv/bin/alembic"), command]
+ ([target] if target else []),
cwd=Path(__file__).resolve().parents[1],
env=os.environ | {"KF_OJ_DATABASE_URL": url},
check=True,
capture_output=True,
)
async def connect(url: str):
return await asyncpg.connect(url.replace("postgresql+asyncpg:", "postgresql:", 1))
+204
View File
@@ -0,0 +1,204 @@
import asyncio
import hashlib
import os
from uuid import uuid4
import pytest
from sqlalchemy import func, select, text, update
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from kf_oj.contracts.judging import ArtifactReference
from kf_oj.infra.artifacts.models import Artifact
from kf_oj.infra.artifacts.repository import ArtifactRepository
from kf_oj.infra.artifacts.store import ArtifactError, ArtifactStore
def test_immutable_store_digest_capacity_and_recovery(tmp_path):
root = tmp_path / "artifacts"
store = ArtifactStore(root, max_bytes=6, max_files=2)
try:
first = store.put(b"abc")
assert store.read(first) == b"abc"
assert store.put(b"abc") == first
with pytest.raises(ArtifactError, match="digest or length"):
store.put(b"changed", first)
second = store.put(b"def")
with pytest.raises(ArtifactError, match="capacity"):
store.put(b"g")
assert store.read(first) == b"abc" and store.read(second) == b"def"
(root / first.digest).write_bytes(b"xyz")
with pytest.raises(ArtifactError, match="digest or length"):
store.read(first)
with pytest.raises(ArtifactError, match="digest or length"):
store.put(b"abc") # Corrupt files are not silently repaired or overwritten.
assert (root / first.digest).read_bytes() == b"xyz"
finally:
store.close()
@pytest.mark.parametrize("attack", ["symlink", "hardlink", "fifo", "traversal"])
def test_links_and_non_regular_entries_refused(tmp_path, attack):
root = tmp_path / "artifacts"
store = ArtifactStore(root)
data = b"private bytes"
ref = ArtifactReference(digest=hashlib.sha256(data).hexdigest(), bytes=len(data))
private = tmp_path / "outside"
private.write_bytes(data)
private.chmod(0o600)
try:
if attack == "symlink":
(root / ref.digest).symlink_to(private)
elif attack == "hardlink":
os.link(private, root / ref.digest)
elif attack == "fifo":
os.mkfifo(root / ref.digest, 0o600)
else:
ref = ref.model_copy(update={"digest": "../outside"})
with pytest.raises(ArtifactError):
store.read(ref)
finally:
store.close()
def test_private_root_and_bounded_temporary_cleanup(tmp_path):
root = tmp_path / "root"
root.mkdir(mode=0o755)
with pytest.raises(ArtifactError, match="private"):
ArtifactStore(root)
linked = tmp_path / "linked"
linked.symlink_to(root)
with pytest.raises(ArtifactError, match="links"):
ArtifactStore(linked)
root.chmod(0o700)
store = ArtifactStore(root)
try:
old = root / (".tmp-" + uuid4().hex)
recent = root / (".tmp-" + uuid4().hex)
for path in (old, recent):
path.write_bytes(b"temporary")
path.chmod(0o600)
os.utime(old, (0, 0))
assert store.clean_temporary(limit=1) == 1
assert not old.exists() and recent.exists()
with pytest.raises(ArtifactError):
store.clean_temporary(min_age_seconds=0)
finally:
store.close()
def test_physical_quota_serializes_parallel_publishers(tmp_path):
root = tmp_path / "root"
first = ArtifactStore(root, max_bytes=6, max_files=10)
second = ArtifactStore(root, max_bytes=6, max_files=10)
async def run():
results = await asyncio.gather(
asyncio.to_thread(first.put, b"abcd"),
asyncio.to_thread(second.put, b"efgh"),
return_exceptions=True,
)
assert sum(isinstance(r, ArtifactReference) for r in results) == 1
assert sum(isinstance(r, ArtifactError) for r in results) == 1
assert sum(p.stat().st_size for p in root.iterdir() if p.name != ".lock") == 4
try:
asyncio.run(run())
finally:
first.close()
second.close()
def test_real_metadata_rollback_reference_fencing_and_collection(postgres_url, tmp_path):
store = ArtifactStore(tmp_path / "bytes")
artifacts = ArtifactRepository(store)
async def run():
engine = create_async_engine(postgres_url)
sessions = async_sessionmaker(engine, expire_on_commit=False)
ref = store.reference(b"rolled back")
try:
with pytest.raises(RuntimeError):
async with sessions.begin() as s:
await s.execute(text("SET LOCAL ROLE kf_oj_platform"))
await artifacts.put(s, b"rolled back")
raise RuntimeError("transaction aborted")
async with sessions() as s:
assert await s.get(Artifact, ref.digest) is None
assert store.read(ref) == b"rolled back" # Durable orphan, never dangling DB reference.
os.utime(tmp_path / "bytes" / ref.digest, (0, 0))
assert await artifacts.collect_orphans(sessions) == 1
with pytest.raises(ArtifactError):
store.read(ref)
async with sessions.begin() as s:
await s.execute(text("SET LOCAL ROLE kf_oj_platform"))
ref = await artifacts.put(s, b"durable unreferenced")
os.utime(tmp_path / "bytes" / ref.digest, (0, 0))
assert await artifacts.collect_orphans(sessions) == 0
async with sessions.begin() as s:
await artifacts.mark_deleting(s, ref.digest)
with pytest.raises(ArtifactError):
async with sessions.begin() as s:
await artifacts.put(s, b"durable unreferenced")
assert await artifacts.finish_deleting(sessions) == 1
async with sessions() as s:
assert list(await s.scalars(select(Artifact))) == []
finally:
await engine.dispose()
try:
asyncio.run(run())
finally:
store.close()
def test_bounded_metadata_collector_preserves_references_and_resumes(postgres_url, tmp_path):
from datetime import timedelta
from test_problems import import_fixture, platform_role
from kf_oj.modules.problems.service import ProblemService
root = tmp_path / "import"
_, policy = import_fixture(root)
store = ArtifactStore(tmp_path / "bytes")
repository = ArtifactRepository(store)
async def run():
engine = create_async_engine(postgres_url)
sessions = async_sessionmaker(engine, expire_on_commit=False)
try:
service = ProblemService(sessions, repository, policy)
version = await service.import_directory(root)
async with sessions.begin() as session:
orphan = await repository.put(session, b"committed without business reference")
await session.execute(
update(Artifact).values(created_at=func.now() - timedelta(days=2))
)
# After trusted fixture preparation, run collection with actual limited platform rights.
await engine.dispose()
platform_role(engine)
marked_total, cursor = 0, None
for _ in range(100):
marked, cursor = await repository.collect_metadata(sessions, limit=1, after=cursor)
marked_total += marked
if cursor is None:
break
else:
pytest.fail("bounded metadata scan failed to finish")
assert marked_total == 1
assert await repository.finish_deleting(sessions, limit=1) == 1
assert await repository.finish_deleting(sessions, limit=1) == 0
async with sessions() as session:
projected = await service.for_submission(session, version)
assert (
await repository.read(session, projected.cases[0].expected)
== b"hidden-expected\x00\xff"
)
assert await session.get(Artifact, orphan.digest) is None
finally:
await engine.dispose()
try:
asyncio.run(run())
finally:
store.close()
+5 -1
View File
@@ -19,7 +19,11 @@ def test_shared_wire_examples(path: Path) -> None:
if ".invalid-" in path.name:
with pytest.raises(ValidationError):
JudgeTaskNotification.model_validate(payload)
assert not validator.is_valid(payload)
if ".invalid-semantic-" in path.name:
# JSON Schema counts 1.0 as an integer; the wire decoder forbids coercion.
assert validator.is_valid(payload)
else:
assert not validator.is_valid(payload)
else:
notification = JudgeTaskNotification.model_validate(payload)
validator.validate(payload)
+612
View File
@@ -0,0 +1,612 @@
"""Actual Rust HTTP + PostgreSQL state tests, using explicitly synthetic reports.
No program is compiled/run by this host suite. Root stop proofs and lease-loss
execution are separately required in the independent VM (F05/F06).
"""
import asyncio
import json
import secrets
import socket
import subprocess
from contextlib import asynccontextmanager
from pathlib import Path
from uuid import UUID, uuid4
import asyncpg
import httpx
from sqlalchemy.engine import make_url
from sqlalchemy.ext.asyncio import create_async_engine
from test_problems import import_fixture, platform_role
from kf_oj.api.app import create_app
from kf_oj.contracts.submissions import CreateSubmission
from kf_oj.core.settings import Settings
ROOT = Path(__file__).resolve().parents[2]
@asynccontextmanager
async def fixture(
postgres_url, tmp_path, monkeypatch, *, import_builder=import_fixture, node_policy=None
):
_, policy = import_builder(tmp_path / "import")
engine = create_async_engine(postgres_url)
platform_role(engine)
monkeypatch.setattr("kf_oj.api.app.build_engine", lambda _: engine)
app = create_app(
Settings(
_env_file=None,
environment="test",
database_url=postgres_url,
artifact_root=tmp_path / "artifacts",
judge_admission=policy,
)
)
url = make_url(postgres_url)
c = await asyncpg.connect(
url.set(drivername="postgresql").render_as_string(hide_password=False)
)
role = "jp_coord_" + uuid4().hex
password = secrets.token_hex(32)
await c.execute(f"CREATE ROLE {role} LOGIN PASSWORD '{password}'")
await c.execute(f"GRANT kf_oj_coordinator TO {role}")
coord_url = url.set(drivername="postgresql", username=role, password=password).render_as_string(
hide_password=False
)
# Only loopback control-plane service is started; finite actual systemd scope.
with socket.socket() as reserved:
reserved.bind(("127.0.0.1", 0))
port = reserved.getsockname()[1]
workers = [dict(id=str(uuid4()), token=secrets.token_hex(32)) for _ in range(2)]
platform = secrets.token_hex(32)
config = tmp_path / "coordinator.json"
config.write_text(
json.dumps(
dict(
listen=f"127.0.0.1:{port}",
database_url=coord_url,
workers=workers,
platform_token=platform,
admission=(node_policy or policy).model_dump(mode="json"),
lease_seconds=5,
recovery_seconds=1,
)
)
)
config.chmod(0o600)
unit = "kf-jp-coord-" + uuid4().hex
binary = ROOT / "judge/target/debug/kf-coordinator"
assert binary.is_file(), "build actual Coordinator before this acceptance"
subprocess.run(
[
"systemd-run",
"--user",
f"--unit={unit}",
"--collect",
"--service-type=exec",
"--property=MemoryMax=128M",
"--property=MemorySwapMax=0",
"--property=CPUQuota=50%",
"--property=TasksMax=32",
"--property=RuntimeMaxSec=180",
"--property=NoNewPrivileges=yes",
str(binary),
"--config",
str(config),
],
check=True,
capture_output=True,
)
try:
async with app.router.lifespan_context(app):
owner = await app.state.identity.create_account(
"coordinator_fixture", secrets.token_urlsafe(24)
)
version = await app.state.problems.import_directory(tmp_path / "import")
async with httpx.AsyncClient(base_url=f"http://127.0.0.1:{port}", timeout=10) as client:
for _ in range(100):
try:
if (await client.get("/health/ready")).status_code == 200:
break
except httpx.TransportError:
pass
await asyncio.sleep(0.05)
else:
raise AssertionError("actual Coordinator not ready")
props = subprocess.check_output(
[
"systemctl",
"--user",
"show",
unit,
"--property=ControlGroup,MemoryMax,MemorySwapMax,TasksMax,CPUQuotaPerSecUSec,RuntimeMaxUSec,MainPID",
],
text=True,
)
values = dict(line.split("=", 1) for line in props.splitlines())
group = Path("/sys/fs/cgroup") / values["ControlGroup"].lstrip("/")
assert {
n: (group / n).read_text().strip()
for n in ["memory.max", "memory.swap.max", "pids.max", "cpu.max"]
} == {
"memory.max": "134217728",
"memory.swap.max": "0",
"pids.max": "32",
"cpu.max": "50000 100000",
}
async def create(language="c17"):
return await app.state.submissions.create(
owner,
CreateSubmission(
problem_version_id=version,
language=language,
idempotency_key=uuid4(),
source="synthetic report fixture; never compiled on host",
),
)
client.fixture_app = app
client.fixture_account = owner
client.fixture_version = version
client.coordinator_unit = unit
client.coordinator_role = role
yield client, c, workers, platform, create
finally:
subprocess.run(["systemctl", "--user", "stop", unit], check=True, capture_output=True)
assert (
subprocess.run(
["systemctl", "--user", "is-active", unit], capture_output=True
).returncode
!= 0
)
await c.execute(f"DROP ROLE {role}")
await c.close()
def notice(eid):
return dict(schema_version=1, kind="judge.evaluation", evaluation_id=str(eid))
def worker_headers(worker, grant=None):
headers = {"Authorization": "Bearer " + worker["token"]}
if grant:
headers["X-Lease-Token"] = grant["lease_token"]
return headers
async def claim(client, created, worker):
r = await client.post(
"/internal/claim", json=notice(created.evaluation_id), headers=worker_headers(worker)
)
assert r.status_code == 200, r.text
return r.json()
async def manifest(c, g):
v = await c.fetchval(
"SELECT manifest FROM judge_attempts WHERE id=$1", UUID(g["binding"]["attempt_id"])
)
return json.loads(v)
async def stages(client, worker, g):
h = worker_headers(worker, g)
for seq, stage, index in [(1, "compiling", None), (2, "running", 0), (3, "running", 1)]:
r = await client.post(
"/internal/stage",
headers=h,
json=dict(
schema_version=1,
kind="judge.stage",
binding=g["binding"],
sequence=seq,
stage=stage,
case_index=index,
),
)
assert r.status_code == 204, r.text
def synthetic_report(g, m, verdict="accepted"):
cases = [
dict(case_id=c["case_id"], verdict=verdict, cpu_time_us=100, memory_peak_bytes=4096)
for c in m["cases"]
]
return dict(
schema_version=1,
kind="judge.result",
binding=g["binding"],
manifest_digest=g["manifest_digest"],
compilation=dict(
status="succeeded",
program=dict(digest="d" * 64, bytes=4096),
diagnostics="",
diagnostics_truncated=False,
cpu_time_us=1000,
memory_peak_bytes=16384,
),
cases=cases,
verdict=verdict,
cleanup_complete=True,
failure_code=None,
run_cpu_time_us=len(cases) * 100,
run_memory_peak_bytes=4096,
)
def test_actual_auth_claim_competition_stage_fence_and_report(postgres_url, tmp_path, monkeypatch):
async def run():
async with fixture(postgres_url, tmp_path, monkeypatch) as (
client,
c,
workers,
platform,
create,
):
created = await create()
n = notice(created.evaluation_id)
assert (await client.post("/internal/claim", json=n)).status_code == 401
assert (
await client.post(
"/internal/claim", json=n, headers={"Authorization": "Bearer " + platform}
)
).status_code == 409
r = await client.post(
"/internal/claim",
content=b"x" * 262145,
headers=worker_headers(workers[0]) | {"Content-Type": "application/json"},
)
assert r.status_code == 413
outcomes = await asyncio.gather(
*(
client.post("/internal/claim", json=n, headers=worker_headers(w))
for w in workers
)
)
assert sorted(r.status_code for r in outcomes) == [200, 204]
g = next(r.json() for r in outcomes if r.status_code == 200)
w = next(w for w in workers if w["id"] == g["binding"]["worker_id"])
other = next(w2 for w2 in workers if w2 != w)
assert await c.fetchval("SELECT count(*) FROM judge_attempts") == 1
assert (
await client.post(
"/internal/renew", json=g["binding"], headers=worker_headers(other, g)
)
).status_code == 409
assert (
await client.post(
"/internal/renew",
json=g["binding"] | {"fencing_version": 2},
headers=worker_headers(w, g),
)
).status_code == 409
bad = dict(
schema_version=1,
kind="judge.stage",
binding=g["binding"],
sequence=1,
stage="running",
case_index=0,
)
assert (
await client.post("/internal/stage", json=bad, headers=worker_headers(w, g))
).status_code == 409
await stages(client, w, g)
progress = await client.post(
"/internal/progress", json=g["binding"], headers=worker_headers(w, g)
)
assert progress.status_code == 200
assert progress.json() == {
"state": "running",
"sequence": 3,
"stage": "running",
"case_index": 1,
}
subprocess.run(
["systemctl", "--user", "restart", client.coordinator_unit],
check=True,
capture_output=True,
)
progress = await client.post(
"/internal/progress", json=g["binding"], headers=worker_headers(w, g)
)
assert progress.status_code == 200 and progress.json()["sequence"] == 3
assert (
await client.post("/internal/stage", json=bad, headers=worker_headers(w, g))
).status_code == 409
assert (
await client.post(
"/internal/renew", json=g["binding"], headers=worker_headers(w, g)
)
).status_code == 200
m = await manifest(c, g)
report = synthetic_report(g, m)
wrong = report | {"cases": list(reversed(report["cases"]))}
assert (
await client.post("/internal/report", json=wrong, headers=worker_headers(w, g))
).status_code == 422
assert await c.fetchval("SELECT count(*) FROM case_results") == 0
good = await client.post("/internal/report", json=report, headers=worker_headers(w, g))
assert good.status_code == 204, good.text
assert (
await client.post("/internal/report", json=report, headers=worker_headers(w, g))
).status_code == 204
assert (
await client.post(
"/internal/report",
json=synthetic_report(g, m, "wrong_answer"),
headers=worker_headers(w, g),
)
).status_code == 409
assert (
await c.fetchval("SELECT state FROM evaluations WHERE id=$1", created.evaluation_id)
== "finished"
)
assert await c.fetchval("SELECT count(*) FROM case_results") == 2
assert (
await client.post("/internal/claim", json=n, headers=worker_headers(w))
).status_code == 204
queued = await create()
await c.execute(
"UPDATE outbox SET delivered_at=clock_timestamp()-interval '31 seconds' "
"WHERE evaluation_id=$1",
queued.evaluation_id,
)
for _ in range(60):
if await c.fetchval(
"SELECT delivered_at IS NULL FROM outbox WHERE evaluation_id=$1",
queued.evaluation_id,
):
break
await asyncio.sleep(0.05)
else:
raise AssertionError("missing notification not compensated")
assert (
await c.fetchval("SELECT state FROM evaluations WHERE id=$1", queued.evaluation_id)
== "queued"
)
await claim(client, queued, w)
await c.execute(f"ALTER ROLE {client.coordinator_role} NOLOGIN")
assert (await client.get("/health/ready")).status_code == 503
assert (await client.get("/health/live")).status_code == 200
await c.execute(f"ALTER ROLE {client.coordinator_role} LOGIN")
asyncio.run(run())
def test_actual_report_transaction_rollback(postgres_url, tmp_path, monkeypatch):
async def run():
async with fixture(postgres_url, tmp_path, monkeypatch) as (client, c, workers, _, create):
created = await create()
w = workers[0]
g = await claim(client, created, w)
await stages(client, w, g)
m = await manifest(c, g)
await c.execute("""CREATE FUNCTION test_case_reject() RETURNS trigger
LANGUAGE plpgsql AS $$
BEGIN IF NEW.position=1 THEN
RAISE EXCEPTION 'synthetic rollback';
END IF; RETURN NEW; END $$""")
await c.execute(
"CREATE TRIGGER test_case_reject BEFORE INSERT ON case_results "
"FOR EACH ROW EXECUTE FUNCTION test_case_reject()"
)
r = await client.post(
"/internal/report", json=synthetic_report(g, m), headers=worker_headers(w, g)
)
assert r.status_code == 503, r.text
assert await c.fetchval("SELECT count(*) FROM case_results") == 0
assert await c.fetchval("SELECT report FROM judge_attempts") is None
assert await c.fetchval("SELECT state FROM evaluations") == "running"
await c.execute("DROP TRIGGER test_case_reject ON case_results")
await c.execute("DROP FUNCTION test_case_reject()")
assert (
await client.post(
"/internal/report", json=synthetic_report(g, m), headers=worker_headers(w, g)
)
).status_code == 204
asyncio.run(run())
def test_actual_expiry_stopping_quarantine_retry_and_old_attempt(
postgres_url, tmp_path, monkeypatch
):
async def run():
async with fixture(postgres_url, tmp_path, monkeypatch) as (client, c, workers, _, create):
created = await create()
other = await create()
w = workers[0]
g = await claim(client, created, w)
# Expired server-side lease (not a client clock claim) causes actual recovery scan.
await c.execute(
"UPDATE judge_attempts SET expires_at=clock_timestamp()-interval '1 second'"
)
assert (
await client.post(
"/internal/renew", json=g["binding"], headers=worker_headers(w, g)
)
).status_code == 409
for _ in range(40):
pending = (
await client.get("/internal/pending-stops", headers=worker_headers(w))
).json()
if pending:
break
await asyncio.sleep(0.05)
assert pending[0]["binding"] == g["binding"]
assert pending == [dict(binding=g["binding"], case_count=2)]
stopped = dict(
schema_version=1, kind="judge.stopped", binding=g["binding"], cleanup_complete=False
)
assert (
await client.post("/internal/stopped", json=stopped, headers=worker_headers(w))
).status_code == 204
assert (
await c.fetchval(
"SELECT failure_code FROM evaluations WHERE id=$1", created.evaluation_id
)
== "cleanup_failed"
)
assert (
await client.post(
"/internal/claim", json=notice(other.evaluation_id), headers=worker_headers(w)
)
).status_code == 429
stopped["cleanup_complete"] = True
# Synthetic stop assertion only tests DB state. Actual Root proof is required in VM.
assert (
await client.post("/internal/stopped", json=stopped, headers=worker_headers(w))
).status_code == 204
assert (
await client.post("/internal/stopped", json=stopped, headers=worker_headers(w))
).status_code == 204
assert (
await c.fetchval("SELECT state FROM evaluations WHERE id=$1", created.evaluation_id)
== "queued"
)
await c.execute(
"UPDATE evaluations SET not_before=clock_timestamp() WHERE id=$1",
created.evaluation_id,
)
fresh = await claim(client, created, w)
assert fresh["attempt_number"] == 2 and fresh["binding"]["fencing_version"] == 2
assert (
await client.post(
"/internal/stage",
json=dict(
schema_version=1,
kind="judge.stage",
binding=g["binding"],
sequence=1,
stage="compiling",
case_index=None,
),
headers=worker_headers(w, g),
)
).status_code == 409
assert (
await client.post("/internal/stopped", json=stopped, headers=worker_headers(w))
).status_code == 409
asyncio.run(run())
def test_actual_user_cancel_all_stages_and_bounded_infrastructure_retry(
postgres_url, tmp_path, monkeypatch
):
async def run():
async with fixture(postgres_url, tmp_path, monkeypatch) as (
client,
c,
workers,
platform,
create,
):
w = workers[0]
for stage in ["queued", "compiling", "running"]:
created = await create()
g = None
if stage != "queued":
g = await claim(client, created, w)
r = dict(
schema_version=1,
kind="judge.stage",
binding=g["binding"],
sequence=1,
stage="compiling",
case_index=None,
)
assert (
await client.post("/internal/stage", json=r, headers=worker_headers(w, g))
).status_code == 204
if stage == "running":
r |= dict(sequence=2, stage="running", case_index=0)
assert (
await client.post(
"/internal/stage", json=r, headers=worker_headers(w, g)
)
).status_code == 204
assert (
await client.post(
"/internal/cancel",
json=notice(created.evaluation_id),
headers={"Authorization": "Bearer " + platform},
)
).status_code == 204
if g:
assert (
await c.fetchval(
"SELECT state FROM evaluations WHERE id=$1", created.evaluation_id
)
== "cancelling"
)
assert (
await client.post(
"/internal/renew", json=g["binding"], headers=worker_headers(w, g)
)
).status_code == 409
r = dict(
schema_version=1,
kind="judge.stopped",
binding=g["binding"],
cleanup_complete=True,
)
assert (
await client.post("/internal/stopped", json=r, headers=worker_headers(w))
).status_code == 204
assert (
await c.fetchval(
"SELECT state FROM evaluations WHERE id=$1", created.evaluation_id
)
== "cancelled"
)
created = await create()
for number in range(1, 4):
g = await claim(client, created, w)
m = await manifest(c, g)
report = synthetic_report(g, m) | dict(
compilation=dict(
status="system_error",
program=None,
diagnostics="",
diagnostics_truncated=False,
cpu_time_us=None,
memory_peak_bytes=None,
),
cases=[],
verdict="system_error",
failure_code="sandbox_unavailable",
run_cpu_time_us=None,
run_memory_peak_bytes=None,
)
assert (
await client.post("/internal/report", json=report, headers=worker_headers(w, g))
).status_code == 204
state = await c.fetchval(
"SELECT state FROM evaluations WHERE id=$1", created.evaluation_id
)
assert state == ("failed" if number == 3 else "queued")
if state == "queued":
await c.execute(
"UPDATE evaluations SET not_before=clock_timestamp() WHERE id=$1",
created.evaluation_id,
)
assert (
await client.post(
"/internal/claim", json=notice(created.evaluation_id), headers=worker_headers(w)
)
).status_code == 204
assert (
await c.fetchval(
"SELECT count(*) FROM judge_attempts WHERE evaluation_id=$1",
created.evaluation_id,
)
== 3
)
asyncio.run(run())
+47
View File
@@ -0,0 +1,47 @@
import asyncio
import json
import secrets
from http.server import BaseHTTPRequestHandler, HTTPServer
from threading import Thread
from uuid import uuid4
from kf_oj.infra.coordinator import CoordinatorControl
def test_real_loopback_cancellation_notification_and_failure():
token, evaluation = secrets.token_hex(32), uuid4()
class Handler(BaseHTTPRequestHandler):
code = 204
seen = []
def do_POST(self):
assert self.path == "/internal/cancel"
assert self.headers["Authorization"] == "Bearer " + token
body = json.loads(self.rfile.read(int(self.headers["Content-Length"])))
assert body == {
"schema_version": 1,
"kind": "judge.evaluation",
"evaluation_id": str(evaluation),
}
self.seen.append(body)
self.send_response(self.code)
self.end_headers()
def log_message(self, *_):
pass
server = HTTPServer(("127.0.0.1", 0), Handler)
thread = Thread(target=server.serve_forever, daemon=True)
thread.start()
control = CoordinatorControl(f"http://127.0.0.1:{server.server_port}/", token)
try:
assert asyncio.run(control.cancel(evaluation)) is True
Handler.code = 503
assert asyncio.run(control.cancel(evaluation)) is False
assert len(Handler.seen) == 2
finally:
server.shutdown()
server.server_close()
thread.join(timeout=2)
assert asyncio.run(control.cancel(evaluation)) is False
+262
View File
@@ -0,0 +1,262 @@
import asyncio
import json
from uuid import uuid4
import asyncpg
import pytest
from conftest import connect, migrate
def test_migration_roundtrip_and_role_safety(postgres_url):
async def inspect():
c = await connect(postgres_url)
try:
tables = set(await c.fetch("SELECT tablename FROM pg_tables WHERE schemaname='public'"))
assert len(tables) == 13 # Twelve business tables and Alembic history.
roles = await c.fetch("SELECT * FROM pg_roles WHERE rolname LIKE 'kf_oj_%'")
assert {row["rolname"] for row in roles} == {"kf_oj_platform", "kf_oj_coordinator"}
assert all(
not any(
r[k]
for k in (
"rolsuper",
"rolcanlogin",
"rolcreatedb",
"rolcreaterole",
"rolbypassrls",
)
)
for r in roles
)
await c.execute("SET ROLE kf_oj_platform")
with pytest.raises(asyncpg.InsufficientPrivilegeError):
await c.execute("CREATE TABLE forbidden_table (x int)")
await c.execute("RESET ROLE")
finally:
await c.close()
asyncio.run(inspect())
migrate(postgres_url, "downgrade", "base")
migrate(postgres_url, "upgrade", "head")
migrate(postgres_url, "check")
asyncio.run(inspect())
async def fixture_rows(c):
account, problem, version, case, submission, evaluation = (uuid4() for _ in range(6))
await c.execute(
"INSERT INTO accounts (id,username,password_hash) VALUES ($1,'fixture_user',$2)",
account,
"synthetic-irreversible-placeholder-hash-" * 2,
)
await c.execute("INSERT INTO artifacts (digest,bytes) VALUES ($1,3),($2,0)", "a" * 64, "b" * 64)
await c.execute("INSERT INTO problems (id,slug) VALUES ($1,'fixture-problem')", problem)
await c.execute(
"""INSERT INTO problem_versions
(id,problem_id,revision,title,statement,languages,toolchain,compile_limits,run_limits,
job_wall_time_us,public_samples)
VALUES ($1,$2,1,'Fixture','Public statement','["c17"]','{}','{}','{}',10000,'[]')""",
version,
problem,
)
await c.execute(
"""INSERT INTO problem_cases
(id,version_id,position,input_digest,expected_digest) VALUES ($1,$2,0,$3,$4)""",
case,
version,
"b" * 64,
"a" * 64,
)
await c.execute("UPDATE problem_versions SET published_at=now() WHERE id=$1", version)
await c.execute("UPDATE problems SET active_version_id=$1 WHERE id=$2", version, problem)
await c.execute(
"""INSERT INTO submissions
(id,account_id,problem_id,problem_version_id,language,source_digest,idempotency_key,
request_digest) VALUES ($1,$2,$3,$4,'c17',$5,$6,$7)""",
submission,
account,
problem,
version,
"a" * 64,
uuid4(),
"c" * 64,
)
await c.execute(
"INSERT INTO evaluations (id,submission_id,problem_version_id) VALUES ($1,$2,$3)",
evaluation,
submission,
version,
)
await c.execute("INSERT INTO outbox (id,evaluation_id) VALUES ($1,$2)", uuid4(), evaluation)
return account, problem, version, case, submission, evaluation
def test_database_write_authorities_and_immutable_versions(postgres_url):
async def run():
c = await connect(postgres_url)
try:
await c.execute("SET ROLE kf_oj_platform")
ids = await fixture_rows(c)
assert await c.fetchval("SELECT state FROM evaluations WHERE id=$1", ids[5]) == "queued"
for command in (
"UPDATE evaluations SET state='finished'",
"INSERT INTO evaluations (id,submission_id,problem_version_id,state) "
"SELECT gen_random_uuid(),submission_id,problem_version_id,'finished' "
"FROM evaluations",
"DELETE FROM evaluations",
"UPDATE problem_versions SET title='altered'",
"INSERT INTO judge_attempts (id) VALUES (gen_random_uuid())",
"UPDATE artifacts SET bytes=42",
):
with pytest.raises(asyncpg.InsufficientPrivilegeError):
await c.execute(command)
with pytest.raises(asyncpg.CheckViolationError):
await c.execute("UPDATE problem_versions SET published_at=now()")
with pytest.raises(asyncpg.CheckViolationError):
await c.execute(
"""INSERT INTO problem_cases
(id,version_id,position,input_digest,expected_digest)
VALUES ($1,$2,1,$3,$3)""",
uuid4(),
ids[2],
"a" * 64,
)
with pytest.raises(asyncpg.ForeignKeyViolationError):
await c.execute("UPDATE artifacts SET state='deleting' WHERE digest=$1", "a" * 64)
await c.execute("RESET ROLE")
await c.execute("SET ROLE kf_oj_coordinator")
for command in (
"UPDATE accounts SET enabled=false",
"INSERT INTO problems (id,slug) VALUES (gen_random_uuid(),'forbidden')",
"UPDATE submissions SET language='cpp20'",
):
with pytest.raises(asyncpg.InsufficientPrivilegeError):
await c.execute(command)
attempt = uuid4()
async with c.transaction():
await c.execute(
"""INSERT INTO judge_attempts
(id,evaluation_id,worker_id,number,fencing_version,lease_token_hash,
expires_at,manifest,manifest_digest)
VALUES ($1,$2,$3,1,1,$4,now()+interval '30 seconds',$5,$6)""",
attempt,
ids[5],
uuid4(),
"d" * 64,
json.dumps({"fixture": True}),
"e" * 64,
)
await c.execute(
"""UPDATE evaluations SET current_attempt_id=$1,state='claimed',
attempts_count=1,fencing_version=1 WHERE id=$2""",
attempt,
ids[5],
)
assert await c.fetchval("SELECT state FROM evaluations") == "claimed"
with pytest.raises(asyncpg.InsufficientPrivilegeError):
await c.execute("UPDATE judge_attempts SET manifest='{}'")
# Active-attempt FK is deferred but still rejects an unrelated/absent attempt at commit.
with pytest.raises(asyncpg.ForeignKeyViolationError):
async with c.transaction():
await c.execute("UPDATE evaluations SET current_attempt_id=$1", uuid4())
finally:
await c.close()
asyncio.run(run())
def test_initial_creation_transaction_rolls_back_and_cross_version_fk_rejects(postgres_url):
async def run():
c = await connect(postgres_url)
try:
with pytest.raises(RuntimeError):
async with c.transaction():
await c.execute("SET LOCAL ROLE kf_oj_platform")
await fixture_rows(c)
raise RuntimeError("simulated transaction failure")
assert await c.fetchval("SELECT count(*) FROM submissions") == 0
assert await c.fetchval("SELECT count(*) FROM outbox") == 0
ids = await fixture_rows(c)
with pytest.raises(asyncpg.ForeignKeyViolationError):
await c.execute(
"UPDATE evaluations SET problem_version_id=$1 WHERE id=$2",
uuid4(),
ids[5],
)
finally:
await c.close()
asyncio.run(run())
def test_publication_serializes_concurrent_case_insertion(postgres_url):
async def run():
c, other = await connect(postgres_url), await connect(postgres_url)
pending = None
transaction = None
try:
ids = await fixture_rows(c)
# A new draft version, separate from the immutable already published fixture.
draft = uuid4()
await c.execute(
"""INSERT INTO problem_versions
(id,problem_id,revision,title,statement,languages,toolchain,compile_limits,
run_limits,job_wall_time_us,public_samples)
SELECT $1,problem_id,2,title,statement,languages,toolchain,compile_limits,
run_limits,job_wall_time_us,public_samples FROM problem_versions WHERE id=$2
""",
draft,
ids[2],
)
await c.execute(
"""INSERT INTO problem_cases
(id,version_id,position,input_digest,expected_digest)
VALUES ($1,$2,0,$3,$3)""",
uuid4(),
draft,
"a" * 64,
)
transaction = c.transaction()
await transaction.start()
await c.execute("UPDATE problem_versions SET published_at=now() WHERE id=$1", draft)
pending = asyncio.create_task(
other.execute(
"""INSERT INTO problem_cases
(id,version_id,position,input_digest,expected_digest)
VALUES ($1,$2,1,$3,$3)""",
uuid4(),
draft,
"a" * 64,
)
)
await asyncio.sleep(0.1)
assert not pending.done() # Actual row lock, not a simulated repository response.
await transaction.commit()
transaction = None
with pytest.raises(asyncpg.CheckViolationError):
await asyncio.wait_for(pending, timeout=5)
assert (
await c.fetchval("SELECT count(*) FROM problem_cases WHERE version_id=$1", draft)
== 1
)
with pytest.raises(asyncpg.CheckViolationError):
await c.execute(
"""INSERT INTO problem_versions
(id,problem_id,revision,title,statement,languages,toolchain,compile_limits,
run_limits,job_wall_time_us,public_samples,published_at)
SELECT $1,problem_id,3,title,statement,languages,toolchain,compile_limits,
run_limits,job_wall_time_us,public_samples,now()
FROM problem_versions WHERE id=$2""",
uuid4(),
draft,
)
finally:
if transaction:
await transaction.rollback()
if pending and not pending.done():
pending.cancel()
await asyncio.gather(pending, return_exceptions=True)
await c.close()
await other.close()
asyncio.run(run())
+101
View File
@@ -0,0 +1,101 @@
"""Offline package data only; no compiler, submitted source or image execution."""
import importlib.util
import json
from pathlib import Path
from types import SimpleNamespace
import pytest
SPEC = importlib.util.spec_from_file_location(
"gnu_image_builder", Path(__file__).resolve().parents[2] / "scripts/build_gnu_image.py"
)
builder = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(builder)
def test_absolute_package_link_is_relocated_inside_verified_image(tmp_path):
root = tmp_path / "image"
target = root / "usr/lib/plugin.so"
target.parent.mkdir(parents=True)
target.write_bytes(b"synthetic pinned package bytes")
link = root / "usr/lib/plugin-link.so"
link.symlink_to("/usr/lib/plugin.so")
records = {
"usr/lib/plugin-link.so": {"kind": "symlink", "target": "/usr/lib/plugin.so"},
"usr/lib/plugin.so": {"kind": "file", "sha256": builder.sha(target)},
}
source, record = builder.resolve_verified(root, records, "usr/lib/plugin-link.so")
assert source == target
assert source.read_bytes() == b"synthetic pinned package bytes"
assert record == records["usr/lib/plugin.so"]
@pytest.mark.parametrize("target", ["../../../../outside", "plugin-link.so"])
def test_package_link_escape_or_cycle_refused(tmp_path, target):
link = tmp_path / "usr/lib/plugin-link.so"
link.parent.mkdir(parents=True)
link.symlink_to(target)
records = {"usr/lib/plugin-link.so": {"kind": "symlink", "target": target}}
with pytest.raises(ValueError):
builder.resolve_verified(tmp_path, records, "usr/lib/plugin-link.so")
def test_replaced_or_unverified_ancestor_link_is_refused(tmp_path):
library = tmp_path / "usr/lib"
library.parent.mkdir()
library.symlink_to("/usr/lib")
records = {"usr/lib/plugin.so": {"kind": "file", "sha256": "a" * 64}}
with pytest.raises(ValueError, match="unverified GNU ancestor"):
builder.resolve_verified(tmp_path, records, "usr/lib/plugin.so")
records["usr/lib"] = {"kind": "symlink", "target": "/different/path"}
with pytest.raises(ValueError, match="differs from verified manifest"):
builder.resolve_verified(tmp_path, records, "usr/lib/plugin.so")
def test_verified_manifest_does_not_authorize_changed_package_bytes(tmp_path):
root = tmp_path / "support"
library = root / "usr/lib/ld-linux-x86-64.so.2"
library.parent.mkdir(parents=True)
library.write_bytes(b"original synthetic library")
manifest = {
"packages": [
{
"name": "gcc",
"version": "16.2.1+r23+gd564253eb6c8-1",
"sha256": "ebac12ceef494c616a36fc6f0f792b365dc59b6ed46ca1a69f138aecc810b6ab",
},
{
"name": "glibc",
"version": "2.44+r50+g1848099f063e-3",
"sha256": "071d807f4d3bf6b921f86b9a91e83a47ecc026c5821634efa46b09d890e06785",
},
{
"name": "binutils",
"version": "2.47-2",
"sha256": "62efd213e87590f025cf861b646793d3cacb03e7e26187b73d33b00adaf7a977",
},
],
"files": [
{
"path": "usr/lib/ld-linux-x86-64.so.2",
"kind": "file",
"bytes": library.stat().st_size,
"sha256": builder.sha(library),
}
],
}
manifest_path = tmp_path / "manifest.json"
manifest_path.write_text(json.dumps(manifest))
library.write_bytes(b"changed synthetic library")
output = tmp_path / "output"
args = SimpleNamespace(
support_dir=root,
support_manifest=manifest_path,
support_manifest_sha256=builder.sha(manifest_path),
output=output,
)
with pytest.raises(ValueError, match="differs from verified public package bytes"):
builder.build(args)
assert not (output / "manifest.json").exists()
assert not list(output.rglob("*.so*"))
+3 -1
View File
@@ -21,7 +21,9 @@ def test_liveness_without_database_or_queue() -> None:
def test_production_disables_api_documentation() -> None:
app = create_app(Settings(_env_file=None, environment="production"))
app = create_app(
Settings(_env_file=None, environment="production", browser_origins=("https://oj.example",))
)
async def request():
async with httpx.AsyncClient(
+284
View File
@@ -0,0 +1,284 @@
import asyncio
import os
import pty
import secrets
import select as io_select
import signal
import sys
import time
from datetime import timedelta
from uuid import uuid4
import httpx
import pytest
from sqlalchemy import event, func, select, update
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from kf_oj.api.app import create_app
from kf_oj.contracts.identity import LoginRequest
from kf_oj.core.settings import Settings
from kf_oj.modules.identity.models import Account, Session
from kf_oj.modules.identity.passwords import hash_password, verify_password
from kf_oj.modules.identity.public import AuthenticationFailed, LoginThrottled
from kf_oj.modules.identity.service import IdentityService, token_digest
def restrict_platform_role(engine):
@event.listens_for(engine.sync_engine, "connect")
def apply_role(connection, _):
cursor = connection.cursor()
cursor.execute("SET ROLE kf_oj_platform")
cursor.close()
def test_password_policy_and_credentials_redacted():
password = secrets.token_hex(16)
hashed = hash_password("fixture_user", password)
assert hashed.startswith("$argon2id$v=19$m=19456,t=2,p=1$")
assert verify_password(hashed, password)
assert not verify_password(hashed, secrets.token_hex(16))
assert not verify_password("$argon2id$v=19$m=999999999,t=2,p=1$invalid", password)
assert password not in repr(LoginRequest(username="fixture_user", password=password))
with pytest.raises(ValueError):
hash_password("not canonical", password)
with pytest.raises(ValueError):
hash_password("fixture_user", "too short")
def test_production_cookie_configuration_requires_explicit_https_origins():
with pytest.raises(ValueError):
Settings(_env_file=None, environment="production")
for invalid in ("https://oj.example/path", "https://user@oj.example", "*", "null"):
with pytest.raises(ValueError):
Settings(_env_file=None, browser_origins=(invalid,))
def test_real_login_session_revocation_expiry_and_throttle(postgres_url):
async def run():
engine = create_async_engine(postgres_url)
restrict_platform_role(engine)
sessions = async_sessionmaker(engine, expire_on_commit=False)
identity = IdentityService(sessions, lifetime_seconds=300)
password = secrets.token_hex(16)
try:
principal = await identity.create_account("fixture_user", password)
with pytest.raises(AuthenticationFailed):
await identity.login("fixture_user", secrets.token_hex(16), peer="fixture-peer")
with pytest.raises(AuthenticationFailed):
await identity.login("absent_user", password, peer="fixture-peer")
issued = await identity.login("fixture_user", password, peer="fixture-peer")
assert await identity.authenticate(issued.token) == principal
assert issued.token not in repr(issued) and issued.csrf_token not in repr(issued)
with pytest.raises(AuthenticationFailed):
await identity.authenticate(str(principal.account_id))
with pytest.raises(AuthenticationFailed):
await identity.authenticate(issued.token, csrf_token="f" * 64)
await identity.logout(issued.token, issued.csrf_token)
with pytest.raises(AuthenticationFailed):
await identity.authenticate(issued.token)
issued = await identity.login("fixture_user", password, peer="expiry-peer")
async with sessions.begin() as session:
# Keep the expiry constraint valid while moving the entire session into the past.
await session.execute(
update(Session).values(
created_at=func.now() - timedelta(minutes=10),
expires_at=func.now() - timedelta(minutes=5),
)
)
with pytest.raises(AuthenticationFailed):
await identity.authenticate(issued.token)
issued = await identity.login("fixture_user", password, peer="disable-peer")
assert await identity.disable_account("fixture_user")
with pytest.raises(AuthenticationFailed):
await identity.authenticate(issued.token)
with pytest.raises(AuthenticationFailed):
await identity.login("fixture_user", password, peer="disable-peer")
assert not await identity.disable_account("unknown_user")
# Exactly ten real failed attempts consume the same persistent peer budget.
for _ in range(10):
with pytest.raises(AuthenticationFailed):
await identity.login("absent_user", password, peer="limited-peer")
with pytest.raises(LoginThrottled):
await identity.login("absent_user", password, peer="limited-peer")
async with sessions() as session:
account = await session.scalar(select(Account))
assert password != account.password_hash and password not in account.password_hash
rows = list(await session.scalars(select(Session)))
assert all(row.token_hash != issued.token for row in rows)
finally:
await engine.dispose()
asyncio.run(run())
def test_real_http_cookie_csrf_and_identity_spoofing_rejection(postgres_url, caplog):
async def run():
app = create_app(
Settings(
_env_file=None,
environment="production",
database_url=postgres_url,
browser_origins=("https://test",),
)
)
async with app.router.lifespan_context(app):
restrict_platform_role(app.state.sessions.kw["bind"])
password = secrets.token_hex(16)
first = await app.state.identity.create_account("first_user", password)
second = await app.state.identity.create_account("second_user", secrets.token_hex(16))
headers = {"Origin": "https://test"}
async with httpx.AsyncClient(
transport=httpx.ASGITransport(app=app), base_url="https://test"
) as client:
body = {"username": "first_user", "password": password}
assert (await client.get("/api/v1/auth/me")).status_code == 401
assert (await client.post("/api/v1/auth/login", json=body)).status_code == 403
assert (
await client.post(
"/api/v1/auth/login",
json=body,
headers={"Origin": "https://hostile.example"},
)
).status_code == 403
invalid = await client.post(
"/api/v1/auth/login",
headers=headers,
json=body | {"user_id": str(second.account_id), "role": "admin"},
)
assert invalid.status_code == 422
assert password not in invalid.text and "admin" not in invalid.text
response = await client.post("/api/v1/auth/login", json=body, headers=headers)
assert response.status_code == 200
assert response.json()["user"]["id"] == str(first.account_id)
assert (
await client.get(
"/api/v1/auth/me", headers={"X-User-ID": str(second.account_id)}
)
).json()["id"] == str(first.account_id)
cookie = response.headers.get_list("set-cookie")[0]
assert "HttpOnly" in cookie and "Secure" in cookie and "SameSite=strict" in cookie
assert response.headers["cache-control"] == "no-store"
assert client.cookies.get("kf_csrf") == response.json()["csrf_token"]
assert (
await client.post("/api/v1/auth/logout", headers=headers)
).status_code == 403
assert (
await client.post(
"/api/v1/auth/logout", headers=headers | {"X-CSRF-Token": "f" * 64}
)
).status_code == 403
assert (
await client.post(
"/api/v1/auth/logout",
headers=headers | {"X-CSRF-Token": response.json()["csrf_token"]},
)
).status_code == 204
assert (await client.get("/api/v1/auth/me")).status_code == 401
assert not client.cookies.get("kf_session") and not client.cookies.get("kf_csrf")
assert (
password not in caplog.text and response.json()["csrf_token"] not in caplog.text
)
asyncio.run(run())
def test_real_concurrent_logins_keep_bounded_active_sessions(postgres_url):
async def run():
engine = create_async_engine(postgres_url)
restrict_platform_role(engine)
sessions = async_sessionmaker(engine, expire_on_commit=False)
identity = IdentityService(sessions)
password = secrets.token_hex(16)
try:
await identity.create_account("fixture_user", password)
# Independent peer budgets exercise session bounds without exhausting login retries.
issued = []
for _ in range(12):
issued.append(await identity.login("fixture_user", password, peer=uuid4().hex))
with pytest.raises(AuthenticationFailed):
await identity.authenticate(issued[0].token)
results = await asyncio.gather(
*(identity.login("fixture_user", password, peer=uuid4().hex) for _ in range(3)),
return_exceptions=True,
)
assert sum(isinstance(result, LoginThrottled) for result in results) == 1
assert sum(not isinstance(result, Exception) for result in results) == 2
async with sessions() as session:
assert await session.scalar(select(func.count()).select_from(Session)) == 10
assert (
await session.scalar(
select(func.count())
.select_from(Session)
.where(Session.token_hash == token_digest("0" * 64))
)
== 0
)
finally:
await engine.dispose()
asyncio.run(run())
def test_account_cli_hidden_password_and_disable(postgres_url):
password = secrets.token_hex(16)
# The CLI prompts through a real private PTY; the password is never an argument/environment.
pid, descriptor = pty.fork()
if pid == 0:
os.environ["KF_OJ_DATABASE_URL"] = postgres_url
os.execv(
sys.executable,
[sys.executable, "-m", "kf_oj.modules.identity.cli", "create", "cli_user"],
)
output = b""
deadline, prompts, status = time.monotonic() + 15, 0, None
try:
while time.monotonic() < deadline:
if io_select.select([descriptor], [], [], 0.1)[0]:
try:
block = os.read(descriptor, 4096)
except OSError:
block = b""
output += block
if prompts == 0 and b"Password (12" in output:
os.write(descriptor, password.encode() + b"\n")
prompts = 1
if prompts == 1 and b"Repeat password:" in output:
os.write(descriptor, password.encode() + b"\n")
prompts = 2
result, status = os.waitpid(pid, os.WNOHANG)
if result:
break
else:
os.kill(pid, signal.SIGTERM)
os.waitpid(pid, 0)
pytest.fail("account CLI deadline exceeded")
assert os.waitstatus_to_exitcode(status) == 0
assert prompts == 2 and b"Account created." in output
assert password.encode() not in output
finally:
os.close(descriptor)
async def inspect():
engine = create_async_engine(postgres_url)
sessions = async_sessionmaker(engine, expire_on_commit=False)
try:
service = IdentityService(sessions)
issued = await service.login("cli_user", password, peer="cli-fixture")
from kf_oj.modules.identity.cli import run
await run("disable", "cli_user")
with pytest.raises(AuthenticationFailed):
await service.authenticate(issued.token)
finally:
await engine.dispose()
# CLI run uses the test DB only for this call; don't reuse a configured business database.
original = os.environ.get("KF_OJ_DATABASE_URL")
os.environ["KF_OJ_DATABASE_URL"] = postgres_url
try:
asyncio.run(inspect())
finally:
if original is None:
os.environ.pop("KF_OJ_DATABASE_URL", None)
else:
os.environ["KF_OJ_DATABASE_URL"] = original
+143
View File
@@ -0,0 +1,143 @@
import json
from pathlib import Path
from uuid import UUID
import pytest
from jsonschema import Draft202012Validator, FormatChecker
from pydantic import ValidationError
from kf_oj.contracts.judge_state import (
EvaluationState,
require_current_lease,
require_transition,
)
from kf_oj.contracts.judging import (
MAX_WIRE_BYTES,
WIRE_MODELS,
AdmissionPolicy,
JobManifest,
JudgementReport,
LeaseGrant,
decode_wire,
)
ROOT = Path(__file__).resolve().parents[2]
FIXTURES = ROOT / "tests/fixtures/contracts"
PATHS = sorted(p for name in WIRE_MODELS for p in FIXTURES.glob(f"{name}.*.json"))
@pytest.mark.parametrize("path", PATHS, ids=lambda p: p.stem)
def test_shared_judging_examples(path: Path) -> None:
name = path.name.split(".")[0]
model = WIRE_MODELS[name]
payload = json.loads(path.read_text())
schema = json.loads((ROOT / f"contracts/schemas/{name}.v1.schema.json").read_text())
validator = Draft202012Validator(schema, format_checker=FormatChecker())
if ".invalid-" in path.name:
with pytest.raises(ValueError):
decode_wire(model, path.read_bytes())
if ".invalid-semantic-" in path.name:
assert validator.is_valid(payload)
else:
assert not validator.is_valid(payload)
else:
decoded = decode_wire(model, path.read_bytes())
validator.validate(payload)
if name == "judge-admission" and "defaults" in path.name:
assert decoded.max_concurrent_jobs == decoded.prefetch == 1
assert decoded.max_retries == 2
else:
assert decoded.model_dump(mode="json") == payload
def fixture(name: str, model):
return decode_wire(model, (FIXTURES / f"{name}.valid.json").read_bytes())
def test_every_state_transition_matches_shared_table() -> None:
transitions = json.loads((FIXTURES / "judge-state.transitions.json").read_text())
assert set(transitions) == {s.value for s in EvaluationState}
for current in EvaluationState:
for target in EvaluationState:
allowed = target.value in transitions[current.value]
assert current.allows(target) == allowed
if allowed:
require_transition(current, target)
else:
with pytest.raises(ValueError):
require_transition(current, target)
def test_lease_fencing_identity_and_expiry_boundary() -> None:
grant = fixture("judge-lease", LeaseGrant)
binding = grant.binding
require_current_lease(binding, binding, grant.expires_at_unix_ms, grant.expires_at_unix_ms - 1)
for now in (grant.expires_at_unix_ms, grant.expires_at_unix_ms + 1, -1):
with pytest.raises(ValueError):
require_current_lease(binding, binding, grant.expires_at_unix_ms, now)
for field, value in (
("fencing_version", 2),
("attempt_id", UUID("aaaaaaaa-aaaa-4aaa-8aaa-000000000009")),
("worker_id", UUID("aaaaaaaa-aaaa-4aaa-8aaa-000000000009")),
("evaluation_id", UUID("aaaaaaaa-aaaa-4aaa-8aaa-000000000009")),
):
supplied = binding.model_copy(update={field: value})
with pytest.raises(ValueError):
require_current_lease(binding, supplied, grant.expires_at_unix_ms, 1)
assert grant.lease_token not in repr(grant)
def test_node_admission_never_accepts_job_or_retry_above_trusted_policy() -> None:
manifest = fixture("judge-manifest", JobManifest)
policy = fixture("judge-admission", AdmissionPolicy)
for attempt in (1, 2, 3):
policy.admit(manifest, attempt)
for attempt in (0, 4, True):
with pytest.raises(ValueError):
policy.admit(manifest, attempt)
for field, value in (("max_package_bytes", 1), ("max_job_wall_time_us", 1000)):
with pytest.raises(ValueError):
policy.model_copy(update={field: value}).admit(manifest, 1)
lower = policy.run_ceiling.model_copy(update={"memory_bytes": 32 * 1024 * 1024})
with pytest.raises(ValueError):
policy.model_copy(update={"run_ceiling": lower}).admit(manifest, 1)
other = manifest.model_copy(
update={"toolchain": manifest.toolchain.model_copy(update={"image_revision": 2})}
)
with pytest.raises(ValueError):
policy.admit(other, 1)
def test_result_requires_authorized_manifest_and_complete_case_order() -> None:
manifest = fixture("judge-manifest", JobManifest)
result = fixture("judge-result", JudgementReport)
result.validate_for_manifest(manifest, result.manifest_digest)
with pytest.raises(ValueError):
result.validate_for_manifest(manifest, "f" * 64)
short = result.model_dump(mode="json")
short.update(cases=short["cases"][:1], run_cpu_time_us=100, run_memory_peak_bytes=256 * 1024)
valid_partial = JudgementReport.model_validate(short)
with pytest.raises(ValueError):
valid_partial.validate_for_manifest(manifest, result.manifest_digest)
wrong_order = result.model_dump(mode="json")
wrong_order["cases"].reverse()
with pytest.raises(ValueError):
JudgementReport.model_validate(wrong_order).validate_for_manifest(
manifest, result.manifest_digest
)
def test_wire_decoding_bounds_ambiguity_and_private_error_output() -> None:
payload = (FIXTURES / "judge-lease.valid.json").read_bytes()
duplicate = payload.replace(b'"schema_version": 1', b'"schema_version": 1, "schema_version": 1')
for invalid in (duplicate, payload.decode().encode("utf-16"), b" " * (MAX_WIRE_BYTES + 1)):
with pytest.raises(ValueError):
decode_wire(LeaseGrant, invalid)
secret = "do-not-echo-private-token"
with pytest.raises(ValueError) as error:
decode_wire(
LeaseGrant, payload.replace(b'"0' + b"0" * 63 + b'"', json.dumps(secret).encode())
)
assert secret not in str(error.value)
with pytest.raises(ValidationError):
LeaseGrant.model_validate({})
@@ -0,0 +1,64 @@
"""Image construction data tests; actual publication/execution needs the independent VM."""
import importlib.util
import io
import json
import tarfile
from pathlib import Path
from types import SimpleNamespace
import pytest
MODULE = Path(__file__).resolve().parents[2] / "scripts/build_judging_images.py"
SPEC = importlib.util.spec_from_file_location("judging_image_builder", MODULE)
builder = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(builder)
def test_catalog_uses_wire_path_order_including_dot_and_directory_boundaries(tmp_path):
image = tmp_path / "image"
image.mkdir()
builder.put(image, "usr/lib/module/a.py", b"data")
builder.put(image, "usr/lib/module.py", b"module")
support = tmp_path / "support"
library = support / "usr/lib/ld-linux-x86-64.so.2"
library.parent.mkdir(parents=True)
library.write_bytes(b"synthetic ordinary loader data")
args = SimpleNamespace(support_dir=support, support_manifest_sha256="a" * 64)
builder.finish(
args,
builder.PINS["python313"],
image,
{"usr/lib/ld-linux-x86-64.so.2": {"kind": "file", "sha256": builder.sha(library)}},
)
manifest = json.loads((image / "manifest.json").read_bytes())
assert manifest["architecture"] == "x86_64_v3"
paths = [item["path"] for item in manifest["files"]]
assert paths == sorted(paths)
assert paths.index("usr/lib/module.py") < paths.index("usr/lib/module/a.py")
def test_public_archive_links_cannot_escape_or_cycle():
buffer = io.BytesIO()
with tarfile.open(fileobj=buffer, mode="w") as archive:
for name, target in [("python/outside", "../../secret"), ("python/cycle", "cycle")]:
item = tarfile.TarInfo(name)
item.type = tarfile.SYMTYPE
item.linkname = target
archive.addfile(item)
buffer.seek(0)
with tarfile.open(fileobj=buffer, mode="r:") as archive:
members = {item.name: item for item in archive.getmembers()}
for name in members:
with pytest.raises(ValueError):
builder.member_bytes(archive, members, name)
def test_corrupt_archive_is_refused_before_output_creation(tmp_path):
archive = tmp_path / "public.tar.gz"
archive.write_bytes(b"untrusted replacement")
output = tmp_path / "image"
args = SimpleNamespace(language="python313", archive=archive, output=output)
with pytest.raises(ValueError, match="fixed upstream digest"):
builder.build(args)
assert not output.exists()
+269
View File
@@ -0,0 +1,269 @@
import hashlib
import json
from pathlib import Path
import pytest
from jsonschema import Draft202012Validator, FormatChecker
from kf_oj.contracts import judging as v1
from kf_oj.contracts import judging_v2 as v2
from kf_oj.contracts import judging_versions as versions
ROOT = Path(__file__).resolve().parents[2]
FIXTURES = ROOT / "tests/fixtures/contracts-v2"
PATHS = sorted(FIXTURES.glob("*.json"))
def fixture(name, model):
return v1.decode_wire(model, (FIXTURES / f"{name}.json").read_bytes())
@pytest.mark.parametrize("path", PATHS, ids=lambda p: p.stem)
def test_multi_runtime_examples_match_schema_and_rust(path):
name = path.name.split(".")[0]
model = v2.WIRE_MODELS[name]
schema = json.loads((ROOT / f"contracts/schemas/{name}.v2.schema.json").read_text())
validator = Draft202012Validator(schema, format_checker=FormatChecker())
original = json.loads(path.read_text())
if ".invalid-" in path.name:
with pytest.raises(ValueError):
v1.decode_wire(model, path.read_bytes())
if ".invalid-semantic-" in path.name:
assert validator.is_valid(original)
else:
assert not validator.is_valid(original)
else:
value = v1.decode_wire(model, path.read_bytes())
validator.validate(original)
assert value.model_dump(mode="json") == original
@pytest.mark.parametrize("language", list(v2.Language))
def test_explicit_version_keeps_old_consumers_closed_and_canonical_bytes_stable(language):
path = FIXTURES / f"judge-manifest.valid-{language}.json"
manifest = v1.decode_wire(v2.JobManifest, path.read_bytes())
canonical = (FIXTURES / f"canonical-manifest.{language}.txt").read_bytes()
assert v1.manifest_bytes(manifest) == canonical
assert (
hashlib.sha256(v1.manifest_bytes(manifest)).digest() == hashlib.sha256(canonical).digest()
)
with pytest.raises(ValueError):
v1.decode_wire(v1.JobManifest, path.read_bytes())
old = ROOT / "tests/fixtures/contracts/judge-manifest.valid.json"
with pytest.raises(ValueError):
v1.decode_wire(v2.JobManifest, old.read_bytes())
old_manifest = v1.decode_wire(v1.JobManifest, old.read_bytes())
assert (
v1.manifest_bytes(old_manifest)
== (ROOT / "tests/fixtures/contracts/canonical-manifest.v1.txt").read_bytes()
)
def test_shared_capabilities_refuse_absent_changed_runtime_and_over_budget():
policy = fixture("judge-admission.valid", v2.AdmissionPolicy)
for language in v2.Language:
manifest = fixture(f"judge-manifest.valid-{language}", v2.JobManifest)
for attempt in (1, 2, 3):
policy.admit(manifest, attempt)
for attempt in (0, 4, True):
with pytest.raises(ValueError):
policy.admit(manifest, attempt)
for field, value in (("image_revision", 2), ("image_digest", "f" * 64)):
other = manifest.model_copy(
update={"toolchain": manifest.toolchain.model_copy(update={field: value})}
)
with pytest.raises(ValueError):
policy.admit(other, 1)
python = fixture("judge-manifest.valid-python313", v2.JobManifest)
without_python = policy.model_copy(
update={
"capabilities": tuple(c for c in policy.capabilities if c.language != python.language)
}
)
with pytest.raises(ValueError):
without_python.admit(python, 1)
too_small = python.model_copy(
update={"run_limits": python.run_limits.model_copy(update={"memory_bytes": 1})}
)
with pytest.raises(ValueError):
policy.admit(too_small, 1)
too_large = python.model_copy(
update={"run_limits": python.run_limits.model_copy(update={"memory_bytes": 128 * v1.MIB})}
)
with pytest.raises(ValueError):
policy.admit(too_large, 1)
@pytest.mark.parametrize("language", list(v2.Language))
def test_results_bind_exact_source_runtime_attempt_and_case_order(language):
manifest = fixture(f"judge-manifest.valid-{language}", v2.JobManifest)
result = fixture(f"judge-result.valid-{language}", v2.JudgementReport)
digest = hashlib.sha256(versions.manifest_bytes(manifest)).hexdigest()
assert digest == (FIXTURES / f"canonical-manifest.{language}.sha256").read_text().strip()
result.validate_for_manifest(manifest, digest)
for field, value in (("digest", "f" * 64), ("bytes", manifest.source.bytes + 1)):
other = manifest.model_copy(
update={"source": manifest.source.model_copy(update={field: value})}
)
with pytest.raises(ValueError):
result.validate_for_manifest(other, digest)
with pytest.raises(ValueError):
result.validate_for_manifest(manifest, "f" * 64)
changed_runtime = manifest.model_copy(
update={"toolchain": manifest.toolchain.model_copy(update={"image_digest": "e" * 64})}
)
with pytest.raises(ValueError):
result.validate_for_manifest(changed_runtime, digest)
short = result.model_dump(mode="json")
short.update(cases=short["cases"][:1], run_cpu_time_us=100, run_memory_peak_bytes=256 * 1024)
valid_partial = v2.JudgementReport.model_validate(short)
with pytest.raises(ValueError):
valid_partial.validate_for_manifest(manifest, digest)
def test_python_check_is_not_an_accepted_result_and_stage_kind_is_bound():
manifest = fixture("judge-manifest.valid-python313", v2.JobManifest)
stage = fixture("judge-stage.valid-python313", v2.StageReport)
stage.validate_for_manifest(manifest)
with pytest.raises(ValueError):
stage.model_copy(
update={"preparation_kind": v2.PreparationKind.COMPILE}
).validate_for_manifest(manifest)
result = fixture("judge-result.valid-python313", v2.JudgementReport).model_dump(mode="json")
result.update(cases=[], run_cpu_time_us=None, run_memory_peak_bytes=None)
with pytest.raises(ValueError):
v2.JudgementReport.model_validate(result)
def test_jvm_heap_is_smaller_than_total_memory_and_threads_share_the_budget():
manifest = fixture("judge-manifest.valid-java21", v2.JobManifest)
limits = manifest.run_limits
assert 32 * v1.MIB <= v2.jvm_heap_bytes(limits) < limits.memory_bytes
more_threads = limits.model_copy(update={"processes": 32})
assert v2.jvm_heap_bytes(more_threads) < v2.jvm_heap_bytes(limits)
with pytest.raises(ValueError):
v2.jvm_heap_bytes(limits.model_copy(update={"memory_bytes": 128 * v1.MIB}))
# Ordinary RuntimeError has no infrastructure failure code or trusted memory-limit claim.
result = fixture("judge-result.valid-java21", v2.JudgementReport).model_dump(mode="json")
result["cases"][0]["verdict"] = "runtime_error"
result["verdict"] = "runtime_error"
v2.JudgementReport.model_validate(result)
def test_v2_wire_rejects_ambiguous_private_values_without_echoing_them():
data = (FIXTURES / "judge-lease.valid.json").read_bytes()
duplicate = data.replace(b'"schema_version": 2', b'"schema_version": 2, "schema_version": 2')
for invalid in (duplicate, b" " * (v1.MAX_WIRE_BYTES + 1), data.decode().encode("utf-16")):
with pytest.raises(ValueError):
v1.decode_wire(v2.LeaseGrant, invalid)
secret = "private-placeholder-never-echo"
value = json.loads(data)
value["lease_token"] = secret
with pytest.raises(ValueError) as error:
v1.decode_wire(v2.LeaseGrant, json.dumps(value).encode())
assert secret not in str(error.value)
def test_storage_boundary_reads_both_versions_without_rewriting_or_cross_version_reports():
old = ROOT / "tests/fixtures/contracts"
old_manifest = versions.decode_manifest((old / "judge-manifest.valid.json").read_bytes())
old_report = versions.decode_report((old / "judge-result.valid.json").read_bytes())
versions.validate_report(old_report, old_manifest, old_report.manifest_digest)
assert versions.manifest_bytes(old_manifest) == (old / "canonical-manifest.v1.txt").read_bytes()
manifest = versions.decode_manifest(
(FIXTURES / "judge-manifest.valid-python313.json").read_bytes()
)
report = versions.decode_report((FIXTURES / "judge-result.valid-python313.json").read_bytes())
versions.validate_report(
report, manifest, hashlib.sha256(versions.manifest_bytes(manifest)).hexdigest()
)
for result, task in ((old_report, manifest), (report, old_manifest)):
with pytest.raises(ValueError):
versions.validate_report(result, task, result.manifest_digest)
payload = (FIXTURES / "judge-manifest.valid-python313.json").read_bytes()
for invalid in (
payload.replace(b'"schema_version": 2', b'"schema_version": 3'),
payload.replace(b'"schema_version": 2', b'"schema_version": true'),
payload.replace(b'"schema_version": 2', b'"schema_version": 2, "schema_version": 2'),
):
with pytest.raises(ValueError):
versions.decode_manifest(invalid)
def test_registry_is_explicit_for_each_protocol_and_retains_legacy_configuration():
from kf_oj.core.settings import Settings
registry = fixture("judge-registry.valid", v2.AdmissionRegistry)
original = ROOT / "tests/fixtures/contracts"
legacy = v1.decode_wire(v1.JobManifest, (original / "judge-manifest.valid.json").read_bytes())
registry.admit(legacy, 1)
assert registry.legacy.model_dump(mode="json") == json.loads(
(original / "judge-admission.valid.json").read_text()
)
for language in v2.Language:
manifest = fixture(f"judge-manifest.valid-{language}", v2.JobManifest)
registry.admit(manifest, 1)
with pytest.raises(ValueError, match="protocol"):
v2.admit_manifest(registry.legacy, manifest, 1)
for field, value in (("image_digest", "f" * 64), ("runtime_version", "other")):
changed = manifest.model_copy(
update={"toolchain": manifest.toolchain.model_copy(update={field: value})}
)
with pytest.raises(ValueError):
registry.admit(changed, 1)
with pytest.raises(ValueError, match="protocol"):
v2.admit_manifest(registry.multi_runtime, legacy, 1)
for policy in (registry.legacy, registry.multi_runtime, registry):
settings = Settings(_env_file=None, judge_admission=policy.model_dump(mode="json"))
assert type(settings.judge_admission) is type(policy)
raw = registry.model_dump(mode="json")
raw["multi_runtime"]["prefetch"] = 2
with pytest.raises(ValueError):
Settings(_env_file=None, judge_admission=raw)
def test_generic_admission_revalidates_constructed_values_in_both_generations():
registry = fixture("judge-registry.valid", v2.AdmissionRegistry)
old = ROOT / "tests/fixtures/contracts/judge-manifest.valid.json"
legacy = v1.decode_wire(v1.JobManifest, old.read_bytes())
for manifest in (legacy, fixture("judge-manifest.valid-python313", v2.JobManifest)):
for wrong in (
manifest.model_copy(update={"schema_version": True}),
manifest.model_copy(update={"job_wall_time_us": 1}),
manifest.model_copy(update={"cases": ()}),
):
with pytest.raises(ValueError):
v2.admit_manifest(registry, wrong, 1)
v2.admit_manifest(registry, legacy, 1)
def test_operator_registry_example_preserves_old_protocol_and_binds_managed_runtime():
policy = v2.AdmissionRegistry.model_validate_json(
(ROOT / "deploy/judge.admission.example.json").read_bytes()
)
old = v1.decode_wire(
v1.JobManifest, (ROOT / "tests/fixtures/contracts/judge-manifest.valid.json").read_bytes()
)
policy.admit(old, 1)
assert policy.multi_runtime is not None
root = json.loads((ROOT / "deploy/sandbox.managed.runtime.example.json").read_text())
for cap in policy.multi_runtime.capabilities:
selected = next(
item
for item in root["runtime_images"]
if item["runtime_identity"] == cap.toolchain.model_dump(mode="json")
)
ceiling = v1.RunLimits.model_validate(selected["per_task_ceiling"])
assert cap.compile_ceiling.fits(ceiling) and cap.run_ceiling.fits(ceiling)
manifest = fixture(f"judge-manifest.valid-{cap.language}", v2.JobManifest).model_copy(
update={"compile_limits": cap.compile_ceiling, "run_limits": cap.run_ceiling}
)
policy.admit(manifest, 1)
changed = manifest.model_copy(
update={"toolchain": manifest.toolchain.model_copy(update={"image_digest": "f" * 64})}
)
with pytest.raises(ValueError):
policy.admit(changed, 1)
legacy_only = v2.AdmissionRegistry(schema_version=2, legacy=policy.legacy, multi_runtime=None)
with pytest.raises(ValueError):
legacy_only.admit(fixture("judge-manifest.valid-java21", v2.JobManifest), 1)
@@ -0,0 +1,399 @@
"""Actual finite Rust Coordinator/PG control acceptance; reports are synthetic, no execution."""
import asyncio
import copy
import hashlib
import json
from uuid import UUID, uuid4
import pytest
from test_coordinator import claim, fixture, manifest, notice, synthetic_report, worker_headers
from test_multi_runtime_versions import multi_import
from kf_oj.contracts import judging_v2 as v2
from kf_oj.contracts import judging_versions as versions
def report_v2(grant, task):
result = synthetic_report(grant, task)
result["schema_version"] = 2
language = v2.Language(task["language"])
kind = v2.runtime_contract(language)[3]
if kind == v2.ProgramKind.PYTHON:
files = [dict(path="main.py", artifact=task["source"])]
elif kind == v2.ProgramKind.JAVA:
files = [
dict(path=name, artifact=dict(digest="d" * 64, bytes=256))
for name in ("Helper.class", "Main$Inner.class", "Main.class")
]
else:
files = [dict(path="main", artifact=dict(digest="d" * 64, bytes=4096))]
result["compilation"]["program"] = dict(
kind=kind.value,
binding=grant["binding"],
source=task["source"],
toolchain=task["toolchain"],
files=files,
)
return result
def stage_v2(grant, language, sequence, stage="compiling", index=None):
return dict(
schema_version=2,
kind="judge.stage",
binding=grant["binding"],
sequence=sequence,
stage=stage,
case_index=index,
preparation_kind=v2.runtime_contract(v2.Language(language))[4].value,
)
def test_actual_versioned_claim_canonical_snapshot_stage_and_report(
postgres_url, tmp_path, monkeypatch
):
async def run():
async with fixture(postgres_url, tmp_path, monkeypatch, import_builder=multi_import) as (
client,
c,
workers,
_,
create,
):
for language in v2.Language:
created = await create(language.value)
grant = await claim(client, created, workers[0])
assert grant["schema_version"] == 2
task = await manifest(c, grant)
assert task["schema_version"] == 2 and task["language"] == language.value
parsed = versions.decode_manifest(json.dumps(task).encode())
canonical = versions.manifest_bytes(parsed)
assert hashlib.sha256(canonical).hexdigest() == grant["manifest_digest"]
async with client.fixture_app.state.sessions() as s:
version = await client.fixture_app.state.problems.for_submission(
s, created.problem_version_id
)
expected = version.manifest(
created.evaluation_id, created.submission_id, language, parsed.source
)
assert versions.manifest_bytes(expected) == canonical
headers = worker_headers(workers[0], grant)
bad = stage_v2(grant, language, 1)
bad["preparation_kind"] = (
"compile" if language == v2.Language.PYTHON313 else "syntax_check"
)
assert (
await client.post("/internal/stage", json=bad, headers=headers)
).status_code == 422
old = stage_v2(grant, language, 1)
old.update(schema_version=1)
old.pop("preparation_kind")
assert (
await client.post("/internal/stage", json=old, headers=headers)
).status_code == 422
for sequence, phase, index in (
(1, "compiling", None),
(2, "running", 0),
(3, "running", 1),
):
stage = stage_v2(grant, language, sequence, phase, index)
assert (
await client.post("/internal/stage", json=stage, headers=headers)
).status_code == 204
assert (
await client.post("/internal/stage", json=stage, headers=headers)
).status_code == 204
result = report_v2(grant, task)
for invalid in (
result | {"manifest_digest": "f" * 64},
result | {"binding": grant["binding"] | {"fencing_version": 2}},
):
assert (
await client.post("/internal/report", json=invalid, headers=headers)
).status_code in (409, 422)
changed = copy.deepcopy(result)
changed["compilation"]["program"]["toolchain"]["image_digest"] = "e" * 64
assert (
await client.post("/internal/report", json=changed, headers=headers)
).status_code == 422
assert (
await client.post("/internal/report", json=result, headers=headers)
).status_code == 204
assert (
await client.post("/internal/report", json=result, headers=headers)
).status_code == 204
final = await c.fetchval(
"SELECT final_report FROM evaluations WHERE id=$1", created.evaluation_id
)
assert json.loads(final) == result
assert (
await c.fetchval(
"SELECT state FROM evaluations WHERE id=$1", created.evaluation_id
)
== "finished"
)
assert (
await c.fetchval(
"SELECT count(*) FROM case_results WHERE evaluation_id=$1",
created.evaluation_id,
)
== 2
)
# No real program/Worker/Root executed; accepted is a synthetic protocol value only.
asyncio.run(run())
@pytest.mark.parametrize("kind", ["legacy", "identity", "budget"])
def test_actual_claim_refuses_missing_protocol_or_changed_capability_without_attempt(
postgres_url,
tmp_path,
monkeypatch,
kind,
):
from pathlib import Path
from kf_oj.contracts.judging import decode_wire
fixtures = Path(__file__).resolve().parents[2] / "tests/fixtures/contracts-v2"
registry = decode_wire(
v2.AdmissionRegistry, (fixtures / "judge-registry.valid.json").read_bytes()
)
raw = registry.model_dump(mode="json")
if kind == "identity":
raw["multi_runtime"]["capabilities"][2]["toolchain"]["image_digest"] = "e" * 64
elif kind == "budget":
raw["multi_runtime"]["capabilities"][2]["run_ceiling"]["memory_bytes"] = 32 * 1024 * 1024
policy = registry.legacy if kind == "legacy" else v2.AdmissionRegistry.model_validate(raw)
async def run():
async with fixture(
postgres_url,
tmp_path,
monkeypatch,
import_builder=multi_import,
node_policy=policy,
) as (client, c, workers, _, create):
created = await create("python313")
response = await client.post(
"/internal/claim",
json=notice(created.evaluation_id),
headers=worker_headers(workers[0]),
)
assert response.status_code == 422
assert await c.fetchval("SELECT count(*) FROM judge_attempts") == 0
assert await c.fetchval("SELECT state FROM evaluations") == "queued"
assert (await client.get("/health/ready")).status_code == 200
asyncio.run(run())
def test_actual_v2_cancel_requires_matching_stop_protocol_and_fence(
postgres_url, tmp_path, monkeypatch
):
async def run():
async with fixture(postgres_url, tmp_path, monkeypatch, import_builder=multi_import) as (
client,
c,
workers,
platform,
create,
):
created = await create("java21")
grant = await claim(client, created, workers[0])
assert (
await client.post(
"/internal/cancel",
json=notice(created.evaluation_id),
headers={"Authorization": "Bearer " + platform},
)
).status_code == 204
pending = await client.get(
"/internal/pending-stops", headers=worker_headers(workers[0])
)
assert pending.status_code == 200
assert pending.json() == [
dict(schema_version=2, binding=grant["binding"], case_count=2)
]
stopped = dict(
schema_version=2,
kind="judge.stopped",
binding=grant["binding"],
cleanup_complete=True,
)
headers = worker_headers(workers[0])
assert (
await client.post(
"/internal/stopped", json=stopped | {"schema_version": 1}, headers=headers
)
).status_code == 422
bad = stopped | {"binding": grant["binding"] | {"attempt_id": str(uuid4())}}
assert (
await client.post("/internal/stopped", json=bad, headers=headers)
).status_code == 409
assert (
await client.post("/internal/stopped", json=stopped, headers=headers)
).status_code == 204
assert await c.fetchval("SELECT state FROM evaluations") == "cancelled"
assert (
await c.fetchval(
"SELECT cleanup_complete FROM judge_attempts WHERE id=$1",
UUID(grant["binding"]["attempt_id"]),
)
is True
)
# This proves stop-report authority only; real JVM cleanup remains VM acceptance.
asyncio.run(run())
@pytest.mark.parametrize("language", ["python313", "java21"])
def test_actual_v2_expiry_late_report_and_finite_infrastructure_retry(
postgres_url, tmp_path, monkeypatch, language
):
"""Real PG/HTTP authority; reports/stop assertions remain synthetic, no user execution."""
async def run():
async with fixture(postgres_url, tmp_path, monkeypatch, import_builder=multi_import) as (
client,
c,
workers,
_,
create,
):
worker = workers[0]
created = await create(language)
old = await claim(client, created, worker)
old_task = await manifest(c, old)
late = report_v2(old, old_task)
await c.execute(
"UPDATE judge_attempts SET expires_at=clock_timestamp()-interval '1 second'"
)
assert (
await client.post(
"/internal/renew", json=old["binding"], headers=worker_headers(worker, old)
)
).status_code == 409
for _ in range(60):
pending = (
await client.get("/internal/pending-stops", headers=worker_headers(worker))
).json()
if pending:
break
await asyncio.sleep(0.05)
assert pending == [dict(schema_version=2, binding=old["binding"], case_count=2)]
stopped = dict(
schema_version=2,
kind="judge.stopped",
binding=old["binding"],
cleanup_complete=True,
)
assert (
await client.post("/internal/stopped", json=stopped, headers=worker_headers(worker))
).status_code == 204
await c.execute(
"UPDATE evaluations SET not_before=clock_timestamp() WHERE id=$1",
created.evaluation_id,
)
fresh = await claim(client, created, worker)
assert fresh["attempt_number"] == 2 and fresh["binding"]["fencing_version"] == 2
assert (
await client.post(
"/internal/report", json=late, headers=worker_headers(worker, old)
)
).status_code == 409
assert (
await client.post(
"/internal/stage",
json=stage_v2(old, language, 1),
headers=worker_headers(worker, old),
)
).status_code == 409
for sequence, stage, index in [
(1, "compiling", None),
(2, "running", 0),
(3, "running", 1),
]:
assert (
await client.post(
"/internal/stage",
json=stage_v2(fresh, language, sequence, stage, index),
headers=worker_headers(worker, fresh),
)
).status_code == 204
result = report_v2(fresh, await manifest(c, fresh))
assert (
await client.post(
"/internal/report", json=result, headers=worker_headers(worker, fresh)
)
).status_code == 204
digest = await c.fetchval(
"SELECT final_report_digest FROM evaluations WHERE id=$1", created.evaluation_id
)
assert (
await client.post(
"/internal/report", json=late, headers=worker_headers(worker, old)
)
).status_code == 409
assert (
await c.fetchval(
"SELECT final_report_digest FROM evaluations WHERE id=$1", created.evaluation_id
)
== digest
)
assert (
await c.fetchval(
"SELECT count(*) FROM case_results WHERE evaluation_id=$1",
created.evaluation_id,
)
== 2
)
retry = await create(language)
for number in range(1, 4):
grant = await claim(client, retry, worker)
result = report_v2(grant, await manifest(c, grant)) | dict(
compilation=dict(
status="system_error",
program=None,
diagnostics="",
diagnostics_truncated=False,
cpu_time_us=None,
memory_peak_bytes=None,
),
cases=[],
verdict="system_error",
failure_code="artifact_unavailable",
run_cpu_time_us=None,
run_memory_peak_bytes=None,
)
assert grant["attempt_number"] == number
assert (
await client.post(
"/internal/report", json=result, headers=worker_headers(worker, grant)
)
).status_code == 204
assert await c.fetchval(
"SELECT state FROM evaluations WHERE id=$1", retry.evaluation_id
) == ("failed" if number == 3 else "queued")
if number < 3:
await c.execute(
"UPDATE evaluations SET not_before=clock_timestamp() WHERE id=$1",
retry.evaluation_id,
)
assert (
await client.post(
"/internal/claim",
json=notice(retry.evaluation_id),
headers=worker_headers(worker),
)
).status_code == 204
assert (
await c.fetchval(
"SELECT count(*) FROM judge_attempts WHERE evaluation_id=$1",
retry.evaluation_id,
)
== 3
)
asyncio.run(run())
@@ -0,0 +1,454 @@
"""Real temporary PostgreSQL data/authority acceptance; no submitted code is executed."""
import asyncio
import copy
import hashlib
import json
import secrets
import subprocess
from pathlib import Path
from uuid import uuid4
import asyncpg
import httpx
import pytest
from conftest import connect, migrate
from sqlalchemy import func, select, text
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from test_database_foundation import fixture_rows
from test_problems import import_fixture, platform_role
from test_worker_artifacts import install_attempt
from kf_oj.api.app import create_app
from kf_oj.contracts import judging as v1
from kf_oj.contracts import judging_v2 as v2
from kf_oj.contracts.judging_versions import manifest_bytes
from kf_oj.contracts.submissions import CreateSubmission
from kf_oj.core.settings import Settings
from kf_oj.infra.artifacts.repository import ArtifactRepository
from kf_oj.infra.artifacts.store import ArtifactStore
from kf_oj.infra.messaging.models import Outbox
from kf_oj.modules.problems.models import ProblemVersion
from kf_oj.modules.problems.service import ProblemService
from kf_oj.modules.submissions.models import Evaluation, Submission
from kf_oj.modules.submissions.public import IdempotencyConflict, SubmissionUnavailable
ROOT = Path(__file__).resolve().parents[2]
FIXTURES = ROOT / "tests/fixtures/contracts-v2"
def configurations():
return [
{
key: json.loads((FIXTURES / f"judge-manifest.valid-{language}.json").read_text())[key]
for key in ("language", "toolchain", "compile_limits", "run_limits", "job_wall_time_us")
}
for language in v2.Language
]
def multi_import(root, *, slug="multi-runtime"):
legacy, _ = import_fixture(root, slug=slug)
manifest = {
key: legacy[key]
for key in ("slug", "title", "statement_file", "comparison", "cases", "samples")
}
manifest.update(schema_version=2, language_configurations=configurations())
(root / "problem.json").write_text(json.dumps(manifest))
registry = v1.decode_wire(
v2.AdmissionRegistry, (FIXTURES / "judge-registry.valid.json").read_bytes()
)
return manifest, registry
def test_upgrade_preserves_published_rows_and_legacy_attempt_bytes(postgres_url):
migrate(postgres_url, "downgrade", "0001_judging_foundation")
async def seed():
c = await connect(postgres_url)
try:
ids = await fixture_rows(c)
manifest = v1.decode_wire(
v1.JobManifest,
(ROOT / "tests/fixtures/contracts/judge-manifest.valid.json").read_bytes(),
)
canonical = v1.manifest_bytes(manifest)
await c.execute("SET ROLE kf_oj_coordinator")
await c.execute(
"""INSERT INTO judge_attempts
(id,evaluation_id,worker_id,number,fencing_version,lease_token_hash,
expires_at,manifest,manifest_digest)
VALUES ($1,$2,$3,1,1,$4,now()+interval '30 seconds',$5,$6)""",
uuid4(),
ids[5],
uuid4(),
"d" * 64,
canonical.decode(),
hashlib.sha256(canonical).hexdigest(),
)
await c.execute("RESET ROLE")
return await snapshot(c)
finally:
await c.close()
async def snapshot(c):
version = await c.fetchval(
"SELECT (to_jsonb(p)-'judging_schema_version'-'language_configurations')::text "
"FROM problem_versions p"
)
submission = await c.fetchval("SELECT to_jsonb(s)::text FROM submissions s")
attempt = await c.fetchrow("SELECT manifest,manifest_digest FROM judge_attempts")
decoded = v1.JobManifest.model_validate_json(attempt["manifest"])
assert hashlib.sha256(v1.manifest_bytes(decoded)).hexdigest() == attempt["manifest_digest"]
return version, submission, dict(attempt)
before = asyncio.run(seed())
migrate(postgres_url, "upgrade", "head")
async def read():
c = await connect(postgres_url)
try:
assert await c.fetchval("SELECT judging_schema_version FROM problem_versions") == 1
assert await c.fetchval("SELECT language_configurations FROM problem_versions") is None
return await snapshot(c)
finally:
await c.close()
assert asyncio.run(read()) == before
migrate(postgres_url, "downgrade", "0001_judging_foundation")
migrate(postgres_url, "upgrade", "head")
assert asyncio.run(read()) == before
migrate(postgres_url, "check")
async def insert_version(c, problem, cfgs, *, languages=None, revision=1):
version = uuid4()
await c.execute(
"""INSERT INTO problem_versions
(id,problem_id,revision,title,statement,languages,judging_schema_version,
language_configurations,public_samples)
VALUES ($1,$2,$3,'Multi runtime','Public',$4,2,$5,'[]')""",
version,
problem,
revision,
json.dumps(languages if languages is not None else [c["language"] for c in cfgs]),
json.dumps(cfgs),
)
return version
def test_sql_configuration_constraints_authority_and_downgrade_refusal(postgres_url):
async def run():
c = await connect(postgres_url)
try:
await c.execute("SET ROLE kf_oj_platform")
problem = uuid4()
await c.execute("INSERT INTO problems(id,slug) VALUES($1,'multi-sql')", problem)
cfgs = configurations()
bad_sets = []
for field, value in (
("language", "shell"),
("toolchain", None),
("compile_limits", None),
("job_wall_time_us", True),
("job_wall_time_us", 1000),
):
bad = copy.deepcopy(cfgs)
bad[0][field] = value
bad_sets.append(bad)
for field, value in (
("image_digest", "invalid"),
("image_revision", True),
("runtime_version", "3.14.0"),
("image_kind", "linux_gnu_cxx_x86_64_v1"),
):
bad = copy.deepcopy(cfgs)
bad[2]["toolchain"][field] = value
bad_sets.append(bad)
for field, value in (
("processes", 8),
("memory_bytes", 64 * v1.MIB),
("cpu_time_us", "1"),
("stack_bytes", 1),
):
bad = copy.deepcopy(cfgs)
bad[3]["run_limits"][field] = value
bad_sets.append(bad)
bad = copy.deepcopy(cfgs)
bad[0]["argv"] = ["/bin/sh"]
bad_sets.append(bad)
bad = copy.deepcopy(cfgs)
bad[0]["toolchain"]["host_path"] = "/host/private"
bad_sets.append(bad)
bad_sets.extend([[], None, [cfgs[0], cfgs[0]]])
for bad in bad_sets:
with pytest.raises(asyncpg.CheckViolationError):
await insert_version(c, problem, bad, languages=[c["language"] for c in cfgs])
with pytest.raises(asyncpg.CheckViolationError):
await insert_version(
c, problem, cfgs, languages=["java21", "python313", "cpp20", "c17"]
)
version = await insert_version(c, problem, cfgs)
with pytest.raises(asyncpg.InsufficientPrivilegeError):
await c.execute("UPDATE problem_versions SET language_configurations='[]'")
await c.execute("RESET ROLE")
with pytest.raises(asyncpg.CheckViolationError):
await c.execute("UPDATE problem_versions SET toolchain='{}'")
await c.execute("SET ROLE kf_oj_coordinator")
with pytest.raises(asyncpg.InsufficientPrivilegeError):
await c.execute("INSERT INTO problems(id,slug) VALUES($1,'forbidden')", uuid4())
with pytest.raises(asyncpg.InsufficientPrivilegeError):
await c.execute("UPDATE problem_versions SET language_configurations='[]'")
return version
finally:
await c.close()
version = asyncio.run(run())
with pytest.raises(subprocess.CalledProcessError) as rejected:
migrate(postgres_url, "downgrade", "0001_judging_foundation")
assert b"cannot downgrade while multi-runtime data exists" in rejected.value.stderr
async def preserved():
c = await connect(postgres_url)
try:
assert (
await c.fetchval("SELECT count(*) FROM problem_versions WHERE id=$1", version) == 1
)
assert (
await c.fetchval("SELECT version_num FROM alembic_version") == "0002_multi_runtime"
)
finally:
await c.close()
asyncio.run(preserved())
def test_real_four_language_import_public_projection_and_initial_submissions(
postgres_url, tmp_path, monkeypatch
):
_, registry = multi_import(tmp_path / "multi")
import_fixture(tmp_path / "legacy", slug="legacy-runtime")
async def run():
engine = create_async_engine(postgres_url)
platform_role(engine)
mod = __import__("kf_oj.api.app", fromlist=["build_engine"])
monkeypatch.setattr(mod, "build_engine", lambda _: engine)
app = create_app(
Settings(
_env_file=None,
environment="test",
database_url=postgres_url,
artifact_root=tmp_path / "artifacts",
judge_admission=registry,
)
)
async with app.router.lifespan_context(app):
owner = await app.state.identity.create_account(
"multi_owner", secrets.token_urlsafe(24)
)
version = await app.state.problems.import_directory(tmp_path / "multi")
old = await app.state.problems.import_directory(tmp_path / "legacy")
async with httpx.AsyncClient(
transport=httpx.ASGITransport(app=app), base_url="http://test"
) as client:
public = await client.get(f"/api/v1/problems/versions/{version}")
assert public.status_code == 200
data = public.json()
assert data["run_limits"] is None
assert data["languages"] == [v.value for v in v2.Language]
assert [c["source_name"] for c in data["language_configurations"]] == [
"main.c",
"main.cpp",
"main.py",
"Main.java",
]
assert data["language_configurations"][2]["preparation_kind"] == "syntax_check"
assert data["language_configurations"][3]["entrypoint"] == "Main"
for private in (
"image_digest",
"image_kind",
"hidden-input",
"hidden-expected",
str(tmp_path),
):
assert private not in public.text
for language in v2.Language:
request = CreateSubmission(
problem_version_id=version,
language=language,
idempotency_key=uuid4(),
source="Synthetic transport bytes; never executed",
)
results = await asyncio.gather(
*(app.state.submissions.create(owner, request) for _ in range(2))
)
assert results[0].submission_id == results[1].submission_id
async with app.state.sessions() as s:
projected = await app.state.problems.for_submission(s, version)
ref = app.state.problems.artifacts.store.reference(request.source.encode())
manifest = projected.manifest(
results[0].evaluation_id, results[0].submission_id, language, ref
)
assert manifest.schema_version == 2 and manifest.toolchain.runtime_version
v2.admit_manifest(registry, manifest, 1)
assert manifest_bytes(manifest)
with pytest.raises(IdempotencyConflict):
await app.state.submissions.create(
owner, request.model_copy(update={"source": "Different"})
)
with pytest.raises(SubmissionUnavailable):
await app.state.submissions.create(
owner,
CreateSubmission(
problem_version_id=old,
language="python313",
idempotency_key=uuid4(),
source="x",
),
)
async with app.state.sessions() as s:
for model in (Submission, Evaluation, Outbox):
assert await s.scalar(select(func.count()).select_from(model)) == 4
row = await s.get(ProblemVersion, version)
assert row.toolchain is None and row.run_limits is None
assert len(row.language_configurations) == 4
# Actual SQL membership guard also rejects new-language rows on old versions.
admin = await connect(postgres_url)
try:
source_digest = await admin.fetchval(
"SELECT source_digest FROM submissions LIMIT 1"
)
problem_id = await admin.fetchval(
"SELECT problem_id FROM problem_versions WHERE id=$1", old
)
await admin.execute("SET ROLE kf_oj_platform")
with pytest.raises(asyncpg.CheckViolationError):
await admin.execute(
"""INSERT INTO submissions
(id,account_id,problem_id,problem_version_id,language,source_digest,
idempotency_key,request_digest)
VALUES($1,$2,$3,$4,'java21',$5,$6,$7)""",
uuid4(),
owner.account_id,
problem_id,
old,
source_digest,
uuid4(),
"a" * 64,
)
await admin.execute("RESET ROLE")
with pytest.raises(asyncpg.CheckViolationError):
await admin.execute(
"UPDATE problem_versions SET language_configurations='[]' WHERE id=$1",
version,
)
finally:
await admin.close()
asyncio.run(run())
def test_import_refuses_missing_protocol_and_runtime_identity_without_writes(
postgres_url, tmp_path
):
manifest, registry = multi_import(tmp_path / "multi")
store = ArtifactStore(tmp_path / "artifacts")
async def run():
engine = create_async_engine(postgres_url)
platform_role(engine)
sessions = async_sessionmaker(engine, expire_on_commit=False)
try:
service = ProblemService(sessions, ArtifactRepository(store), registry.legacy)
with pytest.raises(ValueError, match="protocol"):
await service.import_directory(tmp_path / "multi")
manifest["language_configurations"][2]["toolchain"]["image_digest"] = "f" * 64
(tmp_path / "multi/problem.json").write_text(json.dumps(manifest))
service = ProblemService(sessions, ArtifactRepository(store), registry)
with pytest.raises(ValueError, match="capability"):
await service.import_directory(tmp_path / "multi")
async with sessions() as s:
assert await s.scalar(select(func.count()).select_from(ProblemVersion)) == 0
finally:
await engine.dispose()
try:
asyncio.run(run())
finally:
store.close()
def test_v2_worker_reads_immutable_language_snapshot_with_live_lease(postgres_url, tmp_path):
_, registry = multi_import(tmp_path / "multi")
async def run():
admin = create_async_engine(postgres_url)
sessions = async_sessionmaker(admin, expire_on_commit=False)
app = create_app(
Settings(
_env_file=None,
environment="test",
database_url=postgres_url,
artifact_root=tmp_path / "artifacts",
judge_admission=registry,
)
)
try:
async with app.router.lifespan_context(app):
platform_role(app.state.sessions.kw["bind"])
account = await app.state.identity.create_account(
"worker_owner", secrets.token_urlsafe(24)
)
version = await app.state.problems.import_directory(tmp_path / "multi")
for language in ("python313", "java21"):
binding, token, manifest = await install_attempt(
sessions, app, account, version, language=language
)
# Controlled DB lease fixture, not an actual Coordinator claim or execution.
async with httpx.AsyncClient(
transport=httpx.ASGITransport(app=app), base_url="http://test"
) as client:
path = (
f"/api/v1/internal/evaluations/{binding.evaluation_id}"
f"/attempts/{binding.attempt_id}"
)
headers = {
"Authorization": "Bearer " + token,
"X-Worker-ID": str(binding.worker_id),
"X-Fencing-Version": "1",
}
response = await client.get(path + "/manifest", headers=headers)
assert response.status_code == 200 and response.content == manifest_bytes(
manifest
)
assert manifest.schema_version == 2 and manifest.language == language
response = await client.get(
path + "/artifacts/" + manifest.source.digest, headers=headers
)
assert (
response.status_code == 200 and response.content == b"synthetic source"
)
assert (
await client.get(
path + "/manifest", headers=headers | {"X-Worker-ID": str(uuid4())}
)
).status_code == 404
assert (
await client.get(
path + "/manifest", headers=headers | {"X-Fencing-Version": "2"}
)
).status_code == 404
async with sessions.begin() as s:
await s.execute(text("SET LOCAL ROLE kf_oj_coordinator"))
await s.execute(
text("UPDATE evaluations SET state='cancelling' WHERE id=:id"),
{"id": binding.evaluation_id},
)
assert (
await client.get(path + "/manifest", headers=headers)
).status_code == 404
finally:
await admin.dispose()
asyncio.run(run())
+247
View File
@@ -0,0 +1,247 @@
"""Real private PostgreSQL/RabbitMQ control plane, no user program execution."""
import asyncio
import json
import os
import secrets
import subprocess
import sys
from pathlib import Path
from urllib.parse import urlsplit
from uuid import uuid4
import aio_pika
import pytest
from sqlalchemy import text
from sqlalchemy.ext.asyncio import create_async_engine
from test_problems import import_fixture, platform_role
from kf_oj.api.app import create_app
from kf_oj.contracts.submissions import CreateSubmission
from kf_oj.core.settings import Settings
from kf_oj.infra.messaging.publisher import QUEUE, ConfirmedBroker, OutboxRepository
def test_actual_confirm_duplicate_lock_recovery_and_broker_failure(
postgres_url, tmp_path, monkeypatch
):
asyncio.run(outbox_acceptance(postgres_url, tmp_path, monkeypatch))
async def outbox_acceptance(postgres_url, tmp_path, monkeypatch):
config = os.environ.get("KF_OJ_TEST_AMQP_CONFIG")
if config is None:
pytest.skip("private RabbitMQ control acceptance requires KF_OJ_TEST_AMQP_CONFIG")
url = json.loads(Path(config).read_text())["amqp_url"]
parsed = urlsplit(url)
assert parsed.hostname == "127.0.0.1" and parsed.port == 26572
_, policy = import_fixture(tmp_path / "import")
engine = create_async_engine(postgres_url)
platform_role(engine)
monkeypatch.setattr("kf_oj.api.app.build_engine", lambda _: engine)
app = create_app(
Settings(
_env_file=None,
environment="test",
database_url=postgres_url,
artifact_root=tmp_path / "artifacts",
judge_admission=policy,
)
)
broker = ConfirmedBroker(url)
connection = await aio_pika.connect(url, timeout=5)
try:
async with app.router.lifespan_context(app):
owner = await app.state.identity.create_account(
"outbox_test", secrets.token_urlsafe(24)
)
version = await app.state.problems.import_directory(tmp_path / "import")
created = await app.state.submissions.create(
owner,
CreateSubmission(
problem_version_id=version,
language="c17",
idempotency_key=uuid4(),
source="synthetic control event; not compiled",
),
)
repo = OutboxRepository(app.state.sessions)
claimed = await asyncio.gather(repo.claim(), repo.claim())
assert sum(v is not None for v in claimed) == 1
item = next(v for v in claimed if v is not None)
assert item[1] == created.evaluation_id
# Kill a separate real publisher process after broker confirm, before DB mark.
code = """import asyncio,json,sys,signal
from uuid import UUID
from pathlib import Path
from kf_oj.infra.messaging.publisher import ConfirmedBroker
async def main():
item=tuple(UUID(x) for x in json.loads(sys.stdin.readline()))
broker=ConfirmedBroker(json.loads(Path(sys.argv[1]).read_text())["amqp_url"])
await broker.publish(item)
print("CONFIRMED",flush=True)
await asyncio.Event().wait()
asyncio.run(main())
"""
child = subprocess.Popen(
[sys.executable, "-c", code, config],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
try:
child.stdin.write(json.dumps([str(x) for x in item]) + "\n")
child.stdin.flush()
assert (
await asyncio.wait_for(asyncio.to_thread(child.stdout.readline), 10)
== "CONFIRMED\n"
)
finally:
child.kill()
await asyncio.to_thread(child.wait, 5)
child.stdin.close()
child.stdout.close()
child.stderr.close()
assert child.returncode == -9
channel = await connection.channel()
queue = await channel.get_queue(QUEUE)
first = await queue.get(timeout=5)
assert json.loads(first.body) == {
"schema_version": 1,
"kind": "judge.evaluation",
"evaluation_id": str(created.evaluation_id),
}
assert len(first.body) <= 512 and first.delivery_mode == 2
# A publisher lost after confirm cannot mark DB; actual DB clock lock expires.
assert await repo.claim() is None
await asyncio.sleep(15.1)
retry = await repo.claim()
assert retry[:2] == item[:2] and retry[2] != item[2]
assert not await repo.confirmed(item)
await broker.publish(retry)
assert await repo.confirmed(retry)
second = await queue.get(timeout=5)
assert second.body == first.body and second.message_id == first.message_id
await first.ack()
await second.ack()
assert await repo.claim() is None
async with app.state.sessions() as s:
assert (
await s.scalar(
text("SELECT count(*) FROM outbox WHERE delivered_at IS NOT NULL")
)
== 1
)
# A second transaction succeeds even when a separate broker is unavailable.
await app.state.submissions.create(
owner,
CreateSubmission(
problem_version_id=version,
language="c17",
idempotency_key=uuid4(),
source="broker failure synthetic event",
),
)
failed = await repo.claim()
unavailable = ConfirmedBroker(url.replace(":26572/", ":26579/"))
try:
with pytest.raises((ConnectionError, OSError)):
await unavailable.publish(failed)
finally:
await unavailable.close()
await repo.failed(failed)
assert await repo.claim() is None
async with app.state.sessions() as s:
row = (
await s.execute(
text("""SELECT failures,last_error_code,
next_attempt_at>clock_timestamp() deferred,delivered_at IS NULL pending,
lock_owner IS NULL unlocked FROM outbox WHERE id=:id"""),
{"id": failed[0]},
)
).one()
assert tuple(row) == (1, "broker_unavailable", True, True, True)
# Preserve the broker-failure row for inspection while testing another crash window.
async with app.state.sessions.begin() as s:
await s.execute(
text(
"UPDATE outbox SET next_attempt_at=clock_timestamp()+interval '1 hour' "
"WHERE id=:id"
),
{"id": failed[0]},
)
before = await app.state.submissions.create(
owner,
CreateSubmission(
problem_version_id=version,
language="c17",
idempotency_key=uuid4(),
source="before-confirm synthetic event",
),
)
pending = await repo.claim()
assert pending[1] == before.evaluation_id
preconfirm = code.replace("await broker.publish(item)", "await broker.open()")
child = subprocess.Popen(
[sys.executable, "-c", preconfirm, config],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
try:
child.stdin.write(json.dumps([str(x) for x in pending]) + "\n")
child.stdin.flush()
assert (
await asyncio.wait_for(asyncio.to_thread(child.stdout.readline), 10)
== "CONFIRMED\n"
)
finally:
child.kill()
await asyncio.to_thread(child.wait, 5)
child.stdin.close()
child.stdout.close()
child.stderr.close()
assert child.returncode == -9
assert await queue.get(fail=False, timeout=5) is None
await asyncio.sleep(15.1)
retry = await repo.claim()
assert retry[:2] == pending[:2]
await broker.publish(retry)
assert await repo.confirmed(retry)
delivered = await queue.get(timeout=5)
assert json.loads(delivered.body)["evaluation_id"] == str(before.evaluation_id)
await delivered.ack()
finally:
await broker.close()
await connection.close()
def test_actual_transport_failure_does_not_log_connection_credentials(caplog):
"""Local protocol refusal exercises descendant client loggers without a broker/DB."""
async def refused_connection():
touched = asyncio.Event()
async def refuse(reader, writer):
touched.set()
writer.close()
await writer.wait_closed()
listener = await asyncio.start_server(refuse, "127.0.0.1", 0)
port = listener.sockets[0].getsockname()[1]
broker = ConfirmedBroker(f"amqp://fixture:private-sentinel@127.0.0.1:{port}/")
try:
with pytest.raises(aio_pika.exceptions.AMQPConnectionError):
await broker.publish((uuid4(), uuid4(), uuid4()))
assert touched.is_set(), "must reach actual local transport refusal"
finally:
await broker.close()
listener.close()
await listener.wait_closed()
asyncio.run(refused_connection())
assert "private-sentinel" not in caplog.text
assert "amqp://" not in caplog.text
+262
View File
@@ -0,0 +1,262 @@
import asyncio
import base64
import json
import os
import subprocess
import sys
from pathlib import Path
from uuid import uuid4
import httpx
import pytest
from sqlalchemy import event, func, select, update
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from kf_oj.api.app import create_app
from kf_oj.contracts.judging import AdmissionPolicy
from kf_oj.core.settings import Settings
from kf_oj.infra.artifacts.repository import ArtifactRepository
from kf_oj.infra.artifacts.store import ArtifactError, ArtifactStore
from kf_oj.modules.problems.models import Problem, ProblemVersion
from kf_oj.modules.problems.service import ProblemService
ROOT = Path(__file__).resolve().parents[2]
def import_fixture(root, *, slug="fixture-problem"):
root.mkdir(mode=0o700)
policy = AdmissionPolicy.model_validate_json(
(ROOT / "tests/fixtures/contracts/judge-admission.valid.json").read_bytes()
)
manifest = {
"schema_version": 1,
"slug": slug,
"title": "Synthetic problem",
"statement_file": "statement.md",
"languages": ["c17", "cpp20"],
"toolchain": policy.toolchain.model_dump(),
"comparison": "exact_bytes",
"compile_limits": policy.compile_ceiling.model_dump(),
"run_limits": policy.run_ceiling.model_dump(),
"job_wall_time_us": 60000000,
"cases": [
{"key": "empty", "input_file": "input-1", "expected_file": "expected-1"},
{"key": "binary", "input_file": "input-2", "expected_file": "expected-2"},
],
"samples": [{"input_file": "sample-input", "output_file": "sample-output"}],
}
for filename, data in {
"statement.md": b"Public statement only",
"input-1": b"",
"expected-1": b"hidden-expected\x00\xff",
"input-2": b"hidden-input\x00\xfe",
"expected-2": b"hidden-expected-2\xff",
"sample-input": b"1\n",
"sample-output": b"2\n",
}.items():
(root / filename).write_bytes(data)
(root / "problem.json").write_text(json.dumps(manifest))
return manifest, policy
def platform_role(engine):
@event.listens_for(engine.sync_engine, "connect")
def apply_role(connection, _):
cursor = connection.cursor()
cursor.execute("SET ROLE kf_oj_platform")
cursor.close()
def test_controlled_import_cli_and_unpublished_version_visibility(postgres_url, tmp_path):
root = tmp_path / "import"
_, policy = import_fixture(root)
env = os.environ | {
"KF_OJ_DATABASE_URL": postgres_url,
"KF_OJ_ARTIFACT_ROOT": str(tmp_path / "artifacts"),
"KF_OJ_JUDGE_ADMISSION": policy.model_dump_json(),
"KF_OJ_ENVIRONMENT": "test",
}
result = subprocess.run(
[sys.executable, "-m", "kf_oj.modules.problems.cli", str(root)],
env=env,
capture_output=True,
text=True,
timeout=15,
)
assert result.returncode == 0 and "Problem version published:" in result.stdout
assert "hidden-expected" not in result.stdout + result.stderr
async def run():
engine = create_async_engine(postgres_url)
platform_role(engine)
sessions = async_sessionmaker(engine, expire_on_commit=False)
try:
service = ProblemService(sessions)
page = await service.list()
assert page.items[0].revision == 1
version_id = page.items[0].version_id
draft_id = uuid4()
async with sessions.begin() as session:
published = await session.get(ProblemVersion, version_id)
session.add(
ProblemVersion(
id=draft_id,
problem_id=published.problem_id,
revision=2,
title=published.title,
statement=published.statement,
languages=published.languages,
toolchain=published.toolchain,
compile_limits=published.compile_limits,
run_limits=published.run_limits,
job_wall_time_us=published.job_wall_time_us,
public_samples=[],
)
)
from kf_oj.modules.problems.public import ProblemUnavailable
with pytest.raises(ProblemUnavailable):
await service.detail(draft_id)
assert (await service.list()).items[0].version_id == version_id
finally:
await engine.dispose()
asyncio.run(run())
def test_actual_import_two_immutable_versions_and_public_api(postgres_url, tmp_path):
root = tmp_path / "import"
manifest, policy = import_fixture(root)
async def run():
app = create_app(
Settings(
_env_file=None,
environment="test",
database_url=postgres_url,
artifact_root=tmp_path / "artifacts",
judge_admission=policy,
)
)
async with app.router.lifespan_context(app):
platform_role(app.state.sessions.kw["bind"])
async with httpx.AsyncClient(
transport=httpx.ASGITransport(app=app), base_url="http://test"
) as client:
assert (await client.get("/api/v1/problems")).json() == {
"items": [],
"next_after": None,
}
first = await app.state.problems.import_directory(root)
async with app.state.sessions() as session:
projected = await app.state.problems.for_submission(session, first)
assert len(projected.cases) == 2 and projected.cases[0].input.bytes == 0
binary = await app.state.problems.artifacts.read(
session, projected.cases[0].expected
)
assert binary == b"hidden-expected\x00\xff"
manifest["title"] = "Second immutable version"
(root / "statement.md").write_text("New public statement")
(root / "problem.json").write_text(json.dumps(manifest))
second = await app.state.problems.import_directory(root)
old = await client.get(f"/api/v1/problems/versions/{first}")
new = await client.get(f"/api/v1/problems/versions/{second}")
assert old.json()["revision"] == 1 and new.json()["revision"] == 2
assert old.json()["statement"] == "Public statement only"
assert new.json()["statement"] == "New public statement"
assert new.json()["samples"][0] == {
"encoding": "base64",
"input": base64.b64encode(b"1\n").decode(),
"output": base64.b64encode(b"2\n").decode(),
}
page = (await client.get("/api/v1/problems")).json()
assert page["items"][0]["version_id"] == str(second)
assert (await client.get("/api/v1/problems?limit=0")).status_code == 422
assert (await client.get(f"/api/v1/problems/versions/{uuid4()}")).status_code == 404
public_text = old.text + new.text + json.dumps(page)
assert "hidden-expected" not in public_text and "hidden-input" not in public_text
assert str(root) not in public_text and "digest" not in public_text
async with app.state.sessions.begin() as session:
await session.execute(update(Problem).values(enabled=False))
assert (await client.get(f"/api/v1/problems/versions/{first}")).status_code == 404
assert (await client.get("/api/v1/problems")).json()["items"] == []
asyncio.run(run())
@pytest.mark.parametrize("invalid", ["missing", "duplicate", "deadline", "node_budget"])
def test_import_invalid_package_refused_without_published_rows(postgres_url, tmp_path, invalid):
root = tmp_path / "import"
manifest, policy = import_fixture(root)
if invalid == "missing":
(root / "expected-1").unlink()
elif invalid == "duplicate":
manifest["cases"].append(manifest["cases"][0])
elif invalid == "deadline":
manifest["job_wall_time_us"] = 1000
else:
manifest["run_limits"]["cpu_time_us"] = policy.run_ceiling.cpu_time_us + 1
(root / "problem.json").write_text(json.dumps(manifest))
store = ArtifactStore(tmp_path / "artifacts")
async def run():
engine = create_async_engine(postgres_url)
platform_role(engine)
sessions = async_sessionmaker(engine, expire_on_commit=False)
try:
service = ProblemService(sessions, ArtifactRepository(store), policy)
with pytest.raises(ValueError):
await service.import_directory(root)
async with sessions() as session:
assert await session.scalar(select(func.count()).select_from(ProblemVersion)) == 0
finally:
await engine.dispose()
try:
asyncio.run(run())
finally:
store.close()
def test_concurrent_import_revisions_pagination_and_storage_failure_rollback(
postgres_url, tmp_path
):
root = tmp_path / "import"
_, policy = import_fixture(root)
store = ArtifactStore(tmp_path / "artifacts")
async def run():
engine = create_async_engine(postgres_url)
platform_role(engine)
sessions = async_sessionmaker(engine, expire_on_commit=False)
try:
service = ProblemService(sessions, ArtifactRepository(store), policy)
versions = await asyncio.gather(
service.import_directory(root), service.import_directory(root)
)
assert versions[0] != versions[1]
assert sorted([(await service.detail(v)).revision for v in versions]) == [1, 2]
other = tmp_path / "other"
import_fixture(other, slug="another-problem")
await service.import_directory(other)
page = await service.list(limit=1)
assert page.next_after == "another-problem" and len(page.items) == 1
next_page = await service.list(after=page.next_after, limit=1)
assert next_page.next_after is None and next_page.items[0].slug == "fixture-problem"
limited = ArtifactStore(tmp_path / "limited", max_bytes=1, max_files=5)
try:
with pytest.raises(ArtifactError):
await ProblemService(
sessions, ArtifactRepository(limited), policy
).import_directory(root)
finally:
limited.close()
async with sessions() as session:
assert await session.scalar(select(func.count()).select_from(ProblemVersion)) == 3
finally:
await engine.dispose()
try:
asyncio.run(run())
finally:
store.close()
+222
View File
@@ -0,0 +1,222 @@
import asyncio
import json
import secrets
from uuid import uuid4
import httpx
import pytest
from sqlalchemy import func, select, text
from sqlalchemy.exc import DBAPIError
from sqlalchemy.ext.asyncio import create_async_engine
from test_problems import import_fixture, platform_role
from kf_oj.api.app import create_app
from kf_oj.contracts.judging import MAX_SOURCE_BYTES, Language
from kf_oj.contracts.submissions import CreateSubmission
from kf_oj.core.settings import Settings
from kf_oj.infra.artifacts.models import Artifact
from kf_oj.infra.messaging.models import Outbox
from kf_oj.modules.submissions.models import Evaluation, Submission
from kf_oj.modules.submissions.public import IdempotencyConflict, SubmissionUnavailable
def test_submission_input_utf8_and_unknown_fields():
base = dict(problem_version_id=str(uuid4()), language="c17", idempotency_key=str(uuid4()))
CreateSubmission(**base, source="é" * (MAX_SOURCE_BYTES // 2))
for extra in [
dict(source="é" * (MAX_SOURCE_BYTES // 2 + 1)),
dict(source="x", argv=[]),
dict(source="x", user_id=str(uuid4())),
dict(source=""),
dict(source="\ud800"),
]:
with pytest.raises(ValueError):
CreateSubmission(**base, **extra)
async def setup_app(postgres_url, tmp_path):
_, policy = import_fixture(tmp_path / "import")
app = create_app(
Settings(
_env_file=None,
environment="test",
database_url=postgres_url,
artifact_root=tmp_path / "artifacts",
judge_admission=policy,
)
)
engine = create_async_engine(postgres_url)
platform_role(engine)
# Test creates separate permission-constrained platform connections used by application.
app_engine_build = __import__("kf_oj.api.app", fromlist=["build_engine"])
return app, engine, app_engine_build
def test_actual_concurrent_idempotency_version_and_outbox(postgres_url, tmp_path, monkeypatch):
async def run():
app, engine, mod = await setup_app(postgres_url, tmp_path)
monkeypatch.setattr(mod, "build_engine", lambda _: engine)
async with app.router.lifespan_context(app):
account = await app.state.identity.create_account(
"sub_fixture", secrets.token_urlsafe(24)
)
version = await app.state.problems.import_directory(tmp_path / "import")
request = CreateSubmission(
problem_version_id=version,
language="c17",
idempotency_key=uuid4(),
source="synthetic source, never executed",
)
results = await asyncio.gather(
*(app.state.submissions.create(account, request) for _ in range(8))
)
assert len({r.submission_id for r in results}) == 1
async with app.state.sessions() as s:
for model in [Submission, Evaluation, Outbox]:
assert await s.scalar(select(func.count()).select_from(model)) == 1
stored = await s.get(Submission, results[0].submission_id)
data = await app.state.problems.artifacts.read(
s, app.state.problems.artifacts.store.reference(request.source.encode())
)
assert data == request.source.encode() and stored.language == "c17"
with pytest.raises(IdempotencyConflict):
await app.state.submissions.create(
account, request.model_copy(update={"source": "different source"})
)
admin = create_async_engine(postgres_url)
async with admin.begin() as c:
await c.execute(text("SET LOCAL ROLE kf_oj_coordinator"))
await c.execute(
text("UPDATE evaluations SET state='finished',verdict='accepted' WHERE id=:id"),
{"id": results[0].evaluation_id},
)
await admin.dispose()
assert (await app.state.submissions.create(account, request)).state == "finished"
cpp = await app.state.submissions.create(
account,
request.model_copy(update={"language": Language.CPP20, "idempotency_key": uuid4()}),
)
assert cpp.submission_id != results[0].submission_id
async with app.state.sessions.begin() as s:
await s.execute(text("UPDATE problems SET enabled=false"))
assert (await app.state.submissions.create(account, request)).submission_id == results[
0
].submission_id
with pytest.raises(SubmissionUnavailable):
await app.state.submissions.create(
account, request.model_copy(update={"idempotency_key": uuid4()})
)
asyncio.run(run())
def test_actual_database_failure_rolls_back_entire_creation(postgres_url, tmp_path, monkeypatch):
async def run():
app, engine, mod = await setup_app(postgres_url, tmp_path)
monkeypatch.setattr(mod, "build_engine", lambda _: engine)
async with app.router.lifespan_context(app):
account = await app.state.identity.create_account(
"rollback_fixture", secrets.token_urlsafe(24)
)
version = await app.state.problems.import_directory(tmp_path / "import")
admin = create_async_engine(postgres_url)
async with admin.begin() as c:
await c.execute(
text(
"""CREATE FUNCTION test_reject_outbox() RETURNS trigger
LANGUAGE plpgsql AS $$
BEGIN RAISE EXCEPTION 'synthetic rollback'; END $$"""
)
)
await c.execute(
text(
"CREATE TRIGGER test_reject_outbox BEFORE INSERT ON outbox "
"FOR EACH ROW EXECUTE FUNCTION test_reject_outbox()"
)
)
request = CreateSubmission(
problem_version_id=version,
language="c17",
idempotency_key=uuid4(),
source="rollback synthetic bytes",
)
with pytest.raises(DBAPIError):
await app.state.submissions.create(account, request)
async with app.state.sessions() as s:
for model in [Submission, Evaluation, Outbox]:
assert await s.scalar(select(func.count()).select_from(model)) == 0
ref = app.state.problems.artifacts.store.reference(request.source.encode())
assert await s.get(Artifact, ref.digest) is None
# Bytes can survive rollback as a collectable orphan; no durable reference exists.
assert app.state.problems.artifacts.store.read(ref) == request.source.encode()
async with admin.begin() as c:
await c.execute(text("DROP TRIGGER test_reject_outbox ON outbox"))
await c.execute(text("DROP FUNCTION test_reject_outbox()"))
assert (await app.state.submissions.create(account, request)).state == "queued"
await admin.dispose()
asyncio.run(run())
def test_actual_authenticated_creation_request_boundaries(postgres_url, tmp_path, monkeypatch):
async def run():
app, engine, mod = await setup_app(postgres_url, tmp_path)
monkeypatch.setattr(mod, "build_engine", lambda _: engine)
async with app.router.lifespan_context(app):
password = secrets.token_urlsafe(24)
await app.state.identity.create_account("api_submission", password)
version = await app.state.problems.import_directory(tmp_path / "import")
async with httpx.AsyncClient(
transport=httpx.ASGITransport(app=app), base_url="http://localhost:8000"
) as client:
req = dict(
problem_version_id=str(version),
language="c17",
idempotency_key=str(uuid4()),
source="synthetic ordinary data",
)
assert (await client.post("/api/v1/submissions", json=req)).status_code == 403
login = await client.post(
"/api/v1/auth/login",
json=dict(username="api_submission", password=password),
headers={"Origin": "http://localhost:8000"},
)
assert login.status_code == 200
headers = {
"Origin": "http://localhost:8000",
"X-CSRF-Token": login.json()["csrf_token"],
}
assert (
await client.post("/api/v1/submissions", json=req, headers=headers)
).status_code == 201
assert (
await client.post(
"/api/v1/submissions", json=req | {"user_id": str(uuid4())}, headers=headers
)
).status_code == 422
assert (
await client.post(
"/api/v1/submissions", json=req | {"language": "python"}, headers=headers
)
).status_code == 422
assert (
await client.post(
"/api/v1/submissions", json=req | {"source": "changed"}, headers=headers
)
).status_code == 409
assert (
await client.post(
"/api/v1/submissions",
content=b"x" * (6 * MAX_SOURCE_BYTES + 8193),
headers=headers | {"Content-Type": "application/json"},
)
).status_code == 413
# JSON escaping can exceed old1MiB body ceiling while valid source remains256KiB.
req["idempotency_key"] = str(uuid4())
req["source"] = "\x01" * MAX_SOURCE_BYTES
assert len(json.dumps(req)) > 1024 * 1024
assert (
await client.post("/api/v1/submissions", json=req, headers=headers)
).status_code == 201
asyncio.run(run())
+163
View File
@@ -0,0 +1,163 @@
"""Real PostgreSQL owner projection/pagination acceptance; no program execution."""
import asyncio
import json
import secrets
from pathlib import Path
from uuid import uuid4
import httpx
import pytest
from sqlalchemy import text
from sqlalchemy.ext.asyncio import create_async_engine
from test_submission_create import setup_app
from kf_oj.contracts.submissions import CreateSubmission
from kf_oj.core.settings import Settings
from kf_oj.modules.submissions.queries import cursor_decode
def test_cursor_and_control_configuration_fail_closed():
for value in ["garbage", "_" * 129, "", "eHx4"]:
with pytest.raises(ValueError):
cursor_decode(value)
for url in ["http://localhost:8100", "https://127.0.0.1:8100", "http://127.0.0.1:8100/path"]:
with pytest.raises(ValueError):
Settings(_env_file=None, coordinator_url=url, coordinator_token="a" * 64)
Settings(_env_file=None, coordinator_url="http://127.0.0.1:8100", coordinator_token="a" * 64)
@pytest.mark.parametrize("wire_version", [1, 2])
def test_actual_owned_history_source_result_and_cancel(
postgres_url, tmp_path, monkeypatch, wire_version
):
async def run():
app, engine, mod = await setup_app(postgres_url, tmp_path)
monkeypatch.setattr(mod, "build_engine", lambda _: engine)
async with app.router.lifespan_context(app):
passwords = [secrets.token_urlsafe(24), secrets.token_urlsafe(24)]
owners = [
await app.state.identity.create_account(name, password)
for name, password in zip(["reader_one", "reader_two"], passwords, strict=True)
]
version = await app.state.problems.import_directory(tmp_path / "import")
records = []
for i in range(4):
records.append(
await app.state.submissions.create(
owners[0],
CreateSubmission(
problem_version_id=version,
language="c17",
idempotency_key=uuid4(),
source="synthetic private source " + str(i),
),
)
)
# Synthetic contract projection only, never claimed as a real judgement.
fixture = Path(__file__).parents[2] / (
"tests/fixtures/contracts/judge-result.valid.json"
if wire_version == 1
else "tests/fixtures/contracts-v2/judge-result.valid-c17.json"
)
report = json.loads(fixture.read_text())
report["binding"]["evaluation_id"] = str(records[0].evaluation_id)
if wire_version == 2:
report["compilation"]["program"]["binding"]["evaluation_id"] = str(
records[0].evaluation_id
)
admin = create_async_engine(postgres_url)
async with admin.begin() as conn:
await conn.execute(text("SET LOCAL ROLE kf_oj_coordinator"))
await conn.execute(
text(
"UPDATE evaluations SET state='finished',verdict='accepted',"
"final_report=CAST(:report AS jsonb) WHERE id=:id"
),
{"id": records[0].evaluation_id, "report": json.dumps(report)},
)
await admin.dispose()
class CancelProbe:
accepted = True
seen = []
async def cancel(self, evaluation_id):
self.seen.append(evaluation_id)
return self.accepted
probe = CancelProbe()
app.state.submission_queries.cancellation = probe
async with httpx.AsyncClient(
transport=httpx.ASGITransport(app=app), base_url="http://localhost:8000"
) as client:
first = await client.post(
"/api/v1/auth/login",
json={"username": "reader_one", "password": passwords[0]},
headers={"Origin": "http://localhost:8000"},
)
assert first.status_code == 200
csrf = {
"Origin": "http://localhost:8000",
"X-CSRF-Token": first.json()["csrf_token"],
}
page = await client.get("/api/v1/submissions?limit=2")
assert page.status_code == 200 and page.headers["Cache-Control"] == "no-store"
after = page.json()["next_after"]
# Concurrent insertion must not shift the cursor into repeats or omit old rows.
await app.state.submissions.create(
owners[0],
CreateSubmission(
problem_version_id=version,
language="cpp20",
idempotency_key=uuid4(),
source="new synthetic private source",
),
)
tail = await client.get("/api/v1/submissions", params={"limit": 2, "after": after})
seen = [i["submission_id"] for i in page.json()["items"] + tail.json()["items"]]
assert len(set(seen)) == 4 and set(seen) == {str(r.submission_id) for r in records}
assert tail.json()["next_after"] is None
base = f"/api/v1/submissions/{records[0].submission_id}"
public = await client.get(base)
assert public.status_code == 200 and public.json()["verdict"] == "accepted"
projection = public.json()["result"]
assert projection["cpu_time_us"] == report["run_cpu_time_us"]
assert projection["memory_peak_bytes"] == report["run_memory_peak_bytes"]
assert all(
key not in public.text
for key in ["case_id", "digest", "lease", "expected", "manifest", "program"]
)
source = await client.get(base + "/source")
assert source.json()["source"] == "synthetic private source 0"
waiting = await client.get(f"/api/v1/submissions/{records[1].submission_id}")
assert waiting.json()["result"] is None and waiting.json()["verdict"] is None
cancel = f"/api/v1/submissions/{records[1].submission_id}/cancel"
assert (await client.post(cancel)).status_code == 403
assert (await client.post(cancel, headers=csrf)).status_code == 202
assert probe.seen == [records[1].evaluation_id]
assert (await client.get(cancel.removesuffix("/cancel"))).json()[
"state"
] == "queued"
probe.accepted = False
assert (await client.post(cancel, headers=csrf)).status_code == 503
assert (await client.get("/api/v1/submissions?after=invalid")).status_code == 422
second = await client.post(
"/api/v1/auth/login",
json={"username": "reader_two", "password": passwords[1]},
headers={"Origin": "http://localhost:8000"},
)
csrf["X-CSRF-Token"] = second.json()["csrf_token"]
assert (await client.get("/api/v1/submissions")).json()["items"] == []
for suffix in ["", "/source"]:
response = await client.get(base + suffix)
assert (
response.status_code == 404
and response.headers["Cache-Control"] == "no-store"
)
assert (await client.post(cancel, headers=csrf)).status_code == 404
assert len(probe.seen) == 2
assert (await client.post("/api/v1/auth/logout", headers=csrf)).status_code == 204
assert (await client.get(base)).status_code == 401
asyncio.run(run())
+343
View File
@@ -0,0 +1,343 @@
import asyncio
import hashlib
import secrets
from datetime import timedelta
from pathlib import Path
from uuid import uuid4
import httpx
import pytest
from sqlalchemy import func, select, text, update
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from test_problems import import_fixture, platform_role
from kf_oj.api.app import create_app
from kf_oj.contracts.judging import (
JobManifest,
LeaseBinding,
manifest_bytes,
)
from kf_oj.core.settings import Settings
from kf_oj.modules.identity.service import token_digest
from kf_oj.modules.submissions.models import Evaluation, JudgeAttempt, Submission
from kf_oj.modules.submissions.public import SubmissionUnavailable
from kf_oj.modules.submissions.queries import SubmissionQueries
def test_canonical_manifest_snapshot():
root = Path(__file__).resolve().parents[2] / "tests/fixtures/contracts"
manifest = JobManifest.model_validate_json((root / "judge-manifest.valid.json").read_bytes())
assert manifest_bytes(manifest) == (root / "canonical-manifest.v1.txt").read_bytes()
async def install_attempt(
sessions, app, account, version_id, *, source=b"synthetic source", language="c17"
):
# Explicit controlled DB fixture, not a simulated Coordinator API or claimed execution.
# E implements actual transactional claim; here the real B01 schema protects associations.
submission_id, evaluation_id, attempt_id, worker_id = (uuid4() for _ in range(4))
lease_token = secrets.token_hex(32)
async with app.state.sessions.begin() as session:
source_ref = await app.state.problems.artifacts.put(session, source)
projected = await app.state.problems.for_submission(session, version_id)
session.add(
Submission(
id=submission_id,
account_id=account.account_id,
problem_id=projected.problem_id,
problem_version_id=version_id,
language=language,
source_digest=source_ref.digest,
idempotency_key=uuid4(),
request_digest="e" * 64,
)
)
await session.flush()
# Use column-level INSERT. ORM inserts optional execution fields, which the platform
# deliberately cannot assign; application creation must use the same column projection.
await session.execute(
Evaluation.__table__.insert().values(
id=evaluation_id, submission_id=submission_id, problem_version_id=version_id
)
)
manifest = projected.manifest(evaluation_id, submission_id, language, source_ref)
async with sessions.begin() as session:
await session.execute(text("SET LOCAL ROLE kf_oj_coordinator"))
now = await session.scalar(select(func.now()))
session.add(
JudgeAttempt(
id=attempt_id,
evaluation_id=evaluation_id,
worker_id=worker_id,
number=1,
fencing_version=1,
lease_token_hash=token_digest(lease_token),
expires_at=now + timedelta(minutes=10),
manifest=manifest.model_dump(mode="json"),
manifest_digest=hashlib.sha256(manifest_bytes(manifest)).hexdigest(),
)
)
await session.flush()
await session.execute(
update(Evaluation)
.where(Evaluation.id == evaluation_id)
.values(
current_attempt_id=attempt_id, state="claimed", attempts_count=1, fencing_version=1
)
)
return (
LeaseBinding(
evaluation_id=evaluation_id,
attempt_id=attempt_id,
worker_id=worker_id,
fencing_version=1,
),
lease_token,
manifest,
)
def test_real_two_user_two_version_manifest_and_artifact_authorization(postgres_url, tmp_path):
root = tmp_path / "import"
_, policy = import_fixture(root)
async def run():
owner = create_async_engine(postgres_url)
owner_sessions = async_sessionmaker(owner, expire_on_commit=False)
app = create_app(
Settings(
_env_file=None,
environment="test",
database_url=postgres_url,
artifact_root=tmp_path / "artifacts",
judge_admission=policy,
)
)
try:
async with app.router.lifespan_context(app):
platform_role(app.state.sessions.kw["bind"])
password = secrets.token_hex(16)
first_user = await app.state.identity.create_account("first_user", password)
second_password = secrets.token_hex(16)
second_user = await app.state.identity.create_account(
"second_user", second_password
)
old_version = await app.state.problems.import_directory(root)
first, token, original = await install_attempt(
owner_sessions, app, first_user, old_version
)
(root / "expected-1").write_bytes(b"different hidden version\x00")
new_version = await app.state.problems.import_directory(root)
second, other_token, other_manifest = await install_attempt(
owner_sessions, app, second_user, new_version, source=b"other user's source"
)
queries = SubmissionQueries(app.state.sessions, app.state.problems.artifacts)
async with httpx.AsyncClient(
transport=httpx.ASGITransport(app=app), base_url="http://localhost:8000"
) as client:
login = await client.post(
"/api/v1/auth/login",
headers={"Origin": "http://localhost:8000"},
json={"username": "first_user", "password": password},
)
assert login.status_code == 200
authenticated = await app.state.identity.authenticate(
client.cookies.get("kf_session")
)
assert (
await queries.owned_source(authenticated, original.submission_id)
== b"synthetic source"
)
async with httpx.AsyncClient(
transport=httpx.ASGITransport(app=app), base_url="http://localhost:8000"
) as other_client:
other_login = await other_client.post(
"/api/v1/auth/login",
headers={"Origin": "http://localhost:8000"},
json={"username": "second_user", "password": second_password},
)
assert other_login.status_code == 200
other_principal = await app.state.identity.authenticate(
other_client.cookies.get("kf_session")
)
assert other_principal.account_id == second_user.account_id
with pytest.raises(SubmissionUnavailable):
await queries.owned_source(other_principal, original.submission_id)
path = (
f"/api/v1/internal/evaluations/{first.evaluation_id}"
f"/attempts/{first.attempt_id}"
)
headers = {
"Authorization": "Bearer " + token,
"X-Worker-ID": str(first.worker_id),
"X-Fencing-Version": "1",
}
assert (
await client.get(path + "/manifest", headers=headers)
).content == manifest_bytes(original)
# Changing the active problem version did not change an existing evaluation.
async with owner_sessions() as s:
evaluation = await s.get(Evaluation, first.evaluation_id)
assert evaluation.problem_version_id == old_version
assert (
await client.get(
path + "/manifest",
headers={k: v for k, v in headers.items() if k != "Authorization"},
)
).status_code == 401
assert (
await client.get(
path + "/manifest",
headers=headers | {"Authorization": "Bearer " + other_token},
)
).status_code == 404
assert (
await client.get(
path + "/manifest", headers=headers | {"X-Worker-ID": str(uuid4())}
)
).status_code == 404
assert (
await client.get(
path + "/manifest", headers=headers | {"X-Fencing-Version": "2"}
)
).status_code == 404
for reference in (
original.source,
original.cases[0].input,
original.cases[0].expected,
):
response = await client.get(
path + "/artifacts/" + reference.digest, headers=headers
)
assert response.status_code == 200
assert len(response.content) == reference.bytes
assert hashlib.sha256(response.content).hexdigest() == reference.digest
assert response.headers["cache-control"] == "no-store"
assert (
await client.get(
path + "/artifacts/" + other_manifest.source.digest, headers=headers
)
).status_code == 404
assert (
await client.get(
path + "/artifacts/" + other_manifest.cases[0].expected.digest,
headers=headers,
)
).status_code == 404
assert (
await client.get(path + "/artifacts/" + "f" * 64, headers=headers)
).status_code == 404
second_path = (
f"/api/v1/internal/evaluations/{second.evaluation_id}"
f"/attempts/{second.attempt_id}"
)
assert (
await client.get(second_path + "/manifest", headers=headers)
).status_code == 404
assert (
await client.get(
path + "/manifest",
headers={
"Authorization": "Bearer " + client.cookies.get("kf_session"),
"X-Worker-ID": str(first.worker_id),
"X-Fencing-Version": "1",
},
)
).status_code == 404
finally:
await owner.dispose()
asyncio.run(run())
@pytest.mark.parametrize(
"invalid", ["expired", "revoked", "cancel_requested", "terminal", "old_attempt"]
)
def test_expired_revoked_cancelled_terminal_or_old_attempt_rejected(
postgres_url, tmp_path, invalid
):
root = tmp_path / "import"
_, policy = import_fixture(root)
async def run():
owner = create_async_engine(postgres_url)
sessions = async_sessionmaker(owner, expire_on_commit=False)
app = create_app(
Settings(
_env_file=None,
environment="test",
database_url=postgres_url,
artifact_root=tmp_path / "artifacts",
judge_admission=policy,
)
)
try:
async with app.router.lifespan_context(app):
platform_role(app.state.sessions.kw["bind"])
account = await app.state.identity.create_account(
"fixture_user", secrets.token_hex(16)
)
version = await app.state.problems.import_directory(root)
binding, token, manifest = await install_attempt(sessions, app, account, version)
async with sessions.begin() as s:
await s.execute(text("SET LOCAL ROLE kf_oj_coordinator"))
if invalid == "expired":
await s.execute(
update(JudgeAttempt).values(
expires_at=func.now() - timedelta(seconds=1)
)
)
elif invalid == "revoked":
await s.execute(update(JudgeAttempt).values(revoked_at=func.now()))
elif invalid == "cancel_requested":
await s.execute(update(Evaluation).values(cancel_requested_at=func.now()))
elif invalid == "terminal":
await s.execute(update(Evaluation).values(state="finished"))
else:
new_attempt = uuid4()
now = await s.scalar(select(func.now()))
s.add(
JudgeAttempt(
id=new_attempt,
evaluation_id=binding.evaluation_id,
worker_id=uuid4(),
number=2,
fencing_version=2,
lease_token_hash="e" * 64,
expires_at=now + timedelta(minutes=1),
manifest=manifest.model_dump(mode="json"),
manifest_digest=hashlib.sha256(
manifest_bytes(manifest)
).hexdigest(),
)
)
await s.flush()
await s.execute(
update(Evaluation).values(
current_attempt_id=new_attempt, fencing_version=2, attempts_count=2
)
)
async with httpx.AsyncClient(
transport=httpx.ASGITransport(app=app), base_url="http://test"
) as client:
prefix = (
f"/api/v1/internal/evaluations/{binding.evaluation_id}"
f"/attempts/{binding.attempt_id}"
)
headers = {
"Authorization": "Bearer " + token,
"X-Worker-ID": str(binding.worker_id),
"X-Fencing-Version": "1",
}
assert (
await client.get(prefix + "/manifest", headers=headers)
).status_code == 404
assert (
await client.get(
prefix + "/artifacts/" + manifest.source.digest, headers=headers
)
).status_code == 404
finally:
await owner.dispose()
asyncio.run(run())
+72 -2
View File
@@ -73,6 +73,40 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/12/b8/4bd346e22b28902df4d651910f5242c28d84e4a5c2435ca5c3f797ed7e2e/anyio-4.15.1-py3-none-any.whl", hash = "sha256:6152fdbbf9a77fdec97731721bebf7c4c44f7c29b424b0065826173efc7ed101", size = 132079, upload-time = "2026-09-05T10:42:37.923Z" },
]
[[package]]
name = "argon2-cffi"
version = "25.1.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "argon2-cffi-bindings" },
]
sdist = { url = "https://files.pythonhosted.org/packages/0e/89/ce5af8a7d472a67cc819d5d998aa8c82c5d860608c4db9f46f1162d7dab9/argon2_cffi-25.1.0.tar.gz", hash = "sha256:694ae5cc8a42f4c4e2bf2ca0e64e51e23a040c6a517a85074683d3959e1346c1", size = 45706, upload-time = "2025-06-03T06:55:32.073Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/4f/d3/a8b22fa575b297cd6e3e3b0155c7e25db170edf1c74783d6a31a2490b8d9/argon2_cffi-25.1.0-py3-none-any.whl", hash = "sha256:fdc8b074db390fccb6eb4a3604ae7231f219aa669a2652e0f20e16ba513d5741", size = 14657, upload-time = "2025-06-03T06:55:30.804Z" },
]
[[package]]
name = "argon2-cffi-bindings"
version = "26.1.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "cffi" },
]
sdist = { url = "https://files.pythonhosted.org/packages/0b/43/bb8b6e8708d49a5ab36781333af092d9f483b198a2710d01281204640055/argon2_cffi_bindings-26.1.0.tar.gz", hash = "sha256:63505c71542a44b68b1e38060450fb006404170da375feb31af153e7f9c6205d", size = 1790807, upload-time = "2026-08-20T07:44:22.492Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/e7/d2/0ae991f1b2181e5be49007c574710a800ad36c2978683addb3e67c474e55/argon2_cffi_bindings-26.1.0-cp310-abi3-macosx_11_0_arm64.whl", hash = "sha256:21ca0396fe5ec995dd54431c32698189666f9224810acfa752e50d2bd94d9df2", size = 25521, upload-time = "2026-08-20T07:32:43.019Z" },
{ url = "https://files.pythonhosted.org/packages/7e/e4/ad91d8297638aa2258aad4501c306aca99480dfe76ccd638173fa3702db9/argon2_cffi_bindings-26.1.0-cp310-abi3-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:78de2d65e0b9ea7ce9d1b1c3e87297b2d7305a02c266ee2a2d6910daddd7ee69", size = 27177, upload-time = "2026-08-20T07:32:44.158Z" },
{ url = "https://files.pythonhosted.org/packages/6f/86/5363df11b86d02cf3662208e7406496327649cc90eb365bf6f4e8a54a41f/argon2_cffi_bindings-26.1.0-cp310-abi3-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:27f1821903e2ceadcb88ec2b45ef190897b7682449c772f4d9b53e42c520cf29", size = 26597, upload-time = "2026-08-20T07:32:45.172Z" },
{ url = "https://files.pythonhosted.org/packages/f4/b5/a14dcc592652347dad23ee93b278a4da5d2a25c9ed3ebd10d68eea823a4f/argon2_cffi_bindings-26.1.0-cp310-abi3-manylinux_2_34_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:d88e5f7e60f28ae0b0cc6b2f16c43e87cd642a196a86f85e0d8bb6fe016fc16d", size = 27403, upload-time = "2026-08-20T07:32:46.13Z" },
{ url = "https://files.pythonhosted.org/packages/b3/81/b4a20d4902af7f796390bf9245ff83c5217dfa7367efa1d14986956c482b/argon2_cffi_bindings-26.1.0-cp310-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:34b7d9c24a4165a2c61cc8ae11d44d48c9ce2830fb536cb7914e11fdd9962728", size = 27132, upload-time = "2026-08-20T07:32:47.13Z" },
{ url = "https://files.pythonhosted.org/packages/7e/1b/c8de358af07b1c490e0fcb863ef98e46ddb486e45567aca5a60bd68d9daa/argon2_cffi_bindings-26.1.0-cp310-abi3-musllinux_1_2_riscv64.whl", hash = "sha256:224865cbbcb7a2bd1356741dff12b0134df726b6d44bb7b500df8e303cbd9e81", size = 27588, upload-time = "2026-08-20T07:32:48.087Z" },
{ url = "https://files.pythonhosted.org/packages/48/2f/7ee62a6e79f9309f9d9982d301b22a00010adb580c05c8109b94d7b33de0/argon2_cffi_bindings-26.1.0-cp310-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:ffff613aaa9ce6236766e2fc6dc560bb5abde7a2e2416e3db1f9ae395a2b4dd4", size = 26785, upload-time = "2026-08-20T07:32:48.977Z" },
{ url = "https://files.pythonhosted.org/packages/e9/10/960d0ee93d4897741bcaf4799c697dae2d81499f66fd1ed042a7dd54c1f4/argon2_cffi_bindings-26.1.0-cp310-abi3-win32.whl", hash = "sha256:a86c069c91a747a2c4e5c51473590aeb48172fff9b2130d23729a42d98665ecb", size = 23898, upload-time = "2026-08-20T07:32:50.114Z" },
{ url = "https://files.pythonhosted.org/packages/6d/3a/0cc14a05810e6add9bce5e87693334baa2222de5f647fa31781885b6573f/argon2_cffi_bindings-26.1.0-cp310-abi3-win_amd64.whl", hash = "sha256:2c36ff87b5dfaa477d0bd51e9d7f6abdae7c8955d2983c97419085d842154b3e", size = 25730, upload-time = "2026-08-20T07:32:51.091Z" },
{ url = "https://files.pythonhosted.org/packages/4e/db/d83cf2af140547f0b9cdaece05b2dc2dcbf991be4667331d073eff771435/argon2_cffi_bindings-26.1.0-cp310-abi3-win_arm64.whl", hash = "sha256:f9c4420a7a864fe1b86ce35befc95b8e39fb852493b81cf798671ddc265de638", size = 24478, upload-time = "2026-08-20T07:32:52.111Z" },
{ url = "https://files.pythonhosted.org/packages/bb/5f/f652055e18d2627e2eed94c7f31a792127cfe38df786635395d742321674/argon2_cffi_bindings-26.1.0-cp313-cp313-pyemscripten_2025_0_wasm32.whl", hash = "sha256:af11ac37a7c53dc16cb7950a6190851b0870fe218b6c60c0bb7ac355234e3083", size = 15434, upload-time = "2026-08-20T07:32:53.143Z" },
]
[[package]]
name = "asyncpg"
version = "0.31.0"
@@ -107,6 +141,31 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/0b/a7/71ac2cff56fec219ed242bb11b8efb69fcc4bec75db06fb7bfe35de520e6/certifi-2026.7.22-py3-none-any.whl", hash = "sha256:62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775", size = 136983, upload-time = "2026-07-22T03:35:11.276Z" },
]
[[package]]
name = "cffi"
version = "2.1.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "pycparser", marker = "implementation_name != 'PyPy'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/9e/ef/008a1939e372c06329a3fce4279c02f328488f3526744906eeec3da7ad5f/cffi-2.1.1.tar.gz", hash = "sha256:dd31f52ea1086513bb9df30f8fcee9b8918323ae067a3d5b78bc826a000712be", size = 530807, upload-time = "2026-08-03T21:21:18.939Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/9d/f4/035513d4117049066b4779dc3b7c0c0fdad175fa13731c9f4003f1cd1478/cffi-2.1.1-cp313-cp313-ios_13_0_arm64_iphoneos.whl", hash = "sha256:b5bdfd1c873d4e093aabc0ca84c4ca6dbc4f752afb5c86f146d9742580c9da2e", size = 194248, upload-time = "2026-08-03T21:19:59.399Z" },
{ url = "https://files.pythonhosted.org/packages/76/af/2aeb4dbb5fc41a04161ae9ff1518de7cec08e164f44a8ce6a4cf7fd2cd1d/cffi-2.1.1-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:31348097ff5bbe827ccc41795d4dd099d9f0625e7def00ee653c137a490c2a6c", size = 196908, upload-time = "2026-08-03T21:20:00.746Z" },
{ url = "https://files.pythonhosted.org/packages/a7/46/2e5fdde8555706dd98139a910ca11be02809f3f605ce956f655d0214e100/cffi-2.1.1-cp313-cp313-macosx_10_15_x86_64.whl", hash = "sha256:9d2055050ea716bd38b7f7f1579c275386646b4894c155a3e2f3cd62ed41b7c6", size = 184805, upload-time = "2026-08-03T21:20:02.02Z" },
{ url = "https://files.pythonhosted.org/packages/55/41/4c7042f317b9217502988f0873af87e16ad606dc20f84e546e3e6ce9764c/cffi-2.1.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:19ee6127ee34de7d83ce3d371ebc5ed91addbdcc39f9ab15ce4eb35a4e534971", size = 184764, upload-time = "2026-08-03T21:20:03.141Z" },
{ url = "https://files.pythonhosted.org/packages/43/1f/1c3d90d91811c8f86ced9ed637956c54bfe5b79ca98fe976d7f8c8979f6b/cffi-2.1.1-cp313-cp313-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:6a8dddef476fab96d066d578fc88526767b836ab5ab21754e1d5bf3879c31c7c", size = 214722, upload-time = "2026-08-03T21:20:04.377Z" },
{ url = "https://files.pythonhosted.org/packages/37/6f/3b5ce4c3b2192d250f04908f2bfd91ef34552ec8f7716a5d4abdb8d67bb2/cffi-2.1.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:f16c709686a78c727bbbf059f92b0bf41c6fc60deec706d2dc19f529175a6125", size = 222369, upload-time = "2026-08-03T21:20:05.544Z" },
{ url = "https://files.pythonhosted.org/packages/02/10/4b3c75dde3d9663c9e02ba05c2668b954f671d4bbe346413ca8c696b295a/cffi-2.1.1-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:fcd22650c908d7b7da162bbfaab594a1227a15d1643a98c68b122ac642fa2264", size = 210175, upload-time = "2026-08-03T21:20:06.75Z" },
{ url = "https://files.pythonhosted.org/packages/df/62/14f74b9543e605d17701dc797b815958b8bb70b7624ce1b832ddad48ed6c/cffi-2.1.1-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:aa9511c62d14da7aacc9b4bf51f3f697a621e83b2d6919008243c3aad168eea3", size = 208670, upload-time = "2026-08-03T21:20:08.04Z" },
{ url = "https://files.pythonhosted.org/packages/95/95/86342356ff5953b3fb06f7ef7c5bee212d45e770abc7218d451b9148313c/cffi-2.1.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:a931079504ecc49efed7744c476a5c343a92fabf66dec2db95edb1b2fdc770e2", size = 221824, upload-time = "2026-08-03T21:20:09.274Z" },
{ url = "https://files.pythonhosted.org/packages/eb/ff/7b3429ff53aafe931ed8a5fc69f481bbef7ba6de87ddcbb63d08f483f613/cffi-2.1.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a2d7755bef5a12ed488f4ef1f1b69ee9191d7396083b755a5d2295f6edb4768b", size = 225148, upload-time = "2026-08-03T21:20:10.7Z" },
{ url = "https://files.pythonhosted.org/packages/34/34/a95870b9221e09cf4f2ce3178b1a210abdfe63a1bd357da940418d7b8d15/cffi-2.1.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:e0bcb7e0f677f543555d2adff3bf19c05f66cdb4796e5ff602442ab2fe3c4ef7", size = 223564, upload-time = "2026-08-03T21:20:12.165Z" },
{ url = "https://files.pythonhosted.org/packages/70/ea/839b50531021a647fb5e929f72cf97bc1ff702b5472166164b5b6e76b851/cffi-2.1.1-cp313-cp313-win32.whl", hash = "sha256:334644fbac4eff73d985a17a91226df55d0f394160c4cfb880e084c8f7161cac", size = 175263, upload-time = "2026-08-03T21:20:13.559Z" },
{ url = "https://files.pythonhosted.org/packages/60/a6/8b149b2c3f2e11aaa1618ef64500b45f50f22c57a977a4dff1aff1f91042/cffi-2.1.1-cp313-cp313-win_amd64.whl", hash = "sha256:1aa5645c30469b09530c4ebca77ebf8f17618293c58f8549cb1a543a50236e7d", size = 185688, upload-time = "2026-08-03T21:20:14.69Z" },
{ url = "https://files.pythonhosted.org/packages/01/9a/11f687cb39d6a3504060d5242f04f48c735afb4d3d533958a20594890cb2/cffi-2.1.1-cp313-cp313-win_arm64.whl", hash = "sha256:63bbfd5ded17c4840ac07cd8f1c21ba9d9708141f840b324f422f41b207e3973", size = 180078, upload-time = "2026-08-03T21:20:15.917Z" },
]
[[package]]
name = "click"
version = "8.5.0"
@@ -249,8 +308,10 @@ source = { editable = "." }
dependencies = [
{ name = "aio-pika" },
{ name = "alembic" },
{ name = "argon2-cffi" },
{ name = "asyncpg" },
{ name = "fastapi" },
{ name = "httpx" },
{ name = "pydantic" },
{ name = "pydantic-settings" },
{ name = "sqlalchemy", extra = ["asyncio"] },
@@ -259,7 +320,6 @@ dependencies = [
[package.dev-dependencies]
dev = [
{ name = "httpx" },
{ name = "jsonschema" },
{ name = "pytest" },
{ name = "ruff" },
@@ -269,8 +329,10 @@ dev = [
requires-dist = [
{ name = "aio-pika", specifier = ">=9.5,<10" },
{ name = "alembic", specifier = ">=1.15,<2" },
{ name = "argon2-cffi", specifier = ">=25.1,<26" },
{ name = "asyncpg", specifier = ">=0.30,<1" },
{ name = "fastapi", specifier = ">=0.115,<1" },
{ name = "httpx", specifier = ">=0.28,<1" },
{ name = "pydantic", specifier = ">=2.10,<3" },
{ name = "pydantic-settings", specifier = ">=2.7,<3" },
{ name = "sqlalchemy", extras = ["asyncio"], specifier = ">=2.0,<3" },
@@ -279,7 +341,6 @@ requires-dist = [
[package.metadata.requires-dev]
dev = [
{ name = "httpx", specifier = ">=0.28,<1" },
{ name = "jsonschema", specifier = ">=4.23,<5" },
{ name = "pytest", specifier = ">=8.3,<10" },
{ name = "ruff", specifier = ">=0.11,<1" },
@@ -422,6 +483,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/f5/cd/785c64ed382f3f04201870267b02783f63b4678c2acfddc177a3ebcc2727/propcache-0.5.4-py3-none-any.whl", hash = "sha256:62c60aec739ed00124573cce1178138fd690c7676352d67a37328c1cf51d7468", size = 16338, upload-time = "2026-09-16T00:17:13.106Z" },
]
[[package]]
name = "pycparser"
version = "3.1"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/ac/d3/eb1d3bc30dda12f7e69640ae2ac8cb10240b71fb73024ad528b7d2ae73da/pycparser-3.1.tar.gz", hash = "sha256:b3fc6dec06a8b2fefa0ed4ff92285306a5e3be9987bc5603c9edbdc4e492418f", size = 105946, upload-time = "2026-10-08T12:26:34.349Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/99/ce/b3ae9ee0324d991c860187be2a6ee436d27a3d02397eaddbb101ec901f3d/pycparser-3.1-py3-none-any.whl", hash = "sha256:f09d358c840bd147b79e55f2bc494f18ea869dc897f5852a8f5766b74f787882", size = 48709, upload-time = "2026-10-08T12:26:32.97Z" },
]
[[package]]
name = "pydantic"
version = "2.13.5"
+27 -1
View File
@@ -9,6 +9,32 @@
`make contracts` 从 Python 模型生成 Schema/OpenAPI,并生成前端 API 类型。
`make check` 检查快照未过期及两种语言对样例的接受/拒绝一致。
执行请求/报告、评测状态机、租约与结果提交契约在 JS-01/DR-01 中逐项补充。
执行器私有规格与 FD 协议属于 Sandbox;平台不接收 argv、宿主路径或执行镜像路径。
普通题提交协议现有 `judge-manifest`、`judge-lease`、`judge-stage`、`judge-result`
和本地可信 `judge-admission` v1 模型。清单绑定源码、不可变题目版本、按序工件摘要、
C17/C++20、固定 GNU 工具链版本及编译/运行预算;租约绑定 evaluation/attempt/Worker
与 fencing 版本。Root 权限、执行和 DB 状态转换接口仍由各自服务实现,解析清单或租约
不授予执行权。
Python/Rust `decode_wire` 在解析前限制 256 KiB UTF-8 JSON,拒绝重复字段与非法数据,
传输边界错误不回显输入。令牌及编译诊断的默认调试输出隐藏内容。`serde_json` 的运行期
直接依赖用于此有界解码,不引入 IO、网络或数据库。
Schema 描述结构;模型另外核对规范 UUID、严格整数(拒绝 `1.0`/布尔值)、工件总大小、
重复测试点、预算相容、诊断 UTF-8 字节数、编译状态/产物、清理失败和实际结果汇总。
JSON Schema 的 integer 按数值判断,允许 `1.0`;不能替代协议解码。共享样例中的
`invalid-semantic-*` 结构合法但被两种语言模型拒绝,其他非法样例同时验证结构拒绝。
样例全为合成元数据;其中令牌、摘要不是已签发凭证或实际隐藏测试数据。
Coordinator 必须在事务锁内核对当前租约及状态规则,并将结果和授权清单比较:
完整用例次序、数量和清单摘要不能只靠报告自身校验。已提交报告的幂等确认与过期租约
拒绝分别处理,旧 attempt 无权改写结果。`cancelling` 在实际停止确认前不得转为
`cancelled`;清理失败必须保留独立故障,缺失计量始终为 null。
生成器 `export.py` 是开发工具,允许读取 API 组合层;运行期的契约模型禁止依赖业务模块。
# 任务清单字节身份
Python/Rust 的 `manifest_bytes` 输出同一 UTF-8 JSON:递归排序对象键,无额外空白/换行。
清单摘要是这些确切字节的 SHA-256,不能使用数据库 JSONB 的输出顺序或任意 prettify 后的字节。
共享 `canonical-manifest.v1.txt` 为合成示例的字节验收,平台内部响应复用这些序列化字节。
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,169 @@
{
"$comment": "Structural schema; also run versioned model validation for cross-field budgets, result consistency, UUID canonical form and UTF-8 byte limits.",
"$defs": {
"RunLimits": {
"additionalProperties": false,
"properties": {
"cpu_time_us": {
"maximum": 60000000,
"minimum": 1000,
"title": "Cpu Time Us",
"type": "integer"
},
"file_bytes": {
"maximum": 67108864,
"minimum": 1,
"title": "File Bytes",
"type": "integer"
},
"memory_bytes": {
"maximum": 2147483648,
"minimum": 1,
"title": "Memory Bytes",
"type": "integer"
},
"open_files": {
"maximum": 1024,
"minimum": 3,
"title": "Open Files",
"type": "integer"
},
"processes": {
"maximum": 128,
"minimum": 1,
"title": "Processes",
"type": "integer"
},
"stack_bytes": {
"maximum": 268435456,
"minimum": 1,
"title": "Stack Bytes",
"type": "integer"
},
"stderr_bytes": {
"maximum": 16777216,
"minimum": 1,
"title": "Stderr Bytes",
"type": "integer"
},
"stdout_bytes": {
"maximum": 16777216,
"minimum": 1,
"title": "Stdout Bytes",
"type": "integer"
},
"wall_time_us": {
"maximum": 120000000,
"minimum": 1000,
"title": "Wall Time Us",
"type": "integer"
},
"workdir_bytes": {
"maximum": 268435456,
"minimum": 1,
"title": "Workdir Bytes",
"type": "integer"
}
},
"required": [
"cpu_time_us",
"wall_time_us",
"memory_bytes",
"stack_bytes",
"processes",
"open_files",
"stdout_bytes",
"stderr_bytes",
"file_bytes",
"workdir_bytes"
],
"title": "RunLimits",
"type": "object"
},
"Toolchain": {
"additionalProperties": false,
"properties": {
"image_kind": {
"const": "linux_gnu_cxx_x86_64_v1",
"title": "Image Kind",
"type": "string"
},
"image_revision": {
"maximum": 4294967295,
"minimum": 1,
"title": "Image Revision",
"type": "integer"
},
"toolchain_id": {
"pattern": "^[a-z0-9][a-z0-9_-]{0,63}$",
"title": "Toolchain Id",
"type": "string"
}
},
"required": [
"toolchain_id",
"image_kind",
"image_revision"
],
"title": "Toolchain",
"type": "object"
}
},
"$id": "urn:kf-oj:judge-admission:v1",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false,
"description": "Trusted node policy. A manifest never supplies or replaces this policy.",
"properties": {
"compile_ceiling": {
"$ref": "#/$defs/RunLimits"
},
"max_concurrent_jobs": {
"default": 1,
"maximum": 64,
"minimum": 1,
"title": "Max Concurrent Jobs",
"type": "integer"
},
"max_job_wall_time_us": {
"maximum": 3600000000,
"minimum": 1000,
"title": "Max Job Wall Time Us",
"type": "integer"
},
"max_package_bytes": {
"maximum": 67108864,
"minimum": 1,
"title": "Max Package Bytes",
"type": "integer"
},
"max_retries": {
"default": 2,
"maximum": 4,
"minimum": 0,
"title": "Max Retries",
"type": "integer"
},
"prefetch": {
"default": 1,
"maximum": 64,
"minimum": 1,
"title": "Prefetch",
"type": "integer"
},
"run_ceiling": {
"$ref": "#/$defs/RunLimits"
},
"toolchain": {
"$ref": "#/$defs/Toolchain"
}
},
"required": [
"max_job_wall_time_us",
"max_package_bytes",
"toolchain",
"compile_ceiling",
"run_ceiling"
],
"title": "AdmissionPolicy",
"type": "object"
}
@@ -0,0 +1,226 @@
{
"$comment": "Multi-runtime protocol. Also validate runtime/source/attempt bindings, startup and node budgets, canonical bounded class sets and result consistency.",
"$defs": {
"Language": {
"enum": [
"c17",
"cpp20",
"python313",
"java21"
],
"title": "Language",
"type": "string"
},
"RunLimits": {
"additionalProperties": false,
"properties": {
"cpu_time_us": {
"maximum": 60000000,
"minimum": 1000,
"title": "Cpu Time Us",
"type": "integer"
},
"file_bytes": {
"maximum": 67108864,
"minimum": 1,
"title": "File Bytes",
"type": "integer"
},
"memory_bytes": {
"maximum": 2147483648,
"minimum": 1,
"title": "Memory Bytes",
"type": "integer"
},
"open_files": {
"maximum": 1024,
"minimum": 3,
"title": "Open Files",
"type": "integer"
},
"processes": {
"maximum": 128,
"minimum": 1,
"title": "Processes",
"type": "integer"
},
"stack_bytes": {
"maximum": 268435456,
"minimum": 1,
"title": "Stack Bytes",
"type": "integer"
},
"stderr_bytes": {
"maximum": 16777216,
"minimum": 1,
"title": "Stderr Bytes",
"type": "integer"
},
"stdout_bytes": {
"maximum": 16777216,
"minimum": 1,
"title": "Stdout Bytes",
"type": "integer"
},
"wall_time_us": {
"maximum": 120000000,
"minimum": 1000,
"title": "Wall Time Us",
"type": "integer"
},
"workdir_bytes": {
"maximum": 268435456,
"minimum": 1,
"title": "Workdir Bytes",
"type": "integer"
}
},
"required": [
"cpu_time_us",
"wall_time_us",
"memory_bytes",
"stack_bytes",
"processes",
"open_files",
"stdout_bytes",
"stderr_bytes",
"file_bytes",
"workdir_bytes"
],
"title": "RunLimits",
"type": "object"
},
"RuntimeCapability": {
"additionalProperties": false,
"properties": {
"compile_ceiling": {
"$ref": "#/$defs/RunLimits"
},
"language": {
"$ref": "#/$defs/Language"
},
"run_ceiling": {
"$ref": "#/$defs/RunLimits"
},
"toolchain": {
"$ref": "#/$defs/RuntimeIdentity"
}
},
"required": [
"language",
"toolchain",
"compile_ceiling",
"run_ceiling"
],
"title": "RuntimeCapability",
"type": "object"
},
"RuntimeIdentity": {
"additionalProperties": false,
"description": "An exact identity matched against trusted configuration, never a host path.",
"properties": {
"image_digest": {
"maxLength": 64,
"minLength": 64,
"pattern": "^[0-9a-f]{64}$",
"title": "Image Digest",
"type": "string"
},
"image_kind": {
"enum": [
"linux_gnu_cxx_x86_64_v1",
"linux_cpython313_x86_64_v1",
"linux_openjdk21_x86_64_v1"
],
"title": "Image Kind",
"type": "string"
},
"image_revision": {
"maximum": 4294967295,
"minimum": 1,
"title": "Image Revision",
"type": "integer"
},
"runtime_version": {
"pattern": "^[A-Za-z0-9][A-Za-z0-9_.+-]{0,63}$",
"title": "Runtime Version",
"type": "string"
},
"toolchain_id": {
"pattern": "^[a-z0-9][a-z0-9_-]{0,63}$",
"title": "Toolchain Id",
"type": "string"
}
},
"required": [
"toolchain_id",
"image_kind",
"image_revision",
"image_digest",
"runtime_version"
],
"title": "RuntimeIdentity",
"type": "object"
}
},
"$id": "urn:kf-oj:judge-admission:v2",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false,
"properties": {
"capabilities": {
"items": {
"$ref": "#/$defs/RuntimeCapability"
},
"maxItems": 4,
"minItems": 1,
"title": "Capabilities",
"type": "array"
},
"max_concurrent_jobs": {
"default": 1,
"maximum": 64,
"minimum": 1,
"title": "Max Concurrent Jobs",
"type": "integer"
},
"max_job_wall_time_us": {
"maximum": 3600000000,
"minimum": 1000,
"title": "Max Job Wall Time Us",
"type": "integer"
},
"max_package_bytes": {
"maximum": 67108864,
"minimum": 1,
"title": "Max Package Bytes",
"type": "integer"
},
"max_retries": {
"default": 2,
"maximum": 4,
"minimum": 0,
"title": "Max Retries",
"type": "integer"
},
"prefetch": {
"default": 1,
"maximum": 64,
"minimum": 1,
"title": "Prefetch",
"type": "integer"
},
"schema_version": {
"const": 2,
"title": "Schema Version",
"type": "integer"
}
},
"required": [
"schema_version",
"max_job_wall_time_us",
"max_package_bytes",
"capabilities"
],
"title": "AdmissionPolicy",
"type": "object"
}
@@ -0,0 +1,94 @@
{
"$comment": "Structural schema; also run versioned model validation for cross-field budgets, result consistency, UUID canonical form and UTF-8 byte limits.",
"$defs": {
"LeaseBinding": {
"additionalProperties": false,
"properties": {
"attempt_id": {
"format": "uuid",
"title": "Attempt Id",
"type": "string"
},
"evaluation_id": {
"format": "uuid",
"title": "Evaluation Id",
"type": "string"
},
"fencing_version": {
"maximum": 9223372036854775807,
"minimum": 1,
"title": "Fencing Version",
"type": "integer"
},
"worker_id": {
"format": "uuid",
"title": "Worker Id",
"type": "string"
}
},
"required": [
"evaluation_id",
"attempt_id",
"worker_id",
"fencing_version"
],
"title": "LeaseBinding",
"type": "object"
}
},
"$id": "urn:kf-oj:judge-lease:v1",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false,
"properties": {
"attempt_number": {
"maximum": 5,
"minimum": 1,
"title": "Attempt Number",
"type": "integer"
},
"binding": {
"$ref": "#/$defs/LeaseBinding"
},
"expires_at_unix_ms": {
"maximum": 9223372036854775807,
"minimum": 1,
"title": "Expires At Unix Ms",
"type": "integer"
},
"kind": {
"const": "judge.lease",
"title": "Kind",
"type": "string"
},
"lease_token": {
"maxLength": 64,
"minLength": 64,
"pattern": "^[0-9a-f]{64}$",
"title": "Lease Token",
"type": "string"
},
"manifest_digest": {
"maxLength": 64,
"minLength": 64,
"pattern": "^[0-9a-f]{64}$",
"title": "Manifest Digest",
"type": "string"
},
"schema_version": {
"const": 1,
"title": "Schema Version",
"type": "integer"
}
},
"required": [
"schema_version",
"kind",
"binding",
"lease_token",
"attempt_number",
"expires_at_unix_ms",
"manifest_digest"
],
"title": "LeaseGrant",
"type": "object"
}
@@ -0,0 +1,94 @@
{
"$comment": "Multi-runtime protocol. Also validate runtime/source/attempt bindings, startup and node budgets, canonical bounded class sets and result consistency.",
"$defs": {
"LeaseBinding": {
"additionalProperties": false,
"properties": {
"attempt_id": {
"format": "uuid",
"title": "Attempt Id",
"type": "string"
},
"evaluation_id": {
"format": "uuid",
"title": "Evaluation Id",
"type": "string"
},
"fencing_version": {
"maximum": 9223372036854775807,
"minimum": 1,
"title": "Fencing Version",
"type": "integer"
},
"worker_id": {
"format": "uuid",
"title": "Worker Id",
"type": "string"
}
},
"required": [
"evaluation_id",
"attempt_id",
"worker_id",
"fencing_version"
],
"title": "LeaseBinding",
"type": "object"
}
},
"$id": "urn:kf-oj:judge-lease:v2",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false,
"properties": {
"attempt_number": {
"maximum": 5,
"minimum": 1,
"title": "Attempt Number",
"type": "integer"
},
"binding": {
"$ref": "#/$defs/LeaseBinding"
},
"expires_at_unix_ms": {
"maximum": 9223372036854775807,
"minimum": 1,
"title": "Expires At Unix Ms",
"type": "integer"
},
"kind": {
"const": "judge.lease",
"title": "Kind",
"type": "string"
},
"lease_token": {
"maxLength": 64,
"minLength": 64,
"pattern": "^[0-9a-f]{64}$",
"title": "Lease Token",
"type": "string"
},
"manifest_digest": {
"maxLength": 64,
"minLength": 64,
"pattern": "^[0-9a-f]{64}$",
"title": "Manifest Digest",
"type": "string"
},
"schema_version": {
"const": 2,
"title": "Schema Version",
"type": "integer"
}
},
"required": [
"schema_version",
"kind",
"binding",
"lease_token",
"attempt_number",
"expires_at_unix_ms",
"manifest_digest"
],
"title": "LeaseGrant",
"type": "object"
}
@@ -0,0 +1,249 @@
{
"$comment": "Structural schema; also run versioned model validation for cross-field budgets, result consistency, UUID canonical form and UTF-8 byte limits.",
"$defs": {
"ArtifactReference": {
"additionalProperties": false,
"properties": {
"bytes": {
"maximum": 67108864,
"minimum": 0,
"title": "Bytes",
"type": "integer"
},
"digest": {
"maxLength": 64,
"minLength": 64,
"pattern": "^[0-9a-f]{64}$",
"title": "Digest",
"type": "string"
}
},
"required": [
"digest",
"bytes"
],
"title": "ArtifactReference",
"type": "object"
},
"Language": {
"enum": [
"c17",
"cpp20"
],
"title": "Language",
"type": "string"
},
"RunLimits": {
"additionalProperties": false,
"properties": {
"cpu_time_us": {
"maximum": 60000000,
"minimum": 1000,
"title": "Cpu Time Us",
"type": "integer"
},
"file_bytes": {
"maximum": 67108864,
"minimum": 1,
"title": "File Bytes",
"type": "integer"
},
"memory_bytes": {
"maximum": 2147483648,
"minimum": 1,
"title": "Memory Bytes",
"type": "integer"
},
"open_files": {
"maximum": 1024,
"minimum": 3,
"title": "Open Files",
"type": "integer"
},
"processes": {
"maximum": 128,
"minimum": 1,
"title": "Processes",
"type": "integer"
},
"stack_bytes": {
"maximum": 268435456,
"minimum": 1,
"title": "Stack Bytes",
"type": "integer"
},
"stderr_bytes": {
"maximum": 16777216,
"minimum": 1,
"title": "Stderr Bytes",
"type": "integer"
},
"stdout_bytes": {
"maximum": 16777216,
"minimum": 1,
"title": "Stdout Bytes",
"type": "integer"
},
"wall_time_us": {
"maximum": 120000000,
"minimum": 1000,
"title": "Wall Time Us",
"type": "integer"
},
"workdir_bytes": {
"maximum": 268435456,
"minimum": 1,
"title": "Workdir Bytes",
"type": "integer"
}
},
"required": [
"cpu_time_us",
"wall_time_us",
"memory_bytes",
"stack_bytes",
"processes",
"open_files",
"stdout_bytes",
"stderr_bytes",
"file_bytes",
"workdir_bytes"
],
"title": "RunLimits",
"type": "object"
},
"TestCase": {
"additionalProperties": false,
"properties": {
"case_id": {
"format": "uuid",
"title": "Case Id",
"type": "string"
},
"expected": {
"$ref": "#/$defs/ArtifactReference"
},
"input": {
"$ref": "#/$defs/ArtifactReference"
}
},
"required": [
"case_id",
"input",
"expected"
],
"title": "TestCase",
"type": "object"
},
"Toolchain": {
"additionalProperties": false,
"properties": {
"image_kind": {
"const": "linux_gnu_cxx_x86_64_v1",
"title": "Image Kind",
"type": "string"
},
"image_revision": {
"maximum": 4294967295,
"minimum": 1,
"title": "Image Revision",
"type": "integer"
},
"toolchain_id": {
"pattern": "^[a-z0-9][a-z0-9_-]{0,63}$",
"title": "Toolchain Id",
"type": "string"
}
},
"required": [
"toolchain_id",
"image_kind",
"image_revision"
],
"title": "Toolchain",
"type": "object"
}
},
"$id": "urn:kf-oj:judge-manifest:v1",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false,
"properties": {
"cases": {
"items": {
"$ref": "#/$defs/TestCase"
},
"maxItems": 256,
"minItems": 1,
"title": "Cases",
"type": "array"
},
"comparison": {
"const": "exact_bytes",
"title": "Comparison",
"type": "string"
},
"compile_limits": {
"$ref": "#/$defs/RunLimits"
},
"evaluation_id": {
"format": "uuid",
"title": "Evaluation Id",
"type": "string"
},
"job_wall_time_us": {
"maximum": 3600000000,
"minimum": 1000,
"title": "Job Wall Time Us",
"type": "integer"
},
"kind": {
"const": "judge.manifest",
"title": "Kind",
"type": "string"
},
"language": {
"$ref": "#/$defs/Language"
},
"problem_version_id": {
"format": "uuid",
"title": "Problem Version Id",
"type": "string"
},
"run_limits": {
"$ref": "#/$defs/RunLimits"
},
"schema_version": {
"const": 1,
"title": "Schema Version",
"type": "integer"
},
"source": {
"$ref": "#/$defs/ArtifactReference"
},
"submission_id": {
"format": "uuid",
"title": "Submission Id",
"type": "string"
},
"toolchain": {
"$ref": "#/$defs/Toolchain"
}
},
"required": [
"schema_version",
"kind",
"evaluation_id",
"submission_id",
"problem_version_id",
"language",
"toolchain",
"source",
"cases",
"comparison",
"compile_limits",
"run_limits",
"job_wall_time_us"
],
"title": "JobManifest",
"type": "object"
}
@@ -0,0 +1,270 @@
{
"$comment": "Multi-runtime protocol. Also validate runtime/source/attempt bindings, startup and node budgets, canonical bounded class sets and result consistency.",
"$defs": {
"ArtifactReference": {
"additionalProperties": false,
"properties": {
"bytes": {
"maximum": 67108864,
"minimum": 0,
"title": "Bytes",
"type": "integer"
},
"digest": {
"maxLength": 64,
"minLength": 64,
"pattern": "^[0-9a-f]{64}$",
"title": "Digest",
"type": "string"
}
},
"required": [
"digest",
"bytes"
],
"title": "ArtifactReference",
"type": "object"
},
"Language": {
"enum": [
"c17",
"cpp20",
"python313",
"java21"
],
"title": "Language",
"type": "string"
},
"RunLimits": {
"additionalProperties": false,
"properties": {
"cpu_time_us": {
"maximum": 60000000,
"minimum": 1000,
"title": "Cpu Time Us",
"type": "integer"
},
"file_bytes": {
"maximum": 67108864,
"minimum": 1,
"title": "File Bytes",
"type": "integer"
},
"memory_bytes": {
"maximum": 2147483648,
"minimum": 1,
"title": "Memory Bytes",
"type": "integer"
},
"open_files": {
"maximum": 1024,
"minimum": 3,
"title": "Open Files",
"type": "integer"
},
"processes": {
"maximum": 128,
"minimum": 1,
"title": "Processes",
"type": "integer"
},
"stack_bytes": {
"maximum": 268435456,
"minimum": 1,
"title": "Stack Bytes",
"type": "integer"
},
"stderr_bytes": {
"maximum": 16777216,
"minimum": 1,
"title": "Stderr Bytes",
"type": "integer"
},
"stdout_bytes": {
"maximum": 16777216,
"minimum": 1,
"title": "Stdout Bytes",
"type": "integer"
},
"wall_time_us": {
"maximum": 120000000,
"minimum": 1000,
"title": "Wall Time Us",
"type": "integer"
},
"workdir_bytes": {
"maximum": 268435456,
"minimum": 1,
"title": "Workdir Bytes",
"type": "integer"
}
},
"required": [
"cpu_time_us",
"wall_time_us",
"memory_bytes",
"stack_bytes",
"processes",
"open_files",
"stdout_bytes",
"stderr_bytes",
"file_bytes",
"workdir_bytes"
],
"title": "RunLimits",
"type": "object"
},
"RuntimeIdentity": {
"additionalProperties": false,
"description": "An exact identity matched against trusted configuration, never a host path.",
"properties": {
"image_digest": {
"maxLength": 64,
"minLength": 64,
"pattern": "^[0-9a-f]{64}$",
"title": "Image Digest",
"type": "string"
},
"image_kind": {
"enum": [
"linux_gnu_cxx_x86_64_v1",
"linux_cpython313_x86_64_v1",
"linux_openjdk21_x86_64_v1"
],
"title": "Image Kind",
"type": "string"
},
"image_revision": {
"maximum": 4294967295,
"minimum": 1,
"title": "Image Revision",
"type": "integer"
},
"runtime_version": {
"pattern": "^[A-Za-z0-9][A-Za-z0-9_.+-]{0,63}$",
"title": "Runtime Version",
"type": "string"
},
"toolchain_id": {
"pattern": "^[a-z0-9][a-z0-9_-]{0,63}$",
"title": "Toolchain Id",
"type": "string"
}
},
"required": [
"toolchain_id",
"image_kind",
"image_revision",
"image_digest",
"runtime_version"
],
"title": "RuntimeIdentity",
"type": "object"
},
"TestCase": {
"additionalProperties": false,
"properties": {
"case_id": {
"format": "uuid",
"title": "Case Id",
"type": "string"
},
"expected": {
"$ref": "#/$defs/ArtifactReference"
},
"input": {
"$ref": "#/$defs/ArtifactReference"
}
},
"required": [
"case_id",
"input",
"expected"
],
"title": "TestCase",
"type": "object"
}
},
"$id": "urn:kf-oj:judge-manifest:v2",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false,
"properties": {
"cases": {
"items": {
"$ref": "#/$defs/TestCase"
},
"maxItems": 256,
"minItems": 1,
"title": "Cases",
"type": "array"
},
"comparison": {
"const": "exact_bytes",
"title": "Comparison",
"type": "string"
},
"compile_limits": {
"$ref": "#/$defs/RunLimits"
},
"evaluation_id": {
"format": "uuid",
"title": "Evaluation Id",
"type": "string"
},
"job_wall_time_us": {
"maximum": 3600000000,
"minimum": 1000,
"title": "Job Wall Time Us",
"type": "integer"
},
"kind": {
"const": "judge.manifest",
"title": "Kind",
"type": "string"
},
"language": {
"$ref": "#/$defs/Language"
},
"problem_version_id": {
"format": "uuid",
"title": "Problem Version Id",
"type": "string"
},
"run_limits": {
"$ref": "#/$defs/RunLimits"
},
"schema_version": {
"const": 2,
"title": "Schema Version",
"type": "integer"
},
"source": {
"$ref": "#/$defs/ArtifactReference"
},
"submission_id": {
"format": "uuid",
"title": "Submission Id",
"type": "string"
},
"toolchain": {
"$ref": "#/$defs/RuntimeIdentity"
}
},
"required": [
"schema_version",
"kind",
"evaluation_id",
"submission_id",
"problem_version_id",
"language",
"toolchain",
"source",
"cases",
"comparison",
"compile_limits",
"run_limits",
"job_wall_time_us"
],
"title": "JobManifest",
"type": "object"
}
@@ -0,0 +1,349 @@
{
"$comment": "Multi-runtime protocol. Also validate runtime/source/attempt bindings, startup and node budgets, canonical bounded class sets and result consistency.",
"$defs": {
"Language": {
"enum": [
"c17",
"cpp20",
"python313",
"java21"
],
"title": "Language",
"type": "string"
},
"RunLimits": {
"additionalProperties": false,
"properties": {
"cpu_time_us": {
"maximum": 60000000,
"minimum": 1000,
"title": "Cpu Time Us",
"type": "integer"
},
"file_bytes": {
"maximum": 67108864,
"minimum": 1,
"title": "File Bytes",
"type": "integer"
},
"memory_bytes": {
"maximum": 2147483648,
"minimum": 1,
"title": "Memory Bytes",
"type": "integer"
},
"open_files": {
"maximum": 1024,
"minimum": 3,
"title": "Open Files",
"type": "integer"
},
"processes": {
"maximum": 128,
"minimum": 1,
"title": "Processes",
"type": "integer"
},
"stack_bytes": {
"maximum": 268435456,
"minimum": 1,
"title": "Stack Bytes",
"type": "integer"
},
"stderr_bytes": {
"maximum": 16777216,
"minimum": 1,
"title": "Stderr Bytes",
"type": "integer"
},
"stdout_bytes": {
"maximum": 16777216,
"minimum": 1,
"title": "Stdout Bytes",
"type": "integer"
},
"wall_time_us": {
"maximum": 120000000,
"minimum": 1000,
"title": "Wall Time Us",
"type": "integer"
},
"workdir_bytes": {
"maximum": 268435456,
"minimum": 1,
"title": "Workdir Bytes",
"type": "integer"
}
},
"required": [
"cpu_time_us",
"wall_time_us",
"memory_bytes",
"stack_bytes",
"processes",
"open_files",
"stdout_bytes",
"stderr_bytes",
"file_bytes",
"workdir_bytes"
],
"title": "RunLimits",
"type": "object"
},
"RuntimeCapability": {
"additionalProperties": false,
"properties": {
"compile_ceiling": {
"$ref": "#/$defs/RunLimits"
},
"language": {
"$ref": "#/$defs/Language"
},
"run_ceiling": {
"$ref": "#/$defs/RunLimits"
},
"toolchain": {
"$ref": "#/$defs/RuntimeIdentity"
}
},
"required": [
"language",
"toolchain",
"compile_ceiling",
"run_ceiling"
],
"title": "RuntimeCapability",
"type": "object"
},
"RuntimeIdentity": {
"additionalProperties": false,
"description": "An exact identity matched against trusted configuration, never a host path.",
"properties": {
"image_digest": {
"maxLength": 64,
"minLength": 64,
"pattern": "^[0-9a-f]{64}$",
"title": "Image Digest",
"type": "string"
},
"image_kind": {
"enum": [
"linux_gnu_cxx_x86_64_v1",
"linux_cpython313_x86_64_v1",
"linux_openjdk21_x86_64_v1"
],
"title": "Image Kind",
"type": "string"
},
"image_revision": {
"maximum": 4294967295,
"minimum": 1,
"title": "Image Revision",
"type": "integer"
},
"runtime_version": {
"pattern": "^[A-Za-z0-9][A-Za-z0-9_.+-]{0,63}$",
"title": "Runtime Version",
"type": "string"
},
"toolchain_id": {
"pattern": "^[a-z0-9][a-z0-9_-]{0,63}$",
"title": "Toolchain Id",
"type": "string"
}
},
"required": [
"toolchain_id",
"image_kind",
"image_revision",
"image_digest",
"runtime_version"
],
"title": "RuntimeIdentity",
"type": "object"
},
"Toolchain": {
"additionalProperties": false,
"properties": {
"image_kind": {
"const": "linux_gnu_cxx_x86_64_v1",
"title": "Image Kind",
"type": "string"
},
"image_revision": {
"maximum": 4294967295,
"minimum": 1,
"title": "Image Revision",
"type": "integer"
},
"toolchain_id": {
"pattern": "^[a-z0-9][a-z0-9_-]{0,63}$",
"title": "Toolchain Id",
"type": "string"
}
},
"required": [
"toolchain_id",
"image_kind",
"image_revision"
],
"title": "Toolchain",
"type": "object"
},
"kf_oj__contracts__judging__AdmissionPolicy": {
"additionalProperties": false,
"description": "Trusted node policy. A manifest never supplies or replaces this policy.",
"properties": {
"compile_ceiling": {
"$ref": "#/$defs/RunLimits"
},
"max_concurrent_jobs": {
"default": 1,
"maximum": 64,
"minimum": 1,
"title": "Max Concurrent Jobs",
"type": "integer"
},
"max_job_wall_time_us": {
"maximum": 3600000000,
"minimum": 1000,
"title": "Max Job Wall Time Us",
"type": "integer"
},
"max_package_bytes": {
"maximum": 67108864,
"minimum": 1,
"title": "Max Package Bytes",
"type": "integer"
},
"max_retries": {
"default": 2,
"maximum": 4,
"minimum": 0,
"title": "Max Retries",
"type": "integer"
},
"prefetch": {
"default": 1,
"maximum": 64,
"minimum": 1,
"title": "Prefetch",
"type": "integer"
},
"run_ceiling": {
"$ref": "#/$defs/RunLimits"
},
"toolchain": {
"$ref": "#/$defs/Toolchain"
}
},
"required": [
"max_job_wall_time_us",
"max_package_bytes",
"toolchain",
"compile_ceiling",
"run_ceiling"
],
"title": "AdmissionPolicy",
"type": "object"
},
"kf_oj__contracts__judging_v2__AdmissionPolicy": {
"additionalProperties": false,
"properties": {
"capabilities": {
"items": {
"$ref": "#/$defs/RuntimeCapability"
},
"maxItems": 4,
"minItems": 1,
"title": "Capabilities",
"type": "array"
},
"max_concurrent_jobs": {
"default": 1,
"maximum": 64,
"minimum": 1,
"title": "Max Concurrent Jobs",
"type": "integer"
},
"max_job_wall_time_us": {
"maximum": 3600000000,
"minimum": 1000,
"title": "Max Job Wall Time Us",
"type": "integer"
},
"max_package_bytes": {
"maximum": 67108864,
"minimum": 1,
"title": "Max Package Bytes",
"type": "integer"
},
"max_retries": {
"default": 2,
"maximum": 4,
"minimum": 0,
"title": "Max Retries",
"type": "integer"
},
"prefetch": {
"default": 1,
"maximum": 64,
"minimum": 1,
"title": "Prefetch",
"type": "integer"
},
"schema_version": {
"const": 2,
"title": "Schema Version",
"type": "integer"
}
},
"required": [
"schema_version",
"max_job_wall_time_us",
"max_package_bytes",
"capabilities"
],
"title": "AdmissionPolicy",
"type": "object"
}
},
"$id": "urn:kf-oj:judge-registry:v2",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false,
"description": "Explicit operator opt-in for both protocol generations; no silent conversion.",
"properties": {
"legacy": {
"anyOf": [
{
"$ref": "#/$defs/kf_oj__contracts__judging__AdmissionPolicy"
},
{
"type": "null"
}
]
},
"multi_runtime": {
"anyOf": [
{
"$ref": "#/$defs/kf_oj__contracts__judging_v2__AdmissionPolicy"
},
{
"type": "null"
}
]
},
"schema_version": {
"const": 2,
"title": "Schema Version",
"type": "integer"
}
},
"required": [
"schema_version",
"legacy",
"multi_runtime"
],
"title": "AdmissionRegistry",
"type": "object"
}
@@ -0,0 +1,304 @@
{
"$comment": "Structural schema; also run versioned model validation for cross-field budgets, result consistency, UUID canonical form and UTF-8 byte limits.",
"$defs": {
"ArtifactReference": {
"additionalProperties": false,
"properties": {
"bytes": {
"maximum": 67108864,
"minimum": 0,
"title": "Bytes",
"type": "integer"
},
"digest": {
"maxLength": 64,
"minLength": 64,
"pattern": "^[0-9a-f]{64}$",
"title": "Digest",
"type": "string"
}
},
"required": [
"digest",
"bytes"
],
"title": "ArtifactReference",
"type": "object"
},
"CaseReport": {
"additionalProperties": false,
"properties": {
"case_id": {
"format": "uuid",
"title": "Case Id",
"type": "string"
},
"cpu_time_us": {
"anyOf": [
{
"maximum": 9223372036854775807,
"minimum": 0,
"type": "integer"
},
{
"type": "null"
}
],
"title": "Cpu Time Us"
},
"memory_peak_bytes": {
"anyOf": [
{
"maximum": 9223372036854775807,
"minimum": 0,
"type": "integer"
},
{
"type": "null"
}
],
"title": "Memory Peak Bytes"
},
"verdict": {
"$ref": "#/$defs/Verdict"
}
},
"required": [
"case_id",
"verdict",
"cpu_time_us",
"memory_peak_bytes"
],
"title": "CaseReport",
"type": "object"
},
"CompilationReport": {
"additionalProperties": false,
"properties": {
"cpu_time_us": {
"anyOf": [
{
"maximum": 9223372036854775807,
"minimum": 0,
"type": "integer"
},
{
"type": "null"
}
],
"title": "Cpu Time Us"
},
"diagnostics": {
"maxLength": 65536,
"title": "Diagnostics",
"type": "string"
},
"diagnostics_truncated": {
"title": "Diagnostics Truncated",
"type": "boolean"
},
"memory_peak_bytes": {
"anyOf": [
{
"maximum": 9223372036854775807,
"minimum": 0,
"type": "integer"
},
{
"type": "null"
}
],
"title": "Memory Peak Bytes"
},
"program": {
"anyOf": [
{
"$ref": "#/$defs/ArtifactReference"
},
{
"type": "null"
}
]
},
"status": {
"$ref": "#/$defs/CompilationStatus"
}
},
"required": [
"status",
"program",
"diagnostics",
"diagnostics_truncated",
"cpu_time_us",
"memory_peak_bytes"
],
"title": "CompilationReport",
"type": "object"
},
"CompilationStatus": {
"enum": [
"succeeded",
"compilation_error",
"compilation_limit",
"cancelled",
"system_error"
],
"title": "CompilationStatus",
"type": "string"
},
"FailureCode": {
"enum": [
"artifact_unavailable",
"sandbox_unavailable",
"cleanup_failed",
"lease_lost",
"job_deadline"
],
"title": "FailureCode",
"type": "string"
},
"LeaseBinding": {
"additionalProperties": false,
"properties": {
"attempt_id": {
"format": "uuid",
"title": "Attempt Id",
"type": "string"
},
"evaluation_id": {
"format": "uuid",
"title": "Evaluation Id",
"type": "string"
},
"fencing_version": {
"maximum": 9223372036854775807,
"minimum": 1,
"title": "Fencing Version",
"type": "integer"
},
"worker_id": {
"format": "uuid",
"title": "Worker Id",
"type": "string"
}
},
"required": [
"evaluation_id",
"attempt_id",
"worker_id",
"fencing_version"
],
"title": "LeaseBinding",
"type": "object"
},
"Verdict": {
"enum": [
"accepted",
"wrong_answer",
"compilation_error",
"runtime_error",
"time_limit_exceeded",
"memory_limit_exceeded",
"output_limit_exceeded",
"cancelled",
"system_error"
],
"title": "Verdict",
"type": "string"
}
},
"$id": "urn:kf-oj:judge-result:v1",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false,
"properties": {
"binding": {
"$ref": "#/$defs/LeaseBinding"
},
"cases": {
"items": {
"$ref": "#/$defs/CaseReport"
},
"maxItems": 256,
"title": "Cases",
"type": "array"
},
"cleanup_complete": {
"title": "Cleanup Complete",
"type": "boolean"
},
"compilation": {
"$ref": "#/$defs/CompilationReport"
},
"failure_code": {
"anyOf": [
{
"$ref": "#/$defs/FailureCode"
},
{
"type": "null"
}
]
},
"kind": {
"const": "judge.result",
"title": "Kind",
"type": "string"
},
"manifest_digest": {
"maxLength": 64,
"minLength": 64,
"pattern": "^[0-9a-f]{64}$",
"title": "Manifest Digest",
"type": "string"
},
"run_cpu_time_us": {
"anyOf": [
{
"maximum": 9223372036854775807,
"minimum": 0,
"type": "integer"
},
{
"type": "null"
}
],
"title": "Run Cpu Time Us"
},
"run_memory_peak_bytes": {
"anyOf": [
{
"maximum": 9223372036854775807,
"minimum": 0,
"type": "integer"
},
{
"type": "null"
}
],
"title": "Run Memory Peak Bytes"
},
"schema_version": {
"const": 1,
"title": "Schema Version",
"type": "integer"
},
"verdict": {
"$ref": "#/$defs/Verdict"
}
},
"required": [
"schema_version",
"kind",
"binding",
"manifest_digest",
"compilation",
"cases",
"verdict",
"cleanup_complete",
"failure_code",
"run_cpu_time_us",
"run_memory_peak_bytes"
],
"title": "JudgementReport",
"type": "object"
}
@@ -0,0 +1,416 @@
{
"$comment": "Multi-runtime protocol. Also validate runtime/source/attempt bindings, startup and node budgets, canonical bounded class sets and result consistency.",
"$defs": {
"ArtifactReference": {
"additionalProperties": false,
"properties": {
"bytes": {
"maximum": 67108864,
"minimum": 0,
"title": "Bytes",
"type": "integer"
},
"digest": {
"maxLength": 64,
"minLength": 64,
"pattern": "^[0-9a-f]{64}$",
"title": "Digest",
"type": "string"
}
},
"required": [
"digest",
"bytes"
],
"title": "ArtifactReference",
"type": "object"
},
"CaseReport": {
"additionalProperties": false,
"properties": {
"case_id": {
"format": "uuid",
"title": "Case Id",
"type": "string"
},
"cpu_time_us": {
"anyOf": [
{
"maximum": 9223372036854775807,
"minimum": 0,
"type": "integer"
},
{
"type": "null"
}
],
"title": "Cpu Time Us"
},
"memory_peak_bytes": {
"anyOf": [
{
"maximum": 9223372036854775807,
"minimum": 0,
"type": "integer"
},
{
"type": "null"
}
],
"title": "Memory Peak Bytes"
},
"verdict": {
"$ref": "#/$defs/Verdict"
}
},
"required": [
"case_id",
"verdict",
"cpu_time_us",
"memory_peak_bytes"
],
"title": "CaseReport",
"type": "object"
},
"CompilationReport": {
"additionalProperties": false,
"properties": {
"cpu_time_us": {
"anyOf": [
{
"maximum": 9223372036854775807,
"minimum": 0,
"type": "integer"
},
{
"type": "null"
}
],
"title": "Cpu Time Us"
},
"diagnostics": {
"maxLength": 65536,
"title": "Diagnostics",
"type": "string"
},
"diagnostics_truncated": {
"title": "Diagnostics Truncated",
"type": "boolean"
},
"memory_peak_bytes": {
"anyOf": [
{
"maximum": 9223372036854775807,
"minimum": 0,
"type": "integer"
},
{
"type": "null"
}
],
"title": "Memory Peak Bytes"
},
"program": {
"anyOf": [
{
"$ref": "#/$defs/CompiledProgram"
},
{
"type": "null"
}
]
},
"status": {
"$ref": "#/$defs/CompilationStatus"
}
},
"required": [
"status",
"program",
"diagnostics",
"diagnostics_truncated",
"cpu_time_us",
"memory_peak_bytes"
],
"title": "CompilationReport",
"type": "object"
},
"CompilationStatus": {
"enum": [
"succeeded",
"compilation_error",
"compilation_limit",
"cancelled",
"system_error"
],
"title": "CompilationStatus",
"type": "string"
},
"CompiledProgram": {
"additionalProperties": false,
"description": "Sealed file identities bound to a particular source, runtime and lease attempt.",
"properties": {
"binding": {
"$ref": "#/$defs/LeaseBinding"
},
"files": {
"items": {
"$ref": "#/$defs/ProgramFile"
},
"maxItems": 256,
"minItems": 1,
"title": "Files",
"type": "array"
},
"kind": {
"$ref": "#/$defs/ProgramKind"
},
"source": {
"$ref": "#/$defs/ArtifactReference"
},
"toolchain": {
"$ref": "#/$defs/RuntimeIdentity"
}
},
"required": [
"kind",
"binding",
"source",
"toolchain",
"files"
],
"title": "CompiledProgram",
"type": "object"
},
"FailureCode": {
"enum": [
"artifact_unavailable",
"sandbox_unavailable",
"cleanup_failed",
"lease_lost",
"job_deadline"
],
"title": "FailureCode",
"type": "string"
},
"LeaseBinding": {
"additionalProperties": false,
"properties": {
"attempt_id": {
"format": "uuid",
"title": "Attempt Id",
"type": "string"
},
"evaluation_id": {
"format": "uuid",
"title": "Evaluation Id",
"type": "string"
},
"fencing_version": {
"maximum": 9223372036854775807,
"minimum": 1,
"title": "Fencing Version",
"type": "integer"
},
"worker_id": {
"format": "uuid",
"title": "Worker Id",
"type": "string"
}
},
"required": [
"evaluation_id",
"attempt_id",
"worker_id",
"fencing_version"
],
"title": "LeaseBinding",
"type": "object"
},
"ProgramFile": {
"additionalProperties": false,
"properties": {
"artifact": {
"$ref": "#/$defs/ArtifactReference"
},
"path": {
"maxLength": 128,
"minLength": 1,
"title": "Path",
"type": "string"
}
},
"required": [
"path",
"artifact"
],
"title": "ProgramFile",
"type": "object"
},
"ProgramKind": {
"enum": [
"native_executable",
"python_source",
"java_classes"
],
"title": "ProgramKind",
"type": "string"
},
"RuntimeIdentity": {
"additionalProperties": false,
"description": "An exact identity matched against trusted configuration, never a host path.",
"properties": {
"image_digest": {
"maxLength": 64,
"minLength": 64,
"pattern": "^[0-9a-f]{64}$",
"title": "Image Digest",
"type": "string"
},
"image_kind": {
"enum": [
"linux_gnu_cxx_x86_64_v1",
"linux_cpython313_x86_64_v1",
"linux_openjdk21_x86_64_v1"
],
"title": "Image Kind",
"type": "string"
},
"image_revision": {
"maximum": 4294967295,
"minimum": 1,
"title": "Image Revision",
"type": "integer"
},
"runtime_version": {
"pattern": "^[A-Za-z0-9][A-Za-z0-9_.+-]{0,63}$",
"title": "Runtime Version",
"type": "string"
},
"toolchain_id": {
"pattern": "^[a-z0-9][a-z0-9_-]{0,63}$",
"title": "Toolchain Id",
"type": "string"
}
},
"required": [
"toolchain_id",
"image_kind",
"image_revision",
"image_digest",
"runtime_version"
],
"title": "RuntimeIdentity",
"type": "object"
},
"Verdict": {
"enum": [
"accepted",
"wrong_answer",
"compilation_error",
"runtime_error",
"time_limit_exceeded",
"memory_limit_exceeded",
"output_limit_exceeded",
"cancelled",
"system_error"
],
"title": "Verdict",
"type": "string"
}
},
"$id": "urn:kf-oj:judge-result:v2",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false,
"properties": {
"binding": {
"$ref": "#/$defs/LeaseBinding"
},
"cases": {
"items": {
"$ref": "#/$defs/CaseReport"
},
"maxItems": 256,
"title": "Cases",
"type": "array"
},
"cleanup_complete": {
"title": "Cleanup Complete",
"type": "boolean"
},
"compilation": {
"$ref": "#/$defs/CompilationReport"
},
"failure_code": {
"anyOf": [
{
"$ref": "#/$defs/FailureCode"
},
{
"type": "null"
}
]
},
"kind": {
"const": "judge.result",
"title": "Kind",
"type": "string"
},
"manifest_digest": {
"maxLength": 64,
"minLength": 64,
"pattern": "^[0-9a-f]{64}$",
"title": "Manifest Digest",
"type": "string"
},
"run_cpu_time_us": {
"anyOf": [
{
"maximum": 9223372036854775807,
"minimum": 0,
"type": "integer"
},
{
"type": "null"
}
],
"title": "Run Cpu Time Us"
},
"run_memory_peak_bytes": {
"anyOf": [
{
"maximum": 9223372036854775807,
"minimum": 0,
"type": "integer"
},
{
"type": "null"
}
],
"title": "Run Memory Peak Bytes"
},
"schema_version": {
"const": 2,
"title": "Schema Version",
"type": "integer"
},
"verdict": {
"$ref": "#/$defs/Verdict"
}
},
"required": [
"schema_version",
"kind",
"binding",
"manifest_digest",
"compilation",
"cases",
"verdict",
"cleanup_complete",
"failure_code",
"run_cpu_time_us",
"run_memory_peak_bytes"
],
"title": "JudgementReport",
"type": "object"
}
@@ -0,0 +1,94 @@
{
"$comment": "Structural schema; also run versioned model validation for cross-field budgets, result consistency, UUID canonical form and UTF-8 byte limits.",
"$defs": {
"LeaseBinding": {
"additionalProperties": false,
"properties": {
"attempt_id": {
"format": "uuid",
"title": "Attempt Id",
"type": "string"
},
"evaluation_id": {
"format": "uuid",
"title": "Evaluation Id",
"type": "string"
},
"fencing_version": {
"maximum": 9223372036854775807,
"minimum": 1,
"title": "Fencing Version",
"type": "integer"
},
"worker_id": {
"format": "uuid",
"title": "Worker Id",
"type": "string"
}
},
"required": [
"evaluation_id",
"attempt_id",
"worker_id",
"fencing_version"
],
"title": "LeaseBinding",
"type": "object"
}
},
"$id": "urn:kf-oj:judge-stage:v1",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false,
"properties": {
"binding": {
"$ref": "#/$defs/LeaseBinding"
},
"case_index": {
"anyOf": [
{
"exclusiveMaximum": 256,
"minimum": 0,
"type": "integer"
},
{
"type": "null"
}
],
"title": "Case Index"
},
"kind": {
"const": "judge.stage",
"title": "Kind",
"type": "string"
},
"schema_version": {
"const": 1,
"title": "Schema Version",
"type": "integer"
},
"sequence": {
"maximum": 9223372036854775807,
"minimum": 1,
"title": "Sequence",
"type": "integer"
},
"stage": {
"enum": [
"compiling",
"running"
],
"title": "Stage",
"type": "string"
}
},
"required": [
"schema_version",
"kind",
"binding",
"sequence",
"stage",
"case_index"
],
"title": "StageReport",
"type": "object"
}
@@ -0,0 +1,106 @@
{
"$comment": "Multi-runtime protocol. Also validate runtime/source/attempt bindings, startup and node budgets, canonical bounded class sets and result consistency.",
"$defs": {
"LeaseBinding": {
"additionalProperties": false,
"properties": {
"attempt_id": {
"format": "uuid",
"title": "Attempt Id",
"type": "string"
},
"evaluation_id": {
"format": "uuid",
"title": "Evaluation Id",
"type": "string"
},
"fencing_version": {
"maximum": 9223372036854775807,
"minimum": 1,
"title": "Fencing Version",
"type": "integer"
},
"worker_id": {
"format": "uuid",
"title": "Worker Id",
"type": "string"
}
},
"required": [
"evaluation_id",
"attempt_id",
"worker_id",
"fencing_version"
],
"title": "LeaseBinding",
"type": "object"
},
"PreparationKind": {
"enum": [
"compile",
"syntax_check"
],
"title": "PreparationKind",
"type": "string"
}
},
"$id": "urn:kf-oj:judge-stage:v2",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false,
"properties": {
"binding": {
"$ref": "#/$defs/LeaseBinding"
},
"case_index": {
"anyOf": [
{
"exclusiveMaximum": 256,
"minimum": 0,
"type": "integer"
},
{
"type": "null"
}
],
"title": "Case Index"
},
"kind": {
"const": "judge.stage",
"title": "Kind",
"type": "string"
},
"preparation_kind": {
"$ref": "#/$defs/PreparationKind"
},
"schema_version": {
"const": 2,
"title": "Schema Version",
"type": "integer"
},
"sequence": {
"maximum": 9223372036854775807,
"minimum": 1,
"title": "Sequence",
"type": "integer"
},
"stage": {
"enum": [
"compiling",
"running"
],
"title": "Stage",
"type": "string"
}
},
"required": [
"schema_version",
"kind",
"binding",
"sequence",
"stage",
"case_index",
"preparation_kind"
],
"title": "StageReport",
"type": "object"
}
@@ -0,0 +1,69 @@
{
"$comment": "Structural schema; also run versioned model validation for cross-field budgets, result consistency, UUID canonical form and UTF-8 byte limits.",
"$defs": {
"LeaseBinding": {
"additionalProperties": false,
"properties": {
"attempt_id": {
"format": "uuid",
"title": "Attempt Id",
"type": "string"
},
"evaluation_id": {
"format": "uuid",
"title": "Evaluation Id",
"type": "string"
},
"fencing_version": {
"maximum": 9223372036854775807,
"minimum": 1,
"title": "Fencing Version",
"type": "integer"
},
"worker_id": {
"format": "uuid",
"title": "Worker Id",
"type": "string"
}
},
"required": [
"evaluation_id",
"attempt_id",
"worker_id",
"fencing_version"
],
"title": "LeaseBinding",
"type": "object"
}
},
"$id": "urn:kf-oj:judge-stopped:v1",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false,
"properties": {
"binding": {
"$ref": "#/$defs/LeaseBinding"
},
"cleanup_complete": {
"title": "Cleanup Complete",
"type": "boolean"
},
"kind": {
"const": "judge.stopped",
"title": "Kind",
"type": "string"
},
"schema_version": {
"const": 1,
"title": "Schema Version",
"type": "integer"
}
},
"required": [
"schema_version",
"kind",
"binding",
"cleanup_complete"
],
"title": "StoppedReport",
"type": "object"
}
@@ -0,0 +1,69 @@
{
"$comment": "Multi-runtime protocol. Also validate runtime/source/attempt bindings, startup and node budgets, canonical bounded class sets and result consistency.",
"$defs": {
"LeaseBinding": {
"additionalProperties": false,
"properties": {
"attempt_id": {
"format": "uuid",
"title": "Attempt Id",
"type": "string"
},
"evaluation_id": {
"format": "uuid",
"title": "Evaluation Id",
"type": "string"
},
"fencing_version": {
"maximum": 9223372036854775807,
"minimum": 1,
"title": "Fencing Version",
"type": "integer"
},
"worker_id": {
"format": "uuid",
"title": "Worker Id",
"type": "string"
}
},
"required": [
"evaluation_id",
"attempt_id",
"worker_id",
"fencing_version"
],
"title": "LeaseBinding",
"type": "object"
}
},
"$id": "urn:kf-oj:judge-stopped:v2",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false,
"properties": {
"binding": {
"$ref": "#/$defs/LeaseBinding"
},
"cleanup_complete": {
"title": "Cleanup Complete",
"type": "boolean"
},
"kind": {
"const": "judge.stopped",
"title": "Kind",
"type": "string"
},
"schema_version": {
"const": 2,
"title": "Schema Version",
"type": "integer"
}
},
"required": [
"schema_version",
"kind",
"binding",
"cleanup_complete"
],
"title": "StoppedReport",
"type": "object"
}
+124
View File
@@ -19,3 +19,127 @@ API 存活检查和前端预览不依赖这些容器,可以先独立启动。
官方镜像说明:[PostgreSQL](https://hub.docker.com/_/postgres)、
[RabbitMQ](https://hub.docker.com/_/rabbitmq)。
### 单机 helper 工厂
`kf-sandbox factory --config /etc/kf-judge/runtime.json` 是 Root 服务入口;
普通 Worker 使用配置中 `state_root/factory/control.sock` 的普通用户客户端。
`deploy/kf-sandbox-factory.service` 给出有限服务预算示例,安装前由可信管理员
核对固定二进制、不可写 GNU 镜像/配置、Worker UID/GID,以及**与服务自身 cgroup
分离**的 Root 所有任务委派根。工厂实际读回服务 cgroup 限额,应用/读回任务总预算;
每次编译和测试点仍须通过已有完整隔离门控。服务文件本身不证明这些条件成立。
工厂只接收有界 run ID,无排队:槽位满、重复 ID 或未知字段立即拒绝。
每次申请使用全新单线程 helper,固定继承已认证的连接与工厂生命线;
额外控制帧、断开或生命线消失会终止整棵任务树。ID 先写入有限持久记录
(最多 4096 个,跨重启拒绝重放);容量用尽需要管理员在所有任务停止、已核对
回收后安排新记录周期,不自动删除记录。清理失败保留身份租约并停止新分配;
重启只根据已记录归属和 inode 回收,未知/外来内容保留并拒绝启动。
`kf-worker judge-submission` 是受控本地适配入口。常驻 `kf-worker serve --config`
使用有界 AMQP 通知和经过服务身份认证的 Coordinator claim;配置与依赖见
[判题工程说明](../judge/README.md#单机持久化协调与常驻-worker)。
它验证不可变清单摘要,C17/C++20 仅映射到固定 GNU 命令,编译一次;
成功产物经完整清理后封印并计算 SHA-256,每点分别申请 helper、stdin 与工作区。
期望值逐点读取,留在比较端,不传给工厂/用户文件系统。
新增锁定 `sha2 0.10.9` 仅供 Worker 验证输入及编译产物摘要;
Sandbox 同版本仅为独立 VM 测试数据生成的开发依赖,Engine 不依赖加密或 OS 库。
### 单机提交服务的启动与停止
1. Backend Alembic 迁移;创建分别仅继承平台/协调器 NOLOGIN 角色的专用登录账号,
平台角色只可插入初始评测,执行状态仅由 Coordinator 写入。
2. 普通平台服务持有私有工件目录;使用应用服务导入不可变题目和创建测试账号。
3. 验证固定 GNU 镜像、Root 配置及工厂服务/任务 cgroup 的真实上限,再启动工厂。
4. 独立启动 API、Coordinator、outbox publisher 和普通 Worker;单机 HTTP/AMQP
仅数值 loopback,PostgreSQL loopback/Unix socket。任何入口都不自动创建表。
5. Coordinator readiness 验证数据库和权限;仍需独立 VM 验证真实执行门控。
Root 工厂不可持有平台/MQ/数据库口令,用户运行镜像不含服务文件。
6. 先向 Worker 发送 SIGTERM,使当前任务取消/清理,再停止 publisher/API/Coordinator,
最后停止工厂和本地依赖。强制死亡后的旧 attempt 必须撤销并取得实际停止证明;
`cleanup_failed` 不等于普通终态,先核对保留资源再由管理员恢复工厂/Worker。
平台配置私有 `KF_OJ_ARTIFACT_ROOT`、有界字节/文件容量以及
`KF_OJ_JUDGE_ADMISSION`(完整可信 JSON);其工具链、编译/运行上限、总体作业期限、
并发/prefetch/retry 必须与 Coordinator/Worker 一致。单机使用并发1、prefetch1,
不能仅提高 Worker 数量绕过节点容量。源码最多256KiB,测试点最多256个;
stdout/stderr 各不超过 file_bytes,合计不超过 workdir_bytes。
迁移账号与平台账号分开配置,命令均在 `backend/` 运行:
```sh
uv run --locked alembic upgrade head # 私有迁移账号,仅执行受控 schema 操作
uv run --locked kf-accounts create local_user # 换成平台角色;终端读取密码并二次确认
uv run --locked kf-problems /srv/private-import
uv run --locked uvicorn kf_oj.api.app:create_app --factory --host 127.0.0.1 --port 8000 \
--no-proxy-headers --limit-concurrency 32 --backlog 64
uv run --locked python -m kf_oj.infra.messaging.publisher
```
每个账号操作/导入目录都由管理员准备,不从浏览器接受私有路径。导入格式见
[题目模块](../backend/src/kf_oj/modules/problems/README.md),不可变版本以 UUID
在题库/题面链接和提交请求中传递。更新题目需导入新版本,不覆盖已有提交的题目版本。
临时验收使用合成数据和测试口令,实际业务数据与服务配置都保存在仓库外的私有目录。
取消另需 API 私有配置 `KF_OJ_COORDINATOR_URL=http://127.0.0.1:8100` 与
`KF_OJ_COORDINATOR_TOKEN`(Coordinator 的 platform_token)。所有服务令牌私下生成,
不可互用或放进前端环境。Coordinator/Worker 私有 JSON 为0600、单链接、有界文件;
Root 镜像/配置/二进制由 root 持有,任务委派子树独立于监督器自身 cgroup。
运行前实际读回整体节点、服务和任务 memory/swap/pids/CPU/文件/运行期限上限,
并通过真实隔离门控;配置解析、readiness、存活探测都不能替代执行验收。
前端从 `frontend/` 启动 `pnpm dev`,默认127.0.0.1:5173;平台明确允许相同浏览器
Origin,`/api` 同源代理。生产需另行配置 HTTPS、安全 cookie、SPA 回退与 `/api`
反向代理,本仓库不自动部署。用户以受控账号登录,在题面选择支持语言、提交源码,
随后查看同一 submission UUID 的持久化状态。等待/取消中/系统失败都不等于成功成绩。
broker 暂停时 API 仍能持久化 queued/outbox;恢复后 publisher confirm 与 Coordinator
补偿重新投递。Worker/Coordinator 重启使用原私有配置、工件、日志与 DB,不删除
attempt 或强改状态。旧租约不能覆盖新结果,有限基础设施重试最终收敛。
容量满时返回429或503;先确认待执行/清理状态,恢复服务或由管理员扩充经过核对的
存储容量,不删除仍被提交/题目引用的工件。`kf-artifacts --limit 100` 仅回收满足年龄
要求的无引用工件和已标记删除项;它不会通过删除用户源码来腾出容量。
恢复 Root 必须核对 journal/inode/实际排空;未知资源保留,管理员处理自己确认的
故障来源后再重启。不要递归清空任务目录,也不要把 cleanup_failed 改成已成功。
这些入口和顺序不表示生产部署已完成。隔离/超限与用户程序仅在无 NIC、无宿主共享
挂载、无业务凭据且整体上限已读回的独立测试 VM 内验收。开发 Compose 仍只提供可信
数据库和 broker,不在宿主或业务服务器运行用户代码。workflow 如需启用由指定
GitHub 私有仓库承载,并设置有限并发;Gitea 只保存逐阶段工程提交。
## 四语言节点的受控升级
新增公开示例 `judge.admission.example.json` 为显式 v1/v2 注册表,
`sandbox.managed.runtime.example.json` 为一槽位的共享 Root 配置;原 v1 示例保持原语义。
所有 `aaaaaaaa…` 镜像摘要均是无权限占位值,不能当作已验镜像。管理员制作固定镜像后,
对 manifest.json 的实际字节计算 SHA-256,逐文件验证版本/字节/权限,并将相同完整身份
写入准入注册表与 Root runtime_images。Root 镜像路径仅在私有可信配置中保存。
配置解析通过或 check-runtime 可见仍不允许跳过实际隔离门控。
1. 停止创建新提交/消费投递,向所有普通 Worker 发 SIGTERM;保留 DB、清单、工件和 journal。
2. 取得工厂实际 stopped/cleanup_complete 证明,读回空任务 cgroup 和 owned 资源;
未排空或存在未知资源时停止升级,保留证据,不递归清空任务目录。
3. 使用独立迁移账号执行 Backend Alembic upgrade;新语言数据存在时旧迁移降级会拒绝,
不删除数据以实现回退。平台与 Coordinator 继续使用各自最小权限角色,Worker 无 DB 账号。
4. 安装固定版本的 Backend/Coordinator/Worker/Sandbox;同时更新三个普通服务的同一
`judge_admission`/`admission` 注册表,Root 配置单独由管理员持有。旧消费者不混用新协议。
5. 为整体节点、控制面各服务、工厂和任务设置有限 CPU/memory/swap/pids/文件/期限,
读回内核值。节点至少预留一份512MiB编译任务以及独立控制面;Java运行示例总预算384MiB,
Xmx不能代替整体预算。并发1/prefetch1,共享UID池,不因第二Worker存在而扩大节点槽位。
6. 在独立无NIC、无宿主共享挂载、无业务凭据 VM 用合成题目验证四语言、故障与实际排空,
再恢复工厂/Coordinator/API/publisher/Worker。生产启用需单独安排,仓库不自动部署。
请求不在线安装 Python/Java 依赖,不运行 pip/uvx,不接收自定义解释器参数/JAR/agents。
Worker 私有配置可设置 `control_timeout_ms`(1000–5000,缺省2000)作为 Coordinator
单次控制请求的期限。资源受限节点应依据实际领取/续租/停止耗时选择,并核对租约和
停止恢复;这不改变学生阶段/作业期限、重试次数或工件下载的固定2秒期限,连接仍限1秒。
Python main.py 仅标准库;Java Main.java 使用默认包 Main,最多256个有界编译class。
题目语言与预算以公开版本配置为准,旧题目和草稿保留,Rust继续明确不支持。
新版检查阶段CE/运行RE/真实限额与SystemError的区别见 [判题说明](../judge/README.md#python--java-与明确协议版本)。
停止/故障恢复沿原单机流程:停止Worker使任务全树排空,旧租约/report拒绝后恢复;
publisher/broker中断保留outbox,有限重试而非清空记录。恢复旧软件前应先排空v2任务,
保留不可变v2版本和报告供兼容消费者读取;不能把旧配置静默转换为宽松新能力。
这些公开配置不含业务令牌。数据库/AMQP/服务令牌分别在仓库外私下生成,Root不持有它们。
需要CI时只在GitHub私有仓库 `KiriAky107/KF-OnlineJudge` 运行并设置有限并发,Gitea保存源码。
+185
View File
@@ -0,0 +1,185 @@
{
"schema_version": 2,
"legacy": {
"max_concurrent_jobs": 1,
"prefetch": 1,
"max_retries": 2,
"max_job_wall_time_us": 60000000,
"max_package_bytes": 67108864,
"compile_ceiling": {
"cpu_time_us": 10000000,
"wall_time_us": 30000000,
"memory_bytes": 536870912,
"stack_bytes": 16777216,
"processes": 32,
"open_files": 128,
"stdout_bytes": 1048576,
"stderr_bytes": 1048576,
"file_bytes": 67108864,
"workdir_bytes": 268435456
},
"run_ceiling": {
"cpu_time_us": 1000000,
"wall_time_us": 3000000,
"memory_bytes": 67108864,
"stack_bytes": 8388608,
"processes": 8,
"open_files": 64,
"stdout_bytes": 1048576,
"stderr_bytes": 1048576,
"file_bytes": 16777216,
"workdir_bytes": 67108864
},
"toolchain": {
"toolchain_id": "gnu-cxx-v1",
"image_kind": "linux_gnu_cxx_x86_64_v1",
"image_revision": 1
}
},
"multi_runtime": {
"max_concurrent_jobs": 1,
"prefetch": 1,
"max_retries": 2,
"max_job_wall_time_us": 60000000,
"max_package_bytes": 67108864,
"schema_version": 2,
"capabilities": [
{
"language": "c17",
"toolchain": {
"toolchain_id": "gnu-cxx-v1",
"image_kind": "linux_gnu_cxx_x86_64_v1",
"image_revision": 1,
"image_digest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"runtime_version": "gnu-cxx-v1"
},
"compile_ceiling": {
"cpu_time_us": 10000000,
"wall_time_us": 30000000,
"memory_bytes": 536870912,
"stack_bytes": 16777216,
"processes": 32,
"open_files": 128,
"stdout_bytes": 1048576,
"stderr_bytes": 1048576,
"file_bytes": 67108864,
"workdir_bytes": 268435456
},
"run_ceiling": {
"cpu_time_us": 1000000,
"wall_time_us": 3000000,
"memory_bytes": 67108864,
"stack_bytes": 8388608,
"processes": 8,
"open_files": 64,
"stdout_bytes": 1048576,
"stderr_bytes": 1048576,
"file_bytes": 16777216,
"workdir_bytes": 67108864
}
},
{
"language": "cpp20",
"toolchain": {
"toolchain_id": "gnu-cxx-v1",
"image_kind": "linux_gnu_cxx_x86_64_v1",
"image_revision": 1,
"image_digest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"runtime_version": "gnu-cxx-v1"
},
"compile_ceiling": {
"cpu_time_us": 10000000,
"wall_time_us": 30000000,
"memory_bytes": 536870912,
"stack_bytes": 16777216,
"processes": 32,
"open_files": 128,
"stdout_bytes": 1048576,
"stderr_bytes": 1048576,
"file_bytes": 67108864,
"workdir_bytes": 268435456
},
"run_ceiling": {
"cpu_time_us": 1000000,
"wall_time_us": 3000000,
"memory_bytes": 67108864,
"stack_bytes": 8388608,
"processes": 8,
"open_files": 64,
"stdout_bytes": 1048576,
"stderr_bytes": 1048576,
"file_bytes": 16777216,
"workdir_bytes": 67108864
}
},
{
"language": "python313",
"toolchain": {
"toolchain_id": "cpython313-v1",
"image_kind": "linux_cpython313_x86_64_v1",
"image_revision": 1,
"image_digest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"runtime_version": "3.13.16"
},
"compile_ceiling": {
"cpu_time_us": 1000000,
"wall_time_us": 3000000,
"memory_bytes": 67108864,
"stack_bytes": 8388608,
"processes": 8,
"open_files": 64,
"stdout_bytes": 1048576,
"stderr_bytes": 1048576,
"file_bytes": 16777216,
"workdir_bytes": 67108864
},
"run_ceiling": {
"cpu_time_us": 1000000,
"wall_time_us": 3000000,
"memory_bytes": 67108864,
"stack_bytes": 8388608,
"processes": 8,
"open_files": 64,
"stdout_bytes": 1048576,
"stderr_bytes": 1048576,
"file_bytes": 16777216,
"workdir_bytes": 67108864
}
},
{
"language": "java21",
"toolchain": {
"toolchain_id": "openjdk21-v1",
"image_kind": "linux_openjdk21_x86_64_v1",
"image_revision": 1,
"image_digest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"runtime_version": "21.0.12.1+1-LTS"
},
"compile_ceiling": {
"cpu_time_us": 10000000,
"wall_time_us": 30000000,
"memory_bytes": 536870912,
"stack_bytes": 16777216,
"processes": 32,
"open_files": 128,
"stdout_bytes": 1048576,
"stderr_bytes": 1048576,
"file_bytes": 67108864,
"workdir_bytes": 268435456
},
"run_ceiling": {
"cpu_time_us": 1000000,
"wall_time_us": 3000000,
"memory_bytes": 402653184,
"stack_bytes": 8388608,
"processes": 32,
"open_files": 64,
"stdout_bytes": 1048576,
"stderr_bytes": 1048576,
"file_bytes": 16777216,
"workdir_bytes": 67108864
}
}
]
}
}
+28
View File
@@ -0,0 +1,28 @@
[Unit]
Description=KF bounded Linux helper factory
After=local-fs.target
[Service]
Type=exec
User=root
Group=root
# Trusted operator installs this exact binary/config and separate delegated
# task root. No caller-controlled executable, image or environment is accepted.
ExecStart=/usr/bin/env -i PATH=/usr/bin:/bin LC_ALL=C /usr/lib/kf-judge/kf-sandbox factory --config /etc/kf-judge/runtime.json --lifetime-seconds 3600
MemoryMax=256M
MemorySwapMax=0
TasksMax=32
CPUQuota=50%
LimitFSIZE=64M
RuntimeMaxSec=3700
TimeoutStopSec=30
# First stop the factory; guardians cancel live helpers through their normal
# cleanup path. Remaining processes are terminated at the finite stop deadline.
KillMode=mixed
Restart=no
UMask=0077
# Requires root namespace/mount/cgroup privileges. Do not set NoNewPrivileges or
# an inherited seccomp filter: the existing execution gate intentionally refuses.
[Install]
WantedBy=multi-user.target
+111
View File
@@ -0,0 +1,111 @@
{
"schema_version": 1,
"state_root": "/var/lib/kf-judge",
"tasks_root": "/var/lib/kf-judge/tasks",
"image_root": "/var/lib/kf-judge/images/gnu-cxx-x86_64-v1",
"image_kind": "linux_gnu_cxx_x86_64_v1",
"delegated_cgroup_root": "/sys/fs/cgroup/kf-judge.service",
"identities": {
"worker_uid": 1000,
"worker_gid": 1000,
"first_task_uid": 100000,
"first_task_gid": 100000,
"count": 1
},
"max_concurrent_runs": 1,
"per_task_ceiling": {
"cpu_time_us": 10000000,
"wall_time_us": 30000000,
"memory_bytes": 536870912,
"stack_bytes": 16777216,
"processes": 32,
"open_files": 128,
"stdout_bytes": 1048576,
"stderr_bytes": 1048576,
"file_bytes": 67108864,
"workdir_bytes": 268435456
},
"aggregate": {
"memory_bytes": 805306368,
"supervisor_memory_bytes": 268435456,
"processes": 64,
"supervisor_processes": 32,
"workdir_bytes": 402653184,
"supervisor_workdir_bytes": 67108864,
"cpu_period_us": 100000,
"cpu_quota_us": 150000,
"task_cpu_quota_us": 100000,
"supervisor_cpu_quota_us": 50000
},
"runtime_images": [
{
"image_root": "/var/lib/kf-judge/images/gnu-cxx-v2",
"image_kind": "linux_gnu_cxx_x86_64_v1",
"runtime_identity": {
"toolchain_id": "gnu-cxx-v1",
"image_kind": "linux_gnu_cxx_x86_64_v1",
"image_revision": 1,
"image_digest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"runtime_version": "gnu-cxx-v1"
},
"per_task_ceiling": {
"cpu_time_us": 10000000,
"wall_time_us": 30000000,
"memory_bytes": 536870912,
"stack_bytes": 16777216,
"processes": 32,
"open_files": 128,
"stdout_bytes": 1048576,
"stderr_bytes": 1048576,
"file_bytes": 67108864,
"workdir_bytes": 268435456
}
},
{
"image_root": "/var/lib/kf-judge/images/cpython313-v1",
"image_kind": "linux_cpython313_x86_64_v1",
"runtime_identity": {
"toolchain_id": "cpython313-v1",
"image_kind": "linux_cpython313_x86_64_v1",
"image_revision": 1,
"image_digest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"runtime_version": "3.13.16"
},
"per_task_ceiling": {
"cpu_time_us": 10000000,
"wall_time_us": 30000000,
"memory_bytes": 536870912,
"stack_bytes": 16777216,
"processes": 32,
"open_files": 128,
"stdout_bytes": 1048576,
"stderr_bytes": 1048576,
"file_bytes": 67108864,
"workdir_bytes": 268435456
}
},
{
"image_root": "/var/lib/kf-judge/images/openjdk21-v1",
"image_kind": "linux_openjdk21_x86_64_v1",
"runtime_identity": {
"toolchain_id": "openjdk21-v1",
"image_kind": "linux_openjdk21_x86_64_v1",
"image_revision": 1,
"image_digest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"runtime_version": "21.0.12.1+1-LTS"
},
"per_task_ceiling": {
"cpu_time_us": 10000000,
"wall_time_us": 30000000,
"memory_bytes": 536870912,
"stack_bytes": 16777216,
"processes": 32,
"open_files": 128,
"stdout_bytes": 1048576,
"stderr_bytes": 1048576,
"file_bytes": 67108864,
"workdir_bytes": 268435456
}
}
]
}
+40
View File
@@ -0,0 +1,40 @@
{
"schema_version": 1,
"state_root": "/var/lib/kf-judge",
"tasks_root": "/var/lib/kf-judge/tasks",
"image_root": "/var/lib/kf-judge/images/gnu-cxx-x86_64-v1",
"image_kind": "linux_gnu_cxx_x86_64_v1",
"delegated_cgroup_root": "/sys/fs/cgroup/kf-judge.service",
"identities": {
"worker_uid": 1000,
"worker_gid": 1000,
"first_task_uid": 100000,
"first_task_gid": 100000,
"count": 2
},
"max_concurrent_runs": 2,
"per_task_ceiling": {
"cpu_time_us": 10000000,
"wall_time_us": 30000000,
"memory_bytes": 536870912,
"stack_bytes": 16777216,
"processes": 32,
"open_files": 128,
"stdout_bytes": 1048576,
"stderr_bytes": 1048576,
"file_bytes": 67108864,
"workdir_bytes": 268435456
},
"aggregate": {
"memory_bytes": 1610612736,
"supervisor_memory_bytes": 268435456,
"processes": 128,
"supervisor_processes": 32,
"workdir_bytes": 1073741824,
"supervisor_workdir_bytes": 67108864,
"cpu_period_us": 100000,
"cpu_quota_us": 250000,
"task_cpu_quota_us": 100000,
"supervisor_cpu_quota_us": 50000
}
}
+25 -1
View File
@@ -1,12 +1,14 @@
# Vue 前端工程
Vue 3 + TypeScript + Vite + Element Plus,使用 Vue Router 与 Pinia。
当前页面是开发预览和实际 API 连接检查;没有模拟登录或模拟评测结果。
登录、已发布题库、题面、提交与个人成绩页面使用真实 API。训练、比赛、社区等
本轮以外页面仍为明确的开发预览,不提供这些尚未实现的业务能力。
```bash
pnpm install --frozen-lockfile
pnpm dev
pnpm typecheck
pnpm test
pnpm build
```
@@ -22,3 +24,25 @@ pnpm build
已有代码按实际需要落位;`components/` 和 `stores/` 是后续扩展位置,不放空状态实现。
API 类型更新执行根目录 `make contracts`;前端通过 `pnpm check:api` 拒绝过期类型。
登录凭据、模型密钥和数据库连接不能进入 `VITE_` 环境变量。
已有账号登录使用 HttpOnly/Strict cookie 与单独 CSRF cookie;401清除内存身份,
保护页面重新登录后继续。自助注册暂不开放。密码/会话不存入 localStorage。
浏览器与 API 使用同一 origin 的 `/api`,明确配置 `KF_OJ_BROWSER_ORIGINS`;
生产必须使用 HTTPS、安全 cookie 与独立部署的反向代理。
编辑器按题目版本和语言保留本地草稿,新增 Java 并保留原 C17/C++/Python/Rust 草稿。
可提交语言由服务端已发布版本的能力与语言配置决定,支持 C17/C++20、固定 CPython
3.13.16 的单文件 main.py/标准库,以及固定 OpenJDK 21.0.12.1+1-LTS 的默认包
Main.java/Main;Java 可包含受控的辅助、内部和匿名类。缺失或无效配置不能提交,
Rust 当前保留草稿且不可提交。Python 准备阶段只检查语法,不执行顶层代码;
Java 编译错误显示真实诊断,语言版本与预算取自服务端。请求不在线安装依赖。
提交凭据按账号/版本/内容 SHA-256 绑定,
POST前先保存 UUID,双击共用请求,网络超时/刷新重试复用原键;内容变化产生新键。
草稿和已保存凭据不因失败/退出而清除,本地存储不可用时拒绝发起无法稳定重试的提交。
详情和列表查询每2秒一次,每轮最多240次/10分钟;终态、错误或离页停止,隐藏页面
取消当前查询,恢复可见时继续原预算。路由变化/会话变化后的旧响应不能覆盖当前视图。
耗尽后可手动刷新;不会用预览成绩填补错误。取消受理与最终停止分别展示。
运行指标仅来自完整持久化报告,null显示“—”;诊断和源码按纯文本显示。
`pnpm test` 覆盖草稿、幂等重试、轮询期限/隐藏/终态/失效响应和401处理,
已纳入 `make check-frontend`;真实浏览器编译/判题另在受限独立 VM 验收。

Some files were not shown because too many files have changed in this diff Show More