Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2ad35865c0 | ||
|
|
62132324da |
@@ -57,7 +57,7 @@ format:
|
||||
cd frontend && $(PNPM) format
|
||||
|
||||
api:
|
||||
cd backend && $(UV) run --locked uvicorn kf_oj.api.app:create_app --factory --reload --host 127.0.0.1 --port 8000 --no-proxy-headers --limit-concurrency 32 --backlog 64
|
||||
cd backend && $(UV) run --locked uvicorn kf_oj.api.app:create_app --factory --reload --host 127.0.0.1 --port 8000
|
||||
|
||||
frontend:
|
||||
cd frontend && $(PNPM) dev
|
||||
|
||||
@@ -3,10 +3,8 @@
|
||||
以 Rust 自研判题与 Linux 沙箱、以 Python/FastAPI 提供平台服务的在线评测系统。
|
||||
前端使用 Vue 3、TypeScript、Element Plus 和 Vite。
|
||||
|
||||
当前版本为 **0.0.1**:沙箱已有受控隔离、C/C++ 编译及 native 执行;平台已实现真实身份、
|
||||
不可变题目、工件与幂等提交创建。事务 outbox、持久化 Coordinator、普通 Worker 和受控
|
||||
helper 工厂已接通 C/C++ 编译一次、多测试点评测;登录、题库、提交和个人成绩页面
|
||||
使用真实接口。训练、比赛、社区等扩展页面仍为开发预览。
|
||||
当前版本为 **0.0.1 工程骨架**:平台 API 和协调器提供存活接口,前端可检查实际 API 连接。
|
||||
沙箱隔离执行、判题、账号、题库、提交与学习业务尚未实现。
|
||||
|
||||
## 工程划分
|
||||
|
||||
@@ -27,8 +25,7 @@ Python 业务域分为 `identity`、`problems`、`submissions`、`community`、`
|
||||
|
||||
Rust 分为 `kf-contracts`、`kf-sandbox`、`kf-engine`、`kf-worker`、`kf-coordinator` 五个 crate。
|
||||
协调器拥有执行状态写入权,判题 Worker 不直接写评测数据库。
|
||||
沙箱执行入口每次核验隔离与限额,任何未验证配置继续拒绝执行。
|
||||
用户程序及隔离/超限验收只在独立、整体限额的无业务凭据测试 VM 内运行。
|
||||
沙箱尚未验证隔离能力,当前执行入口明确拒绝运行程序。
|
||||
|
||||
## 安装与检查
|
||||
|
||||
@@ -65,18 +62,11 @@ make sandbox-probe # 只读观察 Linux 环境,不创建隔离资源
|
||||
```
|
||||
|
||||
本地数据库与队列的启动步骤见 [部署说明](deploy/README.md)。
|
||||
业务表和权限分工由后端 Alembic 唯一管理,见 [迁移与权限说明](backend/migrations/README.md)。
|
||||
业务表和数据库 revision 尚未建立;迁移统一由后端 Alembic 管理。
|
||||
模型服务与第三方插件尚未接入。
|
||||
|
||||
开发中的规划文档仅在本地维护,不包含在远端仓库中。
|
||||
单机提交服务的配置、最小权限、停止恢复与独立 VM 验收要求见
|
||||
[判题说明](judge/README.md#单机持久化协调与常驻-worker)和[部署说明](deploy/README.md#单机提交服务的启动与停止)。
|
||||
当前普通题执行支持固定 GNU C17/C++20、CPython 3.13.16 和 OpenJDK 21.0.12.1+1-LTS,
|
||||
使用精确字节比较。Python 固定单文件 main.py 与标准库;Java 固定默认包 Main.java/Main,
|
||||
允许有界辅助/内部/匿名类。题目必须显式声明对应语言能力,旧题目不会自动获得新语言。
|
||||
新版源码与普通 Worker 的沙箱执行已在独立 VM 验收;完整平台/浏览器及故障验收按工程阶段
|
||||
分别记录,生产部署需另行进行。自助注册、自测、SPJ/交互题、Rust 用户程序、分布式调度
|
||||
和学习能力仍待后续实现。
|
||||
后续逐步实现沙箱、单机判题、分布式恢复、并行优化和学习功能。
|
||||
|
||||
## 许可证
|
||||
|
||||
|
||||
+1
-11
@@ -1,15 +1,5 @@
|
||||
KF_OJ_ENVIRONMENT=development
|
||||
KF_OJ_LOG_LEVEL=INFO
|
||||
# Dedicated platform login inherits only kf_oj_platform; replace development credentials.
|
||||
# Optional until database-backed use cases are implemented.
|
||||
KF_OJ_DATABASE_URL=postgresql+asyncpg://kf_oj:local-dev-only@127.0.0.1:5432/kf_oj
|
||||
KF_OJ_AMQP_URL=amqp://kf_oj:local-dev-only@127.0.0.1:5672/kf_oj
|
||||
KF_OJ_BROWSER_ORIGINS='["http://localhost:5173","http://localhost:8000","http://127.0.0.1:5173","http://127.0.0.1:8000"]'
|
||||
KF_OJ_SESSION_LIFETIME_SECONDS=43200
|
||||
# Required for submissions: private absolute artifact root and trusted JSON admission policy.
|
||||
# KF_OJ_ARTIFACT_ROOT=/srv/kf-platform/artifacts
|
||||
KF_OJ_ARTIFACT_MAX_BYTES=536870912
|
||||
KF_OJ_ARTIFACT_MAX_FILES=4096
|
||||
# KF_OJ_JUDGE_ADMISSION=<JSON matching the Coordinator and Worker policy>
|
||||
# Cancellation requires both; token equals Coordinator platform_token, never a Worker token.
|
||||
# KF_OJ_COORDINATOR_URL=http://127.0.0.1:8100
|
||||
# KF_OJ_COORDINATOR_TOKEN=<privately generated 64 lowercase hexadecimal characters>
|
||||
|
||||
+4
-26
@@ -15,33 +15,11 @@ make api
|
||||
make worker-check
|
||||
```
|
||||
|
||||
API 提供存活、真实登录/退出/当前用户、公开题目与有效 Worker 租约的清单/工件读取。
|
||||
配置数据库后在 lifespan 内组合模块,配置缺失时业务端口明确不可用,存活接口仍可用。
|
||||
学习 Worker 目前仅支持 `--check`;Rust 判题 Worker 使用单独的常驻消费入口。
|
||||
提交创建 API 在持久化源码后,同一事务写入初始评测和 outbox;按用户与幂等键
|
||||
防止重复创建,待评测总量和个人待评测数有固定上限。
|
||||
|
||||
待投递事件使用独立进程 `uv run --locked python -m kf_oj.infra.messaging.publisher`;
|
||||
配置 `KF_OJ_DATABASE_URL` 为专用平台角色,`KF_OJ_AMQP_URL` 为内部 broker。
|
||||
发布器一次只锁定一个到期事件;使用持久队列、消息及 mandatory publisher confirm,
|
||||
确认后才提交投递标记。锁过期允许重投,消息不是执行权,Coordinator claim 才是。
|
||||
队列最多 64 条、32 KiB,拒绝溢出,消息有效期 10 分钟;broker 不可用采用有界退避,
|
||||
不阻塞提交事务。日志仅固定错误代码,不打印源码、答案或连接口令。
|
||||
后台每次投递最多 12 秒,关闭连接最多 2 秒,SIGTERM 在当前有界操作结束后退出。
|
||||
账号/题目/工件命令及限制见对应模块 README;模型能力尚未接入。
|
||||
|
||||
个人提交查询 `/api/v1/submissions` 使用 `(created_at,id)` 降序游标,最多100条;
|
||||
`/{id}`、`/{id}/source` 仅允许提交本人读取,统一拒绝外来/不存在记录。
|
||||
成绩投影不包含租约、工件摘要、隐藏用例 ID/输入/答案或程序输出;编译诊断至多64KiB,
|
||||
缺失计量保留 null,未完成报告不展示成绩。身份/提交/内部响应均为 no-store。
|
||||
`POST /{id}/cancel` 验证本人、Origin、会话和 CSRF,再调用 Coordinator 接受取消;
|
||||
202仅表示已持久化意图,实际停止与资源回收由执行服务确认。
|
||||
配置 `KF_OJ_COORDINATOR_URL` 为数值 loopback HTTP origin,
|
||||
`KF_OJ_COORDINATOR_TOKEN` 与私有 Coordinator 的 `platform_token` 一致;
|
||||
缺少配置或通信失败返回503,不直接修改执行状态。新增运行依赖 `httpx` 仅负责这个
|
||||
3秒有界控制调用,禁用代理/重定向,不缓存或打印远端响应及内部凭据。
|
||||
API 当前只有 `/api/v1/health/live`,不连接数据库或消息队列。
|
||||
Worker 目前仅支持 `--check`,默认拒绝进入尚未实现的任务循环。
|
||||
日志、配置和迁移入口已建立;业务表、登录、判题投递和模型接入尚未实现。
|
||||
|
||||
Python 虚拟环境位于 `backend/.venv`,可在 `backend/` 直接使用 `uv sync --locked` 和 `uv run --locked`。
|
||||
数据库由 Alembic 唯一管理;业务模型在 `migrations/env.py` 中显式登记元数据。
|
||||
数据库由 Alembic 唯一管理;业务模型落地后在 `migrations/env.py` 中显式登记元数据。
|
||||
不要用 API 启动时的 `create_all` 代替迁移,也不要让 Rust 维护另一套迁移历史。
|
||||
配置示例在 `.env.example`;执行迁移前复制为 `.env` 并设置实际数据库连接。
|
||||
|
||||
@@ -1,23 +0,0 @@
|
||||
# 数据库所有权
|
||||
|
||||
Alembic 是唯一的 schema 入口。先对专用 PostgreSQL 18 数据库运行 `uv run --locked alembic
|
||||
upgrade head`,API 不在启动时创建表。迁移账号拥有该数据库对象并需要 `CREATEROLE`;它不作为
|
||||
任何运行服务账号。生产配置及登录角色由部署方提供,不在源码里设置默认数据库口令。
|
||||
|
||||
初始迁移建立两个受限的 `NOLOGIN` 权限组:`kf_oj_platform` 和 `kf_oj_coordinator`。
|
||||
平台组创建 submission / queued evaluation / outbox,不能指定或更新执行状态、attempt 或成绩。
|
||||
协调器组领取和更新评测、插入 attempt / 用例成绩及恢复通知,不能修改账号、题库或提交身份。
|
||||
Worker 没有数据库角色,也不需要数据库连接配置。
|
||||
|
||||
服务使用各自独立的普通 `LOGIN` 角色,分别加入对应权限组;登录角色不能拥有数据库/表、
|
||||
不能加入迁移账号或其他高权限组,且必须为 `NOSUPERUSER NOCREATEDB NOCREATEROLE NOBYPASSRLS`。
|
||||
初始迁移拒绝已有的高权限同名组和继承其他角色的同名组。应用启动时不运行 migration。
|
||||
降级删除本项目业务表与触发器,保留权限组及操作方配置的登录账号,避免删除集群级身份。
|
||||
|
||||
发布是一次单向转换:先在事务内创建版本和连续测试点,再设置 `published_at`。
|
||||
数据库触发器拒绝再次发布、修改已发布版本和追加/改写已发布测试点;评测通过复合外键
|
||||
永久绑定 submission 的版本,当前 attempt 和用例成绩也由复合外键绑定同一评测及版本。
|
||||
工件引用具有数据库生成的 `ready` 状态,引用存在时不能转为 `deleting` 或删除元数据。
|
||||
|
||||
真实数据库验收使用显式 `KF_OJ_TEST_POSTGRES` 管理连接。每个测试只创建和删除随机命名的
|
||||
`kf_oj_test_*` 数据库,不清空已有业务库。例行无数据库测试会明确跳过这些验收;跳过不算通过。
|
||||
@@ -5,19 +5,14 @@ from alembic import context
|
||||
from sqlalchemy import Connection
|
||||
|
||||
from kf_oj.core.settings import Settings
|
||||
from kf_oj.infra.artifacts import models as artifact_models # noqa: F401
|
||||
from kf_oj.infra.db.base import Base
|
||||
from kf_oj.infra.db.session import build_engine
|
||||
from kf_oj.infra.messaging import models as messaging_models # noqa: F401
|
||||
from kf_oj.modules.identity import models as identity_models # noqa: F401
|
||||
from kf_oj.modules.problems import models as problem_models # noqa: F401
|
||||
from kf_oj.modules.submissions import models as submission_models # noqa: F401
|
||||
|
||||
config = context.config
|
||||
if config.config_file_name is not None:
|
||||
fileConfig(config.config_file_name)
|
||||
|
||||
# Explicit composition root. Infra and domains do not import foreign ORM models.
|
||||
# Register each business module's ORM models here when its schema is implemented.
|
||||
target_metadata = Base.metadata
|
||||
|
||||
|
||||
|
||||
-588
@@ -1,588 +0,0 @@
|
||||
"""judging foundation and role boundaries
|
||||
|
||||
Revision ID: 0001_judging_foundation
|
||||
Revises:
|
||||
Create Date: 2026-10-09 00:57:34.004943
|
||||
"""
|
||||
|
||||
from collections.abc import Sequence
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
from sqlalchemy.dialects import postgresql
|
||||
|
||||
revision: str = "0001_judging_foundation"
|
||||
down_revision: str | None = None
|
||||
branch_labels: str | Sequence[str] | None = None
|
||||
depends_on: str | Sequence[str] | None = None
|
||||
|
||||
|
||||
def install_guards_and_permissions() -> None:
|
||||
# Serialize case insertion against the one-way publication transition.
|
||||
op.execute("""
|
||||
CREATE FUNCTION public.kf_immutable_case() RETURNS trigger LANGUAGE plpgsql
|
||||
SET search_path = pg_catalog, public AS $$
|
||||
DECLARE published timestamptz;
|
||||
BEGIN
|
||||
IF TG_OP = 'UPDATE' THEN
|
||||
RAISE EXCEPTION 'test cases are immutable' USING ERRCODE = '23514';
|
||||
END IF;
|
||||
SELECT published_at INTO published FROM public.problem_versions
|
||||
WHERE id = CASE WHEN TG_OP = 'DELETE' THEN OLD.version_id ELSE NEW.version_id END
|
||||
FOR UPDATE;
|
||||
IF published IS NOT NULL THEN
|
||||
RAISE EXCEPTION 'published test cases are immutable' USING ERRCODE = '23514';
|
||||
END IF;
|
||||
IF TG_OP = 'DELETE' THEN RETURN OLD; END IF;
|
||||
RETURN NEW;
|
||||
END $$
|
||||
""")
|
||||
op.execute("""
|
||||
CREATE TRIGGER kf_immutable_case BEFORE INSERT OR UPDATE OR DELETE ON public.problem_cases
|
||||
FOR EACH ROW EXECUTE FUNCTION public.kf_immutable_case()
|
||||
""")
|
||||
op.execute("""
|
||||
CREATE FUNCTION public.kf_immutable_version() RETURNS trigger LANGUAGE plpgsql
|
||||
SET search_path = pg_catalog, public AS $$
|
||||
DECLARE count_cases integer; last_position integer;
|
||||
BEGIN
|
||||
IF TG_OP = 'INSERT' THEN
|
||||
IF NEW.published_at IS NOT NULL THEN
|
||||
RAISE EXCEPTION 'publish after inserting cases' USING ERRCODE = '23514';
|
||||
END IF;
|
||||
RETURN NEW;
|
||||
END IF;
|
||||
IF TG_OP = 'DELETE' THEN
|
||||
IF OLD.published_at IS NOT NULL THEN
|
||||
RAISE EXCEPTION 'published version is immutable' USING ERRCODE = '23514';
|
||||
END IF;
|
||||
RETURN OLD;
|
||||
END IF;
|
||||
IF OLD.published_at IS NOT NULL OR NEW.published_at IS NULL
|
||||
OR (to_jsonb(NEW) - 'published_at') != (to_jsonb(OLD) - 'published_at') THEN
|
||||
RAISE EXCEPTION 'version may only be published once' USING ERRCODE = '23514';
|
||||
END IF;
|
||||
SELECT count(*), max(position) INTO count_cases, last_position
|
||||
FROM public.problem_cases WHERE version_id = OLD.id;
|
||||
IF count_cases NOT BETWEEN 1 AND 256 OR last_position != count_cases - 1 THEN
|
||||
RAISE EXCEPTION 'publication requires contiguous ordered cases'
|
||||
USING ERRCODE = '23514';
|
||||
END IF;
|
||||
RETURN NEW;
|
||||
END $$
|
||||
""")
|
||||
op.execute("""
|
||||
CREATE TRIGGER kf_immutable_version BEFORE INSERT OR UPDATE OR DELETE
|
||||
ON public.problem_versions
|
||||
FOR EACH ROW EXECUTE FUNCTION public.kf_immutable_version()
|
||||
""")
|
||||
op.execute("REVOKE ALL ON FUNCTION public.kf_immutable_case() FROM PUBLIC")
|
||||
op.execute("REVOKE ALL ON FUNCTION public.kf_immutable_version() FROM PUBLIC")
|
||||
# Migration owner needs CREATEROLE; runtime logins are separate, unprivileged group members.
|
||||
for role in ("kf_oj_platform", "kf_oj_coordinator"):
|
||||
op.execute(f"""
|
||||
DO $$ BEGIN
|
||||
IF NOT EXISTS (SELECT 1 FROM pg_roles WHERE rolname = '{role}') THEN
|
||||
CREATE ROLE {role} NOLOGIN NOSUPERUSER NOCREATEDB NOCREATEROLE NOBYPASSRLS;
|
||||
ELSIF EXISTS (SELECT 1 FROM pg_roles WHERE rolname = '{role}' AND
|
||||
(rolcanlogin OR rolsuper OR rolcreatedb OR rolcreaterole OR rolbypassrls))
|
||||
OR EXISTS (SELECT 1 FROM pg_auth_members m JOIN pg_roles r ON r.oid=m.member
|
||||
WHERE r.rolname='{role}') THEN
|
||||
RAISE EXCEPTION 'unsafe existing runtime role';
|
||||
END IF;
|
||||
END $$
|
||||
""")
|
||||
tables = (
|
||||
"accounts,sessions,login_throttles,artifacts,problems,problem_versions,problem_cases,"
|
||||
"submissions,evaluations,judge_attempts,case_results,outbox"
|
||||
)
|
||||
op.execute(f"REVOKE ALL ON {tables} FROM PUBLIC, kf_oj_platform, kf_oj_coordinator")
|
||||
op.execute("REVOKE CREATE ON SCHEMA public FROM PUBLIC")
|
||||
op.execute("GRANT USAGE ON SCHEMA public TO kf_oj_platform, kf_oj_coordinator")
|
||||
op.execute(f"GRANT SELECT ON {tables} TO kf_oj_platform")
|
||||
op.execute("GRANT INSERT,UPDATE,DELETE ON accounts,sessions,login_throttles TO kf_oj_platform")
|
||||
op.execute("GRANT INSERT,SELECT,DELETE ON artifacts TO kf_oj_platform")
|
||||
op.execute("GRANT UPDATE(state) ON artifacts TO kf_oj_platform")
|
||||
op.execute(
|
||||
"GRANT INSERT ON problems,problem_versions,problem_cases,submissions,outbox "
|
||||
"TO kf_oj_platform"
|
||||
)
|
||||
op.execute("GRANT UPDATE(active_version_id,enabled) ON problems TO kf_oj_platform")
|
||||
op.execute("GRANT UPDATE(published_at) ON problem_versions TO kf_oj_platform")
|
||||
op.execute("GRANT INSERT(id,submission_id,problem_version_id) ON evaluations TO kf_oj_platform")
|
||||
op.execute(
|
||||
"GRANT UPDATE(failures,next_attempt_at,locked_until,lock_owner,delivered_at,"
|
||||
"last_error_code) ON outbox TO kf_oj_platform"
|
||||
)
|
||||
op.execute(
|
||||
"GRANT SELECT ON problems,problem_versions,problem_cases,artifacts,submissions,"
|
||||
"evaluations,judge_attempts,case_results,outbox TO kf_oj_coordinator"
|
||||
)
|
||||
op.execute(
|
||||
"GRANT UPDATE(state,fencing_version,attempts_count,current_attempt_id,not_before,"
|
||||
"cancel_requested_at,updated_at,verdict,failure_code,final_report,final_report_digest) "
|
||||
"ON evaluations TO kf_oj_coordinator"
|
||||
)
|
||||
op.execute("GRANT INSERT ON judge_attempts,case_results TO kf_oj_coordinator")
|
||||
op.execute(
|
||||
"GRANT UPDATE(expires_at,revoked_at,sequence,stage,case_index,report,report_digest,"
|
||||
"cleanup_complete) ON judge_attempts TO kf_oj_coordinator"
|
||||
)
|
||||
# Recovery may enqueue the same immutable notification, but cannot change identity fields.
|
||||
op.execute("GRANT INSERT ON outbox TO kf_oj_coordinator")
|
||||
op.execute("GRANT UPDATE(next_attempt_at,delivered_at) ON outbox TO kf_oj_coordinator")
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.create_table(
|
||||
"accounts",
|
||||
sa.Column("id", sa.Uuid(), nullable=False),
|
||||
sa.Column("username", sa.String(length=24), nullable=False),
|
||||
sa.Column("password_hash", sa.Text(), nullable=False),
|
||||
sa.Column("enabled", sa.Boolean(), server_default="true", nullable=False),
|
||||
sa.Column(
|
||||
"created_at",
|
||||
sa.DateTime(timezone=True),
|
||||
server_default=sa.text("now()"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.CheckConstraint(
|
||||
"username ~ '^[a-z0-9_]{3,24}$'", name=op.f("ck_accounts_canonical_username")
|
||||
),
|
||||
sa.CheckConstraint(
|
||||
"char_length(password_hash) BETWEEN 40 AND 256",
|
||||
name=op.f("ck_accounts_password_hash_size"),
|
||||
),
|
||||
sa.PrimaryKeyConstraint("id", name=op.f("pk_accounts")),
|
||||
sa.UniqueConstraint("username", name=op.f("uq_accounts_username")),
|
||||
)
|
||||
op.create_table(
|
||||
"artifacts",
|
||||
sa.Column("digest", sa.String(length=64), nullable=False),
|
||||
sa.Column("bytes", sa.BigInteger(), nullable=False),
|
||||
sa.Column("state", sa.String(length=8), server_default="ready", nullable=False),
|
||||
sa.Column(
|
||||
"created_at",
|
||||
sa.DateTime(timezone=True),
|
||||
server_default=sa.text("now()"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.CheckConstraint("digest ~ '^[0-9a-f]{64}$'", name=op.f("ck_artifacts_digest")),
|
||||
sa.CheckConstraint("state IN ('ready', 'deleting')", name=op.f("ck_artifacts_state")),
|
||||
sa.CheckConstraint("bytes BETWEEN 0 AND 67108864", name=op.f("ck_artifacts_bytes")),
|
||||
sa.PrimaryKeyConstraint("digest", name=op.f("pk_artifacts")),
|
||||
sa.UniqueConstraint("digest", "state", name=op.f("uq_artifacts_digest")),
|
||||
)
|
||||
op.create_table(
|
||||
"login_throttles",
|
||||
sa.Column("bucket_hash", sa.String(length=64), nullable=False),
|
||||
sa.Column("attempts", sa.Integer(), nullable=False),
|
||||
sa.Column("reset_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.CheckConstraint(
|
||||
"bucket_hash ~ '^[0-9a-f]{64}$'", name=op.f("ck_login_throttles_bucket_digest")
|
||||
),
|
||||
sa.CheckConstraint(
|
||||
"attempts BETWEEN 1 AND 10", name=op.f("ck_login_throttles_attempt_budget")
|
||||
),
|
||||
sa.PrimaryKeyConstraint("bucket_hash", name=op.f("pk_login_throttles")),
|
||||
)
|
||||
op.create_index("ix_login_throttles_reset", "login_throttles", ["reset_at"], unique=False)
|
||||
op.create_table(
|
||||
"problems",
|
||||
sa.Column("id", sa.Uuid(), nullable=False),
|
||||
sa.Column("slug", sa.String(length=64), nullable=False),
|
||||
sa.Column("active_version_id", sa.Uuid(), nullable=True),
|
||||
sa.Column("enabled", sa.Boolean(), server_default="true", nullable=False),
|
||||
sa.Column(
|
||||
"created_at",
|
||||
sa.DateTime(timezone=True),
|
||||
server_default=sa.text("now()"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.CheckConstraint("slug ~ '^[a-z0-9][a-z0-9_-]{0,63}$'", name=op.f("ck_problems_slug")),
|
||||
sa.PrimaryKeyConstraint("id", name=op.f("pk_problems")),
|
||||
sa.UniqueConstraint("slug", name=op.f("uq_problems_slug")),
|
||||
)
|
||||
op.create_table(
|
||||
"problem_versions",
|
||||
sa.Column("id", sa.Uuid(), nullable=False),
|
||||
sa.Column("problem_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("revision", sa.Integer(), nullable=False),
|
||||
sa.Column("title", sa.String(length=160), nullable=False),
|
||||
sa.Column("statement", sa.Text(), nullable=False),
|
||||
sa.Column("languages", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
|
||||
sa.Column("toolchain", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
|
||||
sa.Column("compile_limits", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
|
||||
sa.Column("run_limits", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
|
||||
sa.Column("job_wall_time_us", sa.BigInteger(), nullable=False),
|
||||
sa.Column("comparison", sa.String(length=16), server_default="exact_bytes", nullable=False),
|
||||
sa.Column("public_samples", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
|
||||
sa.Column("published_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column(
|
||||
"created_at",
|
||||
sa.DateTime(timezone=True),
|
||||
server_default=sa.text("now()"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.CheckConstraint(
|
||||
"comparison = 'exact_bytes'", name=op.f("ck_problem_versions_comparison")
|
||||
),
|
||||
sa.CheckConstraint(
|
||||
"char_length(title) BETWEEN 1 AND 160", name=op.f("ck_problem_versions_title_size")
|
||||
),
|
||||
sa.CheckConstraint(
|
||||
"job_wall_time_us BETWEEN 1000 AND 3600000000",
|
||||
name=op.f("ck_problem_versions_job_wall"),
|
||||
),
|
||||
sa.CheckConstraint(
|
||||
"octet_length(statement) BETWEEN 1 AND 262144",
|
||||
name=op.f("ck_problem_versions_statement_bytes"),
|
||||
),
|
||||
sa.CheckConstraint("revision > 0", name=op.f("ck_problem_versions_revision")),
|
||||
sa.ForeignKeyConstraint(
|
||||
["problem_id"], ["problems.id"], name=op.f("fk_problem_versions_problem_id_problems")
|
||||
),
|
||||
sa.PrimaryKeyConstraint("id", name=op.f("pk_problem_versions")),
|
||||
sa.UniqueConstraint("id", "problem_id", name=op.f("uq_problem_versions_id")),
|
||||
sa.UniqueConstraint("problem_id", "revision", name=op.f("uq_problem_versions_problem_id")),
|
||||
)
|
||||
op.create_index(
|
||||
"ix_problem_versions_public",
|
||||
"problem_versions",
|
||||
["problem_id", "published_at"],
|
||||
unique=False,
|
||||
)
|
||||
op.create_table(
|
||||
"sessions",
|
||||
sa.Column("token_hash", sa.String(length=64), nullable=False),
|
||||
sa.Column("account_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("csrf_hash", sa.String(length=64), nullable=False),
|
||||
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("revoked_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column(
|
||||
"created_at",
|
||||
sa.DateTime(timezone=True),
|
||||
server_default=sa.text("now()"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.CheckConstraint("csrf_hash ~ '^[0-9a-f]{64}$'", name=op.f("ck_sessions_csrf_digest")),
|
||||
sa.CheckConstraint("token_hash ~ '^[0-9a-f]{64}$'", name=op.f("ck_sessions_token_digest")),
|
||||
sa.CheckConstraint("expires_at > created_at", name=op.f("ck_sessions_expiry")),
|
||||
sa.ForeignKeyConstraint(
|
||||
["account_id"], ["accounts.id"], name=op.f("fk_sessions_account_id_accounts")
|
||||
),
|
||||
sa.PrimaryKeyConstraint("token_hash", name=op.f("pk_sessions")),
|
||||
)
|
||||
op.create_index(
|
||||
"ix_sessions_account_expiry", "sessions", ["account_id", "expires_at"], unique=False
|
||||
)
|
||||
op.create_table(
|
||||
"problem_cases",
|
||||
sa.Column("id", sa.Uuid(), nullable=False),
|
||||
sa.Column("version_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("position", sa.Integer(), nullable=False),
|
||||
sa.Column("input_digest", sa.String(length=64), nullable=False),
|
||||
sa.Column("expected_digest", sa.String(length=64), nullable=False),
|
||||
sa.Column(
|
||||
"artifact_state",
|
||||
sa.String(length=8),
|
||||
sa.Computed("'ready'", persisted=True),
|
||||
nullable=False,
|
||||
),
|
||||
sa.CheckConstraint("position BETWEEN 0 AND 255", name=op.f("ck_problem_cases_position")),
|
||||
sa.ForeignKeyConstraint(
|
||||
["expected_digest", "artifact_state"],
|
||||
["artifacts.digest", "artifacts.state"],
|
||||
name="fk_problem_cases_expected_digest_artifacts",
|
||||
),
|
||||
sa.ForeignKeyConstraint(
|
||||
["input_digest", "artifact_state"],
|
||||
["artifacts.digest", "artifacts.state"],
|
||||
name=op.f("fk_problem_cases_input_digest_artifacts"),
|
||||
),
|
||||
sa.ForeignKeyConstraint(
|
||||
["version_id"],
|
||||
["problem_versions.id"],
|
||||
name=op.f("fk_problem_cases_version_id_problem_versions"),
|
||||
),
|
||||
sa.PrimaryKeyConstraint("id", name=op.f("pk_problem_cases")),
|
||||
sa.UniqueConstraint("id", "version_id", name=op.f("uq_problem_cases_id")),
|
||||
sa.UniqueConstraint("version_id", "position", name=op.f("uq_problem_cases_version_id")),
|
||||
)
|
||||
op.create_table(
|
||||
"submissions",
|
||||
sa.Column("id", sa.Uuid(), nullable=False),
|
||||
sa.Column("account_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("problem_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("problem_version_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("language", sa.String(length=8), nullable=False),
|
||||
sa.Column("source_digest", sa.String(length=64), nullable=False),
|
||||
sa.Column(
|
||||
"artifact_state",
|
||||
sa.String(length=8),
|
||||
sa.Computed("'ready'", persisted=True),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column("idempotency_key", sa.Uuid(), nullable=False),
|
||||
sa.Column("request_digest", sa.String(length=64), nullable=False),
|
||||
sa.Column(
|
||||
"created_at",
|
||||
sa.DateTime(timezone=True),
|
||||
server_default=sa.text("now()"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.CheckConstraint("language IN ('c17', 'cpp20')", name=op.f("ck_submissions_language")),
|
||||
sa.CheckConstraint(
|
||||
"request_digest ~ '^[0-9a-f]{64}$'", name=op.f("ck_submissions_request_digest")
|
||||
),
|
||||
sa.ForeignKeyConstraint(
|
||||
["account_id"], ["accounts.id"], name=op.f("fk_submissions_account_id_accounts")
|
||||
),
|
||||
sa.ForeignKeyConstraint(
|
||||
["problem_version_id", "problem_id"],
|
||||
["problem_versions.id", "problem_versions.problem_id"],
|
||||
name=op.f("fk_submissions_problem_version_id_problem_versions"),
|
||||
),
|
||||
sa.ForeignKeyConstraint(
|
||||
["source_digest", "artifact_state"],
|
||||
["artifacts.digest", "artifacts.state"],
|
||||
name=op.f("fk_submissions_source_digest_artifacts"),
|
||||
),
|
||||
sa.PrimaryKeyConstraint("id", name=op.f("pk_submissions")),
|
||||
sa.UniqueConstraint(
|
||||
"account_id", "idempotency_key", name=op.f("uq_submissions_account_id")
|
||||
),
|
||||
sa.UniqueConstraint("id", "problem_version_id", name=op.f("uq_submissions_id")),
|
||||
)
|
||||
op.create_index(
|
||||
"ix_submissions_account_history",
|
||||
"submissions",
|
||||
["account_id", "created_at", "id"],
|
||||
unique=False,
|
||||
)
|
||||
op.create_table(
|
||||
"evaluations",
|
||||
sa.Column("id", sa.Uuid(), nullable=False),
|
||||
sa.Column("submission_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("problem_version_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("state", sa.String(length=16), server_default="queued", nullable=False),
|
||||
sa.Column("fencing_version", sa.BigInteger(), server_default="0", nullable=False),
|
||||
sa.Column("attempts_count", sa.Integer(), server_default="0", nullable=False),
|
||||
sa.Column("max_attempts", sa.Integer(), server_default="3", nullable=False),
|
||||
sa.Column("current_attempt_id", sa.Uuid(), nullable=True),
|
||||
sa.Column(
|
||||
"not_before",
|
||||
sa.DateTime(timezone=True),
|
||||
server_default=sa.text("now()"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column("cancel_requested_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column(
|
||||
"updated_at",
|
||||
sa.DateTime(timezone=True),
|
||||
server_default=sa.text("now()"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column("verdict", sa.String(length=32), nullable=True),
|
||||
sa.Column("failure_code", sa.String(length=32), nullable=True),
|
||||
sa.Column("final_report", postgresql.JSONB(astext_type=sa.Text()), nullable=True),
|
||||
sa.Column("final_report_digest", sa.String(length=64), nullable=True),
|
||||
sa.Column(
|
||||
"created_at",
|
||||
sa.DateTime(timezone=True),
|
||||
server_default=sa.text("now()"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.CheckConstraint(
|
||||
"state IN ('queued','claimed','compiling','running','cancelling',"
|
||||
"'finished','cancelled','failed')",
|
||||
name=op.f("ck_evaluations_state"),
|
||||
),
|
||||
sa.CheckConstraint(
|
||||
"state NOT IN ('claimed','compiling','running','cancelling') "
|
||||
"OR current_attempt_id IS NOT NULL",
|
||||
name=op.f("ck_evaluations_active_attempt"),
|
||||
),
|
||||
sa.CheckConstraint(
|
||||
"attempts_count BETWEEN 0 AND max_attempts", name=op.f("ck_evaluations_attempts_count")
|
||||
),
|
||||
sa.CheckConstraint("fencing_version >= 0", name=op.f("ck_evaluations_fence")),
|
||||
sa.CheckConstraint(
|
||||
"max_attempts BETWEEN 1 AND 5", name=op.f("ck_evaluations_max_attempts")
|
||||
),
|
||||
sa.ForeignKeyConstraint(
|
||||
["submission_id", "problem_version_id"],
|
||||
["submissions.id", "submissions.problem_version_id"],
|
||||
name=op.f("fk_evaluations_submission_id_submissions"),
|
||||
),
|
||||
sa.PrimaryKeyConstraint("id", name=op.f("pk_evaluations")),
|
||||
sa.UniqueConstraint("id", "problem_version_id", name=op.f("uq_evaluations_id")),
|
||||
sa.UniqueConstraint("submission_id", name=op.f("uq_evaluations_submission_id")),
|
||||
)
|
||||
op.create_index(
|
||||
"ix_evaluations_recovery", "evaluations", ["state", "not_before", "id"], unique=False
|
||||
)
|
||||
op.create_table(
|
||||
"judge_attempts",
|
||||
sa.Column("id", sa.Uuid(), nullable=False),
|
||||
sa.Column("evaluation_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("worker_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("number", sa.Integer(), nullable=False),
|
||||
sa.Column("fencing_version", sa.BigInteger(), nullable=False),
|
||||
sa.Column("lease_token_hash", sa.String(length=64), nullable=False),
|
||||
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("revoked_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("manifest", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
|
||||
sa.Column("manifest_digest", sa.String(length=64), nullable=False),
|
||||
sa.Column("sequence", sa.BigInteger(), server_default="0", nullable=False),
|
||||
sa.Column("stage", sa.String(length=16), nullable=True),
|
||||
sa.Column("case_index", sa.Integer(), nullable=True),
|
||||
sa.Column("report", postgresql.JSONB(astext_type=sa.Text()), nullable=True),
|
||||
sa.Column("report_digest", sa.String(length=64), nullable=True),
|
||||
sa.Column("cleanup_complete", sa.Boolean(), nullable=True),
|
||||
sa.Column(
|
||||
"created_at",
|
||||
sa.DateTime(timezone=True),
|
||||
server_default=sa.text("now()"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.CheckConstraint(
|
||||
"lease_token_hash ~ '^[0-9a-f]{64}$'", name=op.f("ck_judge_attempts_token_digest")
|
||||
),
|
||||
sa.CheckConstraint(
|
||||
"manifest_digest ~ '^[0-9a-f]{64}$'", name=op.f("ck_judge_attempts_manifest_digest")
|
||||
),
|
||||
sa.CheckConstraint("fencing_version > 0", name=op.f("ck_judge_attempts_fence")),
|
||||
sa.CheckConstraint("number BETWEEN 1 AND 5", name=op.f("ck_judge_attempts_number")),
|
||||
sa.CheckConstraint("sequence >= 0", name=op.f("ck_judge_attempts_sequence")),
|
||||
sa.ForeignKeyConstraint(
|
||||
["evaluation_id"],
|
||||
["evaluations.id"],
|
||||
name=op.f("fk_judge_attempts_evaluation_id_evaluations"),
|
||||
),
|
||||
sa.PrimaryKeyConstraint("id", name=op.f("pk_judge_attempts")),
|
||||
sa.UniqueConstraint(
|
||||
"evaluation_id", "fencing_version", name="uq_judge_attempts_evaluation_fence"
|
||||
),
|
||||
sa.UniqueConstraint("evaluation_id", "number", name="uq_judge_attempts_evaluation_number"),
|
||||
sa.UniqueConstraint("id", "evaluation_id", name=op.f("uq_judge_attempts_id")),
|
||||
)
|
||||
op.create_index(
|
||||
"ix_judge_attempts_expiry", "judge_attempts", ["expires_at", "evaluation_id"], unique=False
|
||||
)
|
||||
op.create_table(
|
||||
"outbox",
|
||||
sa.Column("id", sa.Uuid(), nullable=False),
|
||||
sa.Column("evaluation_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("kind", sa.String(length=32), server_default="judge.evaluation", nullable=False),
|
||||
sa.Column("failures", sa.Integer(), server_default="0", nullable=False),
|
||||
sa.Column(
|
||||
"next_attempt_at",
|
||||
sa.DateTime(timezone=True),
|
||||
server_default=sa.text("now()"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column("locked_until", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("lock_owner", sa.Uuid(), nullable=True),
|
||||
sa.Column("delivered_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("last_error_code", sa.String(length=32), nullable=True),
|
||||
sa.Column(
|
||||
"created_at",
|
||||
sa.DateTime(timezone=True),
|
||||
server_default=sa.text("now()"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.CheckConstraint("kind = 'judge.evaluation'", name=op.f("ck_outbox_kind")),
|
||||
sa.CheckConstraint("failures BETWEEN 0 AND 100", name=op.f("ck_outbox_failure_budget")),
|
||||
sa.ForeignKeyConstraint(
|
||||
["evaluation_id"], ["evaluations.id"], name=op.f("fk_outbox_evaluation_id_evaluations")
|
||||
),
|
||||
sa.PrimaryKeyConstraint("id", name=op.f("pk_outbox")),
|
||||
sa.UniqueConstraint("evaluation_id", name=op.f("uq_outbox_evaluation_id")),
|
||||
)
|
||||
op.create_index(
|
||||
"ix_outbox_delivery", "outbox", ["delivered_at", "next_attempt_at", "id"], unique=False
|
||||
)
|
||||
op.create_table(
|
||||
"case_results",
|
||||
sa.Column("attempt_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("case_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("evaluation_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("problem_version_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("position", sa.Integer(), nullable=False),
|
||||
sa.Column("verdict", sa.String(length=32), nullable=False),
|
||||
sa.Column("cpu_time_us", sa.BigInteger(), nullable=True),
|
||||
sa.Column("memory_peak_bytes", sa.BigInteger(), nullable=True),
|
||||
sa.CheckConstraint(
|
||||
"cpu_time_us IS NULL OR cpu_time_us >= 0", name=op.f("ck_case_results_cpu")
|
||||
),
|
||||
sa.CheckConstraint(
|
||||
"memory_peak_bytes IS NULL OR memory_peak_bytes >= 0",
|
||||
name=op.f("ck_case_results_memory"),
|
||||
),
|
||||
sa.CheckConstraint("position BETWEEN 0 AND 255", name=op.f("ck_case_results_position")),
|
||||
sa.ForeignKeyConstraint(
|
||||
["attempt_id", "evaluation_id"],
|
||||
["judge_attempts.id", "judge_attempts.evaluation_id"],
|
||||
name=op.f("fk_case_results_attempt_id_judge_attempts"),
|
||||
),
|
||||
sa.ForeignKeyConstraint(
|
||||
["case_id", "problem_version_id"],
|
||||
["problem_cases.id", "problem_cases.version_id"],
|
||||
name=op.f("fk_case_results_case_id_problem_cases"),
|
||||
),
|
||||
sa.ForeignKeyConstraint(
|
||||
["evaluation_id", "problem_version_id"],
|
||||
["evaluations.id", "evaluations.problem_version_id"],
|
||||
name=op.f("fk_case_results_evaluation_id_evaluations"),
|
||||
),
|
||||
sa.PrimaryKeyConstraint("attempt_id", "case_id", name=op.f("pk_case_results")),
|
||||
sa.UniqueConstraint("attempt_id", "position", name=op.f("uq_case_results_attempt_id")),
|
||||
)
|
||||
|
||||
op.create_foreign_key(
|
||||
"fk_problems_active_version",
|
||||
"problems",
|
||||
"problem_versions",
|
||||
["active_version_id", "id"],
|
||||
["id", "problem_id"],
|
||||
deferrable=True,
|
||||
initially="DEFERRED",
|
||||
)
|
||||
op.create_foreign_key(
|
||||
"fk_evaluations_current_attempt",
|
||||
"evaluations",
|
||||
"judge_attempts",
|
||||
["current_attempt_id", "id"],
|
||||
["id", "evaluation_id"],
|
||||
deferrable=True,
|
||||
initially="DEFERRED",
|
||||
)
|
||||
install_guards_and_permissions()
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_constraint("fk_evaluations_current_attempt", "evaluations", type_="foreignkey")
|
||||
op.drop_constraint("fk_problems_active_version", "problems", type_="foreignkey")
|
||||
op.drop_table("case_results")
|
||||
op.drop_index("ix_outbox_delivery", table_name="outbox")
|
||||
op.drop_table("outbox")
|
||||
op.drop_index("ix_judge_attempts_expiry", table_name="judge_attempts")
|
||||
op.drop_table("judge_attempts")
|
||||
op.drop_index("ix_evaluations_recovery", table_name="evaluations")
|
||||
op.drop_table("evaluations")
|
||||
op.drop_index("ix_submissions_account_history", table_name="submissions")
|
||||
op.drop_table("submissions")
|
||||
op.drop_table("problem_cases")
|
||||
op.drop_index("ix_sessions_account_expiry", table_name="sessions")
|
||||
op.drop_table("sessions")
|
||||
op.drop_index("ix_problem_versions_public", table_name="problem_versions")
|
||||
op.drop_table("problem_versions")
|
||||
op.drop_table("problems")
|
||||
op.drop_index("ix_login_throttles_reset", table_name="login_throttles")
|
||||
op.drop_table("login_throttles")
|
||||
op.drop_table("artifacts")
|
||||
op.drop_table("accounts")
|
||||
op.execute("DROP FUNCTION public.kf_immutable_version()")
|
||||
op.execute("DROP FUNCTION public.kf_immutable_case()")
|
||||
# NOLOGIN groups are intentionally retained: a downgrade never deletes operator roles.
|
||||
@@ -1,247 +0,0 @@
|
||||
"""Immutable per-language judging configuration and extended submission language IDs.
|
||||
|
||||
Revision ID: 0002_multi_runtime
|
||||
Revises: 0001_judging_foundation
|
||||
"""
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
|
||||
revision = "0002_multi_runtime"
|
||||
down_revision = "0001_judging_foundation"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
CONFIGURATION_CHECK = """
|
||||
(judging_schema_version = 1 AND language_configurations IS NULL
|
||||
AND toolchain IS NOT NULL AND compile_limits IS NOT NULL
|
||||
AND run_limits IS NOT NULL AND job_wall_time_us IS NOT NULL)
|
||||
OR
|
||||
(judging_schema_version = 2 AND language_configurations IS NOT NULL
|
||||
AND toolchain IS NULL AND compile_limits IS NULL
|
||||
AND run_limits IS NULL AND job_wall_time_us IS NULL
|
||||
AND public.kf_valid_language_configurations(languages, language_configurations))
|
||||
"""
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# Metadata-only additions/widening; do not UPDATE any immutable published rows.
|
||||
op.add_column(
|
||||
"problem_versions",
|
||||
sa.Column("judging_schema_version", sa.SmallInteger(), nullable=False, server_default="1"),
|
||||
)
|
||||
op.add_column(
|
||||
"problem_versions",
|
||||
sa.Column("language_configurations", JSONB(none_as_null=True), nullable=True),
|
||||
)
|
||||
for column, kind in (
|
||||
("toolchain", JSONB()),
|
||||
("compile_limits", JSONB()),
|
||||
("run_limits", JSONB()),
|
||||
("job_wall_time_us", sa.BigInteger()),
|
||||
):
|
||||
op.alter_column("problem_versions", column, existing_type=kind, nullable=True)
|
||||
op.alter_column(
|
||||
"submissions",
|
||||
"language",
|
||||
existing_type=sa.String(8),
|
||||
type_=sa.String(16),
|
||||
existing_nullable=False,
|
||||
)
|
||||
op.drop_constraint(op.f("ck_submissions_language"), "submissions", type_="check")
|
||||
op.create_check_constraint(
|
||||
op.f("ck_submissions_language"),
|
||||
"submissions",
|
||||
"language IN ('c17','cpp20','python313','java21')",
|
||||
)
|
||||
op.execute("""
|
||||
CREATE FUNCTION public.kf_valid_runtime_limits(limits jsonb)
|
||||
RETURNS boolean LANGUAGE plpgsql IMMUTABLE STRICT
|
||||
SET search_path = pg_catalog, public AS $$
|
||||
DECLARE entry record;
|
||||
BEGIN
|
||||
IF jsonb_typeof(limits) != 'object' OR
|
||||
(SELECT count(*) FROM jsonb_object_keys(limits)) != 10 THEN RETURN false; END IF;
|
||||
FOR entry IN SELECT * FROM (VALUES
|
||||
('cpu_time_us',1000::numeric,60000000::numeric),
|
||||
('wall_time_us',1000,120000000), ('memory_bytes',1,2147483648),
|
||||
('stack_bytes',1,268435456), ('processes',1,128), ('open_files',3,1024),
|
||||
('stdout_bytes',1,16777216), ('stderr_bytes',1,16777216),
|
||||
('file_bytes',1,67108864), ('workdir_bytes',1,268435456)
|
||||
) AS bounds(name,lo,hi) LOOP
|
||||
IF NOT limits ? entry.name OR jsonb_typeof(limits->entry.name) != 'number'
|
||||
OR (limits->>entry.name) !~ '^(0|[1-9][0-9]*)$'
|
||||
OR length(limits->>entry.name) > 19 THEN RETURN false; END IF;
|
||||
IF (limits->>entry.name)::numeric NOT BETWEEN entry.lo AND entry.hi
|
||||
THEN RETURN false; END IF;
|
||||
END LOOP;
|
||||
RETURN (limits->>'stack_bytes')::numeric <= (limits->>'memory_bytes')::numeric
|
||||
AND (limits->>'file_bytes')::numeric <= (limits->>'workdir_bytes')::numeric
|
||||
AND (limits->>'stdout_bytes')::numeric <= (limits->>'file_bytes')::numeric
|
||||
AND (limits->>'stderr_bytes')::numeric <= (limits->>'file_bytes')::numeric
|
||||
AND (limits->>'stdout_bytes')::numeric + (limits->>'stderr_bytes')::numeric
|
||||
<= (limits->>'workdir_bytes')::numeric;
|
||||
END $$
|
||||
""")
|
||||
op.execute("""
|
||||
CREATE FUNCTION public.kf_valid_language_configurations(languages jsonb, configs jsonb)
|
||||
RETURNS boolean LANGUAGE plpgsql IMMUTABLE STRICT
|
||||
SET search_path = pg_catalog, public AS $$
|
||||
DECLARE item record; cfg jsonb; runtime jsonb; lang text; seen text[] := ARRAY[]::text[];
|
||||
phase jsonb; preparing boolean;
|
||||
BEGIN
|
||||
IF jsonb_typeof(languages) != 'array' OR jsonb_typeof(configs) != 'array'
|
||||
THEN RETURN false; END IF;
|
||||
IF jsonb_array_length(configs) NOT BETWEEN 1 AND 4 OR
|
||||
jsonb_array_length(languages) != jsonb_array_length(configs)
|
||||
THEN RETURN false; END IF;
|
||||
FOR item IN SELECT value, ordinality FROM jsonb_array_elements(configs)
|
||||
WITH ORDINALITY LOOP
|
||||
cfg := item.value;
|
||||
IF jsonb_typeof(cfg) != 'object' OR
|
||||
(SELECT count(*) FROM jsonb_object_keys(cfg)) != 5 OR NOT cfg ?&
|
||||
ARRAY['language','toolchain','compile_limits','run_limits','job_wall_time_us']
|
||||
THEN RETURN false; END IF;
|
||||
IF jsonb_typeof(cfg->'language') != 'string' THEN RETURN false; END IF;
|
||||
lang := cfg->>'language';
|
||||
IF lang NOT IN ('c17','cpp20','python313','java21') OR lang = ANY(seen)
|
||||
OR languages->(item.ordinality::int-1) != cfg->'language'
|
||||
THEN RETURN false; END IF;
|
||||
seen := array_append(seen,lang); runtime := cfg->'toolchain';
|
||||
IF jsonb_typeof(runtime) != 'object' OR
|
||||
(SELECT count(*) FROM jsonb_object_keys(runtime)) != 5 OR NOT runtime ?&
|
||||
ARRAY['toolchain_id','image_kind','image_revision','image_digest','runtime_version']
|
||||
THEN RETURN false; END IF;
|
||||
IF jsonb_typeof(runtime->'toolchain_id') != 'string' OR
|
||||
jsonb_typeof(runtime->'image_kind') != 'string' OR
|
||||
jsonb_typeof(runtime->'runtime_version') != 'string' OR
|
||||
jsonb_typeof(runtime->'image_digest') != 'string' OR
|
||||
(runtime->>'image_digest') !~ '^[0-9a-f]{64}$' OR
|
||||
(runtime->>'runtime_version') !~ '^[A-Za-z0-9][A-Za-z0-9_.+-]{0,63}$' OR
|
||||
jsonb_typeof(runtime->'image_revision') != 'number' OR
|
||||
(runtime->>'image_revision') !~ '^[1-9][0-9]{0,9}$'
|
||||
THEN RETURN false; END IF;
|
||||
IF (runtime->>'image_revision')::numeric > 4294967295 THEN RETURN false; END IF;
|
||||
IF lang IN ('c17','cpp20') AND (runtime->>'toolchain_id' != 'gnu-cxx-v1' OR
|
||||
runtime->>'image_kind' != 'linux_gnu_cxx_x86_64_v1') THEN RETURN false; END IF;
|
||||
IF lang = 'python313' AND (runtime->>'toolchain_id' != 'cpython313-v1' OR
|
||||
runtime->>'image_kind' != 'linux_cpython313_x86_64_v1' OR
|
||||
runtime->>'runtime_version' != '3.13.16') THEN RETURN false; END IF;
|
||||
IF lang = 'java21' AND (runtime->>'toolchain_id' != 'openjdk21-v1' OR
|
||||
runtime->>'image_kind' != 'linux_openjdk21_x86_64_v1' OR
|
||||
(runtime->>'runtime_version') !~
|
||||
'^21[.][0-9]+[.][0-9]+([.][0-9]+)?[+][1-9][0-9]*(-LTS)?$')
|
||||
THEN RETURN false; END IF;
|
||||
IF NOT public.kf_valid_runtime_limits(cfg->'compile_limits') OR
|
||||
NOT public.kf_valid_runtime_limits(cfg->'run_limits') THEN RETURN false; END IF;
|
||||
IF jsonb_typeof(cfg->'job_wall_time_us') != 'number' OR
|
||||
(cfg->>'job_wall_time_us') !~ '^[1-9][0-9]{0,9}$' THEN RETURN false; END IF;
|
||||
IF (cfg->>'job_wall_time_us')::numeric NOT BETWEEN 1000 AND 3600000000 OR
|
||||
(cfg->'compile_limits'->>'wall_time_us')::numeric +
|
||||
(cfg->'run_limits'->>'wall_time_us')::numeric >
|
||||
(cfg->>'job_wall_time_us')::numeric
|
||||
THEN RETURN false; END IF;
|
||||
FOR preparing, phase IN SELECT true,cfg->'compile_limits' UNION ALL
|
||||
SELECT false,cfg->'run_limits' LOOP
|
||||
IF lang = 'python313' AND ((phase->>'cpu_time_us')::numeric < 10000 OR
|
||||
(phase->>'wall_time_us')::numeric < 50000 OR
|
||||
(phase->>'memory_bytes')::numeric < 33554432 OR
|
||||
(phase->>'open_files')::numeric < 16) THEN RETURN false; END IF;
|
||||
IF lang = 'java21' AND ((phase->>'cpu_time_us')::numeric <
|
||||
CASE WHEN preparing THEN 1000000 ELSE 200000 END OR
|
||||
(phase->>'wall_time_us')::numeric <
|
||||
CASE WHEN preparing THEN 3000000 ELSE 1000000 END OR
|
||||
(phase->>'memory_bytes')::numeric <
|
||||
CASE WHEN preparing THEN 402653184 ELSE 268435456 END OR
|
||||
(phase->>'processes')::numeric < 16 OR
|
||||
(phase->>'open_files')::numeric < 32 OR
|
||||
(phase->>'stack_bytes')::numeric < 262144 OR
|
||||
(phase->>'memory_bytes')::numeric - 167772160 -
|
||||
(phase->>'processes')::numeric * 262144 < 33554432)
|
||||
THEN RETURN false; END IF;
|
||||
END LOOP;
|
||||
END LOOP;
|
||||
RETURN true;
|
||||
END $$
|
||||
""")
|
||||
op.execute("REVOKE ALL ON FUNCTION public.kf_valid_runtime_limits(jsonb) FROM PUBLIC")
|
||||
op.execute(
|
||||
"REVOKE ALL ON FUNCTION public.kf_valid_language_configurations(jsonb,jsonb) FROM PUBLIC"
|
||||
)
|
||||
op.execute("GRANT EXECUTE ON FUNCTION public.kf_valid_runtime_limits(jsonb) TO kf_oj_platform")
|
||||
op.execute(
|
||||
"GRANT EXECUTE ON FUNCTION public.kf_valid_language_configurations(jsonb,jsonb) "
|
||||
"TO kf_oj_platform"
|
||||
)
|
||||
op.create_check_constraint(
|
||||
op.f("ck_problem_versions_judging_schema"),
|
||||
"problem_versions",
|
||||
"judging_schema_version IN (1,2)",
|
||||
)
|
||||
op.create_check_constraint(
|
||||
op.f("ck_problem_versions_language_configurations"), "problem_versions", CONFIGURATION_CHECK
|
||||
)
|
||||
op.execute("""
|
||||
CREATE FUNCTION public.kf_submission_language() RETURNS trigger LANGUAGE plpgsql
|
||||
SET search_path = pg_catalog, public AS $$
|
||||
DECLARE supported jsonb;
|
||||
BEGIN
|
||||
SELECT languages INTO supported FROM public.problem_versions
|
||||
WHERE id=NEW.problem_version_id;
|
||||
IF FOUND AND NOT supported @> jsonb_build_array(NEW.language) THEN
|
||||
RAISE EXCEPTION 'submission language not supported by version'
|
||||
USING ERRCODE = '23514';
|
||||
END IF;
|
||||
RETURN NEW;
|
||||
END $$
|
||||
""")
|
||||
op.execute("REVOKE ALL ON FUNCTION public.kf_submission_language() FROM PUBLIC")
|
||||
op.execute("""
|
||||
CREATE TRIGGER kf_submission_language BEFORE INSERT OR UPDATE OF
|
||||
language,problem_version_id ON public.submissions
|
||||
FOR EACH ROW EXECUTE FUNCTION public.kf_submission_language()
|
||||
""")
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# Losing a runtime configuration is not a rollback strategy. Preserve all user data.
|
||||
op.execute("""
|
||||
DO $$ BEGIN
|
||||
IF EXISTS (SELECT 1 FROM problem_versions WHERE judging_schema_version = 2) OR
|
||||
EXISTS (SELECT 1 FROM submissions WHERE language NOT IN ('c17','cpp20')) THEN
|
||||
RAISE EXCEPTION 'cannot downgrade while multi-runtime data exists'
|
||||
USING ERRCODE = '23514';
|
||||
END IF;
|
||||
END $$
|
||||
""")
|
||||
op.drop_constraint(
|
||||
op.f("ck_problem_versions_language_configurations"), "problem_versions", type_="check"
|
||||
)
|
||||
op.drop_constraint(
|
||||
op.f("ck_problem_versions_judging_schema"), "problem_versions", type_="check"
|
||||
)
|
||||
op.execute("DROP TRIGGER kf_submission_language ON public.submissions")
|
||||
op.execute("DROP FUNCTION public.kf_submission_language()")
|
||||
op.execute("DROP FUNCTION public.kf_valid_language_configurations(jsonb,jsonb)")
|
||||
op.execute("DROP FUNCTION public.kf_valid_runtime_limits(jsonb)")
|
||||
op.drop_column("problem_versions", "language_configurations")
|
||||
op.drop_column("problem_versions", "judging_schema_version")
|
||||
for column, kind in (
|
||||
("toolchain", JSONB()),
|
||||
("compile_limits", JSONB()),
|
||||
("run_limits", JSONB()),
|
||||
("job_wall_time_us", sa.BigInteger()),
|
||||
):
|
||||
op.alter_column("problem_versions", column, existing_type=kind, nullable=False)
|
||||
op.drop_constraint(op.f("ck_submissions_language"), "submissions", type_="check")
|
||||
op.create_check_constraint(
|
||||
op.f("ck_submissions_language"), "submissions", "language IN ('c17','cpp20')"
|
||||
)
|
||||
op.alter_column(
|
||||
"submissions",
|
||||
"language",
|
||||
existing_type=sa.String(16),
|
||||
type_=sa.String(8),
|
||||
existing_nullable=False,
|
||||
)
|
||||
@@ -17,14 +17,9 @@ dependencies = [
|
||||
"asyncpg>=0.30,<1",
|
||||
"alembic>=1.15,<2",
|
||||
"aio-pika>=9.5,<10",
|
||||
"argon2-cffi>=25.1,<26",
|
||||
"httpx>=0.28,<1",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
kf-accounts = "kf_oj.modules.identity.cli:main"
|
||||
kf-problems = "kf_oj.modules.problems.cli:main"
|
||||
kf-artifacts = "kf_oj.infra.artifacts.cli:main"
|
||||
kf-learning-worker = "kf_oj.workers.learning:main"
|
||||
kf-outbox-publisher = "kf_oj.workers.outbox:main"
|
||||
|
||||
@@ -32,6 +27,7 @@ kf-outbox-publisher = "kf_oj.workers.outbox:main"
|
||||
dev = [
|
||||
"ruff>=0.11,<1",
|
||||
"pytest>=8.3,<10",
|
||||
"httpx>=0.28,<1",
|
||||
"jsonschema>=4.23,<5",
|
||||
]
|
||||
|
||||
|
||||
@@ -2,27 +2,11 @@ import logging
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from fastapi import FastAPI
|
||||
from fastapi.exceptions import RequestValidationError
|
||||
from starlette.responses import JSONResponse
|
||||
|
||||
from kf_oj import __version__
|
||||
from kf_oj.api.body_limits import BodyLimitMiddleware
|
||||
from kf_oj.api.v1.identity import router as identity_router
|
||||
from kf_oj.api.v1.problems import router as problems_router
|
||||
from kf_oj.api.v1.router import router
|
||||
from kf_oj.api.v1.submissions import router as submission_router
|
||||
from kf_oj.api.v1.worker_artifacts import router as worker_router
|
||||
from kf_oj.core.logging import configure_logging
|
||||
from kf_oj.core.settings import Settings
|
||||
from kf_oj.infra.artifacts.repository import ArtifactRepository
|
||||
from kf_oj.infra.artifacts.store import ArtifactStore
|
||||
from kf_oj.infra.coordinator import CoordinatorControl
|
||||
from kf_oj.infra.db.session import build_engine, build_session_factory
|
||||
from kf_oj.modules.identity.service import IdentityService
|
||||
from kf_oj.modules.problems.service import ProblemService
|
||||
from kf_oj.modules.submissions.queries import SubmissionQueries
|
||||
from kf_oj.modules.submissions.service import SubmissionService
|
||||
from kf_oj.modules.submissions.worker_access import WorkerAccess
|
||||
|
||||
|
||||
def create_app(settings: Settings | None = None) -> FastAPI:
|
||||
@@ -33,82 +17,20 @@ def create_app(settings: Settings | None = None) -> FastAPI:
|
||||
configure_logging(settings.log_level)
|
||||
logger = logging.getLogger("kf_oj.api")
|
||||
logger.info("Platform API started")
|
||||
engine = None
|
||||
store = None
|
||||
try:
|
||||
if settings.database_url is not None:
|
||||
engine = build_engine(settings)
|
||||
app.state.sessions = build_session_factory(engine)
|
||||
app.state.identity = IdentityService(
|
||||
app.state.sessions, lifetime_seconds=settings.session_lifetime_seconds
|
||||
)
|
||||
artifacts = None
|
||||
if settings.artifact_root is not None:
|
||||
store = ArtifactStore(
|
||||
settings.artifact_root,
|
||||
max_bytes=settings.artifact_max_bytes,
|
||||
max_files=settings.artifact_max_files,
|
||||
)
|
||||
artifacts = ArtifactRepository(store)
|
||||
app.state.problems = ProblemService(
|
||||
app.state.sessions, artifacts, settings.judge_admission
|
||||
)
|
||||
if artifacts is not None:
|
||||
app.state.submissions = SubmissionService(
|
||||
app.state.sessions, app.state.problems, artifacts, settings.judge_admission
|
||||
)
|
||||
app.state.worker_access = WorkerAccess(
|
||||
app.state.sessions, app.state.problems, artifacts
|
||||
)
|
||||
cancellation = (
|
||||
CoordinatorControl(
|
||||
settings.coordinator_url, settings.coordinator_token.get_secret_value()
|
||||
)
|
||||
if settings.coordinator_url and settings.coordinator_token
|
||||
else None
|
||||
)
|
||||
app.state.submission_queries = SubmissionQueries(
|
||||
app.state.sessions, artifacts, cancellation
|
||||
)
|
||||
yield
|
||||
finally:
|
||||
app.state.identity = None
|
||||
app.state.problems = None
|
||||
app.state.worker_access = None
|
||||
app.state.submissions = None
|
||||
app.state.submission_queries = None
|
||||
if engine is not None:
|
||||
await engine.dispose()
|
||||
if store is not None:
|
||||
store.close()
|
||||
logger.info("Platform API stopped")
|
||||
|
||||
app = FastAPI(
|
||||
title="KF-OnlineJudge Platform API",
|
||||
version=__version__,
|
||||
description="平台 API。身份使用真实可撤销会话,业务接口按模块逐项接入。",
|
||||
description="工程骨架。当前只提供进程存活检查,业务接口逐项实现。",
|
||||
docs_url="/api/docs" if settings.environment != "production" else None,
|
||||
redoc_url=None,
|
||||
openapi_url="/api/openapi.json" if settings.environment != "production" else None,
|
||||
lifespan=lifespan,
|
||||
)
|
||||
app.state.settings = settings
|
||||
app.add_middleware(BodyLimitMiddleware)
|
||||
|
||||
@app.middleware("http")
|
||||
async def private_responses(request, call_next):
|
||||
response = await call_next(request)
|
||||
if request.url.path.startswith(("/api/v1/auth", "/api/v1/submissions", "/api/v1/internal")):
|
||||
response.headers["Cache-Control"] = "no-store"
|
||||
return response
|
||||
|
||||
@app.exception_handler(RequestValidationError)
|
||||
async def invalid_request(_, __):
|
||||
return JSONResponse({"detail": "invalid request"}, status_code=422)
|
||||
|
||||
app.include_router(router, prefix="/api/v1")
|
||||
app.include_router(identity_router, prefix="/api/v1")
|
||||
app.include_router(problems_router, prefix="/api/v1")
|
||||
app.include_router(worker_router, prefix="/api/v1")
|
||||
app.include_router(submission_router, prefix="/api/v1")
|
||||
return app
|
||||
|
||||
@@ -1,66 +0,0 @@
|
||||
"""Bound request bytes and upload time before schema parsing can allocate large inputs."""
|
||||
|
||||
import asyncio
|
||||
|
||||
from starlette.responses import JSONResponse
|
||||
|
||||
|
||||
class BodyLimitMiddleware:
|
||||
def __init__(self, app):
|
||||
self.app = app
|
||||
|
||||
async def __call__(self, scope, receive, send):
|
||||
if scope["type"] != "http" or scope["method"] in {"GET", "HEAD", "OPTIONS"}:
|
||||
await self.app(scope, receive, send)
|
||||
return
|
||||
limit = 8192 if scope["path"] == "/api/v1/auth/login" else 1024 * 1024
|
||||
if scope["path"] == "/api/v1/submissions":
|
||||
# Max256KiB UTF-8 source, worst-case JSON escaping plus bounded metadata.
|
||||
limit = 6 * 256 * 1024 + 8192
|
||||
headers = dict(scope.get("headers", []))
|
||||
length = headers.get(b"content-length")
|
||||
if length is not None:
|
||||
try:
|
||||
size = int(length)
|
||||
if size < 0 or size > limit:
|
||||
raise ValueError
|
||||
except ValueError:
|
||||
await JSONResponse({"detail": "request body outside byte policy"}, 413)(
|
||||
scope, receive, send
|
||||
)
|
||||
return
|
||||
# Buffer a finite body so errors can be returned before response headers are sent.
|
||||
chunks, size, received_chunks = [], 0, 0
|
||||
try:
|
||||
async with asyncio.timeout(10):
|
||||
while True:
|
||||
message = await receive()
|
||||
if message["type"] == "http.disconnect":
|
||||
return
|
||||
body = message.get("body", b"")
|
||||
size += len(body)
|
||||
received_chunks += 1
|
||||
if size > limit or received_chunks > 1024:
|
||||
await JSONResponse({"detail": "request body outside byte policy"}, 413)(
|
||||
scope, receive, send
|
||||
)
|
||||
return
|
||||
chunks.append(body)
|
||||
if not message.get("more_body", False):
|
||||
break
|
||||
except TimeoutError:
|
||||
await JSONResponse({"detail": "request body deadline exceeded"}, 408)(
|
||||
scope, receive, send
|
||||
)
|
||||
return
|
||||
body = b"".join(chunks)
|
||||
delivered = False
|
||||
|
||||
async def bounded_receive():
|
||||
nonlocal delivered
|
||||
if not delivered:
|
||||
delivered = True
|
||||
return {"type": "http.request", "body": body, "more_body": False}
|
||||
return await receive()
|
||||
|
||||
await self.app(scope, bounded_receive, send)
|
||||
@@ -1,50 +0,0 @@
|
||||
from typing import Annotated
|
||||
|
||||
from fastapi import Depends, HTTPException, Request
|
||||
|
||||
from kf_oj.core.settings import Settings
|
||||
from kf_oj.modules.identity.public import AuthenticationFailed, Principal
|
||||
from kf_oj.modules.identity.service import IdentityService
|
||||
|
||||
SESSION_COOKIE = "kf_session"
|
||||
|
||||
|
||||
def identity_service(request: Request) -> IdentityService:
|
||||
service = getattr(request.app.state, "identity", None)
|
||||
if service is None:
|
||||
raise HTTPException(503, "identity service unavailable")
|
||||
return service
|
||||
|
||||
|
||||
def trusted_origin(request: Request) -> None:
|
||||
settings: Settings = request.app.state.settings
|
||||
if request.headers.get("origin") not in settings.browser_origins:
|
||||
raise HTTPException(403, "request origin rejected")
|
||||
if request.headers.get("sec-fetch-site") == "cross-site":
|
||||
raise HTTPException(403, "cross-site request rejected")
|
||||
|
||||
|
||||
async def current_principal(
|
||||
request: Request, identity: Annotated[IdentityService, Depends(identity_service)]
|
||||
) -> Principal:
|
||||
try:
|
||||
return await identity.authenticate(request.cookies.get(SESSION_COOKIE))
|
||||
except AuthenticationFailed:
|
||||
raise HTTPException(401, "authentication required") from None
|
||||
|
||||
|
||||
async def unsafe_principal(
|
||||
request: Request, identity: Annotated[IdentityService, Depends(identity_service)]
|
||||
) -> Principal:
|
||||
trusted_origin(request)
|
||||
try:
|
||||
await identity.authenticate(request.cookies.get(SESSION_COOKIE))
|
||||
except AuthenticationFailed:
|
||||
raise HTTPException(401, "authentication required") from None
|
||||
csrf = request.headers.get("x-csrf-token")
|
||||
if csrf is None:
|
||||
raise HTTPException(403, "request token required")
|
||||
try:
|
||||
return await identity.authenticate(request.cookies.get(SESSION_COOKIE), csrf_token=csrf)
|
||||
except AuthenticationFailed:
|
||||
raise HTTPException(403, "request authentication rejected") from None
|
||||
@@ -1,95 +0,0 @@
|
||||
from typing import Annotated
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request, Response
|
||||
|
||||
from kf_oj.api.dependencies import (
|
||||
SESSION_COOKIE,
|
||||
current_principal,
|
||||
identity_service,
|
||||
trusted_origin,
|
||||
unsafe_principal,
|
||||
)
|
||||
from kf_oj.contracts.identity import CurrentUser, LoginRequest, LoginResponse
|
||||
from kf_oj.modules.identity.public import AuthenticationFailed, LoginThrottled, Principal
|
||||
from kf_oj.modules.identity.service import IdentityService
|
||||
|
||||
router = APIRouter(prefix="/auth", tags=["identity"])
|
||||
|
||||
|
||||
@router.post("/login", response_model=LoginResponse, dependencies=[Depends(trusted_origin)])
|
||||
async def login(
|
||||
body: LoginRequest,
|
||||
request: Request,
|
||||
response: Response,
|
||||
identity: Annotated[IdentityService, Depends(identity_service)],
|
||||
) -> LoginResponse:
|
||||
if request.headers.get("content-type", "").split(";", 1)[0] != "application/json":
|
||||
raise HTTPException(415, "JSON request required")
|
||||
try:
|
||||
issued = await identity.login(
|
||||
body.username,
|
||||
body.password.get_secret_value(),
|
||||
peer=request.client.host if request.client else "",
|
||||
)
|
||||
except LoginThrottled:
|
||||
raise HTTPException(
|
||||
429, "login temporarily limited", headers={"Retry-After": "900"}
|
||||
) from None
|
||||
except (AuthenticationFailed, ValueError):
|
||||
raise HTTPException(401, "invalid credentials") from None
|
||||
settings = request.app.state.settings
|
||||
response.set_cookie(
|
||||
SESSION_COOKIE,
|
||||
issued.token,
|
||||
max_age=settings.session_lifetime_seconds,
|
||||
httponly=True,
|
||||
secure=settings.environment == "production",
|
||||
samesite="strict",
|
||||
path="/api/v1",
|
||||
)
|
||||
response.set_cookie(
|
||||
"kf_csrf",
|
||||
issued.csrf_token,
|
||||
max_age=settings.session_lifetime_seconds,
|
||||
httponly=False,
|
||||
secure=settings.environment == "production",
|
||||
samesite="strict",
|
||||
path="/",
|
||||
)
|
||||
response.headers["Cache-Control"] = "no-store"
|
||||
return LoginResponse(
|
||||
user=CurrentUser(id=issued.principal.account_id, username=issued.principal.username),
|
||||
expires_at=issued.expires_at,
|
||||
csrf_token=issued.csrf_token,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/me", response_model=CurrentUser)
|
||||
async def me(response: Response, principal: Annotated[Principal, Depends(current_principal)]):
|
||||
response.headers["Cache-Control"] = "no-store"
|
||||
return CurrentUser(id=principal.account_id, username=principal.username)
|
||||
|
||||
|
||||
@router.post("/logout", status_code=204, dependencies=[Depends(unsafe_principal)])
|
||||
async def logout(
|
||||
request: Request, identity: Annotated[IdentityService, Depends(identity_service)]
|
||||
) -> Response:
|
||||
try:
|
||||
await identity.logout(request.cookies[SESSION_COOKIE], request.headers["x-csrf-token"])
|
||||
except AuthenticationFailed:
|
||||
raise HTTPException(401, "authentication required") from None
|
||||
response = Response(status_code=204, headers={"Cache-Control": "no-store"})
|
||||
response.delete_cookie(
|
||||
SESSION_COOKIE,
|
||||
path="/api/v1",
|
||||
httponly=True,
|
||||
secure=request.app.state.settings.environment == "production",
|
||||
samesite="strict",
|
||||
)
|
||||
response.delete_cookie(
|
||||
"kf_csrf",
|
||||
path="/",
|
||||
secure=request.app.state.settings.environment == "production",
|
||||
samesite="strict",
|
||||
)
|
||||
return response
|
||||
@@ -1,34 +0,0 @@
|
||||
from typing import Annotated
|
||||
from uuid import UUID
|
||||
|
||||
from fastapi import APIRouter, HTTPException, Query, Request
|
||||
|
||||
from kf_oj.contracts.problems import ProblemDetail, ProblemPage
|
||||
from kf_oj.modules.problems.public import ProblemUnavailable
|
||||
from kf_oj.modules.problems.service import ProblemService
|
||||
|
||||
router = APIRouter(prefix="/problems", tags=["problems"])
|
||||
|
||||
|
||||
def service(request: Request) -> ProblemService:
|
||||
component = getattr(request.app.state, "problems", None)
|
||||
if component is None:
|
||||
raise HTTPException(503, "problem service unavailable")
|
||||
return component
|
||||
|
||||
|
||||
@router.get("", response_model=ProblemPage)
|
||||
async def list_problems(
|
||||
request: Request,
|
||||
after: Annotated[str | None, Query(pattern=r"^[a-z0-9][a-z0-9_-]{0,63}$")] = None,
|
||||
limit: Annotated[int, Query(ge=1, le=100)] = 20,
|
||||
) -> ProblemPage:
|
||||
return await service(request).list(after=after, limit=limit)
|
||||
|
||||
|
||||
@router.get("/versions/{version_id}", response_model=ProblemDetail)
|
||||
async def problem_version(version_id: UUID, request: Request) -> ProblemDetail:
|
||||
try:
|
||||
return await service(request).detail(version_id)
|
||||
except ProblemUnavailable:
|
||||
raise HTTPException(404, "problem version unavailable") from None
|
||||
@@ -1,119 +0,0 @@
|
||||
from typing import Annotated
|
||||
from uuid import UUID
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query, Request, Response
|
||||
|
||||
from kf_oj.api.dependencies import current_principal, unsafe_principal
|
||||
from kf_oj.contracts.submissions import (
|
||||
CancellationAccepted,
|
||||
CreateSubmission,
|
||||
SubmissionCreated,
|
||||
SubmissionDetail,
|
||||
SubmissionPage,
|
||||
SubmissionSource,
|
||||
)
|
||||
from kf_oj.infra.artifacts.store import ArtifactError
|
||||
from kf_oj.modules.identity.public import Principal
|
||||
from kf_oj.modules.submissions.public import (
|
||||
CancelUnavailable,
|
||||
IdempotencyConflict,
|
||||
SubmissionBusy,
|
||||
SubmissionUnavailable,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/submissions", tags=["submissions"])
|
||||
|
||||
|
||||
@router.post("", status_code=201, response_model=SubmissionCreated)
|
||||
async def create(
|
||||
body: CreateSubmission,
|
||||
request: Request,
|
||||
response: Response,
|
||||
principal: Annotated[Principal, Depends(unsafe_principal)],
|
||||
):
|
||||
if request.headers.get("content-type", "").split(";", 1)[0] != "application/json":
|
||||
raise HTTPException(415, "JSON request required")
|
||||
service = getattr(request.app.state, "submissions", None)
|
||||
if service is None:
|
||||
raise HTTPException(503, "submission service unavailable")
|
||||
try:
|
||||
result = await service.create(principal, body)
|
||||
except IdempotencyConflict:
|
||||
raise HTTPException(409, "idempotency request conflict") from None
|
||||
except SubmissionBusy:
|
||||
raise HTTPException(429, "pending submission capacity exhausted") from None
|
||||
except SubmissionUnavailable:
|
||||
raise HTTPException(404, "submission capability unavailable") from None
|
||||
except ArtifactError:
|
||||
raise HTTPException(503, "source storage unavailable") from None
|
||||
except ValueError:
|
||||
raise HTTPException(422, "submission outside admission policy") from None
|
||||
response.headers["Cache-Control"] = "no-store"
|
||||
return result
|
||||
|
||||
|
||||
def queries(request: Request):
|
||||
service = getattr(request.app.state, "submission_queries", None)
|
||||
if service is None:
|
||||
raise HTTPException(503, "submission service unavailable")
|
||||
return service
|
||||
|
||||
|
||||
@router.get("", response_model=SubmissionPage)
|
||||
async def history(
|
||||
request: Request,
|
||||
principal: Annotated[Principal, Depends(current_principal)],
|
||||
after: Annotated[str | None, Query(max_length=128)] = None,
|
||||
limit: Annotated[int, Query(ge=1, le=100)] = 20,
|
||||
):
|
||||
try:
|
||||
return await queries(request).page(principal, after, limit)
|
||||
except ValueError:
|
||||
raise HTTPException(422, "invalid history request") from None
|
||||
|
||||
|
||||
@router.get("/{submission_id}", response_model=SubmissionDetail)
|
||||
async def detail(
|
||||
submission_id: UUID,
|
||||
request: Request,
|
||||
principal: Annotated[Principal, Depends(current_principal)],
|
||||
):
|
||||
try:
|
||||
return await queries(request).detail(principal, submission_id)
|
||||
except SubmissionUnavailable:
|
||||
raise HTTPException(404, "submission unavailable") from None
|
||||
except ValueError:
|
||||
raise HTTPException(503, "stored result unavailable") from None
|
||||
|
||||
|
||||
@router.get("/{submission_id}/source", response_model=SubmissionSource)
|
||||
async def source(
|
||||
submission_id: UUID,
|
||||
request: Request,
|
||||
principal: Annotated[Principal, Depends(current_principal)],
|
||||
):
|
||||
service = queries(request)
|
||||
try:
|
||||
result = await service.detail(principal, submission_id)
|
||||
data = await service.owned_source(principal, submission_id)
|
||||
return SubmissionSource(
|
||||
submission_id=submission_id, language=result.language, source=data.decode("utf-8")
|
||||
)
|
||||
except SubmissionUnavailable:
|
||||
raise HTTPException(404, "submission unavailable") from None
|
||||
except ValueError:
|
||||
raise HTTPException(503, "stored source unavailable") from None
|
||||
|
||||
|
||||
@router.post("/{submission_id}/cancel", status_code=202, response_model=CancellationAccepted)
|
||||
async def cancel(
|
||||
submission_id: UUID,
|
||||
request: Request,
|
||||
principal: Annotated[Principal, Depends(unsafe_principal)],
|
||||
):
|
||||
try:
|
||||
return await queries(request).cancel(principal, submission_id)
|
||||
except SubmissionUnavailable:
|
||||
raise HTTPException(404, "submission unavailable") from None
|
||||
except CancelUnavailable:
|
||||
raise HTTPException(503, "cancellation service unavailable") from None
|
||||
@@ -1,96 +0,0 @@
|
||||
from typing import Annotated
|
||||
from uuid import UUID
|
||||
|
||||
from fastapi import APIRouter, Header, HTTPException, Request, Response, Security
|
||||
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
|
||||
|
||||
from kf_oj.contracts.judging import JobManifest, LeaseBinding
|
||||
from kf_oj.modules.submissions.public import WorkerAccessDenied, WorkerReadBusy
|
||||
from kf_oj.modules.submissions.worker_access import WorkerAccess
|
||||
|
||||
router = APIRouter(prefix="/internal/evaluations", tags=["worker artifacts"])
|
||||
bearer = HTTPBearer(auto_error=False, scheme_name="WorkerLease")
|
||||
|
||||
|
||||
def grant(
|
||||
evaluation_id: UUID,
|
||||
attempt_id: UUID,
|
||||
worker: UUID,
|
||||
fence: int,
|
||||
credentials: HTTPAuthorizationCredentials | None,
|
||||
) -> tuple[LeaseBinding, str]:
|
||||
if (
|
||||
credentials is None
|
||||
or credentials.scheme.lower() != "bearer"
|
||||
or len(credentials.credentials) != 64
|
||||
):
|
||||
raise HTTPException(401, "worker lease required")
|
||||
try:
|
||||
return LeaseBinding(
|
||||
evaluation_id=evaluation_id,
|
||||
attempt_id=attempt_id,
|
||||
worker_id=worker,
|
||||
fencing_version=fence,
|
||||
), credentials.credentials
|
||||
except ValueError:
|
||||
raise HTTPException(404, "worker grant rejected") from None
|
||||
|
||||
|
||||
def component(request: Request) -> WorkerAccess:
|
||||
access = getattr(request.app.state, "worker_access", None)
|
||||
if access is None:
|
||||
raise HTTPException(503, "worker artifact service unavailable")
|
||||
return access
|
||||
|
||||
|
||||
@router.get("/{evaluation_id}/attempts/{attempt_id}/manifest", response_model=JobManifest)
|
||||
async def manifest(
|
||||
evaluation_id: UUID,
|
||||
attempt_id: UUID,
|
||||
request: Request,
|
||||
worker: Annotated[UUID, Header(alias="X-Worker-ID")],
|
||||
fence: Annotated[int, Header(alias="X-Fencing-Version", ge=1)],
|
||||
credentials: Annotated[HTTPAuthorizationCredentials | None, Security(bearer)],
|
||||
):
|
||||
binding, token = grant(evaluation_id, attempt_id, worker, fence, credentials)
|
||||
try:
|
||||
payload = await component(request).manifest(binding, token)
|
||||
except WorkerAccessDenied:
|
||||
raise HTTPException(404, "worker grant rejected") from None
|
||||
except WorkerReadBusy:
|
||||
raise HTTPException(429, "worker reads temporarily limited") from None
|
||||
return Response(payload, media_type="application/json", headers={"Cache-Control": "no-store"})
|
||||
|
||||
|
||||
@router.get(
|
||||
"/{evaluation_id}/attempts/{attempt_id}/artifacts/{digest}",
|
||||
response_class=Response,
|
||||
responses={
|
||||
200: {
|
||||
"content": {
|
||||
"application/octet-stream": {"schema": {"type": "string", "format": "binary"}}
|
||||
}
|
||||
}
|
||||
},
|
||||
)
|
||||
async def artifact(
|
||||
evaluation_id: UUID,
|
||||
attempt_id: UUID,
|
||||
digest: str,
|
||||
request: Request,
|
||||
worker: Annotated[UUID, Header(alias="X-Worker-ID")],
|
||||
fence: Annotated[int, Header(alias="X-Fencing-Version", ge=1)],
|
||||
credentials: Annotated[HTTPAuthorizationCredentials | None, Security(bearer)],
|
||||
):
|
||||
binding, token = grant(evaluation_id, attempt_id, worker, fence, credentials)
|
||||
try:
|
||||
payload = await component(request).artifact(binding, token, digest)
|
||||
except WorkerAccessDenied:
|
||||
raise HTTPException(404, "worker grant rejected") from None
|
||||
except WorkerReadBusy:
|
||||
raise HTTPException(429, "worker reads temporarily limited") from None
|
||||
return Response(
|
||||
payload,
|
||||
media_type="application/octet-stream",
|
||||
headers={"Cache-Control": "no-store", "X-Content-Type-Options": "nosniff"},
|
||||
)
|
||||
@@ -6,8 +6,6 @@ from pathlib import Path
|
||||
|
||||
from kf_oj.api.app import create_app
|
||||
from kf_oj.contracts.judge import JudgeTaskNotification
|
||||
from kf_oj.contracts.judging import WIRE_MODELS
|
||||
from kf_oj.contracts.judging_v2 import WIRE_MODELS as WIRE_MODELS_V2
|
||||
from kf_oj.core.settings import Settings
|
||||
|
||||
|
||||
@@ -25,24 +23,6 @@ def main() -> None:
|
||||
Settings(_env_file=None, environment="test")
|
||||
).openapi(),
|
||||
}
|
||||
for name, model in WIRE_MODELS.items():
|
||||
document = model.model_json_schema()
|
||||
document["$schema"] = "https://json-schema.org/draft/2020-12/schema"
|
||||
document["$id"] = f"urn:kf-oj:{name}:v1"
|
||||
document["$comment"] = (
|
||||
"Structural schema; also run versioned model validation for cross-field "
|
||||
"budgets, result consistency, UUID canonical form and UTF-8 byte limits."
|
||||
)
|
||||
outputs[root / f"contracts/schemas/{name}.v1.schema.json"] = document
|
||||
for name, model in WIRE_MODELS_V2.items():
|
||||
document = model.model_json_schema()
|
||||
document["$schema"] = "https://json-schema.org/draft/2020-12/schema"
|
||||
document["$id"] = f"urn:kf-oj:{name}:v2"
|
||||
document["$comment"] = (
|
||||
"Multi-runtime protocol. Also validate runtime/source/attempt bindings, startup "
|
||||
"and node budgets, canonical bounded class sets and result consistency."
|
||||
)
|
||||
outputs[root / f"contracts/schemas/{name}.v2.schema.json"] = document
|
||||
stale = []
|
||||
for path, document in outputs.items():
|
||||
content = json.dumps(document, indent=2, sort_keys=True, ensure_ascii=False) + "\n"
|
||||
|
||||
@@ -1,22 +0,0 @@
|
||||
from datetime import datetime
|
||||
from typing import Annotated
|
||||
|
||||
from pydantic import Field, SecretStr
|
||||
|
||||
from kf_oj.contracts.judging import Digest, Identifier, WireModel
|
||||
|
||||
|
||||
class LoginRequest(WireModel):
|
||||
username: Annotated[str, Field(strict=True, pattern=r"^[a-z0-9_]{3,24}$")]
|
||||
password: Annotated[SecretStr, Field(min_length=12, max_length=128)]
|
||||
|
||||
|
||||
class CurrentUser(WireModel):
|
||||
id: Identifier
|
||||
username: str
|
||||
|
||||
|
||||
class LoginResponse(WireModel):
|
||||
user: CurrentUser
|
||||
expires_at: datetime
|
||||
csrf_token: Digest = Field(repr=False)
|
||||
@@ -1,9 +1,7 @@
|
||||
from typing import Annotated, Literal
|
||||
from typing import Literal
|
||||
from uuid import UUID
|
||||
|
||||
from pydantic import BaseModel, BeforeValidator, ConfigDict
|
||||
|
||||
from kf_oj.contracts.judging import integer_version
|
||||
from pydantic import BaseModel, ConfigDict
|
||||
|
||||
|
||||
class JudgeTaskNotification(BaseModel):
|
||||
@@ -11,6 +9,6 @@ class JudgeTaskNotification(BaseModel):
|
||||
|
||||
model_config = ConfigDict(extra="forbid", frozen=True)
|
||||
|
||||
schema_version: Annotated[Literal[1], BeforeValidator(integer_version)]
|
||||
schema_version: Literal[1]
|
||||
kind: Literal["judge.evaluation"]
|
||||
evaluation_id: UUID
|
||||
|
||||
@@ -1,48 +0,0 @@
|
||||
"""Pure transition/fencing rules; callers still need a locked database transaction."""
|
||||
|
||||
from enum import StrEnum
|
||||
|
||||
from kf_oj.contracts.judging import LeaseBinding
|
||||
|
||||
|
||||
class EvaluationState(StrEnum):
|
||||
QUEUED = "queued"
|
||||
CLAIMED = "claimed"
|
||||
COMPILING = "compiling"
|
||||
RUNNING = "running"
|
||||
CANCELLING = "cancelling"
|
||||
FINISHED = "finished"
|
||||
CANCELLED = "cancelled"
|
||||
FAILED = "failed"
|
||||
|
||||
def allows(self, target: "EvaluationState") -> bool:
|
||||
transitions = {
|
||||
self.QUEUED: {self.CLAIMED, self.CANCELLED},
|
||||
self.CLAIMED: {self.COMPILING, self.CANCELLING, self.FAILED, self.QUEUED},
|
||||
self.COMPILING: {
|
||||
self.RUNNING,
|
||||
self.FINISHED,
|
||||
self.CANCELLING,
|
||||
self.FAILED,
|
||||
self.QUEUED,
|
||||
},
|
||||
self.RUNNING: {self.FINISHED, self.CANCELLING, self.FAILED, self.QUEUED},
|
||||
self.CANCELLING: {self.CANCELLED, self.FAILED, self.QUEUED},
|
||||
}
|
||||
return target in transitions.get(self, set())
|
||||
|
||||
|
||||
def require_current_lease(
|
||||
expected: LeaseBinding,
|
||||
supplied: LeaseBinding,
|
||||
expires_at_unix_ms: int,
|
||||
now_unix_ms: int,
|
||||
) -> None:
|
||||
"""Identity/version equality and a strict deadline; never trust client time."""
|
||||
if expected != supplied or now_unix_ms < 0 or now_unix_ms >= expires_at_unix_ms:
|
||||
raise ValueError("stale or foreign execution authority")
|
||||
|
||||
|
||||
def require_transition(current: EvaluationState, target: EvaluationState) -> None:
|
||||
if not current.allows(target):
|
||||
raise ValueError("invalid evaluation transition")
|
||||
@@ -1,403 +0,0 @@
|
||||
"""Versioned judge wire models. Authorization and persistence belong to services."""
|
||||
|
||||
import json
|
||||
from enum import StrEnum
|
||||
from typing import Annotated, Literal, Self
|
||||
from uuid import UUID
|
||||
|
||||
from pydantic import (
|
||||
BaseModel,
|
||||
BeforeValidator,
|
||||
ConfigDict,
|
||||
Field,
|
||||
StrictBool,
|
||||
model_validator,
|
||||
)
|
||||
|
||||
MIB = 1024 * 1024
|
||||
I64_MAX = (1 << 63) - 1
|
||||
MAX_SOURCE_BYTES = 256 * 1024
|
||||
MAX_PACKAGE_BYTES = 64 * MIB
|
||||
MAX_CASES = 256
|
||||
MAX_JOB_WALL_US = 3_600_000_000
|
||||
MAX_WIRE_BYTES = 256 * 1024
|
||||
|
||||
|
||||
def integer_version(value: object) -> object:
|
||||
if type(value) is not int:
|
||||
raise ValueError("schema version must be an integer")
|
||||
return value
|
||||
|
||||
|
||||
def canonical_id(value: object) -> UUID:
|
||||
try:
|
||||
parsed = UUID(str(value))
|
||||
except (ValueError, TypeError, AttributeError) as error:
|
||||
raise ValueError("invalid identifier") from error
|
||||
if parsed.int == 0 or str(value) != str(parsed):
|
||||
raise ValueError("identifier must be non-nil and canonical")
|
||||
return parsed
|
||||
|
||||
|
||||
Version = Annotated[Literal[1], BeforeValidator(integer_version)]
|
||||
Identifier = Annotated[UUID, BeforeValidator(canonical_id)]
|
||||
Digest = Annotated[str, Field(strict=True, min_length=64, max_length=64, pattern=r"^[0-9a-f]{64}$")]
|
||||
Label = Annotated[str, Field(strict=True, pattern=r"^[a-z0-9][a-z0-9_-]{0,63}$")]
|
||||
Counter = Annotated[int, Field(strict=True, ge=1, le=I64_MAX)]
|
||||
Measurement = Annotated[int, Field(strict=True, ge=0, le=I64_MAX)]
|
||||
|
||||
|
||||
class WireModel(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid", frozen=True)
|
||||
|
||||
|
||||
class Language(StrEnum):
|
||||
C17 = "c17"
|
||||
CPP20 = "cpp20"
|
||||
|
||||
|
||||
class Verdict(StrEnum):
|
||||
ACCEPTED = "accepted"
|
||||
WRONG_ANSWER = "wrong_answer"
|
||||
COMPILATION_ERROR = "compilation_error"
|
||||
RUNTIME_ERROR = "runtime_error"
|
||||
TIME_LIMIT = "time_limit_exceeded"
|
||||
MEMORY_LIMIT = "memory_limit_exceeded"
|
||||
OUTPUT_LIMIT = "output_limit_exceeded"
|
||||
CANCELLED = "cancelled"
|
||||
SYSTEM_ERROR = "system_error"
|
||||
|
||||
|
||||
class CompilationStatus(StrEnum):
|
||||
SUCCEEDED = "succeeded"
|
||||
COMPILATION_ERROR = "compilation_error"
|
||||
COMPILATION_LIMIT = "compilation_limit"
|
||||
CANCELLED = "cancelled"
|
||||
SYSTEM_ERROR = "system_error"
|
||||
|
||||
|
||||
class FailureCode(StrEnum):
|
||||
ARTIFACT_UNAVAILABLE = "artifact_unavailable"
|
||||
SANDBOX_UNAVAILABLE = "sandbox_unavailable"
|
||||
CLEANUP_FAILED = "cleanup_failed"
|
||||
LEASE_LOST = "lease_lost"
|
||||
JOB_DEADLINE = "job_deadline"
|
||||
|
||||
|
||||
class ArtifactReference(WireModel):
|
||||
digest: Digest
|
||||
bytes: Annotated[int, Field(strict=True, ge=0, le=MAX_PACKAGE_BYTES)]
|
||||
|
||||
|
||||
class Toolchain(WireModel):
|
||||
toolchain_id: Label
|
||||
image_kind: Literal["linux_gnu_cxx_x86_64_v1"]
|
||||
image_revision: Annotated[int, Field(strict=True, ge=1, le=(1 << 32) - 1)]
|
||||
|
||||
|
||||
class RunLimits(WireModel):
|
||||
cpu_time_us: Annotated[int, Field(strict=True, ge=1000, le=60_000_000)]
|
||||
wall_time_us: Annotated[int, Field(strict=True, ge=1000, le=120_000_000)]
|
||||
memory_bytes: Annotated[int, Field(strict=True, ge=1, le=2048 * MIB)]
|
||||
stack_bytes: Annotated[int, Field(strict=True, ge=1, le=256 * MIB)]
|
||||
processes: Annotated[int, Field(strict=True, ge=1, le=128)]
|
||||
open_files: Annotated[int, Field(strict=True, ge=3, le=1024)]
|
||||
stdout_bytes: Annotated[int, Field(strict=True, ge=1, le=16 * MIB)]
|
||||
stderr_bytes: Annotated[int, Field(strict=True, ge=1, le=16 * MIB)]
|
||||
file_bytes: Annotated[int, Field(strict=True, ge=1, le=64 * MIB)]
|
||||
workdir_bytes: Annotated[int, Field(strict=True, ge=1, le=256 * MIB)]
|
||||
|
||||
@model_validator(mode="after")
|
||||
def consistent(self) -> Self:
|
||||
if (
|
||||
self.stack_bytes > self.memory_bytes
|
||||
or self.file_bytes > self.workdir_bytes
|
||||
or self.stdout_bytes > self.file_bytes
|
||||
or self.stderr_bytes > self.file_bytes
|
||||
or self.stdout_bytes + self.stderr_bytes > self.workdir_bytes
|
||||
):
|
||||
raise ValueError("inconsistent resource limits")
|
||||
return self
|
||||
|
||||
def fits(self, ceiling: "RunLimits") -> bool:
|
||||
return all(
|
||||
getattr(self, name) <= getattr(ceiling, name) for name in type(self).model_fields
|
||||
)
|
||||
|
||||
|
||||
class TestCase(WireModel):
|
||||
case_id: Identifier
|
||||
input: ArtifactReference
|
||||
expected: ArtifactReference
|
||||
|
||||
|
||||
class JobManifest(WireModel):
|
||||
schema_version: Version
|
||||
kind: Literal["judge.manifest"]
|
||||
evaluation_id: Identifier
|
||||
submission_id: Identifier
|
||||
problem_version_id: Identifier
|
||||
language: Language
|
||||
toolchain: Toolchain
|
||||
source: ArtifactReference
|
||||
cases: Annotated[tuple[TestCase, ...], Field(min_length=1, max_length=MAX_CASES)]
|
||||
comparison: Literal["exact_bytes"]
|
||||
compile_limits: RunLimits
|
||||
run_limits: RunLimits
|
||||
job_wall_time_us: Annotated[int, Field(strict=True, ge=1000, le=MAX_JOB_WALL_US)]
|
||||
|
||||
def package_bytes(self) -> int:
|
||||
# Count references, including duplicates: this bounds actual staged/read bytes.
|
||||
return self.source.bytes + sum(c.input.bytes + c.expected.bytes for c in self.cases)
|
||||
|
||||
@model_validator(mode="after")
|
||||
def consistent(self) -> Self:
|
||||
if not 1 <= self.source.bytes <= MAX_SOURCE_BYTES:
|
||||
raise ValueError("source size outside policy")
|
||||
if len({c.case_id for c in self.cases}) != len(self.cases):
|
||||
raise ValueError("duplicate testcase identifiers")
|
||||
if self.package_bytes() > MAX_PACKAGE_BYTES:
|
||||
raise ValueError("test package outside policy")
|
||||
if any(c.expected.bytes > self.run_limits.stdout_bytes for c in self.cases):
|
||||
raise ValueError("expected output exceeds output budget")
|
||||
required_wall = (
|
||||
self.compile_limits.wall_time_us + len(self.cases) * self.run_limits.wall_time_us
|
||||
)
|
||||
if required_wall > self.job_wall_time_us:
|
||||
raise ValueError("job deadline cannot cover execution budgets")
|
||||
return self
|
||||
|
||||
|
||||
def manifest_bytes(manifest: JobManifest) -> bytes:
|
||||
"""Stable UTF-8 bytes for persisted snapshots and their SHA-256 identity."""
|
||||
return json.dumps(
|
||||
manifest.model_dump(mode="json"),
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
ensure_ascii=False,
|
||||
allow_nan=False,
|
||||
).encode("utf-8")
|
||||
|
||||
|
||||
class LeaseBinding(WireModel):
|
||||
evaluation_id: Identifier
|
||||
attempt_id: Identifier
|
||||
worker_id: Identifier
|
||||
fencing_version: Counter
|
||||
|
||||
|
||||
class LeaseGrant(WireModel):
|
||||
schema_version: Version
|
||||
kind: Literal["judge.lease"]
|
||||
binding: LeaseBinding
|
||||
lease_token: Digest = Field(repr=False)
|
||||
attempt_number: Annotated[int, Field(strict=True, ge=1, le=5)]
|
||||
expires_at_unix_ms: Counter
|
||||
manifest_digest: Digest
|
||||
|
||||
|
||||
class StageReport(WireModel):
|
||||
schema_version: Version
|
||||
kind: Literal["judge.stage"]
|
||||
binding: LeaseBinding
|
||||
sequence: Counter
|
||||
stage: Literal["compiling", "running"]
|
||||
case_index: Annotated[int, Field(strict=True, ge=0, lt=MAX_CASES)] | None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def consistent(self) -> Self:
|
||||
if (self.stage == "running") != (self.case_index is not None):
|
||||
raise ValueError("testcase index only belongs to running phase")
|
||||
return self
|
||||
|
||||
|
||||
class CompilationReport(WireModel):
|
||||
status: CompilationStatus
|
||||
program: ArtifactReference | None
|
||||
diagnostics: Annotated[str, Field(strict=True, max_length=65536)] = Field(repr=False)
|
||||
diagnostics_truncated: StrictBool
|
||||
cpu_time_us: Measurement | None
|
||||
memory_peak_bytes: Measurement | None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def consistent(self) -> Self:
|
||||
if len(self.diagnostics.encode("utf-8")) > 65536:
|
||||
raise ValueError("compiler diagnostics exceed byte budget")
|
||||
if (self.status == CompilationStatus.SUCCEEDED) != (self.program is not None):
|
||||
raise ValueError("compiled artifact only belongs to successful compilation")
|
||||
if self.program is not None and self.program.bytes == 0:
|
||||
raise ValueError("compiled artifact cannot be empty")
|
||||
return self
|
||||
|
||||
|
||||
class CaseReport(WireModel):
|
||||
case_id: Identifier
|
||||
verdict: Verdict
|
||||
cpu_time_us: Measurement | None
|
||||
memory_peak_bytes: Measurement | None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def consistent(self) -> Self:
|
||||
if self.verdict == Verdict.COMPILATION_ERROR:
|
||||
raise ValueError("compilation failure is not a testcase outcome")
|
||||
return self
|
||||
|
||||
|
||||
class JudgementReport(WireModel):
|
||||
schema_version: Version
|
||||
kind: Literal["judge.result"]
|
||||
binding: LeaseBinding
|
||||
manifest_digest: Digest
|
||||
compilation: CompilationReport
|
||||
cases: Annotated[tuple[CaseReport, ...], Field(max_length=MAX_CASES)]
|
||||
verdict: Verdict
|
||||
cleanup_complete: StrictBool
|
||||
failure_code: FailureCode | None
|
||||
run_cpu_time_us: Measurement | None
|
||||
run_memory_peak_bytes: Measurement | None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def consistent(self) -> Self:
|
||||
if len({c.case_id for c in self.cases}) != len(self.cases):
|
||||
raise ValueError("duplicate testcase results")
|
||||
if not self.cleanup_complete and self.failure_code != FailureCode.CLEANUP_FAILED:
|
||||
raise ValueError("incomplete cleanup requires its own failure evidence")
|
||||
if self.cleanup_complete and self.failure_code == FailureCode.CLEANUP_FAILED:
|
||||
raise ValueError("cleanup failure cannot claim completed cleanup")
|
||||
if (self.verdict == Verdict.SYSTEM_ERROR) != (self.failure_code is not None):
|
||||
raise ValueError("infrastructure failure and verdict must agree")
|
||||
terminal = next(
|
||||
(
|
||||
i
|
||||
for i, c in enumerate(self.cases)
|
||||
if c.verdict in (Verdict.SYSTEM_ERROR, Verdict.CANCELLED)
|
||||
),
|
||||
None,
|
||||
)
|
||||
if terminal is not None and terminal != len(self.cases) - 1:
|
||||
raise ValueError("results after fatal testcase outcome")
|
||||
if self.compilation.status != CompilationStatus.SUCCEEDED:
|
||||
mapped = {
|
||||
CompilationStatus.COMPILATION_ERROR: Verdict.COMPILATION_ERROR,
|
||||
CompilationStatus.COMPILATION_LIMIT: Verdict.COMPILATION_ERROR,
|
||||
CompilationStatus.CANCELLED: Verdict.CANCELLED,
|
||||
CompilationStatus.SYSTEM_ERROR: Verdict.SYSTEM_ERROR,
|
||||
}[self.compilation.status]
|
||||
if self.cases or self.verdict != mapped:
|
||||
raise ValueError("failed compilation cannot have runtime results")
|
||||
elif self.failure_code is None:
|
||||
if any(c.verdict == Verdict.SYSTEM_ERROR for c in self.cases):
|
||||
raise ValueError("infrastructure failure requires failure code")
|
||||
if not self.cases and self.verdict != Verdict.CANCELLED:
|
||||
raise ValueError("missing testcase results cannot be a completed judgement")
|
||||
if self.cases and self.verdict != Verdict.CANCELLED:
|
||||
failures = (Verdict.SYSTEM_ERROR, Verdict.CANCELLED)
|
||||
summary = next(
|
||||
(c.verdict for c in self.cases if c.verdict in failures),
|
||||
next(
|
||||
(c.verdict for c in self.cases if c.verdict != Verdict.ACCEPTED),
|
||||
Verdict.ACCEPTED,
|
||||
),
|
||||
)
|
||||
if self.verdict != summary:
|
||||
raise ValueError("testcase summary disagrees")
|
||||
cpu = (
|
||||
sum(c.cpu_time_us for c in self.cases if c.cpu_time_us is not None)
|
||||
if self.cases and all(c.cpu_time_us is not None for c in self.cases)
|
||||
else None
|
||||
)
|
||||
memory = (
|
||||
max(c.memory_peak_bytes for c in self.cases if c.memory_peak_bytes is not None)
|
||||
if self.cases and all(c.memory_peak_bytes is not None for c in self.cases)
|
||||
else None
|
||||
)
|
||||
if self.run_cpu_time_us != cpu or self.run_memory_peak_bytes != memory:
|
||||
raise ValueError("runtime measurement summary disagrees")
|
||||
return self
|
||||
|
||||
def validate_for_manifest(self, manifest: JobManifest, digest: str) -> None:
|
||||
if self.binding.evaluation_id != manifest.evaluation_id or self.manifest_digest != digest:
|
||||
raise ValueError("report does not belong to authorized manifest")
|
||||
if [c.case_id for c in self.cases] != [
|
||||
c.case_id for c in manifest.cases[: len(self.cases)]
|
||||
]:
|
||||
raise ValueError("testcase results differ from authorized order")
|
||||
if self.verdict not in (Verdict.COMPILATION_ERROR, Verdict.CANCELLED, Verdict.SYSTEM_ERROR):
|
||||
if len(self.cases) != len(manifest.cases):
|
||||
raise ValueError("incomplete ordinary judgement")
|
||||
|
||||
|
||||
class AdmissionPolicy(WireModel):
|
||||
"""Trusted node policy. A manifest never supplies or replaces this policy."""
|
||||
|
||||
max_concurrent_jobs: Annotated[int, Field(strict=True, ge=1, le=64)] = 1
|
||||
prefetch: Annotated[int, Field(strict=True, ge=1, le=64)] = 1
|
||||
max_retries: Annotated[int, Field(strict=True, ge=0, le=4)] = 2
|
||||
max_job_wall_time_us: Annotated[int, Field(strict=True, ge=1000, le=MAX_JOB_WALL_US)]
|
||||
max_package_bytes: Annotated[int, Field(strict=True, ge=1, le=MAX_PACKAGE_BYTES)]
|
||||
toolchain: Toolchain
|
||||
compile_ceiling: RunLimits
|
||||
run_ceiling: RunLimits
|
||||
|
||||
@model_validator(mode="after")
|
||||
def consistent(self) -> Self:
|
||||
if self.prefetch > self.max_concurrent_jobs:
|
||||
raise ValueError("prefetch exceeds available job slots")
|
||||
return self
|
||||
|
||||
def admit(self, manifest: JobManifest, attempt_number: int) -> None:
|
||||
if type(attempt_number) is not int or not 1 <= attempt_number <= self.max_retries + 1:
|
||||
raise ValueError("attempt retry budget exhausted")
|
||||
if (
|
||||
manifest.toolchain != self.toolchain
|
||||
or not manifest.compile_limits.fits(self.compile_ceiling)
|
||||
or not manifest.run_limits.fits(self.run_ceiling)
|
||||
or manifest.job_wall_time_us > self.max_job_wall_time_us
|
||||
or manifest.package_bytes() > self.max_package_bytes
|
||||
):
|
||||
raise ValueError("job exceeds trusted node budget")
|
||||
|
||||
|
||||
class StoppedReport(WireModel):
|
||||
schema_version: Version
|
||||
kind: Literal["judge.stopped"]
|
||||
binding: LeaseBinding
|
||||
cleanup_complete: StrictBool
|
||||
|
||||
|
||||
WIRE_MODELS = {
|
||||
"judge-manifest": JobManifest,
|
||||
"judge-lease": LeaseGrant,
|
||||
"judge-stage": StageReport,
|
||||
"judge-result": JudgementReport,
|
||||
"judge-admission": AdmissionPolicy,
|
||||
"judge-stopped": StoppedReport,
|
||||
}
|
||||
|
||||
|
||||
def decode_wire[T: BaseModel](model: type[T], payload: bytes) -> T:
|
||||
"""UTF-8, bounded size and no ambiguous duplicate members before model validation."""
|
||||
if len(payload) > MAX_WIRE_BYTES:
|
||||
raise ValueError("wire payload exceeds byte budget")
|
||||
|
||||
def unique_members(pairs: list[tuple[str, object]]) -> dict[str, object]:
|
||||
result = {}
|
||||
for key, value in pairs:
|
||||
if key in result:
|
||||
raise ValueError("duplicate JSON member")
|
||||
result[key] = value
|
||||
return result
|
||||
|
||||
def invalid_constant(_: str) -> None:
|
||||
raise ValueError("invalid JSON constant")
|
||||
|
||||
try:
|
||||
value = json.loads(
|
||||
payload.decode("utf-8"),
|
||||
object_pairs_hook=unique_members,
|
||||
parse_constant=invalid_constant,
|
||||
)
|
||||
return model.model_validate(value)
|
||||
except (ValueError, TypeError):
|
||||
# Errors at the transport boundary must not echo credentials or hidden inputs.
|
||||
raise ValueError("invalid wire payload") from None
|
||||
@@ -1,358 +0,0 @@
|
||||
"""Explicit multi-runtime protocol; existing v1 snapshots remain byte-for-byte stable."""
|
||||
|
||||
import re
|
||||
from enum import StrEnum
|
||||
from typing import Annotated, Literal, Self
|
||||
|
||||
from pydantic import BeforeValidator, Field, model_validator
|
||||
|
||||
from kf_oj.contracts import judging as v1
|
||||
|
||||
Version = Annotated[Literal[2], BeforeValidator(v1.integer_version)]
|
||||
MAX_CLASS_FILES = 256
|
||||
JVM_NATIVE_RESERVE_BYTES = 160 * v1.MIB
|
||||
JVM_THREAD_STACK_BYTES = 256 * 1024
|
||||
|
||||
|
||||
class Language(StrEnum):
|
||||
C17 = "c17"
|
||||
CPP20 = "cpp20"
|
||||
PYTHON313 = "python313"
|
||||
JAVA21 = "java21"
|
||||
|
||||
|
||||
class PreparationKind(StrEnum):
|
||||
COMPILE = "compile"
|
||||
SYNTAX_CHECK = "syntax_check"
|
||||
|
||||
|
||||
class ProgramKind(StrEnum):
|
||||
NATIVE = "native_executable"
|
||||
PYTHON = "python_source"
|
||||
JAVA = "java_classes"
|
||||
|
||||
|
||||
class RuntimeIdentity(v1.WireModel):
|
||||
"""An exact identity matched against trusted configuration, never a host path."""
|
||||
|
||||
toolchain_id: v1.Label
|
||||
image_kind: Literal[
|
||||
"linux_gnu_cxx_x86_64_v1", "linux_cpython313_x86_64_v1", "linux_openjdk21_x86_64_v1"
|
||||
]
|
||||
image_revision: Annotated[int, Field(strict=True, ge=1, le=(1 << 32) - 1)]
|
||||
image_digest: v1.Digest
|
||||
runtime_version: Annotated[
|
||||
str, Field(strict=True, pattern=r"^[A-Za-z0-9][A-Za-z0-9_.+-]{0,63}$")
|
||||
]
|
||||
|
||||
|
||||
def runtime_contract(language: Language) -> tuple[str, str, str, ProgramKind, PreparationKind]:
|
||||
"""Fixed source/profile names. Runtime callers cannot introduce argv or paths."""
|
||||
return {
|
||||
Language.C17: (
|
||||
"main.c",
|
||||
"compile_c",
|
||||
"native",
|
||||
ProgramKind.NATIVE,
|
||||
PreparationKind.COMPILE,
|
||||
),
|
||||
Language.CPP20: (
|
||||
"main.cpp",
|
||||
"compile_cpp",
|
||||
"native",
|
||||
ProgramKind.NATIVE,
|
||||
PreparationKind.COMPILE,
|
||||
),
|
||||
Language.PYTHON313: (
|
||||
"main.py",
|
||||
"python_check",
|
||||
"python_run",
|
||||
ProgramKind.PYTHON,
|
||||
PreparationKind.SYNTAX_CHECK,
|
||||
),
|
||||
Language.JAVA21: (
|
||||
"Main.java",
|
||||
"java_compile",
|
||||
"java_run",
|
||||
ProgramKind.JAVA,
|
||||
PreparationKind.COMPILE,
|
||||
),
|
||||
}[language]
|
||||
|
||||
|
||||
def validate_runtime(language: Language, identity: RuntimeIdentity) -> None:
|
||||
expected = {
|
||||
Language.C17: ("gnu-cxx-v1", "linux_gnu_cxx_x86_64_v1"),
|
||||
Language.CPP20: ("gnu-cxx-v1", "linux_gnu_cxx_x86_64_v1"),
|
||||
Language.PYTHON313: ("cpython313-v1", "linux_cpython313_x86_64_v1"),
|
||||
Language.JAVA21: ("openjdk21-v1", "linux_openjdk21_x86_64_v1"),
|
||||
}[language]
|
||||
if (identity.toolchain_id, identity.image_kind) != expected:
|
||||
raise ValueError("runtime family differs from language contract")
|
||||
if language == Language.PYTHON313 and identity.runtime_version != "3.13.16":
|
||||
raise ValueError("unsupported CPython version")
|
||||
if language == Language.JAVA21 and not re.fullmatch(
|
||||
r"21\.[0-9]+\.[0-9]+(?:\.[0-9]+)?\+[1-9][0-9]*(?:-LTS)?", identity.runtime_version
|
||||
):
|
||||
raise ValueError("JDK version must identify a fixed Java 21 build")
|
||||
|
||||
|
||||
def validate_startup_budget(language: Language, limits: v1.RunLimits, *, preparing: bool) -> None:
|
||||
"""Admission floors, not multipliers on problem limits; real execution still required."""
|
||||
if language == Language.PYTHON313:
|
||||
floor = (10_000, 50_000, 32 * v1.MIB, 1, 16)
|
||||
elif language == Language.JAVA21:
|
||||
floor = (
|
||||
(1_000_000, 3_000_000, 384 * v1.MIB, 16, 32)
|
||||
if preparing
|
||||
else (200_000, 1_000_000, 256 * v1.MIB, 16, 32)
|
||||
)
|
||||
else:
|
||||
return
|
||||
actual = (
|
||||
limits.cpu_time_us,
|
||||
limits.wall_time_us,
|
||||
limits.memory_bytes,
|
||||
limits.processes,
|
||||
limits.open_files,
|
||||
)
|
||||
if any(value < minimum for value, minimum in zip(actual, floor, strict=True)):
|
||||
raise ValueError("budget below runtime startup policy")
|
||||
if language == Language.JAVA21:
|
||||
jvm_heap_bytes(limits)
|
||||
|
||||
|
||||
def jvm_heap_bytes(limits: v1.RunLimits) -> int:
|
||||
"""Reserve capped metaspace/code/native and every allowed thread stack inside cgroup."""
|
||||
reserve = JVM_NATIVE_RESERVE_BYTES + limits.processes * JVM_THREAD_STACK_BYTES
|
||||
heap = (limits.memory_bytes - reserve) // v1.MIB * v1.MIB
|
||||
if heap < 32 * v1.MIB or limits.stack_bytes < JVM_THREAD_STACK_BYTES:
|
||||
raise ValueError("budget cannot cover JVM heap and explicit non-heap reserve")
|
||||
return heap
|
||||
|
||||
|
||||
class LanguageConfiguration(v1.WireModel):
|
||||
language: Language
|
||||
toolchain: RuntimeIdentity
|
||||
compile_limits: v1.RunLimits
|
||||
run_limits: v1.RunLimits
|
||||
job_wall_time_us: Annotated[int, Field(strict=True, ge=1000, le=v1.MAX_JOB_WALL_US)]
|
||||
|
||||
@model_validator(mode="after")
|
||||
def runtime_and_budget(self) -> Self:
|
||||
validate_runtime(self.language, self.toolchain)
|
||||
validate_startup_budget(self.language, self.compile_limits, preparing=True)
|
||||
validate_startup_budget(self.language, self.run_limits, preparing=False)
|
||||
if self.compile_limits.wall_time_us + self.run_limits.wall_time_us > self.job_wall_time_us:
|
||||
raise ValueError("job deadline cannot cover preparation and execution")
|
||||
return self
|
||||
|
||||
|
||||
class JobManifest(v1.JobManifest):
|
||||
schema_version: Version
|
||||
language: Language
|
||||
toolchain: RuntimeIdentity
|
||||
|
||||
@model_validator(mode="after")
|
||||
def runtime_and_budget(self) -> Self:
|
||||
validate_runtime(self.language, self.toolchain)
|
||||
validate_startup_budget(self.language, self.compile_limits, preparing=True)
|
||||
validate_startup_budget(self.language, self.run_limits, preparing=False)
|
||||
return self
|
||||
|
||||
|
||||
class ProgramFile(v1.WireModel):
|
||||
path: Annotated[str, Field(strict=True, min_length=1, max_length=128)]
|
||||
artifact: v1.ArtifactReference
|
||||
|
||||
|
||||
class CompiledProgram(v1.WireModel):
|
||||
"""Sealed file identities bound to a particular source, runtime and lease attempt."""
|
||||
|
||||
kind: ProgramKind
|
||||
binding: v1.LeaseBinding
|
||||
source: v1.ArtifactReference
|
||||
toolchain: RuntimeIdentity
|
||||
files: Annotated[tuple[ProgramFile, ...], Field(min_length=1, max_length=MAX_CLASS_FILES)]
|
||||
|
||||
@property
|
||||
def bytes(self) -> int:
|
||||
return sum(file.artifact.bytes for file in self.files)
|
||||
|
||||
@model_validator(mode="after")
|
||||
def bounded_files(self) -> Self:
|
||||
names = [file.path for file in self.files]
|
||||
if names != sorted(set(names)):
|
||||
raise ValueError("program files must have unique canonical order")
|
||||
if not 1 <= self.source.bytes <= v1.MAX_SOURCE_BYTES:
|
||||
raise ValueError("program source outside policy")
|
||||
if self.bytes > v1.MAX_PACKAGE_BYTES or any(
|
||||
file.artifact.bytes == 0 for file in self.files
|
||||
):
|
||||
raise ValueError("program files empty or outside byte policy")
|
||||
if self.kind == ProgramKind.NATIVE:
|
||||
validate_runtime(Language.C17, self.toolchain)
|
||||
if names != ["main"] or self.toolchain.image_kind != "linux_gnu_cxx_x86_64_v1":
|
||||
raise ValueError("native program requires its fixed executable and runtime")
|
||||
elif self.kind == ProgramKind.PYTHON:
|
||||
validate_runtime(Language.PYTHON313, self.toolchain)
|
||||
if names != ["main.py"] or self.files[0].artifact != self.source:
|
||||
raise ValueError("Python program must be the exact checked source")
|
||||
else:
|
||||
validate_runtime(Language.JAVA21, self.toolchain)
|
||||
if "Main.class" not in names or any(
|
||||
not re.fullmatch(r"[A-Za-z_$][A-Za-z0-9_$]*\.class", name) for name in names
|
||||
):
|
||||
raise ValueError("Java requires a bounded default-package Main class set")
|
||||
return self
|
||||
|
||||
|
||||
class CompilationReport(v1.CompilationReport):
|
||||
program: CompiledProgram | None
|
||||
|
||||
|
||||
class JudgementReport(v1.JudgementReport):
|
||||
schema_version: Version
|
||||
compilation: CompilationReport
|
||||
|
||||
@model_validator(mode="after")
|
||||
def program_attempt(self) -> Self:
|
||||
if (
|
||||
self.compilation.program is not None
|
||||
and self.compilation.program.binding != self.binding
|
||||
):
|
||||
raise ValueError("program belongs to another attempt")
|
||||
return self
|
||||
|
||||
def validate_for_manifest(self, manifest: JobManifest, digest: str) -> None:
|
||||
super().validate_for_manifest(manifest, digest)
|
||||
program = self.compilation.program
|
||||
if program is not None and (
|
||||
program.source != manifest.source
|
||||
or program.toolchain != manifest.toolchain
|
||||
or program.kind != runtime_contract(manifest.language)[3]
|
||||
):
|
||||
raise ValueError("program differs from authorized source or runtime")
|
||||
|
||||
|
||||
class LeaseGrant(v1.LeaseGrant):
|
||||
schema_version: Version
|
||||
|
||||
|
||||
class StageReport(v1.StageReport):
|
||||
schema_version: Version
|
||||
preparation_kind: PreparationKind
|
||||
|
||||
def validate_for_manifest(self, manifest: JobManifest) -> None:
|
||||
if self.binding.evaluation_id != manifest.evaluation_id or (
|
||||
self.preparation_kind != runtime_contract(manifest.language)[4]
|
||||
):
|
||||
raise ValueError("stage differs from authorized language preparation")
|
||||
|
||||
|
||||
class StoppedReport(v1.StoppedReport):
|
||||
schema_version: Version
|
||||
|
||||
|
||||
class RuntimeCapability(v1.WireModel):
|
||||
language: Language
|
||||
toolchain: RuntimeIdentity
|
||||
compile_ceiling: v1.RunLimits
|
||||
run_ceiling: v1.RunLimits
|
||||
|
||||
@model_validator(mode="after")
|
||||
def runtime_and_budget(self) -> Self:
|
||||
validate_runtime(self.language, self.toolchain)
|
||||
validate_startup_budget(self.language, self.compile_ceiling, preparing=True)
|
||||
validate_startup_budget(self.language, self.run_ceiling, preparing=False)
|
||||
return self
|
||||
|
||||
|
||||
class AdmissionPolicy(v1.WireModel):
|
||||
schema_version: Version
|
||||
max_concurrent_jobs: Annotated[int, Field(strict=True, ge=1, le=64)] = 1
|
||||
prefetch: Annotated[int, Field(strict=True, ge=1, le=64)] = 1
|
||||
max_retries: Annotated[int, Field(strict=True, ge=0, le=4)] = 2
|
||||
max_job_wall_time_us: Annotated[int, Field(strict=True, ge=1000, le=v1.MAX_JOB_WALL_US)]
|
||||
max_package_bytes: Annotated[int, Field(strict=True, ge=1, le=v1.MAX_PACKAGE_BYTES)]
|
||||
capabilities: Annotated[tuple[RuntimeCapability, ...], Field(min_length=1, max_length=4)]
|
||||
|
||||
@model_validator(mode="after")
|
||||
def consistent(self) -> Self:
|
||||
if self.prefetch > self.max_concurrent_jobs:
|
||||
raise ValueError("prefetch exceeds available job slots")
|
||||
if len({cap.language for cap in self.capabilities}) != len(self.capabilities):
|
||||
raise ValueError("duplicate language capability")
|
||||
return self
|
||||
|
||||
def admit(self, manifest: JobManifest, attempt_number: int) -> None:
|
||||
# Revalidate constructed/copied values as well as transport-decoded models.
|
||||
manifest = JobManifest.model_validate(manifest.model_dump(mode="json"))
|
||||
AdmissionPolicy.model_validate(self.model_dump(mode="json"))
|
||||
if type(attempt_number) is not int or not 1 <= attempt_number <= self.max_retries + 1:
|
||||
raise ValueError("attempt retry budget exhausted")
|
||||
cap = next((cap for cap in self.capabilities if cap.language == manifest.language), None)
|
||||
if cap is None or (
|
||||
cap.toolchain != manifest.toolchain
|
||||
or not manifest.compile_limits.fits(cap.compile_ceiling)
|
||||
or not manifest.run_limits.fits(cap.run_ceiling)
|
||||
or manifest.job_wall_time_us > self.max_job_wall_time_us
|
||||
or manifest.package_bytes() > self.max_package_bytes
|
||||
):
|
||||
raise ValueError("job exceeds trusted runtime capability or budget")
|
||||
|
||||
|
||||
class AdmissionRegistry(v1.WireModel):
|
||||
"""Explicit operator opt-in for both protocol generations; no silent conversion."""
|
||||
|
||||
schema_version: Version
|
||||
legacy: v1.AdmissionPolicy | None
|
||||
multi_runtime: AdmissionPolicy | None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def at_least_one_protocol(self) -> Self:
|
||||
if self.legacy is None and self.multi_runtime is None:
|
||||
raise ValueError("node must advertise at least one explicit protocol capability")
|
||||
if self.legacy is not None and self.multi_runtime is not None:
|
||||
for field in ("max_concurrent_jobs", "prefetch", "max_retries"):
|
||||
if getattr(self.legacy, field) != getattr(self.multi_runtime, field):
|
||||
raise ValueError("protocol policies disagree on node capacity or retries")
|
||||
return self
|
||||
|
||||
def admit(self, manifest: v1.JobManifest | JobManifest, attempt_number: int) -> None:
|
||||
AdmissionRegistry.model_validate(self.model_dump(mode="json"))
|
||||
policy = self.legacy if manifest.schema_version == 1 else self.multi_runtime
|
||||
admit_manifest(policy, manifest, attempt_number)
|
||||
|
||||
|
||||
NodeAdmission = v1.AdmissionPolicy | AdmissionPolicy | AdmissionRegistry
|
||||
|
||||
|
||||
def admit_manifest(
|
||||
policy: NodeAdmission | None, manifest: v1.JobManifest | JobManifest, attempt_number: int
|
||||
) -> None:
|
||||
if type(manifest) not in (v1.JobManifest, JobManifest):
|
||||
raise ValueError("unsupported manifest model")
|
||||
# Copied/constructed values must pass the original strict protocol validators too.
|
||||
manifest = type(manifest).model_validate(manifest.model_dump(mode="json"))
|
||||
if isinstance(policy, AdmissionRegistry):
|
||||
policy.admit(manifest, attempt_number)
|
||||
elif manifest.schema_version == 1 and type(policy) is v1.AdmissionPolicy:
|
||||
v1.AdmissionPolicy.model_validate(policy.model_dump(mode="json")).admit(
|
||||
manifest, attempt_number
|
||||
)
|
||||
elif manifest.schema_version == 2 and type(policy) is AdmissionPolicy:
|
||||
policy.admit(manifest, attempt_number)
|
||||
else:
|
||||
raise ValueError("node does not advertise this manifest protocol")
|
||||
|
||||
|
||||
WIRE_MODELS = {
|
||||
"judge-manifest": JobManifest,
|
||||
"judge-lease": LeaseGrant,
|
||||
"judge-stage": StageReport,
|
||||
"judge-result": JudgementReport,
|
||||
"judge-admission": AdmissionPolicy,
|
||||
"judge-stopped": StoppedReport,
|
||||
"judge-registry": AdmissionRegistry,
|
||||
}
|
||||
@@ -1,35 +0,0 @@
|
||||
"""Version dispatch at storage/transport boundaries; never rewrite legacy snapshots."""
|
||||
|
||||
from pydantic import ConfigDict, RootModel
|
||||
|
||||
from kf_oj.contracts import judging as v1
|
||||
from kf_oj.contracts import judging_v2 as v2
|
||||
|
||||
Manifest = v1.JobManifest | v2.JobManifest
|
||||
Report = v1.JudgementReport | v2.JudgementReport
|
||||
|
||||
|
||||
class ManifestEnvelope(RootModel[Manifest]):
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
|
||||
class ReportEnvelope(RootModel[Report]):
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
|
||||
def decode_manifest(payload: bytes) -> Manifest:
|
||||
return v1.decode_wire(ManifestEnvelope, payload).root
|
||||
|
||||
|
||||
def decode_report(payload: bytes) -> Report:
|
||||
return v1.decode_wire(ReportEnvelope, payload).root
|
||||
|
||||
|
||||
def manifest_bytes(manifest: Manifest) -> bytes:
|
||||
return v1.manifest_bytes(manifest)
|
||||
|
||||
|
||||
def validate_report(report: Report, manifest: Manifest, digest: str) -> None:
|
||||
if report.schema_version != manifest.schema_version:
|
||||
raise ValueError("report protocol differs from authorized manifest")
|
||||
report.validate_for_manifest(manifest, digest)
|
||||
@@ -1,58 +0,0 @@
|
||||
from typing import Annotated, Literal
|
||||
|
||||
from pydantic import Field
|
||||
|
||||
from kf_oj.contracts.judging import Identifier, Label, RunLimits, WireModel
|
||||
from kf_oj.contracts.judging_v2 import Language, PreparationKind, runtime_contract
|
||||
|
||||
|
||||
class PublicSample(WireModel):
|
||||
encoding: Literal["base64"] = "base64"
|
||||
input: Annotated[str, Field(max_length=87384)]
|
||||
output: Annotated[str, Field(max_length=87384)]
|
||||
|
||||
|
||||
class ProblemSummary(WireModel):
|
||||
id: Identifier
|
||||
slug: Label
|
||||
version_id: Identifier
|
||||
revision: int
|
||||
title: str
|
||||
languages: tuple[Language, ...]
|
||||
|
||||
|
||||
class PublicLanguageConfiguration(WireModel):
|
||||
language: Language
|
||||
runtime_version: str
|
||||
source_name: str
|
||||
entrypoint: str
|
||||
preparation_kind: PreparationKind
|
||||
compile_limits: RunLimits
|
||||
run_limits: RunLimits
|
||||
job_wall_time_us: int
|
||||
|
||||
@classmethod
|
||||
def from_configuration(cls, configuration):
|
||||
source, _, _, _, preparation = runtime_contract(configuration.language)
|
||||
return cls(
|
||||
language=configuration.language,
|
||||
runtime_version=configuration.toolchain.runtime_version,
|
||||
source_name=source,
|
||||
entrypoint="Main" if configuration.language == Language.JAVA21 else source,
|
||||
preparation_kind=preparation,
|
||||
compile_limits=configuration.compile_limits,
|
||||
run_limits=configuration.run_limits,
|
||||
job_wall_time_us=configuration.job_wall_time_us,
|
||||
)
|
||||
|
||||
|
||||
class ProblemDetail(ProblemSummary):
|
||||
statement: str
|
||||
samples: tuple[PublicSample, ...]
|
||||
run_limits: RunLimits | None
|
||||
language_configurations: tuple[PublicLanguageConfiguration, ...] = ()
|
||||
|
||||
|
||||
class ProblemPage(WireModel):
|
||||
items: tuple[ProblemSummary, ...]
|
||||
next_after: Label | None
|
||||
@@ -1,92 +0,0 @@
|
||||
from datetime import datetime
|
||||
from typing import Annotated, Literal
|
||||
|
||||
from pydantic import Field, model_validator
|
||||
|
||||
from kf_oj.contracts.judging import (
|
||||
MAX_SOURCE_BYTES,
|
||||
CompilationStatus,
|
||||
FailureCode,
|
||||
Identifier,
|
||||
Measurement,
|
||||
Verdict,
|
||||
WireModel,
|
||||
)
|
||||
from kf_oj.contracts.judging_v2 import Language
|
||||
|
||||
EvaluationState = Literal[
|
||||
"queued", "claimed", "compiling", "running", "cancelling", "finished", "cancelled", "failed"
|
||||
]
|
||||
|
||||
|
||||
class CreateSubmission(WireModel):
|
||||
problem_version_id: Identifier
|
||||
language: Language
|
||||
idempotency_key: Identifier
|
||||
source: Annotated[
|
||||
str, Field(strict=True, min_length=1, max_length=MAX_SOURCE_BYTES, repr=False)
|
||||
]
|
||||
|
||||
@model_validator(mode="after")
|
||||
def bounded_source(self):
|
||||
try:
|
||||
if len(self.source.encode("utf-8")) > MAX_SOURCE_BYTES:
|
||||
raise ValueError("source exceeds byte policy")
|
||||
except UnicodeError:
|
||||
raise ValueError("source must be UTF-8") from None
|
||||
return self
|
||||
|
||||
|
||||
class SubmissionCreated(WireModel):
|
||||
submission_id: Identifier
|
||||
evaluation_id: Identifier
|
||||
problem_version_id: Identifier
|
||||
language: Language
|
||||
state: EvaluationState
|
||||
created_at: datetime
|
||||
|
||||
|
||||
class SubmissionSummary(SubmissionCreated):
|
||||
updated_at: datetime
|
||||
attempts: int
|
||||
verdict: Verdict | None
|
||||
failure_code: FailureCode | None
|
||||
|
||||
|
||||
class PublicCaseResult(WireModel):
|
||||
position: int
|
||||
verdict: Verdict
|
||||
cpu_time_us: Measurement | None
|
||||
memory_peak_bytes: Measurement | None
|
||||
|
||||
|
||||
class PublicJudgement(WireModel):
|
||||
compilation_status: CompilationStatus
|
||||
diagnostics: Annotated[str, Field(max_length=65536)]
|
||||
diagnostics_truncated: bool
|
||||
cpu_time_us: Measurement | None
|
||||
memory_peak_bytes: Measurement | None
|
||||
cleanup_complete: bool
|
||||
cases: tuple[PublicCaseResult, ...]
|
||||
|
||||
|
||||
class SubmissionDetail(SubmissionSummary):
|
||||
result: PublicJudgement | None
|
||||
|
||||
|
||||
class SubmissionPage(WireModel):
|
||||
items: tuple[SubmissionSummary, ...]
|
||||
next_after: str | None
|
||||
|
||||
|
||||
class SubmissionSource(WireModel):
|
||||
submission_id: Identifier
|
||||
language: Language
|
||||
source: Annotated[str, Field(max_length=MAX_SOURCE_BYTES, repr=False)]
|
||||
|
||||
|
||||
class CancellationAccepted(WireModel):
|
||||
submission_id: Identifier
|
||||
evaluation_id: Identifier
|
||||
# Acceptance means Coordinator durably accepted intent, not that processes have stopped.
|
||||
accepted: Literal[True] = True
|
||||
@@ -1,11 +1,8 @@
|
||||
from pathlib import Path
|
||||
from typing import Literal
|
||||
|
||||
from pydantic import Field, SecretStr, model_validator
|
||||
from pydantic import SecretStr
|
||||
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||
|
||||
from kf_oj.contracts.judging_v2 import NodeAdmission
|
||||
|
||||
|
||||
class Settings(BaseSettings):
|
||||
model_config = SettingsConfigDict(env_prefix="KF_OJ_", env_file=".env", extra="ignore")
|
||||
@@ -14,65 +11,3 @@ class Settings(BaseSettings):
|
||||
log_level: Literal["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"] = "INFO"
|
||||
database_url: SecretStr | None = None
|
||||
amqp_url: SecretStr | None = None
|
||||
browser_origins: tuple[str, ...] = (
|
||||
"http://localhost:5173",
|
||||
"http://localhost:8000",
|
||||
"http://127.0.0.1:5173",
|
||||
"http://127.0.0.1:8000",
|
||||
)
|
||||
session_lifetime_seconds: int = Field(default=43200, ge=300, le=86400)
|
||||
artifact_root: Path | None = None
|
||||
artifact_max_bytes: int = Field(default=536870912, ge=1, le=16 * 1024**3)
|
||||
artifact_max_files: int = Field(default=4096, ge=1, le=65536)
|
||||
judge_admission: NodeAdmission | None = None
|
||||
coordinator_url: str | None = None
|
||||
coordinator_token: SecretStr | None = None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def secure_origins(self):
|
||||
from ipaddress import ip_address
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
if (self.coordinator_url is None) != (self.coordinator_token is None):
|
||||
raise ValueError("both coordinator URL and token required")
|
||||
if self.coordinator_url is not None:
|
||||
parts = urlsplit(self.coordinator_url)
|
||||
try:
|
||||
valid = parts.hostname is not None and ip_address(parts.hostname).is_loopback
|
||||
port = parts.port
|
||||
except ValueError:
|
||||
valid, port = False, None
|
||||
if (
|
||||
not valid
|
||||
or parts.scheme != "http"
|
||||
or port is None
|
||||
or parts.path not in {"", "/"}
|
||||
or parts.username
|
||||
or parts.password
|
||||
or parts.query
|
||||
or parts.fragment
|
||||
):
|
||||
raise ValueError("coordinator must be an explicit numeric loopback HTTP origin")
|
||||
import re
|
||||
|
||||
if not re.fullmatch(r"[0-9a-f]{64}", self.coordinator_token.get_secret_value()):
|
||||
raise ValueError("invalid coordinator service credential")
|
||||
|
||||
if not self.browser_origins or len(self.browser_origins) > 8:
|
||||
raise ValueError("one to eight browser origins required")
|
||||
for origin in self.browser_origins:
|
||||
parts = urlsplit(origin)
|
||||
if (
|
||||
parts.scheme not in {"http", "https"}
|
||||
or not parts.hostname
|
||||
or parts.username
|
||||
or parts.password
|
||||
or parts.path
|
||||
or parts.query
|
||||
or parts.fragment
|
||||
or (self.environment == "production" and parts.scheme != "https")
|
||||
):
|
||||
raise ValueError(
|
||||
"browser origins must be explicit origins; production requires HTTPS"
|
||||
)
|
||||
return self
|
||||
|
||||
@@ -1,20 +0,0 @@
|
||||
# 不可变工件
|
||||
|
||||
存储根配置于 `KF_OJ_ARTIFACT_ROOT`,由服务账号持有,目录 0700、文件 0600。
|
||||
单件至多 64 MiB;默认总字节 512 MiB / 文件 4096,可在有界配置中调整。
|
||||
发布采用摘要名称、临时文件、文件/目录 fsync 和跨进程锁;已有工件必须通过长度/摘要核验,
|
||||
拒绝静默覆盖/修复,普通读取不跟随链接。锁等待最多五秒。临时文件也计入容量。
|
||||
|
||||
适配器执行已经授权的操作,不自行识别用户、租约或题库。公开样例由 problems 的公开 DTO
|
||||
提供;个人源码由 submissions 的已验证主体与归属端口读取;隐藏数据只经有效当前租约取得。
|
||||
没有匿名工件下载或由文件路径选择的读取接口。
|
||||
|
||||
发布事务先取得 digest 锁,再持久化字节与元数据,业务引用最后提交;事务回滚最多留下
|
||||
无元数据的耐久孤立文件,不留下数据库引用指向未发布字节。回收使用同一 digest 锁,
|
||||
业务引用的数据库生成 ready 状态及复合外键阻止被引用元数据转为 deleting。
|
||||
|
||||
`uv run --locked kf-artifacts --limit 100 [--after <digest>]` 执行有限回收批次:恢复 deleting、
|
||||
标记超过一天的未引用元数据、删除耐久字节/元数据及超过一天的无元数据文件/临时文件。
|
||||
每次扫描至多 100 行;显示游标,后续调用接续,避免固定前页上的保留工件阻塞扫描。
|
||||
引用工件不能被回收。文件删除后事务失败仍保持 deleting,下一批可恢复;新引用无法加入。
|
||||
需要已迁移数据库和有限平台权限;回收不遍历业务域 ORM,也不删除不可变题目版本。
|
||||
@@ -1 +1 @@
|
||||
"""Digest-addressed immutable storage; business ports enforce authorization."""
|
||||
"""Reserved for authenticated artifact access and digest verification."""
|
||||
|
||||
@@ -1,48 +0,0 @@
|
||||
import argparse
|
||||
import asyncio
|
||||
|
||||
from sqlalchemy.exc import SQLAlchemyError
|
||||
|
||||
from kf_oj.core.settings import Settings
|
||||
from kf_oj.infra.artifacts.repository import ArtifactRepository
|
||||
from kf_oj.infra.artifacts.store import ArtifactStore
|
||||
from kf_oj.infra.db.session import build_engine, build_session_factory
|
||||
|
||||
|
||||
async def run(limit: int, after: str | None) -> None:
|
||||
settings = Settings()
|
||||
if settings.artifact_root is None:
|
||||
raise ValueError("artifact root required")
|
||||
store = ArtifactStore(
|
||||
settings.artifact_root,
|
||||
max_bytes=settings.artifact_max_bytes,
|
||||
max_files=settings.artifact_max_files,
|
||||
)
|
||||
engine = build_engine(settings)
|
||||
try:
|
||||
sessions = build_session_factory(engine)
|
||||
repository = ArtifactRepository(store)
|
||||
recovered = await repository.finish_deleting(sessions, limit=limit)
|
||||
marked, cursor = await repository.collect_metadata(sessions, limit=limit, after=after)
|
||||
removed = await repository.finish_deleting(sessions, limit=limit)
|
||||
orphans = await repository.collect_orphans(sessions, limit=limit)
|
||||
print(f"Recovered {recovered}; marked {marked}; removed {removed}; orphans {orphans}.")
|
||||
print(f"Next metadata scan cursor: {cursor or 'complete'}")
|
||||
finally:
|
||||
await engine.dispose()
|
||||
store.close()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Collect bounded old unreferenced artifacts")
|
||||
parser.add_argument("--limit", type=int, default=100, choices=range(1, 101))
|
||||
parser.add_argument("--after")
|
||||
args = parser.parse_args()
|
||||
try:
|
||||
asyncio.run(run(args.limit, args.after))
|
||||
except (ValueError, OSError, SQLAlchemyError):
|
||||
parser.exit(1, "Artifact collection rejected; check storage and database configuration.\n")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,21 +0,0 @@
|
||||
"""Metadata ownership and lifecycle for immutable, digest-addressed bytes."""
|
||||
|
||||
from sqlalchemy import BigInteger, CheckConstraint, String, UniqueConstraint
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from kf_oj.infra.db.base import Base
|
||||
from kf_oj.infra.db.columns import CreatedAt
|
||||
|
||||
|
||||
class Artifact(CreatedAt, Base):
|
||||
__tablename__ = "artifacts"
|
||||
__table_args__ = (
|
||||
CheckConstraint("digest ~ '^[0-9a-f]{64}$'", name="digest"),
|
||||
CheckConstraint("bytes BETWEEN 0 AND 67108864", name="bytes"),
|
||||
CheckConstraint("state IN ('ready', 'deleting')", name="state"),
|
||||
UniqueConstraint("digest", "state"),
|
||||
)
|
||||
|
||||
digest: Mapped[str] = mapped_column(String(64), primary_key=True)
|
||||
bytes: Mapped[int] = mapped_column(BigInteger)
|
||||
state: Mapped[str] = mapped_column(String(8), server_default="ready")
|
||||
@@ -1,144 +0,0 @@
|
||||
"""Byte durability precedes metadata. Digest locks span caller reference transactions."""
|
||||
|
||||
import asyncio
|
||||
from datetime import timedelta
|
||||
|
||||
from sqlalchemy import func, select, text, update
|
||||
from sqlalchemy.dialects.postgresql import insert
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||
|
||||
from kf_oj.contracts.judging import ArtifactReference
|
||||
from kf_oj.infra.artifacts.models import Artifact
|
||||
from kf_oj.infra.artifacts.store import ArtifactError, ArtifactStore
|
||||
|
||||
|
||||
async def lock_digest(session: AsyncSession, digest: str) -> None:
|
||||
await session.execute(
|
||||
text("SELECT pg_advisory_xact_lock(:key)"),
|
||||
{"key": int(digest[:16], 16) & ((1 << 63) - 1)},
|
||||
)
|
||||
|
||||
|
||||
class ArtifactRepository:
|
||||
def __init__(self, store: ArtifactStore):
|
||||
self.store = store
|
||||
|
||||
async def put(
|
||||
self, session: AsyncSession, data: bytes, expected: ArtifactReference | None = None
|
||||
) -> ArtifactReference:
|
||||
reference = self.store.reference(data)
|
||||
if expected is not None and reference != expected:
|
||||
raise ArtifactError("artifact digest or length mismatch")
|
||||
await lock_digest(session, reference.digest)
|
||||
existing = await session.get(Artifact, reference.digest)
|
||||
if existing is not None and (
|
||||
existing.state != "ready" or existing.bytes != reference.bytes
|
||||
):
|
||||
raise ArtifactError("artifact metadata unavailable")
|
||||
await asyncio.to_thread(self.store.put, data, reference)
|
||||
await session.execute(
|
||||
insert(Artifact)
|
||||
.values(digest=reference.digest, bytes=reference.bytes)
|
||||
.on_conflict_do_nothing(index_elements=[Artifact.digest])
|
||||
)
|
||||
return reference
|
||||
|
||||
async def read(self, session: AsyncSession, reference: ArtifactReference) -> bytes:
|
||||
metadata = await session.get(Artifact, reference.digest)
|
||||
if metadata is None or metadata.state != "ready" or metadata.bytes != reference.bytes:
|
||||
raise ArtifactError("artifact metadata unavailable")
|
||||
return await asyncio.to_thread(self.store.read, reference)
|
||||
|
||||
async def check_metadata(self, session: AsyncSession, reference: ArtifactReference) -> None:
|
||||
metadata = await session.get(Artifact, reference.digest)
|
||||
if metadata is None or metadata.state != "ready" or metadata.bytes != reference.bytes:
|
||||
raise ArtifactError("artifact metadata unavailable")
|
||||
|
||||
async def collect_metadata(
|
||||
self,
|
||||
sessions: async_sessionmaker,
|
||||
*,
|
||||
min_age_seconds: int = 86400,
|
||||
limit: int = 100,
|
||||
after: str | None = None,
|
||||
) -> tuple[int, str | None]:
|
||||
if min_age_seconds < 3600 or not 1 <= limit <= 100:
|
||||
raise ArtifactError("invalid collection budget")
|
||||
if after is not None:
|
||||
ArtifactReference(digest=after, bytes=0)
|
||||
async with sessions() as session:
|
||||
query = select(Artifact.digest).where(
|
||||
Artifact.state == "ready",
|
||||
Artifact.created_at < func.now() - timedelta(seconds=min_age_seconds),
|
||||
)
|
||||
if after is not None:
|
||||
query = query.where(Artifact.digest > after)
|
||||
names = list(await session.scalars(query.order_by(Artifact.digest).limit(limit)))
|
||||
marked = 0
|
||||
for digest in names:
|
||||
async with sessions.begin() as session:
|
||||
try:
|
||||
async with session.begin_nested():
|
||||
changed = await self.mark_deleting(session, digest)
|
||||
marked += int(changed)
|
||||
except IntegrityError:
|
||||
# Referenced rows are protected by their generated ready-state foreign key.
|
||||
# No domain ORM imports or reverse dependencies are needed for reclamation.
|
||||
pass
|
||||
return marked, names[-1] if len(names) == limit else None
|
||||
|
||||
async def collect_orphans(
|
||||
self, sessions: async_sessionmaker, *, min_age_seconds: int = 86400, limit: int = 100
|
||||
) -> int:
|
||||
# Only filesystem orphans are collected here. Referenced metadata is never deleted.
|
||||
# A durable row cannot become an orphan while another transaction publishes it:
|
||||
# every writer and collector holds the same transaction-scoped digest lock.
|
||||
names = await asyncio.to_thread(
|
||||
self.store.old_digests, min_age_seconds=min_age_seconds, limit=limit
|
||||
)
|
||||
collected = 0
|
||||
for digest in names:
|
||||
async with sessions.begin() as session:
|
||||
await lock_digest(session, digest)
|
||||
if await session.get(Artifact, digest) is None:
|
||||
await asyncio.to_thread(self.store.remove, digest)
|
||||
collected += 1
|
||||
await asyncio.to_thread(
|
||||
self.store.clean_temporary, min_age_seconds=min_age_seconds, limit=limit
|
||||
)
|
||||
return collected
|
||||
|
||||
async def mark_deleting(self, session: AsyncSession, digest: str) -> bool:
|
||||
# FK(digest, 'ready') on every business reference rejects this transition if referenced.
|
||||
await lock_digest(session, digest)
|
||||
result = await session.scalar(
|
||||
update(Artifact)
|
||||
.where(Artifact.digest == digest, Artifact.state == "ready")
|
||||
.values(state="deleting")
|
||||
.returning(Artifact.digest)
|
||||
)
|
||||
return result is not None
|
||||
|
||||
async def finish_deleting(self, sessions: async_sessionmaker, *, limit: int = 100) -> int:
|
||||
if not 1 <= limit <= 100:
|
||||
raise ArtifactError("invalid collection budget")
|
||||
async with sessions() as session:
|
||||
names = list(
|
||||
await session.scalars(
|
||||
select(Artifact.digest)
|
||||
.where(Artifact.state == "deleting")
|
||||
.order_by(Artifact.digest)
|
||||
.limit(limit)
|
||||
)
|
||||
)
|
||||
count = 0
|
||||
for digest in names:
|
||||
async with sessions.begin() as session:
|
||||
await lock_digest(session, digest)
|
||||
metadata = await session.get(Artifact, digest)
|
||||
if metadata is not None and metadata.state == "deleting":
|
||||
await asyncio.to_thread(self.store.remove, digest)
|
||||
await session.delete(metadata)
|
||||
count += 1
|
||||
return count
|
||||
@@ -1,208 +0,0 @@
|
||||
"""Private immutable byte store. Authorization is applied before calling this adapter."""
|
||||
|
||||
import fcntl
|
||||
import hashlib
|
||||
import os
|
||||
import re
|
||||
import stat
|
||||
import time
|
||||
from collections.abc import Iterator
|
||||
from contextlib import contextmanager
|
||||
from pathlib import Path
|
||||
from uuid import uuid4
|
||||
|
||||
from kf_oj.contracts.judging import MAX_PACKAGE_BYTES, ArtifactReference
|
||||
|
||||
DIGEST = re.compile(r"[0-9a-f]{64}\Z")
|
||||
TEMP = re.compile(r"\.tmp-[0-9a-f]{32}\Z")
|
||||
|
||||
|
||||
class ArtifactError(ValueError):
|
||||
"""Sanitized artifact failure; never includes paths or private byte content."""
|
||||
|
||||
|
||||
class ArtifactStore:
|
||||
def __init__(self, root: Path, *, max_bytes: int = 512 * 1024 * 1024, max_files: int = 4096):
|
||||
if not 1 <= max_bytes <= 16 * 1024**3 or not 1 <= max_files <= 65536:
|
||||
raise ArtifactError("invalid storage capacity")
|
||||
root = root.absolute()
|
||||
root.mkdir(mode=0o700, parents=True, exist_ok=True)
|
||||
if root.resolve() != root:
|
||||
raise ArtifactError("storage root must not contain links")
|
||||
self._directory = os.open(root, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW | os.O_CLOEXEC)
|
||||
self.max_bytes = max_bytes
|
||||
self.max_files = max_files
|
||||
info = os.fstat(self._directory)
|
||||
if info.st_uid != os.getuid() or stat.S_IMODE(info.st_mode) != 0o700:
|
||||
os.close(self._directory)
|
||||
raise ArtifactError("storage root must be private and owned")
|
||||
self._closed = False
|
||||
|
||||
def close(self) -> None:
|
||||
if not self._closed:
|
||||
os.close(self._directory)
|
||||
self._closed = True
|
||||
|
||||
@contextmanager
|
||||
def exclusive(self) -> Iterator[None]:
|
||||
# Open a different description for every operation: flock also serializes threads.
|
||||
descriptor = os.open(
|
||||
".lock",
|
||||
os.O_RDWR | os.O_CREAT | os.O_NOFOLLOW | os.O_CLOEXEC,
|
||||
0o600,
|
||||
dir_fd=self._directory,
|
||||
)
|
||||
try:
|
||||
self._regular(os.fstat(descriptor))
|
||||
deadline = time.monotonic() + 5
|
||||
while True:
|
||||
try:
|
||||
fcntl.flock(descriptor, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
break
|
||||
except BlockingIOError:
|
||||
if time.monotonic() >= deadline:
|
||||
raise ArtifactError("storage lock deadline exceeded") from None
|
||||
time.sleep(0.01)
|
||||
yield
|
||||
finally:
|
||||
os.close(descriptor)
|
||||
|
||||
@staticmethod
|
||||
def _regular(info: os.stat_result) -> None:
|
||||
if (
|
||||
not stat.S_ISREG(info.st_mode)
|
||||
or info.st_nlink != 1
|
||||
or info.st_uid != os.getuid()
|
||||
or stat.S_IMODE(info.st_mode) != 0o600
|
||||
):
|
||||
raise ArtifactError("invalid storage entry")
|
||||
|
||||
def _entries(self) -> list[tuple[str, os.stat_result]]:
|
||||
entries = []
|
||||
with os.scandir(self._directory) as directory:
|
||||
for entry in directory:
|
||||
if entry.name == ".lock":
|
||||
continue
|
||||
if not DIGEST.fullmatch(entry.name) and not TEMP.fullmatch(entry.name):
|
||||
raise ArtifactError("unexpected storage entry")
|
||||
info = entry.stat(follow_symlinks=False)
|
||||
self._regular(info)
|
||||
entries.append((entry.name, info))
|
||||
if len(entries) > self.max_files:
|
||||
raise ArtifactError("storage file capacity exceeded")
|
||||
return entries
|
||||
|
||||
@staticmethod
|
||||
def reference(data: bytes) -> ArtifactReference:
|
||||
if type(data) is not bytes or len(data) > MAX_PACKAGE_BYTES:
|
||||
raise ArtifactError("artifact size outside policy")
|
||||
return ArtifactReference(digest=hashlib.sha256(data).hexdigest(), bytes=len(data))
|
||||
|
||||
def put(self, data: bytes, expected: ArtifactReference | None = None) -> ArtifactReference:
|
||||
reference = self.reference(data)
|
||||
if expected is not None and reference != expected:
|
||||
raise ArtifactError("artifact digest or length mismatch")
|
||||
with self.exclusive():
|
||||
entries = self._entries()
|
||||
if any(name == reference.digest for name, _ in entries):
|
||||
self._read(reference)
|
||||
return reference
|
||||
if len(entries) + 1 > self.max_files or (
|
||||
sum(info.st_size for _, info in entries) + len(data) > self.max_bytes
|
||||
):
|
||||
raise ArtifactError("storage capacity exhausted")
|
||||
temporary = ".tmp-" + uuid4().hex
|
||||
descriptor = os.open(
|
||||
temporary,
|
||||
os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW | os.O_CLOEXEC,
|
||||
0o600,
|
||||
dir_fd=self._directory,
|
||||
)
|
||||
try:
|
||||
with os.fdopen(descriptor, "wb") as output:
|
||||
output.write(data)
|
||||
output.flush()
|
||||
os.fsync(output.fileno())
|
||||
# Locking covers every mutation. Never replace or repair an existing digest.
|
||||
os.rename(
|
||||
temporary,
|
||||
reference.digest,
|
||||
src_dir_fd=self._directory,
|
||||
dst_dir_fd=self._directory,
|
||||
)
|
||||
os.fsync(self._directory)
|
||||
finally:
|
||||
try:
|
||||
os.unlink(temporary, dir_fd=self._directory)
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
return reference
|
||||
|
||||
def _read(self, reference: ArtifactReference) -> bytes:
|
||||
if (
|
||||
type(reference.digest) is not str
|
||||
or not DIGEST.fullmatch(reference.digest)
|
||||
or type(reference.bytes) is not int
|
||||
or not 0 <= reference.bytes <= MAX_PACKAGE_BYTES
|
||||
):
|
||||
raise ArtifactError("invalid artifact reference")
|
||||
try:
|
||||
descriptor = os.open(
|
||||
reference.digest,
|
||||
os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK | os.O_CLOEXEC,
|
||||
dir_fd=self._directory,
|
||||
)
|
||||
with os.fdopen(descriptor, "rb") as source:
|
||||
info = os.fstat(source.fileno())
|
||||
self._regular(info)
|
||||
if info.st_size != reference.bytes:
|
||||
raise ArtifactError("artifact digest or length mismatch")
|
||||
data = source.read(reference.bytes + 1)
|
||||
if self.reference(data) != reference:
|
||||
raise ArtifactError("artifact digest or length mismatch")
|
||||
return data
|
||||
except OSError:
|
||||
raise ArtifactError("artifact unavailable") from None
|
||||
|
||||
def read(self, reference: ArtifactReference) -> bytes:
|
||||
with self.exclusive():
|
||||
return self._read(reference)
|
||||
|
||||
def remove(self, digest: str) -> None:
|
||||
if not DIGEST.fullmatch(digest):
|
||||
raise ArtifactError("invalid artifact identifier")
|
||||
with self.exclusive():
|
||||
try:
|
||||
info = os.stat(digest, dir_fd=self._directory, follow_symlinks=False)
|
||||
self._regular(info)
|
||||
os.unlink(digest, dir_fd=self._directory)
|
||||
os.fsync(self._directory)
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
|
||||
def old_digests(self, *, min_age_seconds: int, limit: int) -> tuple[str, ...]:
|
||||
if not 3600 <= min_age_seconds or not 1 <= limit <= 100:
|
||||
raise ArtifactError("invalid collection budget")
|
||||
with self.exclusive():
|
||||
cutoff = time.time() - min_age_seconds
|
||||
return tuple(
|
||||
name
|
||||
for name, info in self._entries()
|
||||
if DIGEST.fullmatch(name) and info.st_mtime < cutoff
|
||||
)[:limit]
|
||||
|
||||
def clean_temporary(self, *, min_age_seconds: int = 86400, limit: int = 100) -> int:
|
||||
if min_age_seconds < 3600 or not 1 <= limit <= 100:
|
||||
raise ArtifactError("invalid collection budget")
|
||||
with self.exclusive():
|
||||
cutoff = time.time() - min_age_seconds
|
||||
names = [
|
||||
name
|
||||
for name, info in self._entries()
|
||||
if TEMP.fullmatch(name) and info.st_mtime < cutoff
|
||||
][:limit]
|
||||
for name in names:
|
||||
os.unlink(name, dir_fd=self._directory)
|
||||
if names:
|
||||
os.fsync(self._directory)
|
||||
return len(names)
|
||||
@@ -1,30 +0,0 @@
|
||||
"""Bounded loopback control adapter; never owns evaluation database writes."""
|
||||
|
||||
from uuid import UUID
|
||||
|
||||
import httpx
|
||||
|
||||
|
||||
class CoordinatorControl:
|
||||
def __init__(self, url: str, token: str):
|
||||
self.url, self.token = url.rstrip("/"), token
|
||||
|
||||
async def cancel(self, evaluation_id: UUID) -> bool:
|
||||
try:
|
||||
async with httpx.AsyncClient(
|
||||
timeout=3, trust_env=False, follow_redirects=False
|
||||
) as client:
|
||||
# Response has no business payload. Do not buffer or log arbitrary remote bodies.
|
||||
async with client.stream(
|
||||
"POST",
|
||||
self.url + "/internal/cancel",
|
||||
headers={"Authorization": "Bearer " + self.token},
|
||||
json={
|
||||
"schema_version": 1,
|
||||
"kind": "judge.evaluation",
|
||||
"evaluation_id": str(evaluation_id),
|
||||
},
|
||||
) as response:
|
||||
return response.status_code == 204
|
||||
except httpx.HTTPError:
|
||||
return False
|
||||
@@ -1,12 +0,0 @@
|
||||
"""Shared column types; business schemas are registered by composition roots."""
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import DateTime, func
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
|
||||
class CreatedAt:
|
||||
created_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), server_default=func.now(), nullable=False
|
||||
)
|
||||
@@ -6,21 +6,7 @@ from kf_oj.core.settings import Settings
|
||||
def build_engine(settings: Settings) -> AsyncEngine:
|
||||
if settings.database_url is None:
|
||||
raise ValueError("KF_OJ_DATABASE_URL is required for database-backed operations")
|
||||
return create_async_engine(
|
||||
settings.database_url.get_secret_value(),
|
||||
pool_pre_ping=True,
|
||||
pool_size=5,
|
||||
max_overflow=0,
|
||||
pool_timeout=5,
|
||||
connect_args={
|
||||
"command_timeout": 10,
|
||||
"server_settings": {
|
||||
"statement_timeout": "10000",
|
||||
"lock_timeout": "5000",
|
||||
"idle_in_transaction_session_timeout": "30000",
|
||||
},
|
||||
},
|
||||
)
|
||||
return create_async_engine(settings.database_url.get_secret_value(), pool_pre_ping=True)
|
||||
|
||||
|
||||
def build_session_factory(engine: AsyncEngine) -> async_sessionmaker:
|
||||
|
||||
@@ -1,31 +0,0 @@
|
||||
"""Transactional notifications; contains IDs, never source or hidden artifacts."""
|
||||
|
||||
from datetime import datetime
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy import CheckConstraint, DateTime, ForeignKey, Index, String, func
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from kf_oj.infra.db.base import Base
|
||||
from kf_oj.infra.db.columns import CreatedAt
|
||||
|
||||
|
||||
class Outbox(CreatedAt, Base):
|
||||
__tablename__ = "outbox"
|
||||
__table_args__ = (
|
||||
CheckConstraint("failures BETWEEN 0 AND 100", name="failure_budget"),
|
||||
CheckConstraint("kind = 'judge.evaluation'", name="kind"),
|
||||
Index("ix_outbox_delivery", "delivered_at", "next_attempt_at", "id"),
|
||||
)
|
||||
|
||||
id: Mapped[UUID] = mapped_column(primary_key=True)
|
||||
evaluation_id: Mapped[UUID] = mapped_column(ForeignKey("evaluations.id"), unique=True)
|
||||
kind: Mapped[str] = mapped_column(String(32), server_default="judge.evaluation")
|
||||
failures: Mapped[int] = mapped_column(server_default="0")
|
||||
next_attempt_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), server_default=func.now()
|
||||
)
|
||||
locked_until: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
lock_owner: Mapped[UUID | None]
|
||||
delivered_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
last_error_code: Mapped[str | None] = mapped_column(String(32))
|
||||
@@ -1,183 +0,0 @@
|
||||
"""Finite transactional outbox delivery; confirmations are notifications, not grades."""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import signal
|
||||
from uuid import uuid4
|
||||
|
||||
import aio_pika
|
||||
from sqlalchemy import text
|
||||
|
||||
from kf_oj.contracts.judge import JudgeTaskNotification
|
||||
from kf_oj.core.settings import Settings
|
||||
from kf_oj.infra.db.session import build_engine, build_session_factory
|
||||
|
||||
QUEUE = "kf.judge.evaluations.v1"
|
||||
QUEUE_ARGUMENTS = {
|
||||
"x-max-length": 64,
|
||||
"x-max-length-bytes": 32768,
|
||||
"x-overflow": "reject-publish",
|
||||
"x-message-ttl": 600000,
|
||||
}
|
||||
|
||||
|
||||
class OutboxRepository:
|
||||
def __init__(self, sessions):
|
||||
self.sessions = sessions
|
||||
|
||||
async def claim(self):
|
||||
owner = uuid4()
|
||||
async with self.sessions.begin() as s:
|
||||
row = (
|
||||
await s.execute(
|
||||
text("""WITH candidate AS (
|
||||
SELECT id FROM outbox
|
||||
WHERE delivered_at IS NULL AND next_attempt_at<=clock_timestamp()
|
||||
AND (locked_until IS NULL OR locked_until<=clock_timestamp())
|
||||
ORDER BY next_attempt_at,id FOR UPDATE SKIP LOCKED LIMIT 1)
|
||||
UPDATE outbox o SET lock_owner=:owner,
|
||||
locked_until=clock_timestamp()+interval '15 seconds'
|
||||
FROM candidate c WHERE o.id=c.id RETURNING o.id,o.evaluation_id"""),
|
||||
{"owner": owner},
|
||||
)
|
||||
).first()
|
||||
return None if row is None else (row.id, row.evaluation_id, owner)
|
||||
|
||||
async def confirmed(self, item):
|
||||
async with self.sessions.begin() as s:
|
||||
result = await s.execute(
|
||||
text("""UPDATE outbox SET delivered_at=clock_timestamp(),
|
||||
locked_until=NULL,lock_owner=NULL,last_error_code=NULL
|
||||
WHERE id=:id AND lock_owner=:owner AND locked_until>clock_timestamp()"""),
|
||||
{"id": item[0], "owner": item[2]},
|
||||
)
|
||||
return result.rowcount == 1
|
||||
|
||||
async def failed(self, item):
|
||||
async with self.sessions.begin() as s:
|
||||
await s.execute(
|
||||
text("""UPDATE outbox SET failures=least(failures+1,100),
|
||||
next_attempt_at=clock_timestamp()+
|
||||
least(power(2,least(failures+1,6)),60)*interval '1 second',
|
||||
locked_until=NULL,lock_owner=NULL,last_error_code='broker_unavailable'
|
||||
WHERE id=:id AND lock_owner=:owner AND locked_until>clock_timestamp()"""),
|
||||
{"id": item[0], "owner": item[2]},
|
||||
)
|
||||
|
||||
|
||||
class ConfirmedBroker:
|
||||
def __init__(self, url):
|
||||
# Parent disabled does not suppress descendant loggers; set their inherited level.
|
||||
for name in ("aio_pika", "aiormq", "pamqp"):
|
||||
logging.getLogger(name).setLevel(logging.CRITICAL)
|
||||
self.url = url
|
||||
self.connection = None
|
||||
self.channel = None
|
||||
|
||||
async def open(self):
|
||||
async with asyncio.timeout(10):
|
||||
await self._open()
|
||||
|
||||
async def _open(self):
|
||||
# Use a finite connect, not a reconnect loop holding an outbox lock.
|
||||
self.connection = await aio_pika.connect(self.url, timeout=5, heartbeat=30)
|
||||
self.channel = await self.connection.channel(publisher_confirms=True, on_return_raises=True)
|
||||
await self.channel.declare_queue(QUEUE, durable=True, arguments=QUEUE_ARGUMENTS, timeout=5)
|
||||
|
||||
async def publish(self, item):
|
||||
# The complete operation ends before the database claim expires.
|
||||
async with asyncio.timeout(12):
|
||||
await self._publish(item)
|
||||
|
||||
async def _publish(self, item):
|
||||
if self.channel is None or self.channel.is_closed:
|
||||
await self.open()
|
||||
body = (
|
||||
JudgeTaskNotification(schema_version=1, kind="judge.evaluation", evaluation_id=item[1])
|
||||
.model_dump_json()
|
||||
.encode()
|
||||
)
|
||||
assert len(body) <= 512
|
||||
confirmation = await self.channel.default_exchange.publish(
|
||||
aio_pika.Message(
|
||||
body,
|
||||
content_type="application/json",
|
||||
delivery_mode=aio_pika.DeliveryMode.PERSISTENT,
|
||||
message_id=str(item[0]),
|
||||
),
|
||||
routing_key=QUEUE,
|
||||
mandatory=True,
|
||||
timeout=5,
|
||||
)
|
||||
from pamqp.commands import Basic
|
||||
|
||||
if not isinstance(confirmation, Basic.Ack):
|
||||
raise RuntimeError("notification not confirmed")
|
||||
|
||||
async def close(self):
|
||||
if self.connection is not None and not self.connection.is_closed:
|
||||
try:
|
||||
await asyncio.wait_for(self.connection.close(), timeout=2)
|
||||
except Exception:
|
||||
pass
|
||||
self.connection = self.channel = None
|
||||
|
||||
|
||||
async def run(settings: Settings, stop: asyncio.Event):
|
||||
if settings.amqp_url is None:
|
||||
raise ValueError("AMQP configuration required")
|
||||
engine = build_engine(settings)
|
||||
repository = OutboxRepository(build_session_factory(engine))
|
||||
broker = ConfirmedBroker(settings.amqp_url.get_secret_value())
|
||||
logger = logging.getLogger("kf_oj.outbox")
|
||||
# Client exceptions may include connection URIs; suppress transport-library logging.
|
||||
for name in ("aio_pika", "aiormq", "pamqp"):
|
||||
logging.getLogger(name).disabled = True
|
||||
try:
|
||||
async with engine.connect() as c:
|
||||
allowed = await c.scalar(
|
||||
text("""SELECT NOT r.rolsuper AND NOT r.rolbypassrls
|
||||
AND pg_has_role(current_user,'kf_oj_platform','MEMBER')
|
||||
AND NOT has_column_privilege(current_user,'evaluations','state','UPDATE')
|
||||
FROM pg_roles r WHERE r.rolname=current_user""")
|
||||
)
|
||||
if not allowed:
|
||||
raise ValueError("dedicated platform database role required")
|
||||
while not stop.is_set():
|
||||
try:
|
||||
item = await repository.claim()
|
||||
if item is not None:
|
||||
try:
|
||||
await broker.publish(item)
|
||||
await repository.confirmed(item)
|
||||
except Exception:
|
||||
await repository.failed(item)
|
||||
await broker.close()
|
||||
logger.warning("outbox broker unavailable")
|
||||
except Exception:
|
||||
logger.warning("outbox database unavailable")
|
||||
try:
|
||||
await asyncio.wait_for(stop.wait(), timeout=1)
|
||||
except TimeoutError:
|
||||
pass
|
||||
finally:
|
||||
await broker.close()
|
||||
await engine.dispose()
|
||||
|
||||
|
||||
def main():
|
||||
async def entry():
|
||||
stop = asyncio.Event()
|
||||
loop = asyncio.get_running_loop()
|
||||
for sig in (signal.SIGTERM, signal.SIGINT):
|
||||
loop.add_signal_handler(sig, stop.set)
|
||||
await run(Settings(), stop)
|
||||
|
||||
try:
|
||||
asyncio.run(entry())
|
||||
except Exception:
|
||||
raise SystemExit("outbox unavailable") from None
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,26 +1,5 @@
|
||||
# identity
|
||||
|
||||
负责账号、登录凭证、会话与角色;向其他模块提供经过验证的用户身份和授权依据。
|
||||
`public.py` 提供已验证主体,`service.py` 管理账号和可撤销会话,`repository.py` 封装持久化。
|
||||
用户身份不能来自前端自报字段或模型参数。
|
||||
|
||||
本地管理命令为 `uv run --locked kf-accounts create <username>` / `disable <username>`。
|
||||
创建口令由隐藏终端提示读取和确认,不接受命令行参数、环境中的默认口令或自助注册。
|
||||
用户名为 3–24 个小写 ASCII 字母、数字或下划线;口令 12–128 字符,最多 512 UTF-8 字节。
|
||||
密码摘要由新增的 `argon2-cffi` 依赖提供 Argon2id,固定 19 MiB/2 次/1 并行度。
|
||||
参数依据 [OWASP](https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html)
|
||||
和 [argon2-cffi 文档](https://argon2-cffi.readthedocs.io/en/stable/parameters.html)。
|
||||
|
||||
登录接口为 `/api/v1/auth/login`,退出 `/logout`,当前主体 `/me`。会话使用随机 256-bit token,
|
||||
数据库只保存 token / CSRF 摘要;默认 12 小时有效,单账号保留至多 10 个有效会话。
|
||||
账号禁用立即拒绝登录和旧会话。请求只采用 ASGI 连接 peer,不读取客户端转发地址。
|
||||
受控启动使用 `--no-proxy-headers`,避免服务器中间件预先替换 peer;反向代理后的用户共享
|
||||
该连接预算,引入可信代理的独立限流策略之前不能直接打开客户端转发头信任。
|
||||
每 peer 至多 10 次登录/15 分钟,预算通过真实数据库串行更新,失败尝试也消耗预算。
|
||||
摘要计算最多同时 2 次,超出直接 429;限流表至多 4096 桶且每次有界回收过期桶。
|
||||
|
||||
有副作用的浏览器请求必须匹配 `KF_OJ_BROWSER_ORIGINS` 的显式来源;生产只接受 HTTPS 来源,
|
||||
必须提供生产来源配置。会话 Cookie 为 HttpOnly / SameSite=Strict,生产 Secure;CSRF Cookie
|
||||
供前端读出请求 token,在 `X-CSRF-Token` 携带,服务端核对其与当前会话绑定的摘要。
|
||||
登录、主体和退出响应不缓存;请求校验错误不回显密码或输入值。API 层限制请求字节、块数和
|
||||
读取期限。跨服务 Worker 凭证不使用用户 Cookie。
|
||||
后续 `public.py` 暴露身份查询接口,`service.py` 编排用例,`repository.py` 封装持久化。
|
||||
用户身份不能来自前端自报字段或模型参数。目前仅建立模块边界,未实现登录。
|
||||
|
||||
@@ -1,49 +0,0 @@
|
||||
"""Controlled local account management; passwords are read from the terminal only."""
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import getpass
|
||||
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
|
||||
from kf_oj.core.settings import Settings
|
||||
from kf_oj.infra.db.session import build_engine, build_session_factory
|
||||
from kf_oj.modules.identity.service import IdentityService
|
||||
|
||||
|
||||
async def run(command: str, username: str, password: str | None = None) -> None:
|
||||
engine = build_engine(Settings())
|
||||
try:
|
||||
service = IdentityService(build_session_factory(engine))
|
||||
if command == "create":
|
||||
assert password is not None
|
||||
await service.create_account(username, password)
|
||||
print("Account created.")
|
||||
else:
|
||||
if not await service.disable_account(username):
|
||||
raise ValueError("account not found")
|
||||
print("Account disabled; sessions revoked.")
|
||||
finally:
|
||||
await engine.dispose()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Manage local OJ accounts")
|
||||
parser.add_argument("command", choices=("create", "disable"))
|
||||
parser.add_argument("username")
|
||||
args = parser.parse_args()
|
||||
password = None
|
||||
if args.command == "create":
|
||||
password = getpass.getpass("Password (12–128 characters): ")
|
||||
if password != getpass.getpass("Repeat password: "):
|
||||
parser.error("passwords differ")
|
||||
try:
|
||||
asyncio.run(run(args.command, args.username, password))
|
||||
except IntegrityError:
|
||||
parser.exit(1, "Account operation rejected by database constraints.\n")
|
||||
except ValueError as error:
|
||||
parser.exit(1, f"Account operation rejected: {error}.\n")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,52 +0,0 @@
|
||||
"""Identity-owned persistent account and revocable session records."""
|
||||
|
||||
from datetime import datetime
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy import Boolean, CheckConstraint, DateTime, ForeignKey, Index, String, Text
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from kf_oj.infra.db.base import Base
|
||||
from kf_oj.infra.db.columns import CreatedAt
|
||||
|
||||
|
||||
class Account(CreatedAt, Base):
|
||||
__tablename__ = "accounts"
|
||||
__table_args__ = (
|
||||
CheckConstraint("username ~ '^[a-z0-9_]{3,24}$'", name="canonical_username"),
|
||||
CheckConstraint("char_length(password_hash) BETWEEN 40 AND 256", name="password_hash_size"),
|
||||
)
|
||||
|
||||
id: Mapped[UUID] = mapped_column(primary_key=True)
|
||||
username: Mapped[str] = mapped_column(String(24), unique=True)
|
||||
password_hash: Mapped[str] = mapped_column(Text)
|
||||
enabled: Mapped[bool] = mapped_column(Boolean, server_default="true")
|
||||
|
||||
|
||||
class Session(CreatedAt, Base):
|
||||
__tablename__ = "sessions"
|
||||
__table_args__ = (
|
||||
CheckConstraint("token_hash ~ '^[0-9a-f]{64}$'", name="token_digest"),
|
||||
CheckConstraint("csrf_hash ~ '^[0-9a-f]{64}$'", name="csrf_digest"),
|
||||
CheckConstraint("expires_at > created_at", name="expiry"),
|
||||
Index("ix_sessions_account_expiry", "account_id", "expires_at"),
|
||||
)
|
||||
|
||||
token_hash: Mapped[str] = mapped_column(String(64), primary_key=True)
|
||||
account_id: Mapped[UUID] = mapped_column(ForeignKey("accounts.id"))
|
||||
csrf_hash: Mapped[str] = mapped_column(String(64))
|
||||
expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True))
|
||||
revoked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
|
||||
class LoginThrottle(Base):
|
||||
__tablename__ = "login_throttles"
|
||||
__table_args__ = (
|
||||
CheckConstraint("bucket_hash ~ '^[0-9a-f]{64}$'", name="bucket_digest"),
|
||||
CheckConstraint("attempts BETWEEN 1 AND 10", name="attempt_budget"),
|
||||
Index("ix_login_throttles_reset", "reset_at"),
|
||||
)
|
||||
|
||||
bucket_hash: Mapped[str] = mapped_column(String(64), primary_key=True)
|
||||
attempts: Mapped[int]
|
||||
reset_at: Mapped[datetime] = mapped_column(DateTime(timezone=True))
|
||||
@@ -1,32 +0,0 @@
|
||||
"""Argon2id password verification with explicit bounded parameters."""
|
||||
|
||||
import re
|
||||
|
||||
from argon2 import PasswordHasher, Type
|
||||
from argon2.exceptions import InvalidHashError, VerificationError
|
||||
|
||||
USERNAME = re.compile(r"[a-z0-9_]{3,24}\Z")
|
||||
HASHER = PasswordHasher(time_cost=2, memory_cost=19456, parallelism=1, type=Type.ID)
|
||||
|
||||
|
||||
def validate_account(username: str, password: str) -> None:
|
||||
if not USERNAME.fullmatch(username) or not 12 <= len(password) <= 128:
|
||||
raise ValueError("username or password outside account policy")
|
||||
if len(password.encode("utf-8")) > 512:
|
||||
raise ValueError("password outside byte policy")
|
||||
|
||||
|
||||
def hash_password(username: str, password: str) -> str:
|
||||
validate_account(username, password)
|
||||
return HASHER.hash(password)
|
||||
|
||||
|
||||
def verify_password(stored: str, password: str) -> bool:
|
||||
try:
|
||||
# Stored hashes are owner-controlled, but don't allow corrupted parameters to allocate
|
||||
# arbitrary memory. Account creation always uses these exact parameters.
|
||||
if not stored.startswith("$argon2id$v=19$m=19456,t=2,p=1$"):
|
||||
return False
|
||||
return HASHER.verify(stored, password)
|
||||
except (VerificationError, InvalidHashError):
|
||||
return False
|
||||
@@ -1,18 +0,0 @@
|
||||
"""Only verified principals cross business module boundaries."""
|
||||
|
||||
from dataclasses import dataclass
|
||||
from uuid import UUID
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Principal:
|
||||
account_id: UUID
|
||||
username: str
|
||||
|
||||
|
||||
class AuthenticationFailed(ValueError):
|
||||
pass
|
||||
|
||||
|
||||
class LoginThrottled(ValueError):
|
||||
pass
|
||||
@@ -1,85 +0,0 @@
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import delete, func, select, text, update
|
||||
from sqlalchemy.dialects.postgresql import insert
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from kf_oj.modules.identity.models import Account, LoginThrottle, Session
|
||||
from kf_oj.modules.identity.public import LoginThrottled
|
||||
|
||||
|
||||
async def consume_login_budget(
|
||||
session: AsyncSession, bucket: str, now: datetime, reset_at: datetime
|
||||
) -> None:
|
||||
# An upsert establishes the key before FOR UPDATE; requests share one DB-backed budget.
|
||||
inserted = await session.scalar(
|
||||
insert(LoginThrottle)
|
||||
.values(bucket_hash=bucket, attempts=1, reset_at=reset_at)
|
||||
.on_conflict_do_nothing(index_elements=[LoginThrottle.bucket_hash])
|
||||
.returning(LoginThrottle.bucket_hash)
|
||||
)
|
||||
row = await session.scalar(
|
||||
select(LoginThrottle).where(LoginThrottle.bucket_hash == bucket).with_for_update()
|
||||
)
|
||||
assert row is not None
|
||||
# The initial upsert consumes this request only if there was no prior row.
|
||||
# A separate returning flag avoids granting an extra request during concurrency.
|
||||
if inserted is not None:
|
||||
return
|
||||
if row.reset_at <= now:
|
||||
row.attempts, row.reset_at = 1, reset_at
|
||||
elif row.attempts >= 10:
|
||||
raise LoginThrottled("login request budget exhausted")
|
||||
else:
|
||||
row.attempts += 1
|
||||
|
||||
|
||||
async def prune_throttles(session: AsyncSession, now: datetime, *, limit: int = 100) -> None:
|
||||
await session.execute(text("SELECT pg_advisory_xact_lock(-740164777)"))
|
||||
keys = select(LoginThrottle.bucket_hash).where(LoginThrottle.reset_at <= now)
|
||||
keys = keys.order_by(LoginThrottle.reset_at).limit(limit).with_for_update(skip_locked=True)
|
||||
await session.execute(delete(LoginThrottle).where(LoginThrottle.bucket_hash.in_(keys)))
|
||||
|
||||
|
||||
async def check_throttle_capacity(session: AsyncSession, bucket: str) -> None:
|
||||
if await session.get(LoginThrottle, bucket) is None:
|
||||
if await session.scalar(select(func.count()).select_from(LoginThrottle)) >= 4096:
|
||||
raise LoginThrottled("login throttle capacity exhausted")
|
||||
|
||||
|
||||
async def account_by_username(session: AsyncSession, username: str) -> Account | None:
|
||||
return await session.scalar(select(Account).where(Account.username == username))
|
||||
|
||||
|
||||
async def current_session(session: AsyncSession, token_hash: str):
|
||||
return (
|
||||
await session.execute(
|
||||
select(Session, Account)
|
||||
.join(Account)
|
||||
.where(
|
||||
Session.token_hash == token_hash,
|
||||
Session.revoked_at.is_(None),
|
||||
Session.expires_at > func.now(),
|
||||
Account.enabled.is_(True),
|
||||
)
|
||||
)
|
||||
).one_or_none()
|
||||
|
||||
|
||||
async def prune_sessions(session: AsyncSession, account_id) -> None:
|
||||
await session.execute(
|
||||
delete(Session).where(
|
||||
Session.account_id == account_id,
|
||||
(Session.expires_at <= func.now()) | Session.revoked_at.is_not(None),
|
||||
)
|
||||
)
|
||||
# Keep at most nine older valid sessions before adding the new one.
|
||||
old = select(Session.token_hash).where(Session.account_id == account_id)
|
||||
old = old.order_by(Session.created_at.desc(), Session.token_hash).offset(9)
|
||||
await session.execute(delete(Session).where(Session.token_hash.in_(old)))
|
||||
|
||||
|
||||
async def revoke(session: AsyncSession, token_hash: str) -> None:
|
||||
await session.execute(
|
||||
update(Session).where(Session.token_hash == token_hash).values(revoked_at=func.now())
|
||||
)
|
||||
@@ -1,131 +0,0 @@
|
||||
import asyncio
|
||||
import hashlib
|
||||
import hmac
|
||||
import re
|
||||
import secrets
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime, timedelta
|
||||
from uuid import uuid4
|
||||
|
||||
from sqlalchemy import func, select, update
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker
|
||||
|
||||
from kf_oj.modules.identity import repository
|
||||
from kf_oj.modules.identity.models import Account, Session
|
||||
from kf_oj.modules.identity.passwords import (
|
||||
HASHER,
|
||||
hash_password,
|
||||
validate_account,
|
||||
verify_password,
|
||||
)
|
||||
from kf_oj.modules.identity.public import AuthenticationFailed, LoginThrottled, Principal
|
||||
|
||||
TOKEN = re.compile(r"[0-9a-f]{64}\Z")
|
||||
|
||||
|
||||
def token_digest(value: str) -> str:
|
||||
return hashlib.sha256(value.encode("ascii")).hexdigest()
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class IssuedSession:
|
||||
principal: Principal
|
||||
expires_at: datetime
|
||||
token: str = field(repr=False)
|
||||
csrf_token: str = field(repr=False)
|
||||
|
||||
|
||||
class IdentityService:
|
||||
def __init__(self, sessions: async_sessionmaker, *, lifetime_seconds: int = 43200):
|
||||
if not 300 <= lifetime_seconds <= 86400:
|
||||
raise ValueError("invalid session lifetime")
|
||||
self.sessions = sessions
|
||||
self.lifetime_seconds = lifetime_seconds
|
||||
self._hash_slots = asyncio.Semaphore(2)
|
||||
self._dummy_hash = HASHER.hash(secrets.token_hex(32))
|
||||
|
||||
async def create_account(self, username: str, password: str) -> Principal:
|
||||
hashed = await asyncio.to_thread(hash_password, username, password)
|
||||
async with self.sessions.begin() as session:
|
||||
account = Account(id=uuid4(), username=username, password_hash=hashed)
|
||||
session.add(account)
|
||||
return Principal(account.id, username)
|
||||
|
||||
async def disable_account(self, username: str) -> bool:
|
||||
async with self.sessions.begin() as session:
|
||||
account = await repository.account_by_username(session, username)
|
||||
if account is None:
|
||||
return False
|
||||
account.enabled = False
|
||||
await session.execute(
|
||||
update(Session)
|
||||
.where(Session.account_id == account.id)
|
||||
.values(revoked_at=func.now())
|
||||
)
|
||||
return True
|
||||
|
||||
async def login(self, username: str, password: str, *, peer: str) -> IssuedSession:
|
||||
validate_account(username, password)
|
||||
if not peer or len(peer) > 128:
|
||||
raise AuthenticationFailed("invalid login context")
|
||||
# Fail fast instead of allowing an unbounded password-hash queue.
|
||||
if self._hash_slots.locked():
|
||||
raise LoginThrottled("password verification capacity exhausted")
|
||||
async with self._hash_slots:
|
||||
async with self.sessions.begin() as session:
|
||||
now = await session.scalar(select(func.now()))
|
||||
assert now is not None
|
||||
await repository.prune_throttles(session, now)
|
||||
# Direct peer only; caller must never substitute forwarded client headers.
|
||||
bucket = hashlib.sha256(peer.encode("utf-8")).hexdigest()
|
||||
await repository.check_throttle_capacity(session, bucket)
|
||||
await repository.consume_login_budget(
|
||||
session, bucket, now, now + timedelta(minutes=15)
|
||||
)
|
||||
async with self.sessions() as session:
|
||||
account = await repository.account_by_username(session, username)
|
||||
hashed = account.password_hash if account is not None else self._dummy_hash
|
||||
correct = await asyncio.to_thread(verify_password, hashed, password)
|
||||
if not correct or account is None or not account.enabled:
|
||||
raise AuthenticationFailed("invalid credentials")
|
||||
token, csrf = secrets.token_hex(32), secrets.token_hex(32)
|
||||
async with self.sessions.begin() as session:
|
||||
# Serialize with disabling the account and concurrent logins before issuing.
|
||||
account = await session.scalar(
|
||||
select(Account).where(Account.id == account.id).with_for_update()
|
||||
)
|
||||
if account is None or not account.enabled:
|
||||
raise AuthenticationFailed("invalid credentials")
|
||||
now = await session.scalar(select(func.now()))
|
||||
expires = now + timedelta(seconds=self.lifetime_seconds)
|
||||
await repository.prune_sessions(session, account.id)
|
||||
session.add(
|
||||
Session(
|
||||
token_hash=token_digest(token),
|
||||
account_id=account.id,
|
||||
csrf_hash=token_digest(csrf),
|
||||
expires_at=expires,
|
||||
)
|
||||
)
|
||||
principal = Principal(account.id, account.username)
|
||||
return IssuedSession(principal, expires, token, csrf)
|
||||
|
||||
async def authenticate(self, token: str | None, *, csrf_token: str | None = None) -> Principal:
|
||||
if token is None or not TOKEN.fullmatch(token):
|
||||
raise AuthenticationFailed("invalid session")
|
||||
async with self.sessions() as session:
|
||||
row = await repository.current_session(session, token_digest(token))
|
||||
if row is None:
|
||||
raise AuthenticationFailed("invalid session")
|
||||
record, account = row
|
||||
if csrf_token is not None:
|
||||
if not TOKEN.fullmatch(csrf_token) or not hmac.compare_digest(
|
||||
record.csrf_hash, token_digest(csrf_token)
|
||||
):
|
||||
raise AuthenticationFailed("invalid request token")
|
||||
return Principal(account.id, account.username)
|
||||
|
||||
async def logout(self, token: str, csrf_token: str) -> None:
|
||||
await self.authenticate(token, csrf_token=csrf_token)
|
||||
async with self.sessions.begin() as session:
|
||||
await repository.revoke(session, token_digest(token))
|
||||
@@ -1,33 +1,5 @@
|
||||
# problems
|
||||
|
||||
负责题目逻辑身份、不可变版本、导入、标签、限制、发布状态与测试工件元数据。
|
||||
跨模块通过 `public.py` 的不可变判题投影;大文件存取通过 `infra/artifacts` 的适配器。
|
||||
公开题面/显式样例与隐藏输入/答案分别授权。公开 API 不返回工件摘要或存储路径。
|
||||
|
||||
`uv run --locked kf-problems <directory>` 导入并发布一个新版本。目录必须属于当前操作账号,
|
||||
权限 0700;`problem.json` 遵循 `import_format.py`,只引用该目录下的单层普通文件名。
|
||||
不解压归档、不跟随链接、不读取任意路径、不执行导入内容。默认清单可由 `--manifest` 指定。
|
||||
要求配置 `KF_OJ_ARTIFACT_ROOT` 和可信 `KF_OJ_JUDGE_ADMISSION` JSON;同一准入配置应交付给节点,
|
||||
不能由题目放宽。结构示例来自共享 `judge-admission.valid.json`,不是当前宿主隔离验收证明。
|
||||
|
||||
v1 清单固定 C17/C++20、工具链/镜像版本、精确字节比较、编译/运行/整个作业预算,并声明按序
|
||||
测试点 `{key,input_file,expected_file}` 和最多十对公开样例 `{input_file,output_file}`。
|
||||
题面以 `statement_file` 读取 UTF-8。每份样例至多 64 KiB,响应采用 base64 保留二进制数据。
|
||||
同一 slug 受事务锁保护,每次发布递增 revision;发布后题面、限额、样例、测试点不可修改。
|
||||
旧评测继续绑定旧版本;当前列表指向新版本。工件先持久化,再在同一事务建立引用和发布。
|
||||
|
||||
`GET /api/v1/problems` 以 slug 游标分页(`after` / `limit`,最多 100 项),
|
||||
`GET /api/v1/problems/versions/{uuid}` 读取已发布版本。禁用逻辑题目时其版本对公开读取不可用。
|
||||
内部判题投影仅供授权应用编排使用;外部 Worker 读取由 submissions 的有效租约端口控制。
|
||||
|
||||
|
||||
v2 使用 `schema_version: 2` 与 `language_configurations`,每个配置包含 `language`、
|
||||
完整 `toolchain` 身份、`compile_limits`、`run_limits` 和 `job_wall_time_us`。
|
||||
可分别声明 `c17`、`cpp20`、`python313`、`java21`;不再同时提供 v1 顶层工具链或预算。
|
||||
所有语言按可信节点配置准入,缺失能力/错误补丁版本/镜像摘要/过宽预算在发布前拒绝。
|
||||
`judge.admission.example.json` 的镜像摘要为占位值,使用前替换为管理员制作镜像的实际摘要。
|
||||
公开题面仅显示固定语言、版本、入口与预算,不公布宿主镜像路径和隐藏工件。
|
||||
|
||||
导入同一 slug 的 v2 版本保持原 v1 版本与提交不变。已有提交按其保存的协议版本解析,
|
||||
不能重新序列化旧 canonical 数据来补入新字段。Java 使用 Main.java 与默认包 Main;
|
||||
Python 使用 main.py。每个新版本的语言集合独立,浏览器不得为旧版本补开新语言。
|
||||
跨模块访问通过后续 `public.py`;大文件存取通过 `infra/artifacts` 的适配器。
|
||||
隐藏数据与公开样例分别授权。当前未创建题库表或导入接口。
|
||||
|
||||
@@ -1,49 +0,0 @@
|
||||
"""Local, controlled import of ordinary files; no archive extraction or program execution."""
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
from pathlib import Path
|
||||
|
||||
from sqlalchemy.exc import SQLAlchemyError
|
||||
|
||||
from kf_oj.core.settings import Settings
|
||||
from kf_oj.infra.artifacts.repository import ArtifactRepository
|
||||
from kf_oj.infra.artifacts.store import ArtifactStore
|
||||
from kf_oj.infra.db.session import build_engine, build_session_factory
|
||||
from kf_oj.modules.problems.service import ProblemService
|
||||
|
||||
|
||||
async def run(root: Path, manifest: str) -> None:
|
||||
settings = Settings()
|
||||
if settings.artifact_root is None or settings.judge_admission is None:
|
||||
raise ValueError("artifact root and trusted judge admission required")
|
||||
store = ArtifactStore(
|
||||
settings.artifact_root,
|
||||
max_bytes=settings.artifact_max_bytes,
|
||||
max_files=settings.artifact_max_files,
|
||||
)
|
||||
engine = build_engine(settings)
|
||||
try:
|
||||
service = ProblemService(
|
||||
build_session_factory(engine), ArtifactRepository(store), settings.judge_admission
|
||||
)
|
||||
version_id = await service.import_directory(root, manifest)
|
||||
print(f"Problem version published: {version_id}")
|
||||
finally:
|
||||
await engine.dispose()
|
||||
store.close()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Import immutable problem version")
|
||||
parser.add_argument("directory", type=Path)
|
||||
parser.add_argument("--manifest", default="problem.json")
|
||||
args = parser.parse_args()
|
||||
try:
|
||||
asyncio.run(run(args.directory, args.manifest))
|
||||
except (ValueError, OSError, SQLAlchemyError):
|
||||
parser.exit(1, "Problem import rejected; check controlled files, budgets and database.\n")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,86 +0,0 @@
|
||||
"""Local operator format. Filenames never cross into the judge execution protocol."""
|
||||
|
||||
from typing import Annotated, Literal, Self
|
||||
|
||||
from pydantic import Field, RootModel, model_validator
|
||||
|
||||
from kf_oj.contracts import judging_v2 as v2
|
||||
from kf_oj.contracts.judging import (
|
||||
MAX_CASES,
|
||||
MAX_JOB_WALL_US,
|
||||
Label,
|
||||
Language,
|
||||
RunLimits,
|
||||
Toolchain,
|
||||
Version,
|
||||
WireModel,
|
||||
)
|
||||
|
||||
FileName = Annotated[str, Field(strict=True, pattern=r"^[a-zA-Z0-9][a-zA-Z0-9_.-]{0,127}$")]
|
||||
|
||||
|
||||
class ImportCase(WireModel):
|
||||
key: Label
|
||||
input_file: FileName
|
||||
expected_file: FileName
|
||||
|
||||
|
||||
class ImportSample(WireModel):
|
||||
input_file: FileName
|
||||
output_file: FileName
|
||||
|
||||
|
||||
class ProblemImport(WireModel):
|
||||
schema_version: Version
|
||||
slug: Label
|
||||
title: Annotated[str, Field(strict=True, min_length=1, max_length=160)]
|
||||
statement_file: FileName
|
||||
languages: Annotated[tuple[Language, ...], Field(min_length=1, max_length=2)]
|
||||
toolchain: Toolchain
|
||||
comparison: Literal["exact_bytes"]
|
||||
compile_limits: RunLimits
|
||||
run_limits: RunLimits
|
||||
job_wall_time_us: Annotated[int, Field(strict=True, ge=1000, le=MAX_JOB_WALL_US)]
|
||||
cases: Annotated[tuple[ImportCase, ...], Field(min_length=1, max_length=MAX_CASES)]
|
||||
samples: Annotated[tuple[ImportSample, ...], Field(max_length=10)]
|
||||
|
||||
@model_validator(mode="after")
|
||||
def unique(self) -> Self:
|
||||
if len(set(self.languages)) != len(self.languages):
|
||||
raise ValueError("duplicate language")
|
||||
if len({case.key for case in self.cases}) != len(self.cases):
|
||||
raise ValueError("duplicate testcase key")
|
||||
if len({(c.input_file, c.expected_file) for c in self.cases}) != len(self.cases):
|
||||
raise ValueError("duplicate testcase reference")
|
||||
return self
|
||||
|
||||
|
||||
class ProblemImportV2(WireModel):
|
||||
schema_version: v2.Version
|
||||
slug: Label
|
||||
title: Annotated[str, Field(strict=True, min_length=1, max_length=160)]
|
||||
statement_file: FileName
|
||||
language_configurations: Annotated[
|
||||
tuple[v2.LanguageConfiguration, ...], Field(min_length=1, max_length=4)
|
||||
]
|
||||
comparison: Literal["exact_bytes"]
|
||||
cases: Annotated[tuple[ImportCase, ...], Field(min_length=1, max_length=MAX_CASES)]
|
||||
samples: Annotated[tuple[ImportSample, ...], Field(max_length=10)]
|
||||
|
||||
@property
|
||||
def languages(self) -> tuple[v2.Language, ...]:
|
||||
return tuple(config.language for config in self.language_configurations)
|
||||
|
||||
@model_validator(mode="after")
|
||||
def unique(self) -> Self:
|
||||
if len(set(self.languages)) != len(self.languages):
|
||||
raise ValueError("duplicate language configuration")
|
||||
if len({case.key for case in self.cases}) != len(self.cases):
|
||||
raise ValueError("duplicate testcase key")
|
||||
if len({(case.input_file, case.expected_file) for case in self.cases}) != len(self.cases):
|
||||
raise ValueError("duplicate testcase reference")
|
||||
return self
|
||||
|
||||
|
||||
class ImportEnvelope(RootModel[ProblemImport | ProblemImportV2]):
|
||||
pass
|
||||
@@ -1,48 +0,0 @@
|
||||
import os
|
||||
import re
|
||||
import stat
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
class LocalImportFiles:
|
||||
def __init__(self, root: Path):
|
||||
root = root.absolute()
|
||||
if root.resolve() != root:
|
||||
raise ValueError("import directory must not contain links")
|
||||
self.descriptor = os.open(root, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW)
|
||||
info = os.fstat(self.descriptor)
|
||||
if info.st_uid != os.getuid() or stat.S_IMODE(info.st_mode) != 0o700:
|
||||
self.close()
|
||||
raise ValueError("import directory must be private and owned")
|
||||
|
||||
def close(self):
|
||||
os.close(self.descriptor)
|
||||
|
||||
def read(self, filename: str, maximum: int) -> bytes:
|
||||
if not re.fullmatch(r"[a-zA-Z0-9][a-zA-Z0-9_.-]{0,127}", filename):
|
||||
raise ValueError("invalid local filename")
|
||||
try:
|
||||
descriptor = os.open(
|
||||
filename, os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK, dir_fd=self.descriptor
|
||||
)
|
||||
with os.fdopen(descriptor, "rb") as stream:
|
||||
before = os.fstat(stream.fileno())
|
||||
if (
|
||||
not stat.S_ISREG(before.st_mode)
|
||||
or before.st_nlink != 1
|
||||
or before.st_uid != os.getuid()
|
||||
or before.st_size > maximum
|
||||
):
|
||||
raise ValueError("invalid local input file")
|
||||
data = stream.read(maximum + 1)
|
||||
after = os.fstat(stream.fileno())
|
||||
if (
|
||||
len(data) > maximum
|
||||
or before.st_size != len(data)
|
||||
or before.st_mtime_ns != after.st_mtime_ns
|
||||
or before.st_ctime_ns != after.st_ctime_ns
|
||||
):
|
||||
raise ValueError("local input changed during import")
|
||||
return data
|
||||
except OSError:
|
||||
raise ValueError("local input unavailable") from None
|
||||
@@ -1,110 +0,0 @@
|
||||
"""Immutable published versions, ordered cases, and mutable public availability."""
|
||||
|
||||
from datetime import datetime
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy import (
|
||||
BigInteger,
|
||||
Boolean,
|
||||
CheckConstraint,
|
||||
Computed,
|
||||
DateTime,
|
||||
ForeignKey,
|
||||
ForeignKeyConstraint,
|
||||
Index,
|
||||
Integer,
|
||||
SmallInteger,
|
||||
String,
|
||||
Text,
|
||||
UniqueConstraint,
|
||||
)
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from kf_oj.infra.db.base import Base
|
||||
from kf_oj.infra.db.columns import CreatedAt
|
||||
|
||||
|
||||
class Problem(CreatedAt, Base):
|
||||
__tablename__ = "problems"
|
||||
__table_args__ = (
|
||||
CheckConstraint("slug ~ '^[a-z0-9][a-z0-9_-]{0,63}$'", name="slug"),
|
||||
ForeignKeyConstraint(
|
||||
["active_version_id", "id"],
|
||||
["problem_versions.id", "problem_versions.problem_id"],
|
||||
name="fk_problems_active_version",
|
||||
use_alter=True,
|
||||
deferrable=True,
|
||||
initially="DEFERRED",
|
||||
),
|
||||
)
|
||||
|
||||
id: Mapped[UUID] = mapped_column(primary_key=True)
|
||||
slug: Mapped[str] = mapped_column(String(64), unique=True)
|
||||
active_version_id: Mapped[UUID | None]
|
||||
enabled: Mapped[bool] = mapped_column(Boolean, server_default="true")
|
||||
|
||||
|
||||
class ProblemVersion(CreatedAt, Base):
|
||||
__tablename__ = "problem_versions"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("id", "problem_id"),
|
||||
UniqueConstraint("problem_id", "revision"),
|
||||
CheckConstraint("revision > 0", name="revision"),
|
||||
CheckConstraint("octet_length(statement) BETWEEN 1 AND 262144", name="statement_bytes"),
|
||||
CheckConstraint("char_length(title) BETWEEN 1 AND 160", name="title_size"),
|
||||
CheckConstraint("comparison = 'exact_bytes'", name="comparison"),
|
||||
CheckConstraint("job_wall_time_us BETWEEN 1000 AND 3600000000", name="job_wall"),
|
||||
CheckConstraint("judging_schema_version IN (1,2)", name="judging_schema"),
|
||||
CheckConstraint(
|
||||
"(judging_schema_version = 1 AND language_configurations IS NULL "
|
||||
"AND toolchain IS NOT NULL AND compile_limits IS NOT NULL "
|
||||
"AND run_limits IS NOT NULL AND job_wall_time_us IS NOT NULL) OR "
|
||||
"(judging_schema_version = 2 AND language_configurations IS NOT NULL "
|
||||
"AND toolchain IS NULL AND compile_limits IS NULL "
|
||||
"AND run_limits IS NULL AND job_wall_time_us IS NULL "
|
||||
"AND public.kf_valid_language_configurations(languages, language_configurations))",
|
||||
name="language_configurations",
|
||||
),
|
||||
Index("ix_problem_versions_public", "problem_id", "published_at"),
|
||||
)
|
||||
|
||||
id: Mapped[UUID] = mapped_column(primary_key=True)
|
||||
problem_id: Mapped[UUID] = mapped_column(ForeignKey("problems.id"))
|
||||
revision: Mapped[int] = mapped_column(Integer)
|
||||
title: Mapped[str] = mapped_column(String(160))
|
||||
statement: Mapped[str] = mapped_column(Text)
|
||||
languages: Mapped[list[str]] = mapped_column(JSONB)
|
||||
judging_schema_version: Mapped[int] = mapped_column(SmallInteger, server_default="1")
|
||||
language_configurations: Mapped[list[dict] | None] = mapped_column(JSONB(none_as_null=True))
|
||||
toolchain: Mapped[dict | None] = mapped_column(JSONB(none_as_null=True))
|
||||
compile_limits: Mapped[dict | None] = mapped_column(JSONB(none_as_null=True))
|
||||
run_limits: Mapped[dict | None] = mapped_column(JSONB(none_as_null=True))
|
||||
job_wall_time_us: Mapped[int | None] = mapped_column(BigInteger)
|
||||
comparison: Mapped[str] = mapped_column(String(16), server_default="exact_bytes")
|
||||
public_samples: Mapped[list[dict]] = mapped_column(JSONB)
|
||||
published_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
|
||||
class ProblemCase(Base):
|
||||
__tablename__ = "problem_cases"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("id", "version_id"),
|
||||
UniqueConstraint("version_id", "position"),
|
||||
CheckConstraint("position BETWEEN 0 AND 255", name="position"),
|
||||
ForeignKeyConstraint(
|
||||
["input_digest", "artifact_state"], ["artifacts.digest", "artifacts.state"]
|
||||
),
|
||||
ForeignKeyConstraint(
|
||||
["expected_digest", "artifact_state"],
|
||||
["artifacts.digest", "artifacts.state"],
|
||||
name="fk_problem_cases_expected_digest_artifacts",
|
||||
),
|
||||
)
|
||||
|
||||
id: Mapped[UUID] = mapped_column(primary_key=True)
|
||||
version_id: Mapped[UUID] = mapped_column(ForeignKey("problem_versions.id"))
|
||||
position: Mapped[int]
|
||||
input_digest: Mapped[str] = mapped_column(String(64))
|
||||
expected_digest: Mapped[str] = mapped_column(String(64))
|
||||
artifact_state: Mapped[str] = mapped_column(String(8), Computed("'ready'", persisted=True))
|
||||
@@ -1,80 +0,0 @@
|
||||
"""Authorized immutable version projection; hidden references stay out of public DTOs."""
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Protocol
|
||||
from uuid import UUID
|
||||
|
||||
from kf_oj.contracts import judging as v1
|
||||
from kf_oj.contracts import judging_v2 as v2
|
||||
from kf_oj.contracts.judging_versions import Manifest
|
||||
|
||||
|
||||
class ProblemUnavailable(ValueError):
|
||||
pass
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class JudgingVersion:
|
||||
version_id: UUID
|
||||
problem_id: UUID
|
||||
languages: tuple[v1.Language | v2.Language, ...]
|
||||
toolchain: v1.Toolchain | None
|
||||
compile_limits: v1.RunLimits | None
|
||||
run_limits: v1.RunLimits | None
|
||||
job_wall_time_us: int | None
|
||||
cases: tuple[v1.TestCase, ...]
|
||||
judging_schema_version: int = 1
|
||||
language_configurations: tuple[v2.LanguageConfiguration, ...] = ()
|
||||
|
||||
def manifest(
|
||||
self,
|
||||
evaluation_id: UUID,
|
||||
submission_id: UUID,
|
||||
language: v1.Language | v2.Language,
|
||||
source: v1.ArtifactReference,
|
||||
) -> Manifest:
|
||||
"""Build only from the immutable authorized version, preserving v1 wire bytes."""
|
||||
if language not in self.languages:
|
||||
raise ProblemUnavailable("language unavailable")
|
||||
fields = dict(
|
||||
schema_version=self.judging_schema_version,
|
||||
kind="judge.manifest",
|
||||
evaluation_id=evaluation_id,
|
||||
submission_id=submission_id,
|
||||
problem_version_id=self.version_id,
|
||||
language=language,
|
||||
source=source,
|
||||
cases=self.cases,
|
||||
comparison="exact_bytes",
|
||||
)
|
||||
if self.judging_schema_version == 1:
|
||||
if self.language_configurations:
|
||||
raise ProblemUnavailable("inconsistent legacy version")
|
||||
return v1.JobManifest(
|
||||
**fields,
|
||||
toolchain=self.toolchain,
|
||||
compile_limits=self.compile_limits,
|
||||
run_limits=self.run_limits,
|
||||
job_wall_time_us=self.job_wall_time_us,
|
||||
)
|
||||
if self.judging_schema_version != 2 or any(
|
||||
field is not None
|
||||
for field in (
|
||||
self.toolchain,
|
||||
self.compile_limits,
|
||||
self.run_limits,
|
||||
self.job_wall_time_us,
|
||||
)
|
||||
):
|
||||
raise ProblemUnavailable("unsupported judging configuration")
|
||||
if tuple(c.language for c in self.language_configurations) != self.languages:
|
||||
raise ProblemUnavailable("inconsistent language configurations")
|
||||
configuration = next(c for c in self.language_configurations if c.language == language)
|
||||
return v2.JobManifest(
|
||||
**fields,
|
||||
**configuration.model_dump(exclude={"language"}),
|
||||
)
|
||||
|
||||
|
||||
class ProblemPort(Protocol):
|
||||
async def for_submission(self, session, version_id: UUID) -> JudgingVersion: ...
|
||||
@@ -1,59 +0,0 @@
|
||||
import hashlib
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy import func, select, text
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from kf_oj.modules.problems.models import Problem, ProblemCase, ProblemVersion
|
||||
|
||||
|
||||
async def lock_slug(session: AsyncSession, slug: str) -> None:
|
||||
# Two-key namespace is disjoint from the one-key artifact advisory locks.
|
||||
key = int.from_bytes(hashlib.sha256(slug.encode()).digest()[:4], signed=True)
|
||||
await session.execute(text("SELECT pg_advisory_xact_lock(1262907468,:key)"), {"key": key})
|
||||
|
||||
|
||||
async def lock_problem(session: AsyncSession, slug: str) -> Problem | None:
|
||||
return await session.scalar(select(Problem).where(Problem.slug == slug).with_for_update())
|
||||
|
||||
|
||||
async def next_revision(session: AsyncSession, problem_id: UUID) -> int:
|
||||
latest = await session.scalar(
|
||||
select(func.max(ProblemVersion.revision)).where(ProblemVersion.problem_id == problem_id)
|
||||
)
|
||||
return (latest or 0) + 1
|
||||
|
||||
|
||||
async def public_version(session: AsyncSession, version_id: UUID):
|
||||
return (
|
||||
await session.execute(
|
||||
select(ProblemVersion, Problem)
|
||||
.join(Problem, Problem.id == ProblemVersion.problem_id)
|
||||
.where(
|
||||
ProblemVersion.id == version_id,
|
||||
ProblemVersion.published_at.is_not(None),
|
||||
Problem.enabled.is_(True),
|
||||
)
|
||||
)
|
||||
).one_or_none()
|
||||
|
||||
|
||||
async def ordered_cases(session: AsyncSession, version_id: UUID):
|
||||
return list(
|
||||
await session.scalars(
|
||||
select(ProblemCase)
|
||||
.where(ProblemCase.version_id == version_id)
|
||||
.order_by(ProblemCase.position)
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
async def list_public(session: AsyncSession, after: str | None, limit: int):
|
||||
query = (
|
||||
select(Problem, ProblemVersion)
|
||||
.join(ProblemVersion, ProblemVersion.id == Problem.active_version_id)
|
||||
.where(Problem.enabled.is_(True), ProblemVersion.published_at.is_not(None))
|
||||
)
|
||||
if after is not None:
|
||||
query = query.where(Problem.slug > after)
|
||||
return list(await session.execute(query.order_by(Problem.slug).limit(limit)))
|
||||
@@ -1,269 +0,0 @@
|
||||
import base64
|
||||
from pathlib import Path
|
||||
from uuid import UUID, uuid4, uuid5
|
||||
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker
|
||||
|
||||
from kf_oj.contracts import judging_v2 as v2
|
||||
from kf_oj.contracts.judging import (
|
||||
MAX_PACKAGE_BYTES,
|
||||
MAX_SOURCE_BYTES,
|
||||
ArtifactReference,
|
||||
Language,
|
||||
RunLimits,
|
||||
TestCase,
|
||||
Toolchain,
|
||||
decode_wire,
|
||||
)
|
||||
from kf_oj.contracts.problems import (
|
||||
ProblemDetail,
|
||||
ProblemPage,
|
||||
ProblemSummary,
|
||||
PublicLanguageConfiguration,
|
||||
PublicSample,
|
||||
)
|
||||
from kf_oj.infra.artifacts.models import Artifact
|
||||
from kf_oj.infra.artifacts.repository import ArtifactRepository
|
||||
from kf_oj.modules.problems import repository
|
||||
from kf_oj.modules.problems.import_format import ImportEnvelope, ProblemImport
|
||||
from kf_oj.modules.problems.local_files import LocalImportFiles
|
||||
from kf_oj.modules.problems.models import Problem, ProblemCase, ProblemVersion
|
||||
from kf_oj.modules.problems.public import JudgingVersion, ProblemUnavailable
|
||||
|
||||
|
||||
class ProblemService:
|
||||
def __init__(
|
||||
self,
|
||||
sessions: async_sessionmaker,
|
||||
artifacts: ArtifactRepository | None = None,
|
||||
policy: v2.NodeAdmission | None = None,
|
||||
):
|
||||
self.sessions, self.artifacts, self.policy = sessions, artifacts, policy
|
||||
|
||||
async def import_directory(self, root: Path, manifest_file: str = "problem.json") -> UUID:
|
||||
if self.artifacts is None or self.policy is None:
|
||||
raise ValueError("artifact store and trusted node policy required for import")
|
||||
files = LocalImportFiles(root)
|
||||
try:
|
||||
manifest = decode_wire(ImportEnvelope, files.read(manifest_file, 256 * 1024)).root
|
||||
stdout_limit = (
|
||||
manifest.run_limits.stdout_bytes
|
||||
if isinstance(manifest, ProblemImport)
|
||||
else min(c.run_limits.stdout_bytes for c in manifest.language_configurations)
|
||||
)
|
||||
statement = files.read(manifest.statement_file, 256 * 1024).decode("utf-8")
|
||||
if not statement or "\x00" in statement or "\x00" in manifest.title:
|
||||
raise ValueError("invalid public text")
|
||||
# Read the complete bounded package before writing anything. Public samples are
|
||||
# separately declared, at most 64 KiB each, and never inferred from hidden tests.
|
||||
remaining = MAX_PACKAGE_BYTES - MAX_SOURCE_BYTES
|
||||
payloads = []
|
||||
for case in manifest.cases:
|
||||
data = files.read(case.input_file, remaining)
|
||||
remaining -= len(data)
|
||||
answer = files.read(case.expected_file, min(remaining, stdout_limit))
|
||||
remaining -= len(answer)
|
||||
payloads.append((data, answer))
|
||||
samples = tuple(
|
||||
PublicSample(
|
||||
input=base64.b64encode(files.read(s.input_file, 65536)).decode("ascii"),
|
||||
output=base64.b64encode(files.read(s.output_file, 65536)).decode("ascii"),
|
||||
)
|
||||
for s in manifest.samples
|
||||
)
|
||||
finally:
|
||||
files.close()
|
||||
version_id = uuid4()
|
||||
cases = tuple(
|
||||
TestCase(
|
||||
case_id=uuid5(version_id, case.key),
|
||||
input=self.artifacts.store.reference(data),
|
||||
expected=self.artifacts.store.reference(answer),
|
||||
)
|
||||
for case, (data, answer) in zip(manifest.cases, payloads, strict=True)
|
||||
)
|
||||
legacy = isinstance(manifest, ProblemImport)
|
||||
projection = JudgingVersion(
|
||||
version_id,
|
||||
uuid4(),
|
||||
manifest.languages,
|
||||
manifest.toolchain if legacy else None,
|
||||
manifest.compile_limits if legacy else None,
|
||||
manifest.run_limits if legacy else None,
|
||||
manifest.job_wall_time_us if legacy else None,
|
||||
cases,
|
||||
manifest.schema_version,
|
||||
() if legacy else manifest.language_configurations,
|
||||
)
|
||||
for language in manifest.languages:
|
||||
check = projection.manifest(
|
||||
uuid4(),
|
||||
uuid4(),
|
||||
language,
|
||||
ArtifactReference(digest="0" * 64, bytes=MAX_SOURCE_BYTES),
|
||||
)
|
||||
v2.admit_manifest(self.policy, check, 1)
|
||||
async with self.sessions.begin() as session:
|
||||
# Lock existing and not-yet-existing logical slugs before assigning revisions.
|
||||
await repository.lock_slug(session, manifest.slug)
|
||||
problem = await repository.lock_problem(session, manifest.slug)
|
||||
if problem is None:
|
||||
problem = Problem(id=uuid4(), slug=manifest.slug)
|
||||
session.add(problem)
|
||||
await session.flush()
|
||||
if not problem.enabled:
|
||||
raise ProblemUnavailable("problem disabled")
|
||||
revision = await repository.next_revision(session, problem.id)
|
||||
session.add(
|
||||
ProblemVersion(
|
||||
id=version_id,
|
||||
problem_id=problem.id,
|
||||
revision=revision,
|
||||
title=manifest.title,
|
||||
statement=statement,
|
||||
languages=[lang.value for lang in manifest.languages],
|
||||
judging_schema_version=manifest.schema_version,
|
||||
language_configurations=(
|
||||
None
|
||||
if legacy
|
||||
else [c.model_dump(mode="json") for c in manifest.language_configurations]
|
||||
),
|
||||
toolchain=manifest.toolchain.model_dump(mode="json") if legacy else None,
|
||||
compile_limits=manifest.compile_limits.model_dump(mode="json")
|
||||
if legacy
|
||||
else None,
|
||||
run_limits=manifest.run_limits.model_dump(mode="json") if legacy else None,
|
||||
job_wall_time_us=manifest.job_wall_time_us if legacy else None,
|
||||
comparison=manifest.comparison,
|
||||
public_samples=[s.model_dump() for s in samples],
|
||||
)
|
||||
)
|
||||
await session.flush()
|
||||
# Consistent digest lock order avoids deadlocks between overlapping imports/jobs.
|
||||
unique = {
|
||||
ref.digest: (data, ref)
|
||||
for case, (input_bytes, answer_bytes) in zip(cases, payloads, strict=True)
|
||||
for data, ref in [(input_bytes, case.input), (answer_bytes, case.expected)]
|
||||
}
|
||||
for digest in sorted(unique):
|
||||
data, reference = unique[digest]
|
||||
await self.artifacts.put(session, data, reference)
|
||||
for position, case in enumerate(cases):
|
||||
session.add(
|
||||
ProblemCase(
|
||||
id=case.case_id,
|
||||
version_id=version_id,
|
||||
position=position,
|
||||
input_digest=case.input.digest,
|
||||
expected_digest=case.expected.digest,
|
||||
)
|
||||
)
|
||||
await session.flush()
|
||||
version = await session.get(ProblemVersion, version_id)
|
||||
version.published_at = await session.scalar(select(func.now()))
|
||||
problem.active_version_id = version_id
|
||||
return version_id
|
||||
|
||||
@staticmethod
|
||||
def _summary(problem, version) -> ProblemSummary:
|
||||
return ProblemSummary(
|
||||
id=problem.id,
|
||||
slug=problem.slug,
|
||||
version_id=version.id,
|
||||
revision=version.revision,
|
||||
title=version.title,
|
||||
languages=tuple(version.languages),
|
||||
)
|
||||
|
||||
async def list(self, *, after: str | None = None, limit: int = 20) -> ProblemPage:
|
||||
if not 1 <= limit <= 100:
|
||||
raise ValueError("invalid page limit")
|
||||
async with self.sessions() as session:
|
||||
rows = await repository.list_public(session, after, limit + 1)
|
||||
visible = rows[:limit]
|
||||
return ProblemPage(
|
||||
items=tuple(self._summary(*row) for row in visible),
|
||||
next_after=visible[-1][0].slug if len(rows) > limit else None,
|
||||
)
|
||||
|
||||
async def detail(self, version_id: UUID) -> ProblemDetail:
|
||||
async with self.sessions() as session:
|
||||
row = await repository.public_version(session, version_id)
|
||||
if row is None:
|
||||
raise ProblemUnavailable("problem version unavailable")
|
||||
version, problem = row
|
||||
return ProblemDetail(
|
||||
**self._summary(problem, version).model_dump(),
|
||||
statement=version.statement,
|
||||
samples=tuple(PublicSample.model_validate(s) for s in version.public_samples),
|
||||
run_limits=(
|
||||
RunLimits.model_validate(version.run_limits)
|
||||
if version.judging_schema_version == 1
|
||||
else None
|
||||
),
|
||||
language_configurations=tuple(
|
||||
PublicLanguageConfiguration.from_configuration(
|
||||
v2.LanguageConfiguration.model_validate(c)
|
||||
)
|
||||
for c in version.language_configurations or ()
|
||||
),
|
||||
)
|
||||
|
||||
async def for_submission(self, session, version_id: UUID) -> JudgingVersion:
|
||||
if self.artifacts is None:
|
||||
raise ProblemUnavailable("artifact service unavailable")
|
||||
row = await repository.public_version(session, version_id)
|
||||
if row is None:
|
||||
raise ProblemUnavailable("problem version unavailable")
|
||||
version, problem = row
|
||||
cases = await repository.ordered_cases(session, version.id)
|
||||
refs = []
|
||||
digests = {d for case in cases for d in (case.input_digest, case.expected_digest)}
|
||||
records = {
|
||||
a.digest: a
|
||||
for a in await session.scalars(select(Artifact).where(Artifact.digest.in_(digests)))
|
||||
}
|
||||
for case in cases:
|
||||
input_record = records.get(case.input_digest)
|
||||
answer_record = records.get(case.expected_digest)
|
||||
if (
|
||||
not input_record
|
||||
or not answer_record
|
||||
or input_record.state != "ready"
|
||||
or answer_record.state != "ready"
|
||||
):
|
||||
raise ProblemUnavailable("problem artifacts unavailable")
|
||||
refs.append(
|
||||
TestCase(
|
||||
case_id=case.id,
|
||||
input=ArtifactReference(digest=case.input_digest, bytes=input_record.bytes),
|
||||
expected=ArtifactReference(
|
||||
digest=case.expected_digest, bytes=answer_record.bytes
|
||||
),
|
||||
)
|
||||
)
|
||||
return JudgingVersion(
|
||||
version.id,
|
||||
problem.id,
|
||||
tuple(
|
||||
(Language if version.judging_schema_version == 1 else v2.Language)(v)
|
||||
for v in version.languages
|
||||
),
|
||||
Toolchain.model_validate(version.toolchain)
|
||||
if version.judging_schema_version == 1
|
||||
else None,
|
||||
RunLimits.model_validate(version.compile_limits)
|
||||
if version.judging_schema_version == 1
|
||||
else None,
|
||||
RunLimits.model_validate(version.run_limits)
|
||||
if version.judging_schema_version == 1
|
||||
else None,
|
||||
version.job_wall_time_us,
|
||||
tuple(refs),
|
||||
version.judging_schema_version,
|
||||
tuple(
|
||||
v2.LanguageConfiguration.model_validate(c)
|
||||
for c in version.language_configurations or ()
|
||||
),
|
||||
)
|
||||
@@ -3,23 +3,4 @@
|
||||
负责提交幂等请求、不可变源码引用、创建初始评测、历史查询与学生可见结果。
|
||||
创建提交、评测和待投递事件必须处于同一事务。
|
||||
协调器拥有领取后的执行状态与最终结果写入权;本模块不能绕过协议覆盖终态。
|
||||
`queries.py` 提供已验证主体的提交历史、详情与私有源码读取,其他账号不可读取;
|
||||
历史按创建时间/UUID降序游标翻页,一次至多100条。读取完整成绩时严格核对报告契约和
|
||||
评测绑定,仅投影允许公开的诊断/计量/测试点序号。取消通过注入的 Coordinator 控制
|
||||
端口接受意图,不更新执行状态;202与清理完成的终态不同。
|
||||
`worker_access.py` 核对当前 evaluation / attempt / Worker / fencing、未撤销及未过期凭证、
|
||||
固定 submission 与版本及任务清单摘要;只允许读取该清单声明的工件。
|
||||
内部 API 使用 WorkerLease bearer 和 X-Worker-ID / X-Fencing-Version,不使用用户会话授权。
|
||||
允许状态为 claimed / compiling / running;取消请求、终态和旧 attempt 均拒绝。
|
||||
字节准备后再次核对当前租约;最多同时两个有限读取,超过直接 429,不排无界等待队列。
|
||||
|
||||
任务清单为 UTF-8 JSON、对象键递归排序、无额外空白,并以这些确切字节的 SHA-256 标识。
|
||||
Python/Rust `manifest_bytes` 与共享字节样例保持一致;Coordinator 必须用同一序列化定义存快照。
|
||||
读取凭证与客户端提交的身份分开:身份和归属从实际数据库绑定获得,不接受客户端自报 user ID。
|
||||
|
||||
`service.py` 按账号与幂等UUID绑定版本/语言/源码摘要,持久化源码后同一事务创建
|
||||
submission、queued evaluation 和 outbox;同键同内容复用既有提交,不同内容409。
|
||||
待评测固定总量64、个人8,取消清理中也占用容量;配置或容量不足明确拒绝。
|
||||
Coordinator 通过正式 claim 创建短期领取权,Worker 不写业务数据库。
|
||||
真实服务链与安全条件必须在独立 VM 验收;数据库 fixture 的租约或合成报告只能
|
||||
验证控制规则、授权和投影,不能作为真实程序执行/停止证明。
|
||||
后续 `public.py` 提供授权后的学习证据入口。当前未实现提交接口。
|
||||
|
||||
@@ -1,163 +0,0 @@
|
||||
"""Platform creates initial rows; Coordinator exclusively advances execution."""
|
||||
|
||||
from datetime import datetime
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy import (
|
||||
BigInteger,
|
||||
Boolean,
|
||||
CheckConstraint,
|
||||
Computed,
|
||||
DateTime,
|
||||
ForeignKey,
|
||||
ForeignKeyConstraint,
|
||||
Index,
|
||||
String,
|
||||
UniqueConstraint,
|
||||
func,
|
||||
)
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from kf_oj.infra.db.base import Base
|
||||
from kf_oj.infra.db.columns import CreatedAt
|
||||
|
||||
|
||||
class Submission(CreatedAt, Base):
|
||||
__tablename__ = "submissions"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("account_id", "idempotency_key"),
|
||||
UniqueConstraint("id", "problem_version_id"),
|
||||
CheckConstraint("language IN ('c17','cpp20','python313','java21')", name="language"),
|
||||
CheckConstraint("request_digest ~ '^[0-9a-f]{64}$'", name="request_digest"),
|
||||
ForeignKeyConstraint(
|
||||
["problem_version_id", "problem_id"],
|
||||
["problem_versions.id", "problem_versions.problem_id"],
|
||||
),
|
||||
ForeignKeyConstraint(
|
||||
["source_digest", "artifact_state"], ["artifacts.digest", "artifacts.state"]
|
||||
),
|
||||
Index("ix_submissions_account_history", "account_id", "created_at", "id"),
|
||||
)
|
||||
|
||||
id: Mapped[UUID] = mapped_column(primary_key=True)
|
||||
account_id: Mapped[UUID] = mapped_column(ForeignKey("accounts.id"))
|
||||
problem_id: Mapped[UUID]
|
||||
problem_version_id: Mapped[UUID]
|
||||
language: Mapped[str] = mapped_column(String(16))
|
||||
source_digest: Mapped[str] = mapped_column(String(64))
|
||||
artifact_state: Mapped[str] = mapped_column(String(8), Computed("'ready'", persisted=True))
|
||||
idempotency_key: Mapped[UUID]
|
||||
request_digest: Mapped[str] = mapped_column(String(64))
|
||||
|
||||
|
||||
class Evaluation(CreatedAt, Base):
|
||||
__tablename__ = "evaluations"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("id", "problem_version_id"),
|
||||
ForeignKeyConstraint(
|
||||
["submission_id", "problem_version_id"],
|
||||
["submissions.id", "submissions.problem_version_id"],
|
||||
),
|
||||
ForeignKeyConstraint(
|
||||
["current_attempt_id", "id"],
|
||||
["judge_attempts.id", "judge_attempts.evaluation_id"],
|
||||
name="fk_evaluations_current_attempt",
|
||||
use_alter=True,
|
||||
deferrable=True,
|
||||
initially="DEFERRED",
|
||||
),
|
||||
CheckConstraint(
|
||||
"state IN ('queued','claimed','compiling','running','cancelling',"
|
||||
"'finished','cancelled','failed')",
|
||||
name="state",
|
||||
),
|
||||
CheckConstraint("fencing_version >= 0", name="fence"),
|
||||
CheckConstraint("max_attempts BETWEEN 1 AND 5", name="max_attempts"),
|
||||
CheckConstraint("attempts_count BETWEEN 0 AND max_attempts", name="attempts_count"),
|
||||
CheckConstraint(
|
||||
"state NOT IN ('claimed','compiling','running','cancelling') "
|
||||
"OR current_attempt_id IS NOT NULL",
|
||||
name="active_attempt",
|
||||
),
|
||||
Index("ix_evaluations_recovery", "state", "not_before", "id"),
|
||||
)
|
||||
|
||||
id: Mapped[UUID] = mapped_column(primary_key=True)
|
||||
submission_id: Mapped[UUID] = mapped_column(unique=True)
|
||||
problem_version_id: Mapped[UUID]
|
||||
state: Mapped[str] = mapped_column(String(16), server_default="queued")
|
||||
fencing_version: Mapped[int] = mapped_column(BigInteger, server_default="0")
|
||||
attempts_count: Mapped[int] = mapped_column(server_default="0")
|
||||
max_attempts: Mapped[int] = mapped_column(server_default="3")
|
||||
current_attempt_id: Mapped[UUID | None]
|
||||
not_before: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
|
||||
cancel_requested_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
|
||||
verdict: Mapped[str | None] = mapped_column(String(32))
|
||||
failure_code: Mapped[str | None] = mapped_column(String(32))
|
||||
final_report: Mapped[dict | None] = mapped_column(JSONB)
|
||||
final_report_digest: Mapped[str | None] = mapped_column(String(64))
|
||||
|
||||
|
||||
class JudgeAttempt(CreatedAt, Base):
|
||||
__tablename__ = "judge_attempts"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("id", "evaluation_id"),
|
||||
UniqueConstraint("evaluation_id", "number", name="uq_judge_attempts_evaluation_number"),
|
||||
UniqueConstraint(
|
||||
"evaluation_id", "fencing_version", name="uq_judge_attempts_evaluation_fence"
|
||||
),
|
||||
CheckConstraint("number BETWEEN 1 AND 5", name="number"),
|
||||
CheckConstraint("fencing_version > 0", name="fence"),
|
||||
CheckConstraint("lease_token_hash ~ '^[0-9a-f]{64}$'", name="token_digest"),
|
||||
CheckConstraint("manifest_digest ~ '^[0-9a-f]{64}$'", name="manifest_digest"),
|
||||
CheckConstraint("sequence >= 0", name="sequence"),
|
||||
Index("ix_judge_attempts_expiry", "expires_at", "evaluation_id"),
|
||||
)
|
||||
|
||||
id: Mapped[UUID] = mapped_column(primary_key=True)
|
||||
evaluation_id: Mapped[UUID] = mapped_column(ForeignKey("evaluations.id"))
|
||||
worker_id: Mapped[UUID]
|
||||
number: Mapped[int]
|
||||
fencing_version: Mapped[int] = mapped_column(BigInteger)
|
||||
lease_token_hash: Mapped[str] = mapped_column(String(64))
|
||||
expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True))
|
||||
revoked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
manifest: Mapped[dict] = mapped_column(JSONB)
|
||||
manifest_digest: Mapped[str] = mapped_column(String(64))
|
||||
sequence: Mapped[int] = mapped_column(BigInteger, server_default="0")
|
||||
stage: Mapped[str | None] = mapped_column(String(16))
|
||||
case_index: Mapped[int | None]
|
||||
report: Mapped[dict | None] = mapped_column(JSONB)
|
||||
report_digest: Mapped[str | None] = mapped_column(String(64))
|
||||
cleanup_complete: Mapped[bool | None] = mapped_column(Boolean)
|
||||
|
||||
|
||||
class CaseResult(Base):
|
||||
__tablename__ = "case_results"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("attempt_id", "position"),
|
||||
ForeignKeyConstraint(
|
||||
["case_id", "problem_version_id"], ["problem_cases.id", "problem_cases.version_id"]
|
||||
),
|
||||
ForeignKeyConstraint(
|
||||
["evaluation_id", "problem_version_id"],
|
||||
["evaluations.id", "evaluations.problem_version_id"],
|
||||
),
|
||||
ForeignKeyConstraint(
|
||||
["attempt_id", "evaluation_id"], ["judge_attempts.id", "judge_attempts.evaluation_id"]
|
||||
),
|
||||
CheckConstraint("position BETWEEN 0 AND 255", name="position"),
|
||||
CheckConstraint("cpu_time_us IS NULL OR cpu_time_us >= 0", name="cpu"),
|
||||
CheckConstraint("memory_peak_bytes IS NULL OR memory_peak_bytes >= 0", name="memory"),
|
||||
)
|
||||
|
||||
attempt_id: Mapped[UUID] = mapped_column(primary_key=True)
|
||||
case_id: Mapped[UUID] = mapped_column(primary_key=True)
|
||||
evaluation_id: Mapped[UUID]
|
||||
problem_version_id: Mapped[UUID]
|
||||
position: Mapped[int]
|
||||
verdict: Mapped[str] = mapped_column(String(32))
|
||||
cpu_time_us: Mapped[int | None] = mapped_column(BigInteger)
|
||||
memory_peak_bytes: Mapped[int | None] = mapped_column(BigInteger)
|
||||
@@ -1,30 +0,0 @@
|
||||
from typing import Protocol
|
||||
from uuid import UUID
|
||||
|
||||
|
||||
class CancelUnavailable(ValueError):
|
||||
pass
|
||||
|
||||
|
||||
class CancellationPort(Protocol):
|
||||
async def cancel(self, evaluation_id: UUID) -> bool: ...
|
||||
|
||||
|
||||
class SubmissionUnavailable(ValueError):
|
||||
pass
|
||||
|
||||
|
||||
class WorkerAccessDenied(ValueError):
|
||||
"""Never reveal whether the evaluation, attempt or artifact exists."""
|
||||
|
||||
|
||||
class WorkerReadBusy(ValueError):
|
||||
pass
|
||||
|
||||
|
||||
class IdempotencyConflict(ValueError):
|
||||
pass
|
||||
|
||||
|
||||
class SubmissionBusy(ValueError):
|
||||
pass
|
||||
@@ -1,173 +0,0 @@
|
||||
import base64
|
||||
import json
|
||||
from datetime import datetime
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy import select, tuple_
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker
|
||||
|
||||
from kf_oj.contracts.judging import MAX_SOURCE_BYTES, ArtifactReference
|
||||
from kf_oj.contracts.judging_versions import decode_report
|
||||
from kf_oj.contracts.submissions import (
|
||||
CancellationAccepted,
|
||||
PublicCaseResult,
|
||||
PublicJudgement,
|
||||
SubmissionDetail,
|
||||
SubmissionPage,
|
||||
SubmissionSummary,
|
||||
)
|
||||
from kf_oj.infra.artifacts.models import Artifact
|
||||
from kf_oj.infra.artifacts.repository import ArtifactRepository
|
||||
from kf_oj.infra.artifacts.store import ArtifactError
|
||||
from kf_oj.modules.identity.public import Principal
|
||||
from kf_oj.modules.submissions.models import Evaluation, Submission
|
||||
from kf_oj.modules.submissions.public import (
|
||||
CancellationPort,
|
||||
CancelUnavailable,
|
||||
SubmissionUnavailable,
|
||||
)
|
||||
|
||||
|
||||
def cursor_encode(submission: Submission) -> str:
|
||||
return (
|
||||
base64.urlsafe_b64encode(
|
||||
f"{submission.created_at.isoformat()}|{submission.id}".encode("ascii")
|
||||
)
|
||||
.decode("ascii")
|
||||
.rstrip("=")
|
||||
)
|
||||
|
||||
|
||||
def cursor_decode(value: str) -> tuple[datetime, UUID]:
|
||||
try:
|
||||
if not 1 <= len(value) <= 128:
|
||||
raise ValueError
|
||||
date, identifier = (
|
||||
base64.b64decode(value + "=" * (-len(value) % 4), altchars=b"-_", validate=True)
|
||||
.decode("ascii")
|
||||
.split("|")
|
||||
)
|
||||
timestamp = datetime.fromisoformat(date)
|
||||
if timestamp.utcoffset() is None:
|
||||
raise ValueError
|
||||
return timestamp, UUID(identifier)
|
||||
except (ValueError, UnicodeError):
|
||||
raise ValueError("invalid history cursor") from None
|
||||
|
||||
|
||||
def summary(submission: Submission, evaluation: Evaluation) -> SubmissionSummary:
|
||||
terminal = evaluation.state in {"finished", "failed", "cancelled"}
|
||||
return SubmissionSummary(
|
||||
submission_id=submission.id,
|
||||
evaluation_id=evaluation.id,
|
||||
problem_version_id=submission.problem_version_id,
|
||||
language=submission.language,
|
||||
state=evaluation.state,
|
||||
created_at=submission.created_at,
|
||||
updated_at=evaluation.updated_at,
|
||||
attempts=evaluation.attempts_count,
|
||||
verdict=evaluation.verdict if terminal else None,
|
||||
failure_code=evaluation.failure_code,
|
||||
)
|
||||
|
||||
|
||||
class SubmissionQueries:
|
||||
def __init__(
|
||||
self,
|
||||
sessions: async_sessionmaker,
|
||||
artifacts: ArtifactRepository,
|
||||
cancellation: CancellationPort | None = None,
|
||||
):
|
||||
self.sessions, self.artifacts = sessions, artifacts
|
||||
self.cancellation = cancellation
|
||||
|
||||
def owned(self, principal: Principal, submission_id: UUID):
|
||||
return (
|
||||
select(Submission, Evaluation)
|
||||
.join(Evaluation, Evaluation.submission_id == Submission.id)
|
||||
.where(Submission.id == submission_id, Submission.account_id == principal.account_id)
|
||||
)
|
||||
|
||||
async def page(self, principal: Principal, after: str | None, limit: int) -> SubmissionPage:
|
||||
if not 1 <= limit <= 100:
|
||||
raise ValueError("invalid history limit")
|
||||
query = (
|
||||
select(Submission, Evaluation)
|
||||
.join(Evaluation, Evaluation.submission_id == Submission.id)
|
||||
.where(Submission.account_id == principal.account_id)
|
||||
.order_by(Submission.created_at.desc(), Submission.id.desc())
|
||||
.limit(limit + 1)
|
||||
)
|
||||
if after is not None:
|
||||
query = query.where(tuple_(Submission.created_at, Submission.id) < cursor_decode(after))
|
||||
async with self.sessions() as session:
|
||||
rows = (await session.execute(query)).all()
|
||||
return SubmissionPage(
|
||||
items=tuple(summary(s, e) for s, e in rows[:limit]),
|
||||
next_after=cursor_encode(rows[limit - 1][0]) if len(rows) > limit else None,
|
||||
)
|
||||
|
||||
async def detail(self, principal: Principal, submission_id: UUID) -> SubmissionDetail:
|
||||
async with self.sessions() as session:
|
||||
row = (await session.execute(self.owned(principal, submission_id))).one_or_none()
|
||||
if row is None:
|
||||
raise SubmissionUnavailable("submission unavailable")
|
||||
submission, evaluation = row
|
||||
result = None
|
||||
if evaluation.state in {"finished", "failed", "cancelled"} and evaluation.final_report:
|
||||
# A single joined snapshot, strict validation and explicit projection prevent leaking
|
||||
# internal bindings, artifacts, hidden case identifiers or partial attempt reports.
|
||||
report = decode_report(
|
||||
json.dumps(
|
||||
evaluation.final_report, ensure_ascii=False, separators=(",", ":")
|
||||
).encode()
|
||||
)
|
||||
if str(report.binding.evaluation_id) != str(evaluation.id):
|
||||
raise ValueError("stored result binding invalid")
|
||||
if report.verdict.value != evaluation.verdict:
|
||||
raise ValueError("stored result verdict invalid")
|
||||
result = PublicJudgement(
|
||||
compilation_status=report.compilation.status,
|
||||
diagnostics=report.compilation.diagnostics,
|
||||
diagnostics_truncated=report.compilation.diagnostics_truncated,
|
||||
cpu_time_us=report.run_cpu_time_us,
|
||||
memory_peak_bytes=report.run_memory_peak_bytes,
|
||||
cleanup_complete=report.cleanup_complete,
|
||||
cases=tuple(
|
||||
PublicCaseResult(position=i, **c.model_dump(exclude={"case_id"}))
|
||||
for i, c in enumerate(report.cases)
|
||||
),
|
||||
)
|
||||
return SubmissionDetail(**summary(submission, evaluation).model_dump(), result=result)
|
||||
|
||||
async def cancel(self, principal: Principal, submission_id: UUID) -> CancellationAccepted:
|
||||
async with self.sessions() as session:
|
||||
row = (await session.execute(self.owned(principal, submission_id))).one_or_none()
|
||||
if row is None:
|
||||
raise SubmissionUnavailable("submission unavailable")
|
||||
submission, evaluation = row
|
||||
if self.cancellation is None or not await self.cancellation.cancel(evaluation.id):
|
||||
raise CancelUnavailable("cancellation service unavailable")
|
||||
return CancellationAccepted(submission_id=submission.id, evaluation_id=evaluation.id)
|
||||
|
||||
async def owned_source(self, principal: Principal, submission_id: UUID) -> bytes:
|
||||
async with self.sessions() as session:
|
||||
submission = await session.scalar(
|
||||
select(Submission).where(
|
||||
Submission.id == submission_id, Submission.account_id == principal.account_id
|
||||
)
|
||||
)
|
||||
if submission is None:
|
||||
raise SubmissionUnavailable("submission unavailable")
|
||||
metadata = await session.get(Artifact, submission.source_digest)
|
||||
if (
|
||||
metadata is None
|
||||
or metadata.state != "ready"
|
||||
or not 1 <= metadata.bytes <= MAX_SOURCE_BYTES
|
||||
):
|
||||
raise SubmissionUnavailable("submission source unavailable")
|
||||
reference = ArtifactReference(digest=metadata.digest, bytes=metadata.bytes)
|
||||
try:
|
||||
return await self.artifacts.read(session, reference)
|
||||
except ArtifactError:
|
||||
raise SubmissionUnavailable("submission source unavailable") from None
|
||||
@@ -1,27 +0,0 @@
|
||||
from sqlalchemy import func, select
|
||||
|
||||
from kf_oj.contracts.judging import LeaseBinding
|
||||
from kf_oj.modules.submissions.models import Evaluation, JudgeAttempt, Submission
|
||||
|
||||
|
||||
async def leased_submission(session, binding: LeaseBinding, token_hash: str):
|
||||
return (
|
||||
await session.execute(
|
||||
select(Evaluation, JudgeAttempt, Submission)
|
||||
.join(JudgeAttempt, JudgeAttempt.id == Evaluation.current_attempt_id)
|
||||
.join(Submission, Submission.id == Evaluation.submission_id)
|
||||
.where(
|
||||
Evaluation.id == binding.evaluation_id,
|
||||
Evaluation.state.in_(("claimed", "compiling", "running")),
|
||||
Evaluation.cancel_requested_at.is_(None),
|
||||
Evaluation.fencing_version == binding.fencing_version,
|
||||
JudgeAttempt.id == binding.attempt_id,
|
||||
JudgeAttempt.evaluation_id == Evaluation.id,
|
||||
JudgeAttempt.worker_id == binding.worker_id,
|
||||
JudgeAttempt.fencing_version == binding.fencing_version,
|
||||
JudgeAttempt.lease_token_hash == token_hash,
|
||||
JudgeAttempt.expires_at > func.clock_timestamp(),
|
||||
JudgeAttempt.revoked_at.is_(None),
|
||||
)
|
||||
)
|
||||
).one_or_none()
|
||||
@@ -1,123 +0,0 @@
|
||||
"""Only initial queued evaluation/outbox writes; execution belongs to Coordinator."""
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from uuid import uuid4
|
||||
|
||||
from sqlalchemy import func, select, text
|
||||
|
||||
from kf_oj.contracts.judging_v2 import admit_manifest
|
||||
from kf_oj.contracts.submissions import CreateSubmission, SubmissionCreated
|
||||
from kf_oj.infra.artifacts.repository import ArtifactRepository
|
||||
from kf_oj.infra.messaging.models import Outbox
|
||||
from kf_oj.modules.identity.public import Principal
|
||||
from kf_oj.modules.problems.public import ProblemPort, ProblemUnavailable
|
||||
from kf_oj.modules.submissions.models import Evaluation, Submission
|
||||
from kf_oj.modules.submissions.public import (
|
||||
IdempotencyConflict,
|
||||
SubmissionBusy,
|
||||
SubmissionUnavailable,
|
||||
)
|
||||
|
||||
|
||||
class SubmissionService:
|
||||
def __init__(self, sessions, problems: ProblemPort, artifacts: ArtifactRepository, admission):
|
||||
self.sessions, self.problems = sessions, problems
|
||||
self.artifacts, self.admission = artifacts, admission
|
||||
|
||||
@staticmethod
|
||||
def _response(submission, evaluation):
|
||||
return SubmissionCreated(
|
||||
submission_id=submission.id,
|
||||
evaluation_id=evaluation.id,
|
||||
problem_version_id=submission.problem_version_id,
|
||||
language=submission.language,
|
||||
state=evaluation.state,
|
||||
created_at=submission.created_at,
|
||||
)
|
||||
|
||||
async def create(self, principal: Principal, request: CreateSubmission) -> SubmissionCreated:
|
||||
# Validate even trusted non-HTTP callers. No path or argv is accepted.
|
||||
request = CreateSubmission.model_validate(
|
||||
request.model_dump(exclude={"language"}) | {"language": request.language}
|
||||
)
|
||||
source = request.source.encode("utf-8")
|
||||
request_digest = hashlib.sha256(
|
||||
json.dumps(
|
||||
[
|
||||
str(request.problem_version_id),
|
||||
request.language.value,
|
||||
hashlib.sha256(source).hexdigest(),
|
||||
],
|
||||
separators=(",", ":"),
|
||||
).encode()
|
||||
).hexdigest()
|
||||
if self.admission is None:
|
||||
raise SubmissionUnavailable("judge admission unavailable")
|
||||
async with self.sessions.begin() as session:
|
||||
# Bounded pending capacity and per-account idempotency serialize across API processes.
|
||||
# Two-key namespace is separate from artifact/slug advisory locks.
|
||||
await session.execute(text("SELECT pg_advisory_xact_lock(1262907470, 0)"))
|
||||
existing = (
|
||||
await session.execute(
|
||||
select(Submission, Evaluation)
|
||||
.join(Evaluation, Evaluation.submission_id == Submission.id)
|
||||
.where(
|
||||
Submission.account_id == principal.account_id,
|
||||
Submission.idempotency_key == request.idempotency_key,
|
||||
)
|
||||
)
|
||||
).one_or_none()
|
||||
if existing is not None:
|
||||
if existing[0].request_digest != request_digest:
|
||||
raise IdempotencyConflict("idempotency key belongs to another request")
|
||||
return self._response(*existing)
|
||||
pending = ("queued", "claimed", "compiling", "running", "cancelling")
|
||||
total = await session.scalar(
|
||||
select(func.count()).select_from(Evaluation).where(Evaluation.state.in_(pending))
|
||||
)
|
||||
owned = await session.scalar(
|
||||
select(func.count())
|
||||
.select_from(Submission)
|
||||
.join(Evaluation, Evaluation.submission_id == Submission.id)
|
||||
.where(Submission.account_id == principal.account_id, Evaluation.state.in_(pending))
|
||||
)
|
||||
if total >= 64 or owned >= 8:
|
||||
raise SubmissionBusy("pending submission capacity exhausted")
|
||||
try:
|
||||
version = await self.problems.for_submission(session, request.problem_version_id)
|
||||
except ProblemUnavailable:
|
||||
raise SubmissionUnavailable("problem version unavailable") from None
|
||||
if request.language not in version.languages:
|
||||
raise SubmissionUnavailable("language unavailable")
|
||||
# Bytes fsync before metadata/ref transaction. Rollback may leave collectable bytes,
|
||||
# but cannot leave an evaluation that refers to non-durable bytes.
|
||||
reference = await self.artifacts.put(session, source)
|
||||
submission_id, evaluation_id = uuid4(), uuid4()
|
||||
manifest = version.manifest(evaluation_id, submission_id, request.language, reference)
|
||||
admit_manifest(self.admission, manifest, 1)
|
||||
submission = Submission(
|
||||
id=submission_id,
|
||||
account_id=principal.account_id,
|
||||
problem_id=version.problem_id,
|
||||
problem_version_id=version.version_id,
|
||||
language=request.language.value,
|
||||
source_digest=reference.digest,
|
||||
idempotency_key=request.idempotency_key,
|
||||
request_digest=request_digest,
|
||||
)
|
||||
session.add(submission)
|
||||
await session.flush()
|
||||
await session.execute(
|
||||
Evaluation.__table__.insert().values(
|
||||
id=evaluation_id,
|
||||
submission_id=submission_id,
|
||||
problem_version_id=version.version_id,
|
||||
)
|
||||
)
|
||||
session.add(Outbox(id=uuid4(), evaluation_id=evaluation_id))
|
||||
await session.flush()
|
||||
evaluation = await session.get(Evaluation, evaluation_id)
|
||||
response = self._response(submission, evaluation)
|
||||
# The transaction commit must succeed before any success is returned.
|
||||
return response
|
||||
@@ -1,108 +0,0 @@
|
||||
import asyncio
|
||||
import hashlib
|
||||
import hmac
|
||||
import json
|
||||
import re
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker
|
||||
|
||||
from kf_oj.contracts.judging import LeaseBinding
|
||||
from kf_oj.contracts.judging_versions import (
|
||||
Manifest,
|
||||
decode_manifest,
|
||||
manifest_bytes,
|
||||
)
|
||||
from kf_oj.infra.artifacts.repository import ArtifactRepository
|
||||
from kf_oj.infra.artifacts.store import ArtifactError
|
||||
from kf_oj.modules.problems.public import ProblemPort, ProblemUnavailable
|
||||
from kf_oj.modules.submissions import repository
|
||||
from kf_oj.modules.submissions.public import WorkerAccessDenied, WorkerReadBusy
|
||||
|
||||
|
||||
class WorkerAccess:
|
||||
def __init__(
|
||||
self, sessions: async_sessionmaker, problems: ProblemPort, artifacts: ArtifactRepository
|
||||
):
|
||||
self.sessions, self.problems, self.artifacts = sessions, problems, artifacts
|
||||
self._reads = asyncio.Semaphore(2)
|
||||
|
||||
@asynccontextmanager
|
||||
async def _read_slot(self):
|
||||
if self._reads.locked():
|
||||
raise WorkerReadBusy("worker read capacity exhausted")
|
||||
async with self._reads:
|
||||
yield
|
||||
|
||||
@staticmethod
|
||||
def _token_hash(token: str) -> str:
|
||||
if not re.fullmatch(r"[0-9a-f]{64}", token):
|
||||
raise WorkerAccessDenied("worker grant rejected")
|
||||
return hashlib.sha256(token.encode("ascii")).hexdigest()
|
||||
|
||||
async def _manifest(self, session, binding: LeaseBinding, token: str) -> Manifest:
|
||||
row = await repository.leased_submission(session, binding, self._token_hash(token))
|
||||
if row is None:
|
||||
raise WorkerAccessDenied("worker grant rejected")
|
||||
evaluation, attempt, submission = row
|
||||
# Use a sanitized decoder at this trust boundary. JSONB snapshots are immutable for
|
||||
# runtime roles; use canonical bytes for both retrieval and digest verification.
|
||||
|
||||
try:
|
||||
manifest = decode_manifest(json.dumps(attempt.manifest).encode())
|
||||
if (
|
||||
manifest.evaluation_id != evaluation.id
|
||||
or manifest.submission_id != submission.id
|
||||
or manifest.problem_version_id != evaluation.problem_version_id
|
||||
or manifest.problem_version_id != submission.problem_version_id
|
||||
or manifest.language.value != submission.language
|
||||
or manifest.source.digest != submission.source_digest
|
||||
or not hmac.compare_digest(
|
||||
hashlib.sha256(manifest_bytes(manifest)).hexdigest(), attempt.manifest_digest
|
||||
)
|
||||
):
|
||||
raise WorkerAccessDenied("worker snapshot rejected")
|
||||
version = await self.problems.for_submission(session, manifest.problem_version_id)
|
||||
await self.artifacts.check_metadata(session, manifest.source)
|
||||
expected = version.manifest(
|
||||
manifest.evaluation_id, manifest.submission_id, manifest.language, manifest.source
|
||||
)
|
||||
if version.problem_id != submission.problem_id or manifest_bytes(
|
||||
manifest
|
||||
) != manifest_bytes(expected):
|
||||
raise WorkerAccessDenied("worker snapshot rejected")
|
||||
return manifest
|
||||
except (ValueError, ProblemUnavailable):
|
||||
raise WorkerAccessDenied("worker snapshot rejected") from None
|
||||
|
||||
async def manifest(self, binding: LeaseBinding, token: str) -> bytes:
|
||||
async with self._read_slot(), self.sessions() as session:
|
||||
manifest = await self._manifest(session, binding, token)
|
||||
payload = manifest_bytes(manifest)
|
||||
# READ COMMITTED: recheck after all preparation and just before releasing bytes.
|
||||
if (
|
||||
await repository.leased_submission(session, binding, self._token_hash(token))
|
||||
is None
|
||||
):
|
||||
raise WorkerAccessDenied("worker grant rejected")
|
||||
return payload
|
||||
|
||||
async def artifact(self, binding: LeaseBinding, token: str, digest: str) -> bytes:
|
||||
async with self._read_slot(), self.sessions() as session:
|
||||
manifest = await self._manifest(session, binding, token)
|
||||
references = (manifest.source,) + tuple(
|
||||
reference for case in manifest.cases for reference in (case.input, case.expected)
|
||||
)
|
||||
reference = next((ref for ref in references if ref.digest == digest), None)
|
||||
if reference is None:
|
||||
raise WorkerAccessDenied("artifact grant rejected")
|
||||
try:
|
||||
payload = await self.artifacts.read(session, reference)
|
||||
except ArtifactError:
|
||||
raise WorkerAccessDenied("artifact unavailable") from None
|
||||
if (
|
||||
await repository.leased_submission(session, binding, self._token_hash(token))
|
||||
is None
|
||||
):
|
||||
raise WorkerAccessDenied("worker grant rejected")
|
||||
return payload
|
||||
@@ -1,53 +0,0 @@
|
||||
"""Optional real PostgreSQL acceptance; never reuse or reset an existing business DB."""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
from uuid import uuid4
|
||||
|
||||
import asyncpg
|
||||
import pytest
|
||||
from sqlalchemy.engine import make_url
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def postgres_url():
|
||||
admin = os.environ.get("KF_OJ_TEST_POSTGRES")
|
||||
if admin is None:
|
||||
pytest.skip("real PostgreSQL acceptance requires KF_OJ_TEST_POSTGRES")
|
||||
url = make_url(admin)
|
||||
database = "kf_oj_test_" + uuid4().hex
|
||||
assert url.drivername == "postgresql+asyncpg"
|
||||
test_url = url.set(database=database).render_as_string(hide_password=False)
|
||||
|
||||
async def manage(command):
|
||||
connection = await asyncpg.connect(
|
||||
url.set(drivername="postgresql").render_as_string(hide_password=False)
|
||||
)
|
||||
try:
|
||||
await connection.execute(command)
|
||||
finally:
|
||||
await connection.close()
|
||||
|
||||
asyncio.run(manage(f'CREATE DATABASE "{database}"'))
|
||||
try:
|
||||
migrate(test_url, "upgrade", "head")
|
||||
yield test_url
|
||||
finally:
|
||||
asyncio.run(manage(f'DROP DATABASE "{database}"'))
|
||||
|
||||
|
||||
def migrate(url: str, command: str, target: str | None = None) -> None:
|
||||
subprocess.run(
|
||||
[str(Path(__file__).resolve().parents[1] / ".venv/bin/alembic"), command]
|
||||
+ ([target] if target else []),
|
||||
cwd=Path(__file__).resolve().parents[1],
|
||||
env=os.environ | {"KF_OJ_DATABASE_URL": url},
|
||||
check=True,
|
||||
capture_output=True,
|
||||
)
|
||||
|
||||
|
||||
async def connect(url: str):
|
||||
return await asyncpg.connect(url.replace("postgresql+asyncpg:", "postgresql:", 1))
|
||||
@@ -1,204 +0,0 @@
|
||||
import asyncio
|
||||
import hashlib
|
||||
import os
|
||||
from uuid import uuid4
|
||||
|
||||
import pytest
|
||||
from sqlalchemy import func, select, text, update
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
|
||||
|
||||
from kf_oj.contracts.judging import ArtifactReference
|
||||
from kf_oj.infra.artifacts.models import Artifact
|
||||
from kf_oj.infra.artifacts.repository import ArtifactRepository
|
||||
from kf_oj.infra.artifacts.store import ArtifactError, ArtifactStore
|
||||
|
||||
|
||||
def test_immutable_store_digest_capacity_and_recovery(tmp_path):
|
||||
root = tmp_path / "artifacts"
|
||||
store = ArtifactStore(root, max_bytes=6, max_files=2)
|
||||
try:
|
||||
first = store.put(b"abc")
|
||||
assert store.read(first) == b"abc"
|
||||
assert store.put(b"abc") == first
|
||||
with pytest.raises(ArtifactError, match="digest or length"):
|
||||
store.put(b"changed", first)
|
||||
second = store.put(b"def")
|
||||
with pytest.raises(ArtifactError, match="capacity"):
|
||||
store.put(b"g")
|
||||
assert store.read(first) == b"abc" and store.read(second) == b"def"
|
||||
(root / first.digest).write_bytes(b"xyz")
|
||||
with pytest.raises(ArtifactError, match="digest or length"):
|
||||
store.read(first)
|
||||
with pytest.raises(ArtifactError, match="digest or length"):
|
||||
store.put(b"abc") # Corrupt files are not silently repaired or overwritten.
|
||||
assert (root / first.digest).read_bytes() == b"xyz"
|
||||
finally:
|
||||
store.close()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("attack", ["symlink", "hardlink", "fifo", "traversal"])
|
||||
def test_links_and_non_regular_entries_refused(tmp_path, attack):
|
||||
root = tmp_path / "artifacts"
|
||||
store = ArtifactStore(root)
|
||||
data = b"private bytes"
|
||||
ref = ArtifactReference(digest=hashlib.sha256(data).hexdigest(), bytes=len(data))
|
||||
private = tmp_path / "outside"
|
||||
private.write_bytes(data)
|
||||
private.chmod(0o600)
|
||||
try:
|
||||
if attack == "symlink":
|
||||
(root / ref.digest).symlink_to(private)
|
||||
elif attack == "hardlink":
|
||||
os.link(private, root / ref.digest)
|
||||
elif attack == "fifo":
|
||||
os.mkfifo(root / ref.digest, 0o600)
|
||||
else:
|
||||
ref = ref.model_copy(update={"digest": "../outside"})
|
||||
with pytest.raises(ArtifactError):
|
||||
store.read(ref)
|
||||
finally:
|
||||
store.close()
|
||||
|
||||
|
||||
def test_private_root_and_bounded_temporary_cleanup(tmp_path):
|
||||
root = tmp_path / "root"
|
||||
root.mkdir(mode=0o755)
|
||||
with pytest.raises(ArtifactError, match="private"):
|
||||
ArtifactStore(root)
|
||||
linked = tmp_path / "linked"
|
||||
linked.symlink_to(root)
|
||||
with pytest.raises(ArtifactError, match="links"):
|
||||
ArtifactStore(linked)
|
||||
root.chmod(0o700)
|
||||
store = ArtifactStore(root)
|
||||
try:
|
||||
old = root / (".tmp-" + uuid4().hex)
|
||||
recent = root / (".tmp-" + uuid4().hex)
|
||||
for path in (old, recent):
|
||||
path.write_bytes(b"temporary")
|
||||
path.chmod(0o600)
|
||||
os.utime(old, (0, 0))
|
||||
assert store.clean_temporary(limit=1) == 1
|
||||
assert not old.exists() and recent.exists()
|
||||
with pytest.raises(ArtifactError):
|
||||
store.clean_temporary(min_age_seconds=0)
|
||||
finally:
|
||||
store.close()
|
||||
|
||||
|
||||
def test_physical_quota_serializes_parallel_publishers(tmp_path):
|
||||
root = tmp_path / "root"
|
||||
first = ArtifactStore(root, max_bytes=6, max_files=10)
|
||||
second = ArtifactStore(root, max_bytes=6, max_files=10)
|
||||
|
||||
async def run():
|
||||
results = await asyncio.gather(
|
||||
asyncio.to_thread(first.put, b"abcd"),
|
||||
asyncio.to_thread(second.put, b"efgh"),
|
||||
return_exceptions=True,
|
||||
)
|
||||
assert sum(isinstance(r, ArtifactReference) for r in results) == 1
|
||||
assert sum(isinstance(r, ArtifactError) for r in results) == 1
|
||||
assert sum(p.stat().st_size for p in root.iterdir() if p.name != ".lock") == 4
|
||||
|
||||
try:
|
||||
asyncio.run(run())
|
||||
finally:
|
||||
first.close()
|
||||
second.close()
|
||||
|
||||
|
||||
def test_real_metadata_rollback_reference_fencing_and_collection(postgres_url, tmp_path):
|
||||
store = ArtifactStore(tmp_path / "bytes")
|
||||
artifacts = ArtifactRepository(store)
|
||||
|
||||
async def run():
|
||||
engine = create_async_engine(postgres_url)
|
||||
sessions = async_sessionmaker(engine, expire_on_commit=False)
|
||||
ref = store.reference(b"rolled back")
|
||||
try:
|
||||
with pytest.raises(RuntimeError):
|
||||
async with sessions.begin() as s:
|
||||
await s.execute(text("SET LOCAL ROLE kf_oj_platform"))
|
||||
await artifacts.put(s, b"rolled back")
|
||||
raise RuntimeError("transaction aborted")
|
||||
async with sessions() as s:
|
||||
assert await s.get(Artifact, ref.digest) is None
|
||||
assert store.read(ref) == b"rolled back" # Durable orphan, never dangling DB reference.
|
||||
os.utime(tmp_path / "bytes" / ref.digest, (0, 0))
|
||||
assert await artifacts.collect_orphans(sessions) == 1
|
||||
with pytest.raises(ArtifactError):
|
||||
store.read(ref)
|
||||
async with sessions.begin() as s:
|
||||
await s.execute(text("SET LOCAL ROLE kf_oj_platform"))
|
||||
ref = await artifacts.put(s, b"durable unreferenced")
|
||||
os.utime(tmp_path / "bytes" / ref.digest, (0, 0))
|
||||
assert await artifacts.collect_orphans(sessions) == 0
|
||||
async with sessions.begin() as s:
|
||||
await artifacts.mark_deleting(s, ref.digest)
|
||||
with pytest.raises(ArtifactError):
|
||||
async with sessions.begin() as s:
|
||||
await artifacts.put(s, b"durable unreferenced")
|
||||
assert await artifacts.finish_deleting(sessions) == 1
|
||||
async with sessions() as s:
|
||||
assert list(await s.scalars(select(Artifact))) == []
|
||||
finally:
|
||||
await engine.dispose()
|
||||
|
||||
try:
|
||||
asyncio.run(run())
|
||||
finally:
|
||||
store.close()
|
||||
|
||||
|
||||
def test_bounded_metadata_collector_preserves_references_and_resumes(postgres_url, tmp_path):
|
||||
from datetime import timedelta
|
||||
|
||||
from test_problems import import_fixture, platform_role
|
||||
|
||||
from kf_oj.modules.problems.service import ProblemService
|
||||
|
||||
root = tmp_path / "import"
|
||||
_, policy = import_fixture(root)
|
||||
store = ArtifactStore(tmp_path / "bytes")
|
||||
repository = ArtifactRepository(store)
|
||||
|
||||
async def run():
|
||||
engine = create_async_engine(postgres_url)
|
||||
sessions = async_sessionmaker(engine, expire_on_commit=False)
|
||||
try:
|
||||
service = ProblemService(sessions, repository, policy)
|
||||
version = await service.import_directory(root)
|
||||
async with sessions.begin() as session:
|
||||
orphan = await repository.put(session, b"committed without business reference")
|
||||
await session.execute(
|
||||
update(Artifact).values(created_at=func.now() - timedelta(days=2))
|
||||
)
|
||||
# After trusted fixture preparation, run collection with actual limited platform rights.
|
||||
await engine.dispose()
|
||||
platform_role(engine)
|
||||
marked_total, cursor = 0, None
|
||||
for _ in range(100):
|
||||
marked, cursor = await repository.collect_metadata(sessions, limit=1, after=cursor)
|
||||
marked_total += marked
|
||||
if cursor is None:
|
||||
break
|
||||
else:
|
||||
pytest.fail("bounded metadata scan failed to finish")
|
||||
assert marked_total == 1
|
||||
assert await repository.finish_deleting(sessions, limit=1) == 1
|
||||
assert await repository.finish_deleting(sessions, limit=1) == 0
|
||||
async with sessions() as session:
|
||||
projected = await service.for_submission(session, version)
|
||||
assert (
|
||||
await repository.read(session, projected.cases[0].expected)
|
||||
== b"hidden-expected\x00\xff"
|
||||
)
|
||||
assert await session.get(Artifact, orphan.digest) is None
|
||||
finally:
|
||||
await engine.dispose()
|
||||
|
||||
try:
|
||||
asyncio.run(run())
|
||||
finally:
|
||||
store.close()
|
||||
@@ -19,11 +19,7 @@ def test_shared_wire_examples(path: Path) -> None:
|
||||
if ".invalid-" in path.name:
|
||||
with pytest.raises(ValidationError):
|
||||
JudgeTaskNotification.model_validate(payload)
|
||||
if ".invalid-semantic-" in path.name:
|
||||
# JSON Schema counts 1.0 as an integer; the wire decoder forbids coercion.
|
||||
assert validator.is_valid(payload)
|
||||
else:
|
||||
assert not validator.is_valid(payload)
|
||||
assert not validator.is_valid(payload)
|
||||
else:
|
||||
notification = JudgeTaskNotification.model_validate(payload)
|
||||
validator.validate(payload)
|
||||
|
||||
@@ -1,612 +0,0 @@
|
||||
"""Actual Rust HTTP + PostgreSQL state tests, using explicitly synthetic reports.
|
||||
|
||||
No program is compiled/run by this host suite. Root stop proofs and lease-loss
|
||||
execution are separately required in the independent VM (F05/F06).
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import secrets
|
||||
import socket
|
||||
import subprocess
|
||||
from contextlib import asynccontextmanager
|
||||
from pathlib import Path
|
||||
from uuid import UUID, uuid4
|
||||
|
||||
import asyncpg
|
||||
import httpx
|
||||
from sqlalchemy.engine import make_url
|
||||
from sqlalchemy.ext.asyncio import create_async_engine
|
||||
from test_problems import import_fixture, platform_role
|
||||
|
||||
from kf_oj.api.app import create_app
|
||||
from kf_oj.contracts.submissions import CreateSubmission
|
||||
from kf_oj.core.settings import Settings
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def fixture(
|
||||
postgres_url, tmp_path, monkeypatch, *, import_builder=import_fixture, node_policy=None
|
||||
):
|
||||
_, policy = import_builder(tmp_path / "import")
|
||||
engine = create_async_engine(postgres_url)
|
||||
platform_role(engine)
|
||||
monkeypatch.setattr("kf_oj.api.app.build_engine", lambda _: engine)
|
||||
app = create_app(
|
||||
Settings(
|
||||
_env_file=None,
|
||||
environment="test",
|
||||
database_url=postgres_url,
|
||||
artifact_root=tmp_path / "artifacts",
|
||||
judge_admission=policy,
|
||||
)
|
||||
)
|
||||
url = make_url(postgres_url)
|
||||
c = await asyncpg.connect(
|
||||
url.set(drivername="postgresql").render_as_string(hide_password=False)
|
||||
)
|
||||
role = "jp_coord_" + uuid4().hex
|
||||
password = secrets.token_hex(32)
|
||||
await c.execute(f"CREATE ROLE {role} LOGIN PASSWORD '{password}'")
|
||||
await c.execute(f"GRANT kf_oj_coordinator TO {role}")
|
||||
coord_url = url.set(drivername="postgresql", username=role, password=password).render_as_string(
|
||||
hide_password=False
|
||||
)
|
||||
# Only loopback control-plane service is started; finite actual systemd scope.
|
||||
with socket.socket() as reserved:
|
||||
reserved.bind(("127.0.0.1", 0))
|
||||
port = reserved.getsockname()[1]
|
||||
workers = [dict(id=str(uuid4()), token=secrets.token_hex(32)) for _ in range(2)]
|
||||
platform = secrets.token_hex(32)
|
||||
config = tmp_path / "coordinator.json"
|
||||
config.write_text(
|
||||
json.dumps(
|
||||
dict(
|
||||
listen=f"127.0.0.1:{port}",
|
||||
database_url=coord_url,
|
||||
workers=workers,
|
||||
platform_token=platform,
|
||||
admission=(node_policy or policy).model_dump(mode="json"),
|
||||
lease_seconds=5,
|
||||
recovery_seconds=1,
|
||||
)
|
||||
)
|
||||
)
|
||||
config.chmod(0o600)
|
||||
unit = "kf-jp-coord-" + uuid4().hex
|
||||
binary = ROOT / "judge/target/debug/kf-coordinator"
|
||||
assert binary.is_file(), "build actual Coordinator before this acceptance"
|
||||
subprocess.run(
|
||||
[
|
||||
"systemd-run",
|
||||
"--user",
|
||||
f"--unit={unit}",
|
||||
"--collect",
|
||||
"--service-type=exec",
|
||||
"--property=MemoryMax=128M",
|
||||
"--property=MemorySwapMax=0",
|
||||
"--property=CPUQuota=50%",
|
||||
"--property=TasksMax=32",
|
||||
"--property=RuntimeMaxSec=180",
|
||||
"--property=NoNewPrivileges=yes",
|
||||
str(binary),
|
||||
"--config",
|
||||
str(config),
|
||||
],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
)
|
||||
try:
|
||||
async with app.router.lifespan_context(app):
|
||||
owner = await app.state.identity.create_account(
|
||||
"coordinator_fixture", secrets.token_urlsafe(24)
|
||||
)
|
||||
version = await app.state.problems.import_directory(tmp_path / "import")
|
||||
async with httpx.AsyncClient(base_url=f"http://127.0.0.1:{port}", timeout=10) as client:
|
||||
for _ in range(100):
|
||||
try:
|
||||
if (await client.get("/health/ready")).status_code == 200:
|
||||
break
|
||||
except httpx.TransportError:
|
||||
pass
|
||||
await asyncio.sleep(0.05)
|
||||
else:
|
||||
raise AssertionError("actual Coordinator not ready")
|
||||
props = subprocess.check_output(
|
||||
[
|
||||
"systemctl",
|
||||
"--user",
|
||||
"show",
|
||||
unit,
|
||||
"--property=ControlGroup,MemoryMax,MemorySwapMax,TasksMax,CPUQuotaPerSecUSec,RuntimeMaxUSec,MainPID",
|
||||
],
|
||||
text=True,
|
||||
)
|
||||
values = dict(line.split("=", 1) for line in props.splitlines())
|
||||
group = Path("/sys/fs/cgroup") / values["ControlGroup"].lstrip("/")
|
||||
assert {
|
||||
n: (group / n).read_text().strip()
|
||||
for n in ["memory.max", "memory.swap.max", "pids.max", "cpu.max"]
|
||||
} == {
|
||||
"memory.max": "134217728",
|
||||
"memory.swap.max": "0",
|
||||
"pids.max": "32",
|
||||
"cpu.max": "50000 100000",
|
||||
}
|
||||
|
||||
async def create(language="c17"):
|
||||
return await app.state.submissions.create(
|
||||
owner,
|
||||
CreateSubmission(
|
||||
problem_version_id=version,
|
||||
language=language,
|
||||
idempotency_key=uuid4(),
|
||||
source="synthetic report fixture; never compiled on host",
|
||||
),
|
||||
)
|
||||
|
||||
client.fixture_app = app
|
||||
client.fixture_account = owner
|
||||
client.fixture_version = version
|
||||
client.coordinator_unit = unit
|
||||
client.coordinator_role = role
|
||||
yield client, c, workers, platform, create
|
||||
finally:
|
||||
subprocess.run(["systemctl", "--user", "stop", unit], check=True, capture_output=True)
|
||||
assert (
|
||||
subprocess.run(
|
||||
["systemctl", "--user", "is-active", unit], capture_output=True
|
||||
).returncode
|
||||
!= 0
|
||||
)
|
||||
await c.execute(f"DROP ROLE {role}")
|
||||
await c.close()
|
||||
|
||||
|
||||
def notice(eid):
|
||||
return dict(schema_version=1, kind="judge.evaluation", evaluation_id=str(eid))
|
||||
|
||||
|
||||
def worker_headers(worker, grant=None):
|
||||
headers = {"Authorization": "Bearer " + worker["token"]}
|
||||
if grant:
|
||||
headers["X-Lease-Token"] = grant["lease_token"]
|
||||
return headers
|
||||
|
||||
|
||||
async def claim(client, created, worker):
|
||||
r = await client.post(
|
||||
"/internal/claim", json=notice(created.evaluation_id), headers=worker_headers(worker)
|
||||
)
|
||||
assert r.status_code == 200, r.text
|
||||
return r.json()
|
||||
|
||||
|
||||
async def manifest(c, g):
|
||||
v = await c.fetchval(
|
||||
"SELECT manifest FROM judge_attempts WHERE id=$1", UUID(g["binding"]["attempt_id"])
|
||||
)
|
||||
return json.loads(v)
|
||||
|
||||
|
||||
async def stages(client, worker, g):
|
||||
h = worker_headers(worker, g)
|
||||
for seq, stage, index in [(1, "compiling", None), (2, "running", 0), (3, "running", 1)]:
|
||||
r = await client.post(
|
||||
"/internal/stage",
|
||||
headers=h,
|
||||
json=dict(
|
||||
schema_version=1,
|
||||
kind="judge.stage",
|
||||
binding=g["binding"],
|
||||
sequence=seq,
|
||||
stage=stage,
|
||||
case_index=index,
|
||||
),
|
||||
)
|
||||
assert r.status_code == 204, r.text
|
||||
|
||||
|
||||
def synthetic_report(g, m, verdict="accepted"):
|
||||
cases = [
|
||||
dict(case_id=c["case_id"], verdict=verdict, cpu_time_us=100, memory_peak_bytes=4096)
|
||||
for c in m["cases"]
|
||||
]
|
||||
return dict(
|
||||
schema_version=1,
|
||||
kind="judge.result",
|
||||
binding=g["binding"],
|
||||
manifest_digest=g["manifest_digest"],
|
||||
compilation=dict(
|
||||
status="succeeded",
|
||||
program=dict(digest="d" * 64, bytes=4096),
|
||||
diagnostics="",
|
||||
diagnostics_truncated=False,
|
||||
cpu_time_us=1000,
|
||||
memory_peak_bytes=16384,
|
||||
),
|
||||
cases=cases,
|
||||
verdict=verdict,
|
||||
cleanup_complete=True,
|
||||
failure_code=None,
|
||||
run_cpu_time_us=len(cases) * 100,
|
||||
run_memory_peak_bytes=4096,
|
||||
)
|
||||
|
||||
|
||||
def test_actual_auth_claim_competition_stage_fence_and_report(postgres_url, tmp_path, monkeypatch):
|
||||
async def run():
|
||||
async with fixture(postgres_url, tmp_path, monkeypatch) as (
|
||||
client,
|
||||
c,
|
||||
workers,
|
||||
platform,
|
||||
create,
|
||||
):
|
||||
created = await create()
|
||||
n = notice(created.evaluation_id)
|
||||
assert (await client.post("/internal/claim", json=n)).status_code == 401
|
||||
assert (
|
||||
await client.post(
|
||||
"/internal/claim", json=n, headers={"Authorization": "Bearer " + platform}
|
||||
)
|
||||
).status_code == 409
|
||||
r = await client.post(
|
||||
"/internal/claim",
|
||||
content=b"x" * 262145,
|
||||
headers=worker_headers(workers[0]) | {"Content-Type": "application/json"},
|
||||
)
|
||||
assert r.status_code == 413
|
||||
outcomes = await asyncio.gather(
|
||||
*(
|
||||
client.post("/internal/claim", json=n, headers=worker_headers(w))
|
||||
for w in workers
|
||||
)
|
||||
)
|
||||
assert sorted(r.status_code for r in outcomes) == [200, 204]
|
||||
g = next(r.json() for r in outcomes if r.status_code == 200)
|
||||
w = next(w for w in workers if w["id"] == g["binding"]["worker_id"])
|
||||
other = next(w2 for w2 in workers if w2 != w)
|
||||
assert await c.fetchval("SELECT count(*) FROM judge_attempts") == 1
|
||||
assert (
|
||||
await client.post(
|
||||
"/internal/renew", json=g["binding"], headers=worker_headers(other, g)
|
||||
)
|
||||
).status_code == 409
|
||||
assert (
|
||||
await client.post(
|
||||
"/internal/renew",
|
||||
json=g["binding"] | {"fencing_version": 2},
|
||||
headers=worker_headers(w, g),
|
||||
)
|
||||
).status_code == 409
|
||||
bad = dict(
|
||||
schema_version=1,
|
||||
kind="judge.stage",
|
||||
binding=g["binding"],
|
||||
sequence=1,
|
||||
stage="running",
|
||||
case_index=0,
|
||||
)
|
||||
assert (
|
||||
await client.post("/internal/stage", json=bad, headers=worker_headers(w, g))
|
||||
).status_code == 409
|
||||
await stages(client, w, g)
|
||||
progress = await client.post(
|
||||
"/internal/progress", json=g["binding"], headers=worker_headers(w, g)
|
||||
)
|
||||
assert progress.status_code == 200
|
||||
assert progress.json() == {
|
||||
"state": "running",
|
||||
"sequence": 3,
|
||||
"stage": "running",
|
||||
"case_index": 1,
|
||||
}
|
||||
subprocess.run(
|
||||
["systemctl", "--user", "restart", client.coordinator_unit],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
)
|
||||
progress = await client.post(
|
||||
"/internal/progress", json=g["binding"], headers=worker_headers(w, g)
|
||||
)
|
||||
assert progress.status_code == 200 and progress.json()["sequence"] == 3
|
||||
|
||||
assert (
|
||||
await client.post("/internal/stage", json=bad, headers=worker_headers(w, g))
|
||||
).status_code == 409
|
||||
assert (
|
||||
await client.post(
|
||||
"/internal/renew", json=g["binding"], headers=worker_headers(w, g)
|
||||
)
|
||||
).status_code == 200
|
||||
m = await manifest(c, g)
|
||||
report = synthetic_report(g, m)
|
||||
wrong = report | {"cases": list(reversed(report["cases"]))}
|
||||
assert (
|
||||
await client.post("/internal/report", json=wrong, headers=worker_headers(w, g))
|
||||
).status_code == 422
|
||||
assert await c.fetchval("SELECT count(*) FROM case_results") == 0
|
||||
good = await client.post("/internal/report", json=report, headers=worker_headers(w, g))
|
||||
assert good.status_code == 204, good.text
|
||||
assert (
|
||||
await client.post("/internal/report", json=report, headers=worker_headers(w, g))
|
||||
).status_code == 204
|
||||
assert (
|
||||
await client.post(
|
||||
"/internal/report",
|
||||
json=synthetic_report(g, m, "wrong_answer"),
|
||||
headers=worker_headers(w, g),
|
||||
)
|
||||
).status_code == 409
|
||||
assert (
|
||||
await c.fetchval("SELECT state FROM evaluations WHERE id=$1", created.evaluation_id)
|
||||
== "finished"
|
||||
)
|
||||
assert await c.fetchval("SELECT count(*) FROM case_results") == 2
|
||||
assert (
|
||||
await client.post("/internal/claim", json=n, headers=worker_headers(w))
|
||||
).status_code == 204
|
||||
|
||||
queued = await create()
|
||||
await c.execute(
|
||||
"UPDATE outbox SET delivered_at=clock_timestamp()-interval '31 seconds' "
|
||||
"WHERE evaluation_id=$1",
|
||||
queued.evaluation_id,
|
||||
)
|
||||
for _ in range(60):
|
||||
if await c.fetchval(
|
||||
"SELECT delivered_at IS NULL FROM outbox WHERE evaluation_id=$1",
|
||||
queued.evaluation_id,
|
||||
):
|
||||
break
|
||||
await asyncio.sleep(0.05)
|
||||
else:
|
||||
raise AssertionError("missing notification not compensated")
|
||||
assert (
|
||||
await c.fetchval("SELECT state FROM evaluations WHERE id=$1", queued.evaluation_id)
|
||||
== "queued"
|
||||
)
|
||||
await claim(client, queued, w)
|
||||
await c.execute(f"ALTER ROLE {client.coordinator_role} NOLOGIN")
|
||||
assert (await client.get("/health/ready")).status_code == 503
|
||||
assert (await client.get("/health/live")).status_code == 200
|
||||
await c.execute(f"ALTER ROLE {client.coordinator_role} LOGIN")
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_actual_report_transaction_rollback(postgres_url, tmp_path, monkeypatch):
|
||||
async def run():
|
||||
async with fixture(postgres_url, tmp_path, monkeypatch) as (client, c, workers, _, create):
|
||||
created = await create()
|
||||
w = workers[0]
|
||||
g = await claim(client, created, w)
|
||||
await stages(client, w, g)
|
||||
m = await manifest(c, g)
|
||||
await c.execute("""CREATE FUNCTION test_case_reject() RETURNS trigger
|
||||
LANGUAGE plpgsql AS $$
|
||||
BEGIN IF NEW.position=1 THEN
|
||||
RAISE EXCEPTION 'synthetic rollback';
|
||||
END IF; RETURN NEW; END $$""")
|
||||
await c.execute(
|
||||
"CREATE TRIGGER test_case_reject BEFORE INSERT ON case_results "
|
||||
"FOR EACH ROW EXECUTE FUNCTION test_case_reject()"
|
||||
)
|
||||
r = await client.post(
|
||||
"/internal/report", json=synthetic_report(g, m), headers=worker_headers(w, g)
|
||||
)
|
||||
assert r.status_code == 503, r.text
|
||||
assert await c.fetchval("SELECT count(*) FROM case_results") == 0
|
||||
assert await c.fetchval("SELECT report FROM judge_attempts") is None
|
||||
assert await c.fetchval("SELECT state FROM evaluations") == "running"
|
||||
await c.execute("DROP TRIGGER test_case_reject ON case_results")
|
||||
await c.execute("DROP FUNCTION test_case_reject()")
|
||||
assert (
|
||||
await client.post(
|
||||
"/internal/report", json=synthetic_report(g, m), headers=worker_headers(w, g)
|
||||
)
|
||||
).status_code == 204
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_actual_expiry_stopping_quarantine_retry_and_old_attempt(
|
||||
postgres_url, tmp_path, monkeypatch
|
||||
):
|
||||
async def run():
|
||||
async with fixture(postgres_url, tmp_path, monkeypatch) as (client, c, workers, _, create):
|
||||
created = await create()
|
||||
other = await create()
|
||||
w = workers[0]
|
||||
g = await claim(client, created, w)
|
||||
# Expired server-side lease (not a client clock claim) causes actual recovery scan.
|
||||
await c.execute(
|
||||
"UPDATE judge_attempts SET expires_at=clock_timestamp()-interval '1 second'"
|
||||
)
|
||||
assert (
|
||||
await client.post(
|
||||
"/internal/renew", json=g["binding"], headers=worker_headers(w, g)
|
||||
)
|
||||
).status_code == 409
|
||||
for _ in range(40):
|
||||
pending = (
|
||||
await client.get("/internal/pending-stops", headers=worker_headers(w))
|
||||
).json()
|
||||
if pending:
|
||||
break
|
||||
await asyncio.sleep(0.05)
|
||||
assert pending[0]["binding"] == g["binding"]
|
||||
assert pending == [dict(binding=g["binding"], case_count=2)]
|
||||
stopped = dict(
|
||||
schema_version=1, kind="judge.stopped", binding=g["binding"], cleanup_complete=False
|
||||
)
|
||||
assert (
|
||||
await client.post("/internal/stopped", json=stopped, headers=worker_headers(w))
|
||||
).status_code == 204
|
||||
assert (
|
||||
await c.fetchval(
|
||||
"SELECT failure_code FROM evaluations WHERE id=$1", created.evaluation_id
|
||||
)
|
||||
== "cleanup_failed"
|
||||
)
|
||||
assert (
|
||||
await client.post(
|
||||
"/internal/claim", json=notice(other.evaluation_id), headers=worker_headers(w)
|
||||
)
|
||||
).status_code == 429
|
||||
stopped["cleanup_complete"] = True
|
||||
# Synthetic stop assertion only tests DB state. Actual Root proof is required in VM.
|
||||
assert (
|
||||
await client.post("/internal/stopped", json=stopped, headers=worker_headers(w))
|
||||
).status_code == 204
|
||||
assert (
|
||||
await client.post("/internal/stopped", json=stopped, headers=worker_headers(w))
|
||||
).status_code == 204
|
||||
assert (
|
||||
await c.fetchval("SELECT state FROM evaluations WHERE id=$1", created.evaluation_id)
|
||||
== "queued"
|
||||
)
|
||||
await c.execute(
|
||||
"UPDATE evaluations SET not_before=clock_timestamp() WHERE id=$1",
|
||||
created.evaluation_id,
|
||||
)
|
||||
fresh = await claim(client, created, w)
|
||||
assert fresh["attempt_number"] == 2 and fresh["binding"]["fencing_version"] == 2
|
||||
assert (
|
||||
await client.post(
|
||||
"/internal/stage",
|
||||
json=dict(
|
||||
schema_version=1,
|
||||
kind="judge.stage",
|
||||
binding=g["binding"],
|
||||
sequence=1,
|
||||
stage="compiling",
|
||||
case_index=None,
|
||||
),
|
||||
headers=worker_headers(w, g),
|
||||
)
|
||||
).status_code == 409
|
||||
assert (
|
||||
await client.post("/internal/stopped", json=stopped, headers=worker_headers(w))
|
||||
).status_code == 409
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_actual_user_cancel_all_stages_and_bounded_infrastructure_retry(
|
||||
postgres_url, tmp_path, monkeypatch
|
||||
):
|
||||
async def run():
|
||||
async with fixture(postgres_url, tmp_path, monkeypatch) as (
|
||||
client,
|
||||
c,
|
||||
workers,
|
||||
platform,
|
||||
create,
|
||||
):
|
||||
w = workers[0]
|
||||
for stage in ["queued", "compiling", "running"]:
|
||||
created = await create()
|
||||
g = None
|
||||
if stage != "queued":
|
||||
g = await claim(client, created, w)
|
||||
r = dict(
|
||||
schema_version=1,
|
||||
kind="judge.stage",
|
||||
binding=g["binding"],
|
||||
sequence=1,
|
||||
stage="compiling",
|
||||
case_index=None,
|
||||
)
|
||||
assert (
|
||||
await client.post("/internal/stage", json=r, headers=worker_headers(w, g))
|
||||
).status_code == 204
|
||||
if stage == "running":
|
||||
r |= dict(sequence=2, stage="running", case_index=0)
|
||||
assert (
|
||||
await client.post(
|
||||
"/internal/stage", json=r, headers=worker_headers(w, g)
|
||||
)
|
||||
).status_code == 204
|
||||
assert (
|
||||
await client.post(
|
||||
"/internal/cancel",
|
||||
json=notice(created.evaluation_id),
|
||||
headers={"Authorization": "Bearer " + platform},
|
||||
)
|
||||
).status_code == 204
|
||||
if g:
|
||||
assert (
|
||||
await c.fetchval(
|
||||
"SELECT state FROM evaluations WHERE id=$1", created.evaluation_id
|
||||
)
|
||||
== "cancelling"
|
||||
)
|
||||
assert (
|
||||
await client.post(
|
||||
"/internal/renew", json=g["binding"], headers=worker_headers(w, g)
|
||||
)
|
||||
).status_code == 409
|
||||
r = dict(
|
||||
schema_version=1,
|
||||
kind="judge.stopped",
|
||||
binding=g["binding"],
|
||||
cleanup_complete=True,
|
||||
)
|
||||
assert (
|
||||
await client.post("/internal/stopped", json=r, headers=worker_headers(w))
|
||||
).status_code == 204
|
||||
assert (
|
||||
await c.fetchval(
|
||||
"SELECT state FROM evaluations WHERE id=$1", created.evaluation_id
|
||||
)
|
||||
== "cancelled"
|
||||
)
|
||||
created = await create()
|
||||
for number in range(1, 4):
|
||||
g = await claim(client, created, w)
|
||||
m = await manifest(c, g)
|
||||
report = synthetic_report(g, m) | dict(
|
||||
compilation=dict(
|
||||
status="system_error",
|
||||
program=None,
|
||||
diagnostics="",
|
||||
diagnostics_truncated=False,
|
||||
cpu_time_us=None,
|
||||
memory_peak_bytes=None,
|
||||
),
|
||||
cases=[],
|
||||
verdict="system_error",
|
||||
failure_code="sandbox_unavailable",
|
||||
run_cpu_time_us=None,
|
||||
run_memory_peak_bytes=None,
|
||||
)
|
||||
assert (
|
||||
await client.post("/internal/report", json=report, headers=worker_headers(w, g))
|
||||
).status_code == 204
|
||||
state = await c.fetchval(
|
||||
"SELECT state FROM evaluations WHERE id=$1", created.evaluation_id
|
||||
)
|
||||
assert state == ("failed" if number == 3 else "queued")
|
||||
if state == "queued":
|
||||
await c.execute(
|
||||
"UPDATE evaluations SET not_before=clock_timestamp() WHERE id=$1",
|
||||
created.evaluation_id,
|
||||
)
|
||||
assert (
|
||||
await client.post(
|
||||
"/internal/claim", json=notice(created.evaluation_id), headers=worker_headers(w)
|
||||
)
|
||||
).status_code == 204
|
||||
assert (
|
||||
await c.fetchval(
|
||||
"SELECT count(*) FROM judge_attempts WHERE evaluation_id=$1",
|
||||
created.evaluation_id,
|
||||
)
|
||||
== 3
|
||||
)
|
||||
|
||||
asyncio.run(run())
|
||||
@@ -1,47 +0,0 @@
|
||||
import asyncio
|
||||
import json
|
||||
import secrets
|
||||
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||
from threading import Thread
|
||||
from uuid import uuid4
|
||||
|
||||
from kf_oj.infra.coordinator import CoordinatorControl
|
||||
|
||||
|
||||
def test_real_loopback_cancellation_notification_and_failure():
|
||||
token, evaluation = secrets.token_hex(32), uuid4()
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
code = 204
|
||||
seen = []
|
||||
|
||||
def do_POST(self):
|
||||
assert self.path == "/internal/cancel"
|
||||
assert self.headers["Authorization"] == "Bearer " + token
|
||||
body = json.loads(self.rfile.read(int(self.headers["Content-Length"])))
|
||||
assert body == {
|
||||
"schema_version": 1,
|
||||
"kind": "judge.evaluation",
|
||||
"evaluation_id": str(evaluation),
|
||||
}
|
||||
self.seen.append(body)
|
||||
self.send_response(self.code)
|
||||
self.end_headers()
|
||||
|
||||
def log_message(self, *_):
|
||||
pass
|
||||
|
||||
server = HTTPServer(("127.0.0.1", 0), Handler)
|
||||
thread = Thread(target=server.serve_forever, daemon=True)
|
||||
thread.start()
|
||||
control = CoordinatorControl(f"http://127.0.0.1:{server.server_port}/", token)
|
||||
try:
|
||||
assert asyncio.run(control.cancel(evaluation)) is True
|
||||
Handler.code = 503
|
||||
assert asyncio.run(control.cancel(evaluation)) is False
|
||||
assert len(Handler.seen) == 2
|
||||
finally:
|
||||
server.shutdown()
|
||||
server.server_close()
|
||||
thread.join(timeout=2)
|
||||
assert asyncio.run(control.cancel(evaluation)) is False
|
||||
@@ -1,262 +0,0 @@
|
||||
import asyncio
|
||||
import json
|
||||
from uuid import uuid4
|
||||
|
||||
import asyncpg
|
||||
import pytest
|
||||
from conftest import connect, migrate
|
||||
|
||||
|
||||
def test_migration_roundtrip_and_role_safety(postgres_url):
|
||||
async def inspect():
|
||||
c = await connect(postgres_url)
|
||||
try:
|
||||
tables = set(await c.fetch("SELECT tablename FROM pg_tables WHERE schemaname='public'"))
|
||||
assert len(tables) == 13 # Twelve business tables and Alembic history.
|
||||
roles = await c.fetch("SELECT * FROM pg_roles WHERE rolname LIKE 'kf_oj_%'")
|
||||
assert {row["rolname"] for row in roles} == {"kf_oj_platform", "kf_oj_coordinator"}
|
||||
assert all(
|
||||
not any(
|
||||
r[k]
|
||||
for k in (
|
||||
"rolsuper",
|
||||
"rolcanlogin",
|
||||
"rolcreatedb",
|
||||
"rolcreaterole",
|
||||
"rolbypassrls",
|
||||
)
|
||||
)
|
||||
for r in roles
|
||||
)
|
||||
await c.execute("SET ROLE kf_oj_platform")
|
||||
with pytest.raises(asyncpg.InsufficientPrivilegeError):
|
||||
await c.execute("CREATE TABLE forbidden_table (x int)")
|
||||
await c.execute("RESET ROLE")
|
||||
finally:
|
||||
await c.close()
|
||||
|
||||
asyncio.run(inspect())
|
||||
migrate(postgres_url, "downgrade", "base")
|
||||
migrate(postgres_url, "upgrade", "head")
|
||||
migrate(postgres_url, "check")
|
||||
asyncio.run(inspect())
|
||||
|
||||
|
||||
async def fixture_rows(c):
|
||||
account, problem, version, case, submission, evaluation = (uuid4() for _ in range(6))
|
||||
await c.execute(
|
||||
"INSERT INTO accounts (id,username,password_hash) VALUES ($1,'fixture_user',$2)",
|
||||
account,
|
||||
"synthetic-irreversible-placeholder-hash-" * 2,
|
||||
)
|
||||
await c.execute("INSERT INTO artifacts (digest,bytes) VALUES ($1,3),($2,0)", "a" * 64, "b" * 64)
|
||||
await c.execute("INSERT INTO problems (id,slug) VALUES ($1,'fixture-problem')", problem)
|
||||
await c.execute(
|
||||
"""INSERT INTO problem_versions
|
||||
(id,problem_id,revision,title,statement,languages,toolchain,compile_limits,run_limits,
|
||||
job_wall_time_us,public_samples)
|
||||
VALUES ($1,$2,1,'Fixture','Public statement','["c17"]','{}','{}','{}',10000,'[]')""",
|
||||
version,
|
||||
problem,
|
||||
)
|
||||
await c.execute(
|
||||
"""INSERT INTO problem_cases
|
||||
(id,version_id,position,input_digest,expected_digest) VALUES ($1,$2,0,$3,$4)""",
|
||||
case,
|
||||
version,
|
||||
"b" * 64,
|
||||
"a" * 64,
|
||||
)
|
||||
await c.execute("UPDATE problem_versions SET published_at=now() WHERE id=$1", version)
|
||||
await c.execute("UPDATE problems SET active_version_id=$1 WHERE id=$2", version, problem)
|
||||
await c.execute(
|
||||
"""INSERT INTO submissions
|
||||
(id,account_id,problem_id,problem_version_id,language,source_digest,idempotency_key,
|
||||
request_digest) VALUES ($1,$2,$3,$4,'c17',$5,$6,$7)""",
|
||||
submission,
|
||||
account,
|
||||
problem,
|
||||
version,
|
||||
"a" * 64,
|
||||
uuid4(),
|
||||
"c" * 64,
|
||||
)
|
||||
await c.execute(
|
||||
"INSERT INTO evaluations (id,submission_id,problem_version_id) VALUES ($1,$2,$3)",
|
||||
evaluation,
|
||||
submission,
|
||||
version,
|
||||
)
|
||||
await c.execute("INSERT INTO outbox (id,evaluation_id) VALUES ($1,$2)", uuid4(), evaluation)
|
||||
return account, problem, version, case, submission, evaluation
|
||||
|
||||
|
||||
def test_database_write_authorities_and_immutable_versions(postgres_url):
|
||||
async def run():
|
||||
c = await connect(postgres_url)
|
||||
try:
|
||||
await c.execute("SET ROLE kf_oj_platform")
|
||||
ids = await fixture_rows(c)
|
||||
assert await c.fetchval("SELECT state FROM evaluations WHERE id=$1", ids[5]) == "queued"
|
||||
for command in (
|
||||
"UPDATE evaluations SET state='finished'",
|
||||
"INSERT INTO evaluations (id,submission_id,problem_version_id,state) "
|
||||
"SELECT gen_random_uuid(),submission_id,problem_version_id,'finished' "
|
||||
"FROM evaluations",
|
||||
"DELETE FROM evaluations",
|
||||
"UPDATE problem_versions SET title='altered'",
|
||||
"INSERT INTO judge_attempts (id) VALUES (gen_random_uuid())",
|
||||
"UPDATE artifacts SET bytes=42",
|
||||
):
|
||||
with pytest.raises(asyncpg.InsufficientPrivilegeError):
|
||||
await c.execute(command)
|
||||
with pytest.raises(asyncpg.CheckViolationError):
|
||||
await c.execute("UPDATE problem_versions SET published_at=now()")
|
||||
with pytest.raises(asyncpg.CheckViolationError):
|
||||
await c.execute(
|
||||
"""INSERT INTO problem_cases
|
||||
(id,version_id,position,input_digest,expected_digest)
|
||||
VALUES ($1,$2,1,$3,$3)""",
|
||||
uuid4(),
|
||||
ids[2],
|
||||
"a" * 64,
|
||||
)
|
||||
with pytest.raises(asyncpg.ForeignKeyViolationError):
|
||||
await c.execute("UPDATE artifacts SET state='deleting' WHERE digest=$1", "a" * 64)
|
||||
await c.execute("RESET ROLE")
|
||||
await c.execute("SET ROLE kf_oj_coordinator")
|
||||
for command in (
|
||||
"UPDATE accounts SET enabled=false",
|
||||
"INSERT INTO problems (id,slug) VALUES (gen_random_uuid(),'forbidden')",
|
||||
"UPDATE submissions SET language='cpp20'",
|
||||
):
|
||||
with pytest.raises(asyncpg.InsufficientPrivilegeError):
|
||||
await c.execute(command)
|
||||
attempt = uuid4()
|
||||
async with c.transaction():
|
||||
await c.execute(
|
||||
"""INSERT INTO judge_attempts
|
||||
(id,evaluation_id,worker_id,number,fencing_version,lease_token_hash,
|
||||
expires_at,manifest,manifest_digest)
|
||||
VALUES ($1,$2,$3,1,1,$4,now()+interval '30 seconds',$5,$6)""",
|
||||
attempt,
|
||||
ids[5],
|
||||
uuid4(),
|
||||
"d" * 64,
|
||||
json.dumps({"fixture": True}),
|
||||
"e" * 64,
|
||||
)
|
||||
await c.execute(
|
||||
"""UPDATE evaluations SET current_attempt_id=$1,state='claimed',
|
||||
attempts_count=1,fencing_version=1 WHERE id=$2""",
|
||||
attempt,
|
||||
ids[5],
|
||||
)
|
||||
assert await c.fetchval("SELECT state FROM evaluations") == "claimed"
|
||||
with pytest.raises(asyncpg.InsufficientPrivilegeError):
|
||||
await c.execute("UPDATE judge_attempts SET manifest='{}'")
|
||||
# Active-attempt FK is deferred but still rejects an unrelated/absent attempt at commit.
|
||||
with pytest.raises(asyncpg.ForeignKeyViolationError):
|
||||
async with c.transaction():
|
||||
await c.execute("UPDATE evaluations SET current_attempt_id=$1", uuid4())
|
||||
finally:
|
||||
await c.close()
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_initial_creation_transaction_rolls_back_and_cross_version_fk_rejects(postgres_url):
|
||||
async def run():
|
||||
c = await connect(postgres_url)
|
||||
try:
|
||||
with pytest.raises(RuntimeError):
|
||||
async with c.transaction():
|
||||
await c.execute("SET LOCAL ROLE kf_oj_platform")
|
||||
await fixture_rows(c)
|
||||
raise RuntimeError("simulated transaction failure")
|
||||
assert await c.fetchval("SELECT count(*) FROM submissions") == 0
|
||||
assert await c.fetchval("SELECT count(*) FROM outbox") == 0
|
||||
ids = await fixture_rows(c)
|
||||
with pytest.raises(asyncpg.ForeignKeyViolationError):
|
||||
await c.execute(
|
||||
"UPDATE evaluations SET problem_version_id=$1 WHERE id=$2",
|
||||
uuid4(),
|
||||
ids[5],
|
||||
)
|
||||
finally:
|
||||
await c.close()
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_publication_serializes_concurrent_case_insertion(postgres_url):
|
||||
async def run():
|
||||
c, other = await connect(postgres_url), await connect(postgres_url)
|
||||
pending = None
|
||||
transaction = None
|
||||
try:
|
||||
ids = await fixture_rows(c)
|
||||
# A new draft version, separate from the immutable already published fixture.
|
||||
draft = uuid4()
|
||||
await c.execute(
|
||||
"""INSERT INTO problem_versions
|
||||
(id,problem_id,revision,title,statement,languages,toolchain,compile_limits,
|
||||
run_limits,job_wall_time_us,public_samples)
|
||||
SELECT $1,problem_id,2,title,statement,languages,toolchain,compile_limits,
|
||||
run_limits,job_wall_time_us,public_samples FROM problem_versions WHERE id=$2
|
||||
""",
|
||||
draft,
|
||||
ids[2],
|
||||
)
|
||||
await c.execute(
|
||||
"""INSERT INTO problem_cases
|
||||
(id,version_id,position,input_digest,expected_digest)
|
||||
VALUES ($1,$2,0,$3,$3)""",
|
||||
uuid4(),
|
||||
draft,
|
||||
"a" * 64,
|
||||
)
|
||||
transaction = c.transaction()
|
||||
await transaction.start()
|
||||
await c.execute("UPDATE problem_versions SET published_at=now() WHERE id=$1", draft)
|
||||
pending = asyncio.create_task(
|
||||
other.execute(
|
||||
"""INSERT INTO problem_cases
|
||||
(id,version_id,position,input_digest,expected_digest)
|
||||
VALUES ($1,$2,1,$3,$3)""",
|
||||
uuid4(),
|
||||
draft,
|
||||
"a" * 64,
|
||||
)
|
||||
)
|
||||
await asyncio.sleep(0.1)
|
||||
assert not pending.done() # Actual row lock, not a simulated repository response.
|
||||
await transaction.commit()
|
||||
transaction = None
|
||||
with pytest.raises(asyncpg.CheckViolationError):
|
||||
await asyncio.wait_for(pending, timeout=5)
|
||||
assert (
|
||||
await c.fetchval("SELECT count(*) FROM problem_cases WHERE version_id=$1", draft)
|
||||
== 1
|
||||
)
|
||||
with pytest.raises(asyncpg.CheckViolationError):
|
||||
await c.execute(
|
||||
"""INSERT INTO problem_versions
|
||||
(id,problem_id,revision,title,statement,languages,toolchain,compile_limits,
|
||||
run_limits,job_wall_time_us,public_samples,published_at)
|
||||
SELECT $1,problem_id,3,title,statement,languages,toolchain,compile_limits,
|
||||
run_limits,job_wall_time_us,public_samples,now()
|
||||
FROM problem_versions WHERE id=$2""",
|
||||
uuid4(),
|
||||
draft,
|
||||
)
|
||||
finally:
|
||||
if transaction:
|
||||
await transaction.rollback()
|
||||
if pending and not pending.done():
|
||||
pending.cancel()
|
||||
await asyncio.gather(pending, return_exceptions=True)
|
||||
await c.close()
|
||||
await other.close()
|
||||
|
||||
asyncio.run(run())
|
||||
@@ -1,101 +0,0 @@
|
||||
"""Offline package data only; no compiler, submitted source or image execution."""
|
||||
|
||||
import importlib.util
|
||||
import json
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
SPEC = importlib.util.spec_from_file_location(
|
||||
"gnu_image_builder", Path(__file__).resolve().parents[2] / "scripts/build_gnu_image.py"
|
||||
)
|
||||
builder = importlib.util.module_from_spec(SPEC)
|
||||
SPEC.loader.exec_module(builder)
|
||||
|
||||
|
||||
def test_absolute_package_link_is_relocated_inside_verified_image(tmp_path):
|
||||
root = tmp_path / "image"
|
||||
target = root / "usr/lib/plugin.so"
|
||||
target.parent.mkdir(parents=True)
|
||||
target.write_bytes(b"synthetic pinned package bytes")
|
||||
link = root / "usr/lib/plugin-link.so"
|
||||
link.symlink_to("/usr/lib/plugin.so")
|
||||
records = {
|
||||
"usr/lib/plugin-link.so": {"kind": "symlink", "target": "/usr/lib/plugin.so"},
|
||||
"usr/lib/plugin.so": {"kind": "file", "sha256": builder.sha(target)},
|
||||
}
|
||||
source, record = builder.resolve_verified(root, records, "usr/lib/plugin-link.so")
|
||||
assert source == target
|
||||
assert source.read_bytes() == b"synthetic pinned package bytes"
|
||||
assert record == records["usr/lib/plugin.so"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("target", ["../../../../outside", "plugin-link.so"])
|
||||
def test_package_link_escape_or_cycle_refused(tmp_path, target):
|
||||
link = tmp_path / "usr/lib/plugin-link.so"
|
||||
link.parent.mkdir(parents=True)
|
||||
link.symlink_to(target)
|
||||
records = {"usr/lib/plugin-link.so": {"kind": "symlink", "target": target}}
|
||||
with pytest.raises(ValueError):
|
||||
builder.resolve_verified(tmp_path, records, "usr/lib/plugin-link.so")
|
||||
|
||||
|
||||
def test_replaced_or_unverified_ancestor_link_is_refused(tmp_path):
|
||||
library = tmp_path / "usr/lib"
|
||||
library.parent.mkdir()
|
||||
library.symlink_to("/usr/lib")
|
||||
records = {"usr/lib/plugin.so": {"kind": "file", "sha256": "a" * 64}}
|
||||
with pytest.raises(ValueError, match="unverified GNU ancestor"):
|
||||
builder.resolve_verified(tmp_path, records, "usr/lib/plugin.so")
|
||||
records["usr/lib"] = {"kind": "symlink", "target": "/different/path"}
|
||||
with pytest.raises(ValueError, match="differs from verified manifest"):
|
||||
builder.resolve_verified(tmp_path, records, "usr/lib/plugin.so")
|
||||
|
||||
|
||||
def test_verified_manifest_does_not_authorize_changed_package_bytes(tmp_path):
|
||||
root = tmp_path / "support"
|
||||
library = root / "usr/lib/ld-linux-x86-64.so.2"
|
||||
library.parent.mkdir(parents=True)
|
||||
library.write_bytes(b"original synthetic library")
|
||||
manifest = {
|
||||
"packages": [
|
||||
{
|
||||
"name": "gcc",
|
||||
"version": "16.2.1+r23+gd564253eb6c8-1",
|
||||
"sha256": "ebac12ceef494c616a36fc6f0f792b365dc59b6ed46ca1a69f138aecc810b6ab",
|
||||
},
|
||||
{
|
||||
"name": "glibc",
|
||||
"version": "2.44+r50+g1848099f063e-3",
|
||||
"sha256": "071d807f4d3bf6b921f86b9a91e83a47ecc026c5821634efa46b09d890e06785",
|
||||
},
|
||||
{
|
||||
"name": "binutils",
|
||||
"version": "2.47-2",
|
||||
"sha256": "62efd213e87590f025cf861b646793d3cacb03e7e26187b73d33b00adaf7a977",
|
||||
},
|
||||
],
|
||||
"files": [
|
||||
{
|
||||
"path": "usr/lib/ld-linux-x86-64.so.2",
|
||||
"kind": "file",
|
||||
"bytes": library.stat().st_size,
|
||||
"sha256": builder.sha(library),
|
||||
}
|
||||
],
|
||||
}
|
||||
manifest_path = tmp_path / "manifest.json"
|
||||
manifest_path.write_text(json.dumps(manifest))
|
||||
library.write_bytes(b"changed synthetic library")
|
||||
output = tmp_path / "output"
|
||||
args = SimpleNamespace(
|
||||
support_dir=root,
|
||||
support_manifest=manifest_path,
|
||||
support_manifest_sha256=builder.sha(manifest_path),
|
||||
output=output,
|
||||
)
|
||||
with pytest.raises(ValueError, match="differs from verified public package bytes"):
|
||||
builder.build(args)
|
||||
assert not (output / "manifest.json").exists()
|
||||
assert not list(output.rglob("*.so*"))
|
||||
@@ -21,9 +21,7 @@ def test_liveness_without_database_or_queue() -> None:
|
||||
|
||||
|
||||
def test_production_disables_api_documentation() -> None:
|
||||
app = create_app(
|
||||
Settings(_env_file=None, environment="production", browser_origins=("https://oj.example",))
|
||||
)
|
||||
app = create_app(Settings(_env_file=None, environment="production"))
|
||||
|
||||
async def request():
|
||||
async with httpx.AsyncClient(
|
||||
|
||||
@@ -1,284 +0,0 @@
|
||||
import asyncio
|
||||
import os
|
||||
import pty
|
||||
import secrets
|
||||
import select as io_select
|
||||
import signal
|
||||
import sys
|
||||
import time
|
||||
from datetime import timedelta
|
||||
from uuid import uuid4
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from sqlalchemy import event, func, select, update
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
|
||||
|
||||
from kf_oj.api.app import create_app
|
||||
from kf_oj.contracts.identity import LoginRequest
|
||||
from kf_oj.core.settings import Settings
|
||||
from kf_oj.modules.identity.models import Account, Session
|
||||
from kf_oj.modules.identity.passwords import hash_password, verify_password
|
||||
from kf_oj.modules.identity.public import AuthenticationFailed, LoginThrottled
|
||||
from kf_oj.modules.identity.service import IdentityService, token_digest
|
||||
|
||||
|
||||
def restrict_platform_role(engine):
|
||||
@event.listens_for(engine.sync_engine, "connect")
|
||||
def apply_role(connection, _):
|
||||
cursor = connection.cursor()
|
||||
cursor.execute("SET ROLE kf_oj_platform")
|
||||
cursor.close()
|
||||
|
||||
|
||||
def test_password_policy_and_credentials_redacted():
|
||||
password = secrets.token_hex(16)
|
||||
hashed = hash_password("fixture_user", password)
|
||||
assert hashed.startswith("$argon2id$v=19$m=19456,t=2,p=1$")
|
||||
assert verify_password(hashed, password)
|
||||
assert not verify_password(hashed, secrets.token_hex(16))
|
||||
assert not verify_password("$argon2id$v=19$m=999999999,t=2,p=1$invalid", password)
|
||||
assert password not in repr(LoginRequest(username="fixture_user", password=password))
|
||||
with pytest.raises(ValueError):
|
||||
hash_password("not canonical", password)
|
||||
with pytest.raises(ValueError):
|
||||
hash_password("fixture_user", "too short")
|
||||
|
||||
|
||||
def test_production_cookie_configuration_requires_explicit_https_origins():
|
||||
with pytest.raises(ValueError):
|
||||
Settings(_env_file=None, environment="production")
|
||||
for invalid in ("https://oj.example/path", "https://user@oj.example", "*", "null"):
|
||||
with pytest.raises(ValueError):
|
||||
Settings(_env_file=None, browser_origins=(invalid,))
|
||||
|
||||
|
||||
def test_real_login_session_revocation_expiry_and_throttle(postgres_url):
|
||||
async def run():
|
||||
engine = create_async_engine(postgres_url)
|
||||
restrict_platform_role(engine)
|
||||
sessions = async_sessionmaker(engine, expire_on_commit=False)
|
||||
identity = IdentityService(sessions, lifetime_seconds=300)
|
||||
password = secrets.token_hex(16)
|
||||
try:
|
||||
principal = await identity.create_account("fixture_user", password)
|
||||
with pytest.raises(AuthenticationFailed):
|
||||
await identity.login("fixture_user", secrets.token_hex(16), peer="fixture-peer")
|
||||
with pytest.raises(AuthenticationFailed):
|
||||
await identity.login("absent_user", password, peer="fixture-peer")
|
||||
issued = await identity.login("fixture_user", password, peer="fixture-peer")
|
||||
assert await identity.authenticate(issued.token) == principal
|
||||
assert issued.token not in repr(issued) and issued.csrf_token not in repr(issued)
|
||||
with pytest.raises(AuthenticationFailed):
|
||||
await identity.authenticate(str(principal.account_id))
|
||||
with pytest.raises(AuthenticationFailed):
|
||||
await identity.authenticate(issued.token, csrf_token="f" * 64)
|
||||
await identity.logout(issued.token, issued.csrf_token)
|
||||
with pytest.raises(AuthenticationFailed):
|
||||
await identity.authenticate(issued.token)
|
||||
issued = await identity.login("fixture_user", password, peer="expiry-peer")
|
||||
async with sessions.begin() as session:
|
||||
# Keep the expiry constraint valid while moving the entire session into the past.
|
||||
await session.execute(
|
||||
update(Session).values(
|
||||
created_at=func.now() - timedelta(minutes=10),
|
||||
expires_at=func.now() - timedelta(minutes=5),
|
||||
)
|
||||
)
|
||||
with pytest.raises(AuthenticationFailed):
|
||||
await identity.authenticate(issued.token)
|
||||
issued = await identity.login("fixture_user", password, peer="disable-peer")
|
||||
assert await identity.disable_account("fixture_user")
|
||||
with pytest.raises(AuthenticationFailed):
|
||||
await identity.authenticate(issued.token)
|
||||
with pytest.raises(AuthenticationFailed):
|
||||
await identity.login("fixture_user", password, peer="disable-peer")
|
||||
assert not await identity.disable_account("unknown_user")
|
||||
# Exactly ten real failed attempts consume the same persistent peer budget.
|
||||
for _ in range(10):
|
||||
with pytest.raises(AuthenticationFailed):
|
||||
await identity.login("absent_user", password, peer="limited-peer")
|
||||
with pytest.raises(LoginThrottled):
|
||||
await identity.login("absent_user", password, peer="limited-peer")
|
||||
async with sessions() as session:
|
||||
account = await session.scalar(select(Account))
|
||||
assert password != account.password_hash and password not in account.password_hash
|
||||
rows = list(await session.scalars(select(Session)))
|
||||
assert all(row.token_hash != issued.token for row in rows)
|
||||
finally:
|
||||
await engine.dispose()
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_real_http_cookie_csrf_and_identity_spoofing_rejection(postgres_url, caplog):
|
||||
async def run():
|
||||
app = create_app(
|
||||
Settings(
|
||||
_env_file=None,
|
||||
environment="production",
|
||||
database_url=postgres_url,
|
||||
browser_origins=("https://test",),
|
||||
)
|
||||
)
|
||||
async with app.router.lifespan_context(app):
|
||||
restrict_platform_role(app.state.sessions.kw["bind"])
|
||||
password = secrets.token_hex(16)
|
||||
first = await app.state.identity.create_account("first_user", password)
|
||||
second = await app.state.identity.create_account("second_user", secrets.token_hex(16))
|
||||
headers = {"Origin": "https://test"}
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="https://test"
|
||||
) as client:
|
||||
body = {"username": "first_user", "password": password}
|
||||
assert (await client.get("/api/v1/auth/me")).status_code == 401
|
||||
assert (await client.post("/api/v1/auth/login", json=body)).status_code == 403
|
||||
assert (
|
||||
await client.post(
|
||||
"/api/v1/auth/login",
|
||||
json=body,
|
||||
headers={"Origin": "https://hostile.example"},
|
||||
)
|
||||
).status_code == 403
|
||||
invalid = await client.post(
|
||||
"/api/v1/auth/login",
|
||||
headers=headers,
|
||||
json=body | {"user_id": str(second.account_id), "role": "admin"},
|
||||
)
|
||||
assert invalid.status_code == 422
|
||||
assert password not in invalid.text and "admin" not in invalid.text
|
||||
response = await client.post("/api/v1/auth/login", json=body, headers=headers)
|
||||
assert response.status_code == 200
|
||||
assert response.json()["user"]["id"] == str(first.account_id)
|
||||
assert (
|
||||
await client.get(
|
||||
"/api/v1/auth/me", headers={"X-User-ID": str(second.account_id)}
|
||||
)
|
||||
).json()["id"] == str(first.account_id)
|
||||
cookie = response.headers.get_list("set-cookie")[0]
|
||||
assert "HttpOnly" in cookie and "Secure" in cookie and "SameSite=strict" in cookie
|
||||
assert response.headers["cache-control"] == "no-store"
|
||||
assert client.cookies.get("kf_csrf") == response.json()["csrf_token"]
|
||||
assert (
|
||||
await client.post("/api/v1/auth/logout", headers=headers)
|
||||
).status_code == 403
|
||||
assert (
|
||||
await client.post(
|
||||
"/api/v1/auth/logout", headers=headers | {"X-CSRF-Token": "f" * 64}
|
||||
)
|
||||
).status_code == 403
|
||||
assert (
|
||||
await client.post(
|
||||
"/api/v1/auth/logout",
|
||||
headers=headers | {"X-CSRF-Token": response.json()["csrf_token"]},
|
||||
)
|
||||
).status_code == 204
|
||||
assert (await client.get("/api/v1/auth/me")).status_code == 401
|
||||
assert not client.cookies.get("kf_session") and not client.cookies.get("kf_csrf")
|
||||
assert (
|
||||
password not in caplog.text and response.json()["csrf_token"] not in caplog.text
|
||||
)
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_real_concurrent_logins_keep_bounded_active_sessions(postgres_url):
|
||||
async def run():
|
||||
engine = create_async_engine(postgres_url)
|
||||
restrict_platform_role(engine)
|
||||
sessions = async_sessionmaker(engine, expire_on_commit=False)
|
||||
identity = IdentityService(sessions)
|
||||
password = secrets.token_hex(16)
|
||||
try:
|
||||
await identity.create_account("fixture_user", password)
|
||||
# Independent peer budgets exercise session bounds without exhausting login retries.
|
||||
issued = []
|
||||
for _ in range(12):
|
||||
issued.append(await identity.login("fixture_user", password, peer=uuid4().hex))
|
||||
with pytest.raises(AuthenticationFailed):
|
||||
await identity.authenticate(issued[0].token)
|
||||
results = await asyncio.gather(
|
||||
*(identity.login("fixture_user", password, peer=uuid4().hex) for _ in range(3)),
|
||||
return_exceptions=True,
|
||||
)
|
||||
assert sum(isinstance(result, LoginThrottled) for result in results) == 1
|
||||
assert sum(not isinstance(result, Exception) for result in results) == 2
|
||||
async with sessions() as session:
|
||||
assert await session.scalar(select(func.count()).select_from(Session)) == 10
|
||||
assert (
|
||||
await session.scalar(
|
||||
select(func.count())
|
||||
.select_from(Session)
|
||||
.where(Session.token_hash == token_digest("0" * 64))
|
||||
)
|
||||
== 0
|
||||
)
|
||||
finally:
|
||||
await engine.dispose()
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_account_cli_hidden_password_and_disable(postgres_url):
|
||||
password = secrets.token_hex(16)
|
||||
# The CLI prompts through a real private PTY; the password is never an argument/environment.
|
||||
pid, descriptor = pty.fork()
|
||||
if pid == 0:
|
||||
os.environ["KF_OJ_DATABASE_URL"] = postgres_url
|
||||
os.execv(
|
||||
sys.executable,
|
||||
[sys.executable, "-m", "kf_oj.modules.identity.cli", "create", "cli_user"],
|
||||
)
|
||||
output = b""
|
||||
deadline, prompts, status = time.monotonic() + 15, 0, None
|
||||
try:
|
||||
while time.monotonic() < deadline:
|
||||
if io_select.select([descriptor], [], [], 0.1)[0]:
|
||||
try:
|
||||
block = os.read(descriptor, 4096)
|
||||
except OSError:
|
||||
block = b""
|
||||
output += block
|
||||
if prompts == 0 and b"Password (12" in output:
|
||||
os.write(descriptor, password.encode() + b"\n")
|
||||
prompts = 1
|
||||
if prompts == 1 and b"Repeat password:" in output:
|
||||
os.write(descriptor, password.encode() + b"\n")
|
||||
prompts = 2
|
||||
result, status = os.waitpid(pid, os.WNOHANG)
|
||||
if result:
|
||||
break
|
||||
else:
|
||||
os.kill(pid, signal.SIGTERM)
|
||||
os.waitpid(pid, 0)
|
||||
pytest.fail("account CLI deadline exceeded")
|
||||
assert os.waitstatus_to_exitcode(status) == 0
|
||||
assert prompts == 2 and b"Account created." in output
|
||||
assert password.encode() not in output
|
||||
finally:
|
||||
os.close(descriptor)
|
||||
|
||||
async def inspect():
|
||||
engine = create_async_engine(postgres_url)
|
||||
sessions = async_sessionmaker(engine, expire_on_commit=False)
|
||||
try:
|
||||
service = IdentityService(sessions)
|
||||
issued = await service.login("cli_user", password, peer="cli-fixture")
|
||||
from kf_oj.modules.identity.cli import run
|
||||
|
||||
await run("disable", "cli_user")
|
||||
with pytest.raises(AuthenticationFailed):
|
||||
await service.authenticate(issued.token)
|
||||
finally:
|
||||
await engine.dispose()
|
||||
|
||||
# CLI run uses the test DB only for this call; don't reuse a configured business database.
|
||||
original = os.environ.get("KF_OJ_DATABASE_URL")
|
||||
os.environ["KF_OJ_DATABASE_URL"] = postgres_url
|
||||
try:
|
||||
asyncio.run(inspect())
|
||||
finally:
|
||||
if original is None:
|
||||
os.environ.pop("KF_OJ_DATABASE_URL", None)
|
||||
else:
|
||||
os.environ["KF_OJ_DATABASE_URL"] = original
|
||||
@@ -1,143 +0,0 @@
|
||||
import json
|
||||
from pathlib import Path
|
||||
from uuid import UUID
|
||||
|
||||
import pytest
|
||||
from jsonschema import Draft202012Validator, FormatChecker
|
||||
from pydantic import ValidationError
|
||||
|
||||
from kf_oj.contracts.judge_state import (
|
||||
EvaluationState,
|
||||
require_current_lease,
|
||||
require_transition,
|
||||
)
|
||||
from kf_oj.contracts.judging import (
|
||||
MAX_WIRE_BYTES,
|
||||
WIRE_MODELS,
|
||||
AdmissionPolicy,
|
||||
JobManifest,
|
||||
JudgementReport,
|
||||
LeaseGrant,
|
||||
decode_wire,
|
||||
)
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
FIXTURES = ROOT / "tests/fixtures/contracts"
|
||||
PATHS = sorted(p for name in WIRE_MODELS for p in FIXTURES.glob(f"{name}.*.json"))
|
||||
|
||||
|
||||
@pytest.mark.parametrize("path", PATHS, ids=lambda p: p.stem)
|
||||
def test_shared_judging_examples(path: Path) -> None:
|
||||
name = path.name.split(".")[0]
|
||||
model = WIRE_MODELS[name]
|
||||
payload = json.loads(path.read_text())
|
||||
schema = json.loads((ROOT / f"contracts/schemas/{name}.v1.schema.json").read_text())
|
||||
validator = Draft202012Validator(schema, format_checker=FormatChecker())
|
||||
if ".invalid-" in path.name:
|
||||
with pytest.raises(ValueError):
|
||||
decode_wire(model, path.read_bytes())
|
||||
if ".invalid-semantic-" in path.name:
|
||||
assert validator.is_valid(payload)
|
||||
else:
|
||||
assert not validator.is_valid(payload)
|
||||
else:
|
||||
decoded = decode_wire(model, path.read_bytes())
|
||||
validator.validate(payload)
|
||||
if name == "judge-admission" and "defaults" in path.name:
|
||||
assert decoded.max_concurrent_jobs == decoded.prefetch == 1
|
||||
assert decoded.max_retries == 2
|
||||
else:
|
||||
assert decoded.model_dump(mode="json") == payload
|
||||
|
||||
|
||||
def fixture(name: str, model):
|
||||
return decode_wire(model, (FIXTURES / f"{name}.valid.json").read_bytes())
|
||||
|
||||
|
||||
def test_every_state_transition_matches_shared_table() -> None:
|
||||
transitions = json.loads((FIXTURES / "judge-state.transitions.json").read_text())
|
||||
assert set(transitions) == {s.value for s in EvaluationState}
|
||||
for current in EvaluationState:
|
||||
for target in EvaluationState:
|
||||
allowed = target.value in transitions[current.value]
|
||||
assert current.allows(target) == allowed
|
||||
if allowed:
|
||||
require_transition(current, target)
|
||||
else:
|
||||
with pytest.raises(ValueError):
|
||||
require_transition(current, target)
|
||||
|
||||
|
||||
def test_lease_fencing_identity_and_expiry_boundary() -> None:
|
||||
grant = fixture("judge-lease", LeaseGrant)
|
||||
binding = grant.binding
|
||||
require_current_lease(binding, binding, grant.expires_at_unix_ms, grant.expires_at_unix_ms - 1)
|
||||
for now in (grant.expires_at_unix_ms, grant.expires_at_unix_ms + 1, -1):
|
||||
with pytest.raises(ValueError):
|
||||
require_current_lease(binding, binding, grant.expires_at_unix_ms, now)
|
||||
for field, value in (
|
||||
("fencing_version", 2),
|
||||
("attempt_id", UUID("aaaaaaaa-aaaa-4aaa-8aaa-000000000009")),
|
||||
("worker_id", UUID("aaaaaaaa-aaaa-4aaa-8aaa-000000000009")),
|
||||
("evaluation_id", UUID("aaaaaaaa-aaaa-4aaa-8aaa-000000000009")),
|
||||
):
|
||||
supplied = binding.model_copy(update={field: value})
|
||||
with pytest.raises(ValueError):
|
||||
require_current_lease(binding, supplied, grant.expires_at_unix_ms, 1)
|
||||
assert grant.lease_token not in repr(grant)
|
||||
|
||||
|
||||
def test_node_admission_never_accepts_job_or_retry_above_trusted_policy() -> None:
|
||||
manifest = fixture("judge-manifest", JobManifest)
|
||||
policy = fixture("judge-admission", AdmissionPolicy)
|
||||
for attempt in (1, 2, 3):
|
||||
policy.admit(manifest, attempt)
|
||||
for attempt in (0, 4, True):
|
||||
with pytest.raises(ValueError):
|
||||
policy.admit(manifest, attempt)
|
||||
for field, value in (("max_package_bytes", 1), ("max_job_wall_time_us", 1000)):
|
||||
with pytest.raises(ValueError):
|
||||
policy.model_copy(update={field: value}).admit(manifest, 1)
|
||||
lower = policy.run_ceiling.model_copy(update={"memory_bytes": 32 * 1024 * 1024})
|
||||
with pytest.raises(ValueError):
|
||||
policy.model_copy(update={"run_ceiling": lower}).admit(manifest, 1)
|
||||
other = manifest.model_copy(
|
||||
update={"toolchain": manifest.toolchain.model_copy(update={"image_revision": 2})}
|
||||
)
|
||||
with pytest.raises(ValueError):
|
||||
policy.admit(other, 1)
|
||||
|
||||
|
||||
def test_result_requires_authorized_manifest_and_complete_case_order() -> None:
|
||||
manifest = fixture("judge-manifest", JobManifest)
|
||||
result = fixture("judge-result", JudgementReport)
|
||||
result.validate_for_manifest(manifest, result.manifest_digest)
|
||||
with pytest.raises(ValueError):
|
||||
result.validate_for_manifest(manifest, "f" * 64)
|
||||
short = result.model_dump(mode="json")
|
||||
short.update(cases=short["cases"][:1], run_cpu_time_us=100, run_memory_peak_bytes=256 * 1024)
|
||||
valid_partial = JudgementReport.model_validate(short)
|
||||
with pytest.raises(ValueError):
|
||||
valid_partial.validate_for_manifest(manifest, result.manifest_digest)
|
||||
wrong_order = result.model_dump(mode="json")
|
||||
wrong_order["cases"].reverse()
|
||||
with pytest.raises(ValueError):
|
||||
JudgementReport.model_validate(wrong_order).validate_for_manifest(
|
||||
manifest, result.manifest_digest
|
||||
)
|
||||
|
||||
|
||||
def test_wire_decoding_bounds_ambiguity_and_private_error_output() -> None:
|
||||
payload = (FIXTURES / "judge-lease.valid.json").read_bytes()
|
||||
duplicate = payload.replace(b'"schema_version": 1', b'"schema_version": 1, "schema_version": 1')
|
||||
for invalid in (duplicate, payload.decode().encode("utf-16"), b" " * (MAX_WIRE_BYTES + 1)):
|
||||
with pytest.raises(ValueError):
|
||||
decode_wire(LeaseGrant, invalid)
|
||||
secret = "do-not-echo-private-token"
|
||||
with pytest.raises(ValueError) as error:
|
||||
decode_wire(
|
||||
LeaseGrant, payload.replace(b'"0' + b"0" * 63 + b'"', json.dumps(secret).encode())
|
||||
)
|
||||
assert secret not in str(error.value)
|
||||
with pytest.raises(ValidationError):
|
||||
LeaseGrant.model_validate({})
|
||||
@@ -1,64 +0,0 @@
|
||||
"""Image construction data tests; actual publication/execution needs the independent VM."""
|
||||
|
||||
import importlib.util
|
||||
import io
|
||||
import json
|
||||
import tarfile
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
MODULE = Path(__file__).resolve().parents[2] / "scripts/build_judging_images.py"
|
||||
SPEC = importlib.util.spec_from_file_location("judging_image_builder", MODULE)
|
||||
builder = importlib.util.module_from_spec(SPEC)
|
||||
SPEC.loader.exec_module(builder)
|
||||
|
||||
|
||||
def test_catalog_uses_wire_path_order_including_dot_and_directory_boundaries(tmp_path):
|
||||
image = tmp_path / "image"
|
||||
image.mkdir()
|
||||
builder.put(image, "usr/lib/module/a.py", b"data")
|
||||
builder.put(image, "usr/lib/module.py", b"module")
|
||||
support = tmp_path / "support"
|
||||
library = support / "usr/lib/ld-linux-x86-64.so.2"
|
||||
library.parent.mkdir(parents=True)
|
||||
library.write_bytes(b"synthetic ordinary loader data")
|
||||
args = SimpleNamespace(support_dir=support, support_manifest_sha256="a" * 64)
|
||||
builder.finish(
|
||||
args,
|
||||
builder.PINS["python313"],
|
||||
image,
|
||||
{"usr/lib/ld-linux-x86-64.so.2": {"kind": "file", "sha256": builder.sha(library)}},
|
||||
)
|
||||
manifest = json.loads((image / "manifest.json").read_bytes())
|
||||
assert manifest["architecture"] == "x86_64_v3"
|
||||
paths = [item["path"] for item in manifest["files"]]
|
||||
assert paths == sorted(paths)
|
||||
assert paths.index("usr/lib/module.py") < paths.index("usr/lib/module/a.py")
|
||||
|
||||
|
||||
def test_public_archive_links_cannot_escape_or_cycle():
|
||||
buffer = io.BytesIO()
|
||||
with tarfile.open(fileobj=buffer, mode="w") as archive:
|
||||
for name, target in [("python/outside", "../../secret"), ("python/cycle", "cycle")]:
|
||||
item = tarfile.TarInfo(name)
|
||||
item.type = tarfile.SYMTYPE
|
||||
item.linkname = target
|
||||
archive.addfile(item)
|
||||
buffer.seek(0)
|
||||
with tarfile.open(fileobj=buffer, mode="r:") as archive:
|
||||
members = {item.name: item for item in archive.getmembers()}
|
||||
for name in members:
|
||||
with pytest.raises(ValueError):
|
||||
builder.member_bytes(archive, members, name)
|
||||
|
||||
|
||||
def test_corrupt_archive_is_refused_before_output_creation(tmp_path):
|
||||
archive = tmp_path / "public.tar.gz"
|
||||
archive.write_bytes(b"untrusted replacement")
|
||||
output = tmp_path / "image"
|
||||
args = SimpleNamespace(language="python313", archive=archive, output=output)
|
||||
with pytest.raises(ValueError, match="fixed upstream digest"):
|
||||
builder.build(args)
|
||||
assert not output.exists()
|
||||
@@ -1,269 +0,0 @@
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from jsonschema import Draft202012Validator, FormatChecker
|
||||
|
||||
from kf_oj.contracts import judging as v1
|
||||
from kf_oj.contracts import judging_v2 as v2
|
||||
from kf_oj.contracts import judging_versions as versions
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
FIXTURES = ROOT / "tests/fixtures/contracts-v2"
|
||||
PATHS = sorted(FIXTURES.glob("*.json"))
|
||||
|
||||
|
||||
def fixture(name, model):
|
||||
return v1.decode_wire(model, (FIXTURES / f"{name}.json").read_bytes())
|
||||
|
||||
|
||||
@pytest.mark.parametrize("path", PATHS, ids=lambda p: p.stem)
|
||||
def test_multi_runtime_examples_match_schema_and_rust(path):
|
||||
name = path.name.split(".")[0]
|
||||
model = v2.WIRE_MODELS[name]
|
||||
schema = json.loads((ROOT / f"contracts/schemas/{name}.v2.schema.json").read_text())
|
||||
validator = Draft202012Validator(schema, format_checker=FormatChecker())
|
||||
original = json.loads(path.read_text())
|
||||
if ".invalid-" in path.name:
|
||||
with pytest.raises(ValueError):
|
||||
v1.decode_wire(model, path.read_bytes())
|
||||
if ".invalid-semantic-" in path.name:
|
||||
assert validator.is_valid(original)
|
||||
else:
|
||||
assert not validator.is_valid(original)
|
||||
else:
|
||||
value = v1.decode_wire(model, path.read_bytes())
|
||||
validator.validate(original)
|
||||
assert value.model_dump(mode="json") == original
|
||||
|
||||
|
||||
@pytest.mark.parametrize("language", list(v2.Language))
|
||||
def test_explicit_version_keeps_old_consumers_closed_and_canonical_bytes_stable(language):
|
||||
path = FIXTURES / f"judge-manifest.valid-{language}.json"
|
||||
manifest = v1.decode_wire(v2.JobManifest, path.read_bytes())
|
||||
canonical = (FIXTURES / f"canonical-manifest.{language}.txt").read_bytes()
|
||||
assert v1.manifest_bytes(manifest) == canonical
|
||||
assert (
|
||||
hashlib.sha256(v1.manifest_bytes(manifest)).digest() == hashlib.sha256(canonical).digest()
|
||||
)
|
||||
with pytest.raises(ValueError):
|
||||
v1.decode_wire(v1.JobManifest, path.read_bytes())
|
||||
old = ROOT / "tests/fixtures/contracts/judge-manifest.valid.json"
|
||||
with pytest.raises(ValueError):
|
||||
v1.decode_wire(v2.JobManifest, old.read_bytes())
|
||||
old_manifest = v1.decode_wire(v1.JobManifest, old.read_bytes())
|
||||
assert (
|
||||
v1.manifest_bytes(old_manifest)
|
||||
== (ROOT / "tests/fixtures/contracts/canonical-manifest.v1.txt").read_bytes()
|
||||
)
|
||||
|
||||
|
||||
def test_shared_capabilities_refuse_absent_changed_runtime_and_over_budget():
|
||||
policy = fixture("judge-admission.valid", v2.AdmissionPolicy)
|
||||
for language in v2.Language:
|
||||
manifest = fixture(f"judge-manifest.valid-{language}", v2.JobManifest)
|
||||
for attempt in (1, 2, 3):
|
||||
policy.admit(manifest, attempt)
|
||||
for attempt in (0, 4, True):
|
||||
with pytest.raises(ValueError):
|
||||
policy.admit(manifest, attempt)
|
||||
for field, value in (("image_revision", 2), ("image_digest", "f" * 64)):
|
||||
other = manifest.model_copy(
|
||||
update={"toolchain": manifest.toolchain.model_copy(update={field: value})}
|
||||
)
|
||||
with pytest.raises(ValueError):
|
||||
policy.admit(other, 1)
|
||||
python = fixture("judge-manifest.valid-python313", v2.JobManifest)
|
||||
without_python = policy.model_copy(
|
||||
update={
|
||||
"capabilities": tuple(c for c in policy.capabilities if c.language != python.language)
|
||||
}
|
||||
)
|
||||
with pytest.raises(ValueError):
|
||||
without_python.admit(python, 1)
|
||||
too_small = python.model_copy(
|
||||
update={"run_limits": python.run_limits.model_copy(update={"memory_bytes": 1})}
|
||||
)
|
||||
with pytest.raises(ValueError):
|
||||
policy.admit(too_small, 1)
|
||||
too_large = python.model_copy(
|
||||
update={"run_limits": python.run_limits.model_copy(update={"memory_bytes": 128 * v1.MIB})}
|
||||
)
|
||||
with pytest.raises(ValueError):
|
||||
policy.admit(too_large, 1)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("language", list(v2.Language))
|
||||
def test_results_bind_exact_source_runtime_attempt_and_case_order(language):
|
||||
manifest = fixture(f"judge-manifest.valid-{language}", v2.JobManifest)
|
||||
result = fixture(f"judge-result.valid-{language}", v2.JudgementReport)
|
||||
digest = hashlib.sha256(versions.manifest_bytes(manifest)).hexdigest()
|
||||
assert digest == (FIXTURES / f"canonical-manifest.{language}.sha256").read_text().strip()
|
||||
result.validate_for_manifest(manifest, digest)
|
||||
for field, value in (("digest", "f" * 64), ("bytes", manifest.source.bytes + 1)):
|
||||
other = manifest.model_copy(
|
||||
update={"source": manifest.source.model_copy(update={field: value})}
|
||||
)
|
||||
with pytest.raises(ValueError):
|
||||
result.validate_for_manifest(other, digest)
|
||||
with pytest.raises(ValueError):
|
||||
result.validate_for_manifest(manifest, "f" * 64)
|
||||
changed_runtime = manifest.model_copy(
|
||||
update={"toolchain": manifest.toolchain.model_copy(update={"image_digest": "e" * 64})}
|
||||
)
|
||||
with pytest.raises(ValueError):
|
||||
result.validate_for_manifest(changed_runtime, digest)
|
||||
short = result.model_dump(mode="json")
|
||||
short.update(cases=short["cases"][:1], run_cpu_time_us=100, run_memory_peak_bytes=256 * 1024)
|
||||
valid_partial = v2.JudgementReport.model_validate(short)
|
||||
with pytest.raises(ValueError):
|
||||
valid_partial.validate_for_manifest(manifest, digest)
|
||||
|
||||
|
||||
def test_python_check_is_not_an_accepted_result_and_stage_kind_is_bound():
|
||||
manifest = fixture("judge-manifest.valid-python313", v2.JobManifest)
|
||||
stage = fixture("judge-stage.valid-python313", v2.StageReport)
|
||||
stage.validate_for_manifest(manifest)
|
||||
with pytest.raises(ValueError):
|
||||
stage.model_copy(
|
||||
update={"preparation_kind": v2.PreparationKind.COMPILE}
|
||||
).validate_for_manifest(manifest)
|
||||
result = fixture("judge-result.valid-python313", v2.JudgementReport).model_dump(mode="json")
|
||||
result.update(cases=[], run_cpu_time_us=None, run_memory_peak_bytes=None)
|
||||
with pytest.raises(ValueError):
|
||||
v2.JudgementReport.model_validate(result)
|
||||
|
||||
|
||||
def test_jvm_heap_is_smaller_than_total_memory_and_threads_share_the_budget():
|
||||
manifest = fixture("judge-manifest.valid-java21", v2.JobManifest)
|
||||
limits = manifest.run_limits
|
||||
assert 32 * v1.MIB <= v2.jvm_heap_bytes(limits) < limits.memory_bytes
|
||||
more_threads = limits.model_copy(update={"processes": 32})
|
||||
assert v2.jvm_heap_bytes(more_threads) < v2.jvm_heap_bytes(limits)
|
||||
with pytest.raises(ValueError):
|
||||
v2.jvm_heap_bytes(limits.model_copy(update={"memory_bytes": 128 * v1.MIB}))
|
||||
# Ordinary RuntimeError has no infrastructure failure code or trusted memory-limit claim.
|
||||
result = fixture("judge-result.valid-java21", v2.JudgementReport).model_dump(mode="json")
|
||||
result["cases"][0]["verdict"] = "runtime_error"
|
||||
result["verdict"] = "runtime_error"
|
||||
v2.JudgementReport.model_validate(result)
|
||||
|
||||
|
||||
def test_v2_wire_rejects_ambiguous_private_values_without_echoing_them():
|
||||
data = (FIXTURES / "judge-lease.valid.json").read_bytes()
|
||||
duplicate = data.replace(b'"schema_version": 2', b'"schema_version": 2, "schema_version": 2')
|
||||
for invalid in (duplicate, b" " * (v1.MAX_WIRE_BYTES + 1), data.decode().encode("utf-16")):
|
||||
with pytest.raises(ValueError):
|
||||
v1.decode_wire(v2.LeaseGrant, invalid)
|
||||
secret = "private-placeholder-never-echo"
|
||||
value = json.loads(data)
|
||||
value["lease_token"] = secret
|
||||
with pytest.raises(ValueError) as error:
|
||||
v1.decode_wire(v2.LeaseGrant, json.dumps(value).encode())
|
||||
assert secret not in str(error.value)
|
||||
|
||||
|
||||
def test_storage_boundary_reads_both_versions_without_rewriting_or_cross_version_reports():
|
||||
old = ROOT / "tests/fixtures/contracts"
|
||||
old_manifest = versions.decode_manifest((old / "judge-manifest.valid.json").read_bytes())
|
||||
old_report = versions.decode_report((old / "judge-result.valid.json").read_bytes())
|
||||
versions.validate_report(old_report, old_manifest, old_report.manifest_digest)
|
||||
assert versions.manifest_bytes(old_manifest) == (old / "canonical-manifest.v1.txt").read_bytes()
|
||||
manifest = versions.decode_manifest(
|
||||
(FIXTURES / "judge-manifest.valid-python313.json").read_bytes()
|
||||
)
|
||||
report = versions.decode_report((FIXTURES / "judge-result.valid-python313.json").read_bytes())
|
||||
versions.validate_report(
|
||||
report, manifest, hashlib.sha256(versions.manifest_bytes(manifest)).hexdigest()
|
||||
)
|
||||
for result, task in ((old_report, manifest), (report, old_manifest)):
|
||||
with pytest.raises(ValueError):
|
||||
versions.validate_report(result, task, result.manifest_digest)
|
||||
payload = (FIXTURES / "judge-manifest.valid-python313.json").read_bytes()
|
||||
for invalid in (
|
||||
payload.replace(b'"schema_version": 2', b'"schema_version": 3'),
|
||||
payload.replace(b'"schema_version": 2', b'"schema_version": true'),
|
||||
payload.replace(b'"schema_version": 2', b'"schema_version": 2, "schema_version": 2'),
|
||||
):
|
||||
with pytest.raises(ValueError):
|
||||
versions.decode_manifest(invalid)
|
||||
|
||||
|
||||
def test_registry_is_explicit_for_each_protocol_and_retains_legacy_configuration():
|
||||
from kf_oj.core.settings import Settings
|
||||
|
||||
registry = fixture("judge-registry.valid", v2.AdmissionRegistry)
|
||||
original = ROOT / "tests/fixtures/contracts"
|
||||
legacy = v1.decode_wire(v1.JobManifest, (original / "judge-manifest.valid.json").read_bytes())
|
||||
registry.admit(legacy, 1)
|
||||
assert registry.legacy.model_dump(mode="json") == json.loads(
|
||||
(original / "judge-admission.valid.json").read_text()
|
||||
)
|
||||
for language in v2.Language:
|
||||
manifest = fixture(f"judge-manifest.valid-{language}", v2.JobManifest)
|
||||
registry.admit(manifest, 1)
|
||||
with pytest.raises(ValueError, match="protocol"):
|
||||
v2.admit_manifest(registry.legacy, manifest, 1)
|
||||
for field, value in (("image_digest", "f" * 64), ("runtime_version", "other")):
|
||||
changed = manifest.model_copy(
|
||||
update={"toolchain": manifest.toolchain.model_copy(update={field: value})}
|
||||
)
|
||||
with pytest.raises(ValueError):
|
||||
registry.admit(changed, 1)
|
||||
with pytest.raises(ValueError, match="protocol"):
|
||||
v2.admit_manifest(registry.multi_runtime, legacy, 1)
|
||||
for policy in (registry.legacy, registry.multi_runtime, registry):
|
||||
settings = Settings(_env_file=None, judge_admission=policy.model_dump(mode="json"))
|
||||
assert type(settings.judge_admission) is type(policy)
|
||||
raw = registry.model_dump(mode="json")
|
||||
raw["multi_runtime"]["prefetch"] = 2
|
||||
with pytest.raises(ValueError):
|
||||
Settings(_env_file=None, judge_admission=raw)
|
||||
|
||||
|
||||
def test_generic_admission_revalidates_constructed_values_in_both_generations():
|
||||
registry = fixture("judge-registry.valid", v2.AdmissionRegistry)
|
||||
old = ROOT / "tests/fixtures/contracts/judge-manifest.valid.json"
|
||||
legacy = v1.decode_wire(v1.JobManifest, old.read_bytes())
|
||||
for manifest in (legacy, fixture("judge-manifest.valid-python313", v2.JobManifest)):
|
||||
for wrong in (
|
||||
manifest.model_copy(update={"schema_version": True}),
|
||||
manifest.model_copy(update={"job_wall_time_us": 1}),
|
||||
manifest.model_copy(update={"cases": ()}),
|
||||
):
|
||||
with pytest.raises(ValueError):
|
||||
v2.admit_manifest(registry, wrong, 1)
|
||||
v2.admit_manifest(registry, legacy, 1)
|
||||
|
||||
|
||||
def test_operator_registry_example_preserves_old_protocol_and_binds_managed_runtime():
|
||||
policy = v2.AdmissionRegistry.model_validate_json(
|
||||
(ROOT / "deploy/judge.admission.example.json").read_bytes()
|
||||
)
|
||||
old = v1.decode_wire(
|
||||
v1.JobManifest, (ROOT / "tests/fixtures/contracts/judge-manifest.valid.json").read_bytes()
|
||||
)
|
||||
policy.admit(old, 1)
|
||||
assert policy.multi_runtime is not None
|
||||
root = json.loads((ROOT / "deploy/sandbox.managed.runtime.example.json").read_text())
|
||||
for cap in policy.multi_runtime.capabilities:
|
||||
selected = next(
|
||||
item
|
||||
for item in root["runtime_images"]
|
||||
if item["runtime_identity"] == cap.toolchain.model_dump(mode="json")
|
||||
)
|
||||
ceiling = v1.RunLimits.model_validate(selected["per_task_ceiling"])
|
||||
assert cap.compile_ceiling.fits(ceiling) and cap.run_ceiling.fits(ceiling)
|
||||
manifest = fixture(f"judge-manifest.valid-{cap.language}", v2.JobManifest).model_copy(
|
||||
update={"compile_limits": cap.compile_ceiling, "run_limits": cap.run_ceiling}
|
||||
)
|
||||
policy.admit(manifest, 1)
|
||||
changed = manifest.model_copy(
|
||||
update={"toolchain": manifest.toolchain.model_copy(update={"image_digest": "f" * 64})}
|
||||
)
|
||||
with pytest.raises(ValueError):
|
||||
policy.admit(changed, 1)
|
||||
legacy_only = v2.AdmissionRegistry(schema_version=2, legacy=policy.legacy, multi_runtime=None)
|
||||
with pytest.raises(ValueError):
|
||||
legacy_only.admit(fixture("judge-manifest.valid-java21", v2.JobManifest), 1)
|
||||
@@ -1,399 +0,0 @@
|
||||
"""Actual finite Rust Coordinator/PG control acceptance; reports are synthetic, no execution."""
|
||||
|
||||
import asyncio
|
||||
import copy
|
||||
import hashlib
|
||||
import json
|
||||
from uuid import UUID, uuid4
|
||||
|
||||
import pytest
|
||||
from test_coordinator import claim, fixture, manifest, notice, synthetic_report, worker_headers
|
||||
from test_multi_runtime_versions import multi_import
|
||||
|
||||
from kf_oj.contracts import judging_v2 as v2
|
||||
from kf_oj.contracts import judging_versions as versions
|
||||
|
||||
|
||||
def report_v2(grant, task):
|
||||
result = synthetic_report(grant, task)
|
||||
result["schema_version"] = 2
|
||||
language = v2.Language(task["language"])
|
||||
kind = v2.runtime_contract(language)[3]
|
||||
if kind == v2.ProgramKind.PYTHON:
|
||||
files = [dict(path="main.py", artifact=task["source"])]
|
||||
elif kind == v2.ProgramKind.JAVA:
|
||||
files = [
|
||||
dict(path=name, artifact=dict(digest="d" * 64, bytes=256))
|
||||
for name in ("Helper.class", "Main$Inner.class", "Main.class")
|
||||
]
|
||||
else:
|
||||
files = [dict(path="main", artifact=dict(digest="d" * 64, bytes=4096))]
|
||||
result["compilation"]["program"] = dict(
|
||||
kind=kind.value,
|
||||
binding=grant["binding"],
|
||||
source=task["source"],
|
||||
toolchain=task["toolchain"],
|
||||
files=files,
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
def stage_v2(grant, language, sequence, stage="compiling", index=None):
|
||||
return dict(
|
||||
schema_version=2,
|
||||
kind="judge.stage",
|
||||
binding=grant["binding"],
|
||||
sequence=sequence,
|
||||
stage=stage,
|
||||
case_index=index,
|
||||
preparation_kind=v2.runtime_contract(v2.Language(language))[4].value,
|
||||
)
|
||||
|
||||
|
||||
def test_actual_versioned_claim_canonical_snapshot_stage_and_report(
|
||||
postgres_url, tmp_path, monkeypatch
|
||||
):
|
||||
async def run():
|
||||
async with fixture(postgres_url, tmp_path, monkeypatch, import_builder=multi_import) as (
|
||||
client,
|
||||
c,
|
||||
workers,
|
||||
_,
|
||||
create,
|
||||
):
|
||||
for language in v2.Language:
|
||||
created = await create(language.value)
|
||||
grant = await claim(client, created, workers[0])
|
||||
assert grant["schema_version"] == 2
|
||||
task = await manifest(c, grant)
|
||||
assert task["schema_version"] == 2 and task["language"] == language.value
|
||||
parsed = versions.decode_manifest(json.dumps(task).encode())
|
||||
canonical = versions.manifest_bytes(parsed)
|
||||
assert hashlib.sha256(canonical).hexdigest() == grant["manifest_digest"]
|
||||
async with client.fixture_app.state.sessions() as s:
|
||||
version = await client.fixture_app.state.problems.for_submission(
|
||||
s, created.problem_version_id
|
||||
)
|
||||
expected = version.manifest(
|
||||
created.evaluation_id, created.submission_id, language, parsed.source
|
||||
)
|
||||
assert versions.manifest_bytes(expected) == canonical
|
||||
headers = worker_headers(workers[0], grant)
|
||||
bad = stage_v2(grant, language, 1)
|
||||
bad["preparation_kind"] = (
|
||||
"compile" if language == v2.Language.PYTHON313 else "syntax_check"
|
||||
)
|
||||
assert (
|
||||
await client.post("/internal/stage", json=bad, headers=headers)
|
||||
).status_code == 422
|
||||
old = stage_v2(grant, language, 1)
|
||||
old.update(schema_version=1)
|
||||
old.pop("preparation_kind")
|
||||
assert (
|
||||
await client.post("/internal/stage", json=old, headers=headers)
|
||||
).status_code == 422
|
||||
for sequence, phase, index in (
|
||||
(1, "compiling", None),
|
||||
(2, "running", 0),
|
||||
(3, "running", 1),
|
||||
):
|
||||
stage = stage_v2(grant, language, sequence, phase, index)
|
||||
assert (
|
||||
await client.post("/internal/stage", json=stage, headers=headers)
|
||||
).status_code == 204
|
||||
assert (
|
||||
await client.post("/internal/stage", json=stage, headers=headers)
|
||||
).status_code == 204
|
||||
result = report_v2(grant, task)
|
||||
for invalid in (
|
||||
result | {"manifest_digest": "f" * 64},
|
||||
result | {"binding": grant["binding"] | {"fencing_version": 2}},
|
||||
):
|
||||
assert (
|
||||
await client.post("/internal/report", json=invalid, headers=headers)
|
||||
).status_code in (409, 422)
|
||||
changed = copy.deepcopy(result)
|
||||
changed["compilation"]["program"]["toolchain"]["image_digest"] = "e" * 64
|
||||
assert (
|
||||
await client.post("/internal/report", json=changed, headers=headers)
|
||||
).status_code == 422
|
||||
assert (
|
||||
await client.post("/internal/report", json=result, headers=headers)
|
||||
).status_code == 204
|
||||
assert (
|
||||
await client.post("/internal/report", json=result, headers=headers)
|
||||
).status_code == 204
|
||||
final = await c.fetchval(
|
||||
"SELECT final_report FROM evaluations WHERE id=$1", created.evaluation_id
|
||||
)
|
||||
assert json.loads(final) == result
|
||||
assert (
|
||||
await c.fetchval(
|
||||
"SELECT state FROM evaluations WHERE id=$1", created.evaluation_id
|
||||
)
|
||||
== "finished"
|
||||
)
|
||||
assert (
|
||||
await c.fetchval(
|
||||
"SELECT count(*) FROM case_results WHERE evaluation_id=$1",
|
||||
created.evaluation_id,
|
||||
)
|
||||
== 2
|
||||
)
|
||||
# No real program/Worker/Root executed; accepted is a synthetic protocol value only.
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
@pytest.mark.parametrize("kind", ["legacy", "identity", "budget"])
|
||||
def test_actual_claim_refuses_missing_protocol_or_changed_capability_without_attempt(
|
||||
postgres_url,
|
||||
tmp_path,
|
||||
monkeypatch,
|
||||
kind,
|
||||
):
|
||||
from pathlib import Path
|
||||
|
||||
from kf_oj.contracts.judging import decode_wire
|
||||
|
||||
fixtures = Path(__file__).resolve().parents[2] / "tests/fixtures/contracts-v2"
|
||||
registry = decode_wire(
|
||||
v2.AdmissionRegistry, (fixtures / "judge-registry.valid.json").read_bytes()
|
||||
)
|
||||
raw = registry.model_dump(mode="json")
|
||||
if kind == "identity":
|
||||
raw["multi_runtime"]["capabilities"][2]["toolchain"]["image_digest"] = "e" * 64
|
||||
elif kind == "budget":
|
||||
raw["multi_runtime"]["capabilities"][2]["run_ceiling"]["memory_bytes"] = 32 * 1024 * 1024
|
||||
policy = registry.legacy if kind == "legacy" else v2.AdmissionRegistry.model_validate(raw)
|
||||
|
||||
async def run():
|
||||
async with fixture(
|
||||
postgres_url,
|
||||
tmp_path,
|
||||
monkeypatch,
|
||||
import_builder=multi_import,
|
||||
node_policy=policy,
|
||||
) as (client, c, workers, _, create):
|
||||
created = await create("python313")
|
||||
response = await client.post(
|
||||
"/internal/claim",
|
||||
json=notice(created.evaluation_id),
|
||||
headers=worker_headers(workers[0]),
|
||||
)
|
||||
assert response.status_code == 422
|
||||
assert await c.fetchval("SELECT count(*) FROM judge_attempts") == 0
|
||||
assert await c.fetchval("SELECT state FROM evaluations") == "queued"
|
||||
assert (await client.get("/health/ready")).status_code == 200
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_actual_v2_cancel_requires_matching_stop_protocol_and_fence(
|
||||
postgres_url, tmp_path, monkeypatch
|
||||
):
|
||||
async def run():
|
||||
async with fixture(postgres_url, tmp_path, monkeypatch, import_builder=multi_import) as (
|
||||
client,
|
||||
c,
|
||||
workers,
|
||||
platform,
|
||||
create,
|
||||
):
|
||||
created = await create("java21")
|
||||
grant = await claim(client, created, workers[0])
|
||||
assert (
|
||||
await client.post(
|
||||
"/internal/cancel",
|
||||
json=notice(created.evaluation_id),
|
||||
headers={"Authorization": "Bearer " + platform},
|
||||
)
|
||||
).status_code == 204
|
||||
pending = await client.get(
|
||||
"/internal/pending-stops", headers=worker_headers(workers[0])
|
||||
)
|
||||
assert pending.status_code == 200
|
||||
assert pending.json() == [
|
||||
dict(schema_version=2, binding=grant["binding"], case_count=2)
|
||||
]
|
||||
stopped = dict(
|
||||
schema_version=2,
|
||||
kind="judge.stopped",
|
||||
binding=grant["binding"],
|
||||
cleanup_complete=True,
|
||||
)
|
||||
headers = worker_headers(workers[0])
|
||||
assert (
|
||||
await client.post(
|
||||
"/internal/stopped", json=stopped | {"schema_version": 1}, headers=headers
|
||||
)
|
||||
).status_code == 422
|
||||
bad = stopped | {"binding": grant["binding"] | {"attempt_id": str(uuid4())}}
|
||||
assert (
|
||||
await client.post("/internal/stopped", json=bad, headers=headers)
|
||||
).status_code == 409
|
||||
assert (
|
||||
await client.post("/internal/stopped", json=stopped, headers=headers)
|
||||
).status_code == 204
|
||||
assert await c.fetchval("SELECT state FROM evaluations") == "cancelled"
|
||||
assert (
|
||||
await c.fetchval(
|
||||
"SELECT cleanup_complete FROM judge_attempts WHERE id=$1",
|
||||
UUID(grant["binding"]["attempt_id"]),
|
||||
)
|
||||
is True
|
||||
)
|
||||
# This proves stop-report authority only; real JVM cleanup remains VM acceptance.
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
@pytest.mark.parametrize("language", ["python313", "java21"])
|
||||
def test_actual_v2_expiry_late_report_and_finite_infrastructure_retry(
|
||||
postgres_url, tmp_path, monkeypatch, language
|
||||
):
|
||||
"""Real PG/HTTP authority; reports/stop assertions remain synthetic, no user execution."""
|
||||
|
||||
async def run():
|
||||
async with fixture(postgres_url, tmp_path, monkeypatch, import_builder=multi_import) as (
|
||||
client,
|
||||
c,
|
||||
workers,
|
||||
_,
|
||||
create,
|
||||
):
|
||||
worker = workers[0]
|
||||
created = await create(language)
|
||||
old = await claim(client, created, worker)
|
||||
old_task = await manifest(c, old)
|
||||
late = report_v2(old, old_task)
|
||||
await c.execute(
|
||||
"UPDATE judge_attempts SET expires_at=clock_timestamp()-interval '1 second'"
|
||||
)
|
||||
assert (
|
||||
await client.post(
|
||||
"/internal/renew", json=old["binding"], headers=worker_headers(worker, old)
|
||||
)
|
||||
).status_code == 409
|
||||
for _ in range(60):
|
||||
pending = (
|
||||
await client.get("/internal/pending-stops", headers=worker_headers(worker))
|
||||
).json()
|
||||
if pending:
|
||||
break
|
||||
await asyncio.sleep(0.05)
|
||||
assert pending == [dict(schema_version=2, binding=old["binding"], case_count=2)]
|
||||
stopped = dict(
|
||||
schema_version=2,
|
||||
kind="judge.stopped",
|
||||
binding=old["binding"],
|
||||
cleanup_complete=True,
|
||||
)
|
||||
assert (
|
||||
await client.post("/internal/stopped", json=stopped, headers=worker_headers(worker))
|
||||
).status_code == 204
|
||||
await c.execute(
|
||||
"UPDATE evaluations SET not_before=clock_timestamp() WHERE id=$1",
|
||||
created.evaluation_id,
|
||||
)
|
||||
fresh = await claim(client, created, worker)
|
||||
assert fresh["attempt_number"] == 2 and fresh["binding"]["fencing_version"] == 2
|
||||
assert (
|
||||
await client.post(
|
||||
"/internal/report", json=late, headers=worker_headers(worker, old)
|
||||
)
|
||||
).status_code == 409
|
||||
assert (
|
||||
await client.post(
|
||||
"/internal/stage",
|
||||
json=stage_v2(old, language, 1),
|
||||
headers=worker_headers(worker, old),
|
||||
)
|
||||
).status_code == 409
|
||||
for sequence, stage, index in [
|
||||
(1, "compiling", None),
|
||||
(2, "running", 0),
|
||||
(3, "running", 1),
|
||||
]:
|
||||
assert (
|
||||
await client.post(
|
||||
"/internal/stage",
|
||||
json=stage_v2(fresh, language, sequence, stage, index),
|
||||
headers=worker_headers(worker, fresh),
|
||||
)
|
||||
).status_code == 204
|
||||
result = report_v2(fresh, await manifest(c, fresh))
|
||||
assert (
|
||||
await client.post(
|
||||
"/internal/report", json=result, headers=worker_headers(worker, fresh)
|
||||
)
|
||||
).status_code == 204
|
||||
digest = await c.fetchval(
|
||||
"SELECT final_report_digest FROM evaluations WHERE id=$1", created.evaluation_id
|
||||
)
|
||||
assert (
|
||||
await client.post(
|
||||
"/internal/report", json=late, headers=worker_headers(worker, old)
|
||||
)
|
||||
).status_code == 409
|
||||
assert (
|
||||
await c.fetchval(
|
||||
"SELECT final_report_digest FROM evaluations WHERE id=$1", created.evaluation_id
|
||||
)
|
||||
== digest
|
||||
)
|
||||
assert (
|
||||
await c.fetchval(
|
||||
"SELECT count(*) FROM case_results WHERE evaluation_id=$1",
|
||||
created.evaluation_id,
|
||||
)
|
||||
== 2
|
||||
)
|
||||
retry = await create(language)
|
||||
for number in range(1, 4):
|
||||
grant = await claim(client, retry, worker)
|
||||
result = report_v2(grant, await manifest(c, grant)) | dict(
|
||||
compilation=dict(
|
||||
status="system_error",
|
||||
program=None,
|
||||
diagnostics="",
|
||||
diagnostics_truncated=False,
|
||||
cpu_time_us=None,
|
||||
memory_peak_bytes=None,
|
||||
),
|
||||
cases=[],
|
||||
verdict="system_error",
|
||||
failure_code="artifact_unavailable",
|
||||
run_cpu_time_us=None,
|
||||
run_memory_peak_bytes=None,
|
||||
)
|
||||
assert grant["attempt_number"] == number
|
||||
assert (
|
||||
await client.post(
|
||||
"/internal/report", json=result, headers=worker_headers(worker, grant)
|
||||
)
|
||||
).status_code == 204
|
||||
assert await c.fetchval(
|
||||
"SELECT state FROM evaluations WHERE id=$1", retry.evaluation_id
|
||||
) == ("failed" if number == 3 else "queued")
|
||||
if number < 3:
|
||||
await c.execute(
|
||||
"UPDATE evaluations SET not_before=clock_timestamp() WHERE id=$1",
|
||||
retry.evaluation_id,
|
||||
)
|
||||
assert (
|
||||
await client.post(
|
||||
"/internal/claim",
|
||||
json=notice(retry.evaluation_id),
|
||||
headers=worker_headers(worker),
|
||||
)
|
||||
).status_code == 204
|
||||
assert (
|
||||
await c.fetchval(
|
||||
"SELECT count(*) FROM judge_attempts WHERE evaluation_id=$1",
|
||||
retry.evaluation_id,
|
||||
)
|
||||
== 3
|
||||
)
|
||||
|
||||
asyncio.run(run())
|
||||
@@ -1,454 +0,0 @@
|
||||
"""Real temporary PostgreSQL data/authority acceptance; no submitted code is executed."""
|
||||
|
||||
import asyncio
|
||||
import copy
|
||||
import hashlib
|
||||
import json
|
||||
import secrets
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
from uuid import uuid4
|
||||
|
||||
import asyncpg
|
||||
import httpx
|
||||
import pytest
|
||||
from conftest import connect, migrate
|
||||
from sqlalchemy import func, select, text
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
|
||||
from test_database_foundation import fixture_rows
|
||||
from test_problems import import_fixture, platform_role
|
||||
from test_worker_artifacts import install_attempt
|
||||
|
||||
from kf_oj.api.app import create_app
|
||||
from kf_oj.contracts import judging as v1
|
||||
from kf_oj.contracts import judging_v2 as v2
|
||||
from kf_oj.contracts.judging_versions import manifest_bytes
|
||||
from kf_oj.contracts.submissions import CreateSubmission
|
||||
from kf_oj.core.settings import Settings
|
||||
from kf_oj.infra.artifacts.repository import ArtifactRepository
|
||||
from kf_oj.infra.artifacts.store import ArtifactStore
|
||||
from kf_oj.infra.messaging.models import Outbox
|
||||
from kf_oj.modules.problems.models import ProblemVersion
|
||||
from kf_oj.modules.problems.service import ProblemService
|
||||
from kf_oj.modules.submissions.models import Evaluation, Submission
|
||||
from kf_oj.modules.submissions.public import IdempotencyConflict, SubmissionUnavailable
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
FIXTURES = ROOT / "tests/fixtures/contracts-v2"
|
||||
|
||||
|
||||
def configurations():
|
||||
return [
|
||||
{
|
||||
key: json.loads((FIXTURES / f"judge-manifest.valid-{language}.json").read_text())[key]
|
||||
for key in ("language", "toolchain", "compile_limits", "run_limits", "job_wall_time_us")
|
||||
}
|
||||
for language in v2.Language
|
||||
]
|
||||
|
||||
|
||||
def multi_import(root, *, slug="multi-runtime"):
|
||||
legacy, _ = import_fixture(root, slug=slug)
|
||||
manifest = {
|
||||
key: legacy[key]
|
||||
for key in ("slug", "title", "statement_file", "comparison", "cases", "samples")
|
||||
}
|
||||
manifest.update(schema_version=2, language_configurations=configurations())
|
||||
(root / "problem.json").write_text(json.dumps(manifest))
|
||||
registry = v1.decode_wire(
|
||||
v2.AdmissionRegistry, (FIXTURES / "judge-registry.valid.json").read_bytes()
|
||||
)
|
||||
return manifest, registry
|
||||
|
||||
|
||||
def test_upgrade_preserves_published_rows_and_legacy_attempt_bytes(postgres_url):
|
||||
migrate(postgres_url, "downgrade", "0001_judging_foundation")
|
||||
|
||||
async def seed():
|
||||
c = await connect(postgres_url)
|
||||
try:
|
||||
ids = await fixture_rows(c)
|
||||
manifest = v1.decode_wire(
|
||||
v1.JobManifest,
|
||||
(ROOT / "tests/fixtures/contracts/judge-manifest.valid.json").read_bytes(),
|
||||
)
|
||||
canonical = v1.manifest_bytes(manifest)
|
||||
await c.execute("SET ROLE kf_oj_coordinator")
|
||||
await c.execute(
|
||||
"""INSERT INTO judge_attempts
|
||||
(id,evaluation_id,worker_id,number,fencing_version,lease_token_hash,
|
||||
expires_at,manifest,manifest_digest)
|
||||
VALUES ($1,$2,$3,1,1,$4,now()+interval '30 seconds',$5,$6)""",
|
||||
uuid4(),
|
||||
ids[5],
|
||||
uuid4(),
|
||||
"d" * 64,
|
||||
canonical.decode(),
|
||||
hashlib.sha256(canonical).hexdigest(),
|
||||
)
|
||||
await c.execute("RESET ROLE")
|
||||
return await snapshot(c)
|
||||
finally:
|
||||
await c.close()
|
||||
|
||||
async def snapshot(c):
|
||||
version = await c.fetchval(
|
||||
"SELECT (to_jsonb(p)-'judging_schema_version'-'language_configurations')::text "
|
||||
"FROM problem_versions p"
|
||||
)
|
||||
submission = await c.fetchval("SELECT to_jsonb(s)::text FROM submissions s")
|
||||
attempt = await c.fetchrow("SELECT manifest,manifest_digest FROM judge_attempts")
|
||||
decoded = v1.JobManifest.model_validate_json(attempt["manifest"])
|
||||
assert hashlib.sha256(v1.manifest_bytes(decoded)).hexdigest() == attempt["manifest_digest"]
|
||||
return version, submission, dict(attempt)
|
||||
|
||||
before = asyncio.run(seed())
|
||||
migrate(postgres_url, "upgrade", "head")
|
||||
|
||||
async def read():
|
||||
c = await connect(postgres_url)
|
||||
try:
|
||||
assert await c.fetchval("SELECT judging_schema_version FROM problem_versions") == 1
|
||||
assert await c.fetchval("SELECT language_configurations FROM problem_versions") is None
|
||||
return await snapshot(c)
|
||||
finally:
|
||||
await c.close()
|
||||
|
||||
assert asyncio.run(read()) == before
|
||||
migrate(postgres_url, "downgrade", "0001_judging_foundation")
|
||||
migrate(postgres_url, "upgrade", "head")
|
||||
assert asyncio.run(read()) == before
|
||||
migrate(postgres_url, "check")
|
||||
|
||||
|
||||
async def insert_version(c, problem, cfgs, *, languages=None, revision=1):
|
||||
version = uuid4()
|
||||
await c.execute(
|
||||
"""INSERT INTO problem_versions
|
||||
(id,problem_id,revision,title,statement,languages,judging_schema_version,
|
||||
language_configurations,public_samples)
|
||||
VALUES ($1,$2,$3,'Multi runtime','Public',$4,2,$5,'[]')""",
|
||||
version,
|
||||
problem,
|
||||
revision,
|
||||
json.dumps(languages if languages is not None else [c["language"] for c in cfgs]),
|
||||
json.dumps(cfgs),
|
||||
)
|
||||
return version
|
||||
|
||||
|
||||
def test_sql_configuration_constraints_authority_and_downgrade_refusal(postgres_url):
|
||||
async def run():
|
||||
c = await connect(postgres_url)
|
||||
try:
|
||||
await c.execute("SET ROLE kf_oj_platform")
|
||||
problem = uuid4()
|
||||
await c.execute("INSERT INTO problems(id,slug) VALUES($1,'multi-sql')", problem)
|
||||
cfgs = configurations()
|
||||
bad_sets = []
|
||||
for field, value in (
|
||||
("language", "shell"),
|
||||
("toolchain", None),
|
||||
("compile_limits", None),
|
||||
("job_wall_time_us", True),
|
||||
("job_wall_time_us", 1000),
|
||||
):
|
||||
bad = copy.deepcopy(cfgs)
|
||||
bad[0][field] = value
|
||||
bad_sets.append(bad)
|
||||
for field, value in (
|
||||
("image_digest", "invalid"),
|
||||
("image_revision", True),
|
||||
("runtime_version", "3.14.0"),
|
||||
("image_kind", "linux_gnu_cxx_x86_64_v1"),
|
||||
):
|
||||
bad = copy.deepcopy(cfgs)
|
||||
bad[2]["toolchain"][field] = value
|
||||
bad_sets.append(bad)
|
||||
for field, value in (
|
||||
("processes", 8),
|
||||
("memory_bytes", 64 * v1.MIB),
|
||||
("cpu_time_us", "1"),
|
||||
("stack_bytes", 1),
|
||||
):
|
||||
bad = copy.deepcopy(cfgs)
|
||||
bad[3]["run_limits"][field] = value
|
||||
bad_sets.append(bad)
|
||||
bad = copy.deepcopy(cfgs)
|
||||
bad[0]["argv"] = ["/bin/sh"]
|
||||
bad_sets.append(bad)
|
||||
bad = copy.deepcopy(cfgs)
|
||||
bad[0]["toolchain"]["host_path"] = "/host/private"
|
||||
bad_sets.append(bad)
|
||||
bad_sets.extend([[], None, [cfgs[0], cfgs[0]]])
|
||||
for bad in bad_sets:
|
||||
with pytest.raises(asyncpg.CheckViolationError):
|
||||
await insert_version(c, problem, bad, languages=[c["language"] for c in cfgs])
|
||||
with pytest.raises(asyncpg.CheckViolationError):
|
||||
await insert_version(
|
||||
c, problem, cfgs, languages=["java21", "python313", "cpp20", "c17"]
|
||||
)
|
||||
version = await insert_version(c, problem, cfgs)
|
||||
with pytest.raises(asyncpg.InsufficientPrivilegeError):
|
||||
await c.execute("UPDATE problem_versions SET language_configurations='[]'")
|
||||
await c.execute("RESET ROLE")
|
||||
with pytest.raises(asyncpg.CheckViolationError):
|
||||
await c.execute("UPDATE problem_versions SET toolchain='{}'")
|
||||
await c.execute("SET ROLE kf_oj_coordinator")
|
||||
with pytest.raises(asyncpg.InsufficientPrivilegeError):
|
||||
await c.execute("INSERT INTO problems(id,slug) VALUES($1,'forbidden')", uuid4())
|
||||
with pytest.raises(asyncpg.InsufficientPrivilegeError):
|
||||
await c.execute("UPDATE problem_versions SET language_configurations='[]'")
|
||||
return version
|
||||
finally:
|
||||
await c.close()
|
||||
|
||||
version = asyncio.run(run())
|
||||
with pytest.raises(subprocess.CalledProcessError) as rejected:
|
||||
migrate(postgres_url, "downgrade", "0001_judging_foundation")
|
||||
assert b"cannot downgrade while multi-runtime data exists" in rejected.value.stderr
|
||||
|
||||
async def preserved():
|
||||
c = await connect(postgres_url)
|
||||
try:
|
||||
assert (
|
||||
await c.fetchval("SELECT count(*) FROM problem_versions WHERE id=$1", version) == 1
|
||||
)
|
||||
assert (
|
||||
await c.fetchval("SELECT version_num FROM alembic_version") == "0002_multi_runtime"
|
||||
)
|
||||
finally:
|
||||
await c.close()
|
||||
|
||||
asyncio.run(preserved())
|
||||
|
||||
|
||||
def test_real_four_language_import_public_projection_and_initial_submissions(
|
||||
postgres_url, tmp_path, monkeypatch
|
||||
):
|
||||
_, registry = multi_import(tmp_path / "multi")
|
||||
import_fixture(tmp_path / "legacy", slug="legacy-runtime")
|
||||
|
||||
async def run():
|
||||
engine = create_async_engine(postgres_url)
|
||||
platform_role(engine)
|
||||
mod = __import__("kf_oj.api.app", fromlist=["build_engine"])
|
||||
monkeypatch.setattr(mod, "build_engine", lambda _: engine)
|
||||
app = create_app(
|
||||
Settings(
|
||||
_env_file=None,
|
||||
environment="test",
|
||||
database_url=postgres_url,
|
||||
artifact_root=tmp_path / "artifacts",
|
||||
judge_admission=registry,
|
||||
)
|
||||
)
|
||||
async with app.router.lifespan_context(app):
|
||||
owner = await app.state.identity.create_account(
|
||||
"multi_owner", secrets.token_urlsafe(24)
|
||||
)
|
||||
version = await app.state.problems.import_directory(tmp_path / "multi")
|
||||
old = await app.state.problems.import_directory(tmp_path / "legacy")
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="http://test"
|
||||
) as client:
|
||||
public = await client.get(f"/api/v1/problems/versions/{version}")
|
||||
assert public.status_code == 200
|
||||
data = public.json()
|
||||
assert data["run_limits"] is None
|
||||
assert data["languages"] == [v.value for v in v2.Language]
|
||||
assert [c["source_name"] for c in data["language_configurations"]] == [
|
||||
"main.c",
|
||||
"main.cpp",
|
||||
"main.py",
|
||||
"Main.java",
|
||||
]
|
||||
assert data["language_configurations"][2]["preparation_kind"] == "syntax_check"
|
||||
assert data["language_configurations"][3]["entrypoint"] == "Main"
|
||||
for private in (
|
||||
"image_digest",
|
||||
"image_kind",
|
||||
"hidden-input",
|
||||
"hidden-expected",
|
||||
str(tmp_path),
|
||||
):
|
||||
assert private not in public.text
|
||||
for language in v2.Language:
|
||||
request = CreateSubmission(
|
||||
problem_version_id=version,
|
||||
language=language,
|
||||
idempotency_key=uuid4(),
|
||||
source="Synthetic transport bytes; never executed",
|
||||
)
|
||||
results = await asyncio.gather(
|
||||
*(app.state.submissions.create(owner, request) for _ in range(2))
|
||||
)
|
||||
assert results[0].submission_id == results[1].submission_id
|
||||
async with app.state.sessions() as s:
|
||||
projected = await app.state.problems.for_submission(s, version)
|
||||
ref = app.state.problems.artifacts.store.reference(request.source.encode())
|
||||
manifest = projected.manifest(
|
||||
results[0].evaluation_id, results[0].submission_id, language, ref
|
||||
)
|
||||
assert manifest.schema_version == 2 and manifest.toolchain.runtime_version
|
||||
v2.admit_manifest(registry, manifest, 1)
|
||||
assert manifest_bytes(manifest)
|
||||
with pytest.raises(IdempotencyConflict):
|
||||
await app.state.submissions.create(
|
||||
owner, request.model_copy(update={"source": "Different"})
|
||||
)
|
||||
with pytest.raises(SubmissionUnavailable):
|
||||
await app.state.submissions.create(
|
||||
owner,
|
||||
CreateSubmission(
|
||||
problem_version_id=old,
|
||||
language="python313",
|
||||
idempotency_key=uuid4(),
|
||||
source="x",
|
||||
),
|
||||
)
|
||||
async with app.state.sessions() as s:
|
||||
for model in (Submission, Evaluation, Outbox):
|
||||
assert await s.scalar(select(func.count()).select_from(model)) == 4
|
||||
row = await s.get(ProblemVersion, version)
|
||||
assert row.toolchain is None and row.run_limits is None
|
||||
assert len(row.language_configurations) == 4
|
||||
# Actual SQL membership guard also rejects new-language rows on old versions.
|
||||
admin = await connect(postgres_url)
|
||||
try:
|
||||
source_digest = await admin.fetchval(
|
||||
"SELECT source_digest FROM submissions LIMIT 1"
|
||||
)
|
||||
problem_id = await admin.fetchval(
|
||||
"SELECT problem_id FROM problem_versions WHERE id=$1", old
|
||||
)
|
||||
await admin.execute("SET ROLE kf_oj_platform")
|
||||
with pytest.raises(asyncpg.CheckViolationError):
|
||||
await admin.execute(
|
||||
"""INSERT INTO submissions
|
||||
(id,account_id,problem_id,problem_version_id,language,source_digest,
|
||||
idempotency_key,request_digest)
|
||||
VALUES($1,$2,$3,$4,'java21',$5,$6,$7)""",
|
||||
uuid4(),
|
||||
owner.account_id,
|
||||
problem_id,
|
||||
old,
|
||||
source_digest,
|
||||
uuid4(),
|
||||
"a" * 64,
|
||||
)
|
||||
await admin.execute("RESET ROLE")
|
||||
with pytest.raises(asyncpg.CheckViolationError):
|
||||
await admin.execute(
|
||||
"UPDATE problem_versions SET language_configurations='[]' WHERE id=$1",
|
||||
version,
|
||||
)
|
||||
finally:
|
||||
await admin.close()
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_import_refuses_missing_protocol_and_runtime_identity_without_writes(
|
||||
postgres_url, tmp_path
|
||||
):
|
||||
manifest, registry = multi_import(tmp_path / "multi")
|
||||
store = ArtifactStore(tmp_path / "artifacts")
|
||||
|
||||
async def run():
|
||||
engine = create_async_engine(postgres_url)
|
||||
platform_role(engine)
|
||||
sessions = async_sessionmaker(engine, expire_on_commit=False)
|
||||
try:
|
||||
service = ProblemService(sessions, ArtifactRepository(store), registry.legacy)
|
||||
with pytest.raises(ValueError, match="protocol"):
|
||||
await service.import_directory(tmp_path / "multi")
|
||||
manifest["language_configurations"][2]["toolchain"]["image_digest"] = "f" * 64
|
||||
(tmp_path / "multi/problem.json").write_text(json.dumps(manifest))
|
||||
service = ProblemService(sessions, ArtifactRepository(store), registry)
|
||||
with pytest.raises(ValueError, match="capability"):
|
||||
await service.import_directory(tmp_path / "multi")
|
||||
async with sessions() as s:
|
||||
assert await s.scalar(select(func.count()).select_from(ProblemVersion)) == 0
|
||||
finally:
|
||||
await engine.dispose()
|
||||
|
||||
try:
|
||||
asyncio.run(run())
|
||||
finally:
|
||||
store.close()
|
||||
|
||||
|
||||
def test_v2_worker_reads_immutable_language_snapshot_with_live_lease(postgres_url, tmp_path):
|
||||
_, registry = multi_import(tmp_path / "multi")
|
||||
|
||||
async def run():
|
||||
admin = create_async_engine(postgres_url)
|
||||
sessions = async_sessionmaker(admin, expire_on_commit=False)
|
||||
app = create_app(
|
||||
Settings(
|
||||
_env_file=None,
|
||||
environment="test",
|
||||
database_url=postgres_url,
|
||||
artifact_root=tmp_path / "artifacts",
|
||||
judge_admission=registry,
|
||||
)
|
||||
)
|
||||
try:
|
||||
async with app.router.lifespan_context(app):
|
||||
platform_role(app.state.sessions.kw["bind"])
|
||||
account = await app.state.identity.create_account(
|
||||
"worker_owner", secrets.token_urlsafe(24)
|
||||
)
|
||||
version = await app.state.problems.import_directory(tmp_path / "multi")
|
||||
for language in ("python313", "java21"):
|
||||
binding, token, manifest = await install_attempt(
|
||||
sessions, app, account, version, language=language
|
||||
)
|
||||
# Controlled DB lease fixture, not an actual Coordinator claim or execution.
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="http://test"
|
||||
) as client:
|
||||
path = (
|
||||
f"/api/v1/internal/evaluations/{binding.evaluation_id}"
|
||||
f"/attempts/{binding.attempt_id}"
|
||||
)
|
||||
headers = {
|
||||
"Authorization": "Bearer " + token,
|
||||
"X-Worker-ID": str(binding.worker_id),
|
||||
"X-Fencing-Version": "1",
|
||||
}
|
||||
response = await client.get(path + "/manifest", headers=headers)
|
||||
assert response.status_code == 200 and response.content == manifest_bytes(
|
||||
manifest
|
||||
)
|
||||
assert manifest.schema_version == 2 and manifest.language == language
|
||||
response = await client.get(
|
||||
path + "/artifacts/" + manifest.source.digest, headers=headers
|
||||
)
|
||||
assert (
|
||||
response.status_code == 200 and response.content == b"synthetic source"
|
||||
)
|
||||
assert (
|
||||
await client.get(
|
||||
path + "/manifest", headers=headers | {"X-Worker-ID": str(uuid4())}
|
||||
)
|
||||
).status_code == 404
|
||||
assert (
|
||||
await client.get(
|
||||
path + "/manifest", headers=headers | {"X-Fencing-Version": "2"}
|
||||
)
|
||||
).status_code == 404
|
||||
async with sessions.begin() as s:
|
||||
await s.execute(text("SET LOCAL ROLE kf_oj_coordinator"))
|
||||
await s.execute(
|
||||
text("UPDATE evaluations SET state='cancelling' WHERE id=:id"),
|
||||
{"id": binding.evaluation_id},
|
||||
)
|
||||
assert (
|
||||
await client.get(path + "/manifest", headers=headers)
|
||||
).status_code == 404
|
||||
finally:
|
||||
await admin.dispose()
|
||||
|
||||
asyncio.run(run())
|
||||
@@ -1,247 +0,0 @@
|
||||
"""Real private PostgreSQL/RabbitMQ control plane, no user program execution."""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import secrets
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from urllib.parse import urlsplit
|
||||
from uuid import uuid4
|
||||
|
||||
import aio_pika
|
||||
import pytest
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.ext.asyncio import create_async_engine
|
||||
from test_problems import import_fixture, platform_role
|
||||
|
||||
from kf_oj.api.app import create_app
|
||||
from kf_oj.contracts.submissions import CreateSubmission
|
||||
from kf_oj.core.settings import Settings
|
||||
from kf_oj.infra.messaging.publisher import QUEUE, ConfirmedBroker, OutboxRepository
|
||||
|
||||
|
||||
def test_actual_confirm_duplicate_lock_recovery_and_broker_failure(
|
||||
postgres_url, tmp_path, monkeypatch
|
||||
):
|
||||
asyncio.run(outbox_acceptance(postgres_url, tmp_path, monkeypatch))
|
||||
|
||||
|
||||
async def outbox_acceptance(postgres_url, tmp_path, monkeypatch):
|
||||
config = os.environ.get("KF_OJ_TEST_AMQP_CONFIG")
|
||||
if config is None:
|
||||
pytest.skip("private RabbitMQ control acceptance requires KF_OJ_TEST_AMQP_CONFIG")
|
||||
url = json.loads(Path(config).read_text())["amqp_url"]
|
||||
parsed = urlsplit(url)
|
||||
assert parsed.hostname == "127.0.0.1" and parsed.port == 26572
|
||||
_, policy = import_fixture(tmp_path / "import")
|
||||
engine = create_async_engine(postgres_url)
|
||||
platform_role(engine)
|
||||
monkeypatch.setattr("kf_oj.api.app.build_engine", lambda _: engine)
|
||||
app = create_app(
|
||||
Settings(
|
||||
_env_file=None,
|
||||
environment="test",
|
||||
database_url=postgres_url,
|
||||
artifact_root=tmp_path / "artifacts",
|
||||
judge_admission=policy,
|
||||
)
|
||||
)
|
||||
broker = ConfirmedBroker(url)
|
||||
connection = await aio_pika.connect(url, timeout=5)
|
||||
try:
|
||||
async with app.router.lifespan_context(app):
|
||||
owner = await app.state.identity.create_account(
|
||||
"outbox_test", secrets.token_urlsafe(24)
|
||||
)
|
||||
version = await app.state.problems.import_directory(tmp_path / "import")
|
||||
created = await app.state.submissions.create(
|
||||
owner,
|
||||
CreateSubmission(
|
||||
problem_version_id=version,
|
||||
language="c17",
|
||||
idempotency_key=uuid4(),
|
||||
source="synthetic control event; not compiled",
|
||||
),
|
||||
)
|
||||
repo = OutboxRepository(app.state.sessions)
|
||||
claimed = await asyncio.gather(repo.claim(), repo.claim())
|
||||
assert sum(v is not None for v in claimed) == 1
|
||||
item = next(v for v in claimed if v is not None)
|
||||
assert item[1] == created.evaluation_id
|
||||
# Kill a separate real publisher process after broker confirm, before DB mark.
|
||||
code = """import asyncio,json,sys,signal
|
||||
from uuid import UUID
|
||||
from pathlib import Path
|
||||
from kf_oj.infra.messaging.publisher import ConfirmedBroker
|
||||
async def main():
|
||||
item=tuple(UUID(x) for x in json.loads(sys.stdin.readline()))
|
||||
broker=ConfirmedBroker(json.loads(Path(sys.argv[1]).read_text())["amqp_url"])
|
||||
await broker.publish(item)
|
||||
print("CONFIRMED",flush=True)
|
||||
await asyncio.Event().wait()
|
||||
asyncio.run(main())
|
||||
"""
|
||||
child = subprocess.Popen(
|
||||
[sys.executable, "-c", code, config],
|
||||
stdin=subprocess.PIPE,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
text=True,
|
||||
)
|
||||
try:
|
||||
child.stdin.write(json.dumps([str(x) for x in item]) + "\n")
|
||||
child.stdin.flush()
|
||||
assert (
|
||||
await asyncio.wait_for(asyncio.to_thread(child.stdout.readline), 10)
|
||||
== "CONFIRMED\n"
|
||||
)
|
||||
finally:
|
||||
child.kill()
|
||||
await asyncio.to_thread(child.wait, 5)
|
||||
child.stdin.close()
|
||||
child.stdout.close()
|
||||
child.stderr.close()
|
||||
assert child.returncode == -9
|
||||
channel = await connection.channel()
|
||||
queue = await channel.get_queue(QUEUE)
|
||||
first = await queue.get(timeout=5)
|
||||
assert json.loads(first.body) == {
|
||||
"schema_version": 1,
|
||||
"kind": "judge.evaluation",
|
||||
"evaluation_id": str(created.evaluation_id),
|
||||
}
|
||||
assert len(first.body) <= 512 and first.delivery_mode == 2
|
||||
# A publisher lost after confirm cannot mark DB; actual DB clock lock expires.
|
||||
assert await repo.claim() is None
|
||||
await asyncio.sleep(15.1)
|
||||
retry = await repo.claim()
|
||||
assert retry[:2] == item[:2] and retry[2] != item[2]
|
||||
assert not await repo.confirmed(item)
|
||||
await broker.publish(retry)
|
||||
assert await repo.confirmed(retry)
|
||||
second = await queue.get(timeout=5)
|
||||
assert second.body == first.body and second.message_id == first.message_id
|
||||
await first.ack()
|
||||
await second.ack()
|
||||
assert await repo.claim() is None
|
||||
async with app.state.sessions() as s:
|
||||
assert (
|
||||
await s.scalar(
|
||||
text("SELECT count(*) FROM outbox WHERE delivered_at IS NOT NULL")
|
||||
)
|
||||
== 1
|
||||
)
|
||||
# A second transaction succeeds even when a separate broker is unavailable.
|
||||
await app.state.submissions.create(
|
||||
owner,
|
||||
CreateSubmission(
|
||||
problem_version_id=version,
|
||||
language="c17",
|
||||
idempotency_key=uuid4(),
|
||||
source="broker failure synthetic event",
|
||||
),
|
||||
)
|
||||
failed = await repo.claim()
|
||||
unavailable = ConfirmedBroker(url.replace(":26572/", ":26579/"))
|
||||
try:
|
||||
with pytest.raises((ConnectionError, OSError)):
|
||||
await unavailable.publish(failed)
|
||||
finally:
|
||||
await unavailable.close()
|
||||
await repo.failed(failed)
|
||||
assert await repo.claim() is None
|
||||
async with app.state.sessions() as s:
|
||||
row = (
|
||||
await s.execute(
|
||||
text("""SELECT failures,last_error_code,
|
||||
next_attempt_at>clock_timestamp() deferred,delivered_at IS NULL pending,
|
||||
lock_owner IS NULL unlocked FROM outbox WHERE id=:id"""),
|
||||
{"id": failed[0]},
|
||||
)
|
||||
).one()
|
||||
assert tuple(row) == (1, "broker_unavailable", True, True, True)
|
||||
# Preserve the broker-failure row for inspection while testing another crash window.
|
||||
async with app.state.sessions.begin() as s:
|
||||
await s.execute(
|
||||
text(
|
||||
"UPDATE outbox SET next_attempt_at=clock_timestamp()+interval '1 hour' "
|
||||
"WHERE id=:id"
|
||||
),
|
||||
{"id": failed[0]},
|
||||
)
|
||||
before = await app.state.submissions.create(
|
||||
owner,
|
||||
CreateSubmission(
|
||||
problem_version_id=version,
|
||||
language="c17",
|
||||
idempotency_key=uuid4(),
|
||||
source="before-confirm synthetic event",
|
||||
),
|
||||
)
|
||||
pending = await repo.claim()
|
||||
assert pending[1] == before.evaluation_id
|
||||
preconfirm = code.replace("await broker.publish(item)", "await broker.open()")
|
||||
child = subprocess.Popen(
|
||||
[sys.executable, "-c", preconfirm, config],
|
||||
stdin=subprocess.PIPE,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
text=True,
|
||||
)
|
||||
try:
|
||||
child.stdin.write(json.dumps([str(x) for x in pending]) + "\n")
|
||||
child.stdin.flush()
|
||||
assert (
|
||||
await asyncio.wait_for(asyncio.to_thread(child.stdout.readline), 10)
|
||||
== "CONFIRMED\n"
|
||||
)
|
||||
finally:
|
||||
child.kill()
|
||||
await asyncio.to_thread(child.wait, 5)
|
||||
child.stdin.close()
|
||||
child.stdout.close()
|
||||
child.stderr.close()
|
||||
assert child.returncode == -9
|
||||
assert await queue.get(fail=False, timeout=5) is None
|
||||
await asyncio.sleep(15.1)
|
||||
retry = await repo.claim()
|
||||
assert retry[:2] == pending[:2]
|
||||
await broker.publish(retry)
|
||||
assert await repo.confirmed(retry)
|
||||
delivered = await queue.get(timeout=5)
|
||||
assert json.loads(delivered.body)["evaluation_id"] == str(before.evaluation_id)
|
||||
await delivered.ack()
|
||||
|
||||
finally:
|
||||
await broker.close()
|
||||
await connection.close()
|
||||
|
||||
|
||||
def test_actual_transport_failure_does_not_log_connection_credentials(caplog):
|
||||
"""Local protocol refusal exercises descendant client loggers without a broker/DB."""
|
||||
|
||||
async def refused_connection():
|
||||
touched = asyncio.Event()
|
||||
|
||||
async def refuse(reader, writer):
|
||||
touched.set()
|
||||
writer.close()
|
||||
await writer.wait_closed()
|
||||
|
||||
listener = await asyncio.start_server(refuse, "127.0.0.1", 0)
|
||||
port = listener.sockets[0].getsockname()[1]
|
||||
broker = ConfirmedBroker(f"amqp://fixture:private-sentinel@127.0.0.1:{port}/")
|
||||
try:
|
||||
with pytest.raises(aio_pika.exceptions.AMQPConnectionError):
|
||||
await broker.publish((uuid4(), uuid4(), uuid4()))
|
||||
assert touched.is_set(), "must reach actual local transport refusal"
|
||||
finally:
|
||||
await broker.close()
|
||||
listener.close()
|
||||
await listener.wait_closed()
|
||||
|
||||
asyncio.run(refused_connection())
|
||||
assert "private-sentinel" not in caplog.text
|
||||
assert "amqp://" not in caplog.text
|
||||
@@ -1,262 +0,0 @@
|
||||
import asyncio
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from uuid import uuid4
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from sqlalchemy import event, func, select, update
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
|
||||
|
||||
from kf_oj.api.app import create_app
|
||||
from kf_oj.contracts.judging import AdmissionPolicy
|
||||
from kf_oj.core.settings import Settings
|
||||
from kf_oj.infra.artifacts.repository import ArtifactRepository
|
||||
from kf_oj.infra.artifacts.store import ArtifactError, ArtifactStore
|
||||
from kf_oj.modules.problems.models import Problem, ProblemVersion
|
||||
from kf_oj.modules.problems.service import ProblemService
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
|
||||
|
||||
def import_fixture(root, *, slug="fixture-problem"):
|
||||
root.mkdir(mode=0o700)
|
||||
policy = AdmissionPolicy.model_validate_json(
|
||||
(ROOT / "tests/fixtures/contracts/judge-admission.valid.json").read_bytes()
|
||||
)
|
||||
manifest = {
|
||||
"schema_version": 1,
|
||||
"slug": slug,
|
||||
"title": "Synthetic problem",
|
||||
"statement_file": "statement.md",
|
||||
"languages": ["c17", "cpp20"],
|
||||
"toolchain": policy.toolchain.model_dump(),
|
||||
"comparison": "exact_bytes",
|
||||
"compile_limits": policy.compile_ceiling.model_dump(),
|
||||
"run_limits": policy.run_ceiling.model_dump(),
|
||||
"job_wall_time_us": 60000000,
|
||||
"cases": [
|
||||
{"key": "empty", "input_file": "input-1", "expected_file": "expected-1"},
|
||||
{"key": "binary", "input_file": "input-2", "expected_file": "expected-2"},
|
||||
],
|
||||
"samples": [{"input_file": "sample-input", "output_file": "sample-output"}],
|
||||
}
|
||||
for filename, data in {
|
||||
"statement.md": b"Public statement only",
|
||||
"input-1": b"",
|
||||
"expected-1": b"hidden-expected\x00\xff",
|
||||
"input-2": b"hidden-input\x00\xfe",
|
||||
"expected-2": b"hidden-expected-2\xff",
|
||||
"sample-input": b"1\n",
|
||||
"sample-output": b"2\n",
|
||||
}.items():
|
||||
(root / filename).write_bytes(data)
|
||||
(root / "problem.json").write_text(json.dumps(manifest))
|
||||
return manifest, policy
|
||||
|
||||
|
||||
def platform_role(engine):
|
||||
@event.listens_for(engine.sync_engine, "connect")
|
||||
def apply_role(connection, _):
|
||||
cursor = connection.cursor()
|
||||
cursor.execute("SET ROLE kf_oj_platform")
|
||||
cursor.close()
|
||||
|
||||
|
||||
def test_controlled_import_cli_and_unpublished_version_visibility(postgres_url, tmp_path):
|
||||
root = tmp_path / "import"
|
||||
_, policy = import_fixture(root)
|
||||
env = os.environ | {
|
||||
"KF_OJ_DATABASE_URL": postgres_url,
|
||||
"KF_OJ_ARTIFACT_ROOT": str(tmp_path / "artifacts"),
|
||||
"KF_OJ_JUDGE_ADMISSION": policy.model_dump_json(),
|
||||
"KF_OJ_ENVIRONMENT": "test",
|
||||
}
|
||||
result = subprocess.run(
|
||||
[sys.executable, "-m", "kf_oj.modules.problems.cli", str(root)],
|
||||
env=env,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=15,
|
||||
)
|
||||
assert result.returncode == 0 and "Problem version published:" in result.stdout
|
||||
assert "hidden-expected" not in result.stdout + result.stderr
|
||||
|
||||
async def run():
|
||||
engine = create_async_engine(postgres_url)
|
||||
platform_role(engine)
|
||||
sessions = async_sessionmaker(engine, expire_on_commit=False)
|
||||
try:
|
||||
service = ProblemService(sessions)
|
||||
page = await service.list()
|
||||
assert page.items[0].revision == 1
|
||||
version_id = page.items[0].version_id
|
||||
draft_id = uuid4()
|
||||
async with sessions.begin() as session:
|
||||
published = await session.get(ProblemVersion, version_id)
|
||||
session.add(
|
||||
ProblemVersion(
|
||||
id=draft_id,
|
||||
problem_id=published.problem_id,
|
||||
revision=2,
|
||||
title=published.title,
|
||||
statement=published.statement,
|
||||
languages=published.languages,
|
||||
toolchain=published.toolchain,
|
||||
compile_limits=published.compile_limits,
|
||||
run_limits=published.run_limits,
|
||||
job_wall_time_us=published.job_wall_time_us,
|
||||
public_samples=[],
|
||||
)
|
||||
)
|
||||
from kf_oj.modules.problems.public import ProblemUnavailable
|
||||
|
||||
with pytest.raises(ProblemUnavailable):
|
||||
await service.detail(draft_id)
|
||||
assert (await service.list()).items[0].version_id == version_id
|
||||
finally:
|
||||
await engine.dispose()
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_actual_import_two_immutable_versions_and_public_api(postgres_url, tmp_path):
|
||||
root = tmp_path / "import"
|
||||
manifest, policy = import_fixture(root)
|
||||
|
||||
async def run():
|
||||
app = create_app(
|
||||
Settings(
|
||||
_env_file=None,
|
||||
environment="test",
|
||||
database_url=postgres_url,
|
||||
artifact_root=tmp_path / "artifacts",
|
||||
judge_admission=policy,
|
||||
)
|
||||
)
|
||||
async with app.router.lifespan_context(app):
|
||||
platform_role(app.state.sessions.kw["bind"])
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="http://test"
|
||||
) as client:
|
||||
assert (await client.get("/api/v1/problems")).json() == {
|
||||
"items": [],
|
||||
"next_after": None,
|
||||
}
|
||||
first = await app.state.problems.import_directory(root)
|
||||
async with app.state.sessions() as session:
|
||||
projected = await app.state.problems.for_submission(session, first)
|
||||
assert len(projected.cases) == 2 and projected.cases[0].input.bytes == 0
|
||||
binary = await app.state.problems.artifacts.read(
|
||||
session, projected.cases[0].expected
|
||||
)
|
||||
assert binary == b"hidden-expected\x00\xff"
|
||||
manifest["title"] = "Second immutable version"
|
||||
(root / "statement.md").write_text("New public statement")
|
||||
(root / "problem.json").write_text(json.dumps(manifest))
|
||||
second = await app.state.problems.import_directory(root)
|
||||
old = await client.get(f"/api/v1/problems/versions/{first}")
|
||||
new = await client.get(f"/api/v1/problems/versions/{second}")
|
||||
assert old.json()["revision"] == 1 and new.json()["revision"] == 2
|
||||
assert old.json()["statement"] == "Public statement only"
|
||||
assert new.json()["statement"] == "New public statement"
|
||||
assert new.json()["samples"][0] == {
|
||||
"encoding": "base64",
|
||||
"input": base64.b64encode(b"1\n").decode(),
|
||||
"output": base64.b64encode(b"2\n").decode(),
|
||||
}
|
||||
page = (await client.get("/api/v1/problems")).json()
|
||||
assert page["items"][0]["version_id"] == str(second)
|
||||
assert (await client.get("/api/v1/problems?limit=0")).status_code == 422
|
||||
assert (await client.get(f"/api/v1/problems/versions/{uuid4()}")).status_code == 404
|
||||
public_text = old.text + new.text + json.dumps(page)
|
||||
assert "hidden-expected" not in public_text and "hidden-input" not in public_text
|
||||
assert str(root) not in public_text and "digest" not in public_text
|
||||
async with app.state.sessions.begin() as session:
|
||||
await session.execute(update(Problem).values(enabled=False))
|
||||
assert (await client.get(f"/api/v1/problems/versions/{first}")).status_code == 404
|
||||
assert (await client.get("/api/v1/problems")).json()["items"] == []
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
@pytest.mark.parametrize("invalid", ["missing", "duplicate", "deadline", "node_budget"])
|
||||
def test_import_invalid_package_refused_without_published_rows(postgres_url, tmp_path, invalid):
|
||||
root = tmp_path / "import"
|
||||
manifest, policy = import_fixture(root)
|
||||
if invalid == "missing":
|
||||
(root / "expected-1").unlink()
|
||||
elif invalid == "duplicate":
|
||||
manifest["cases"].append(manifest["cases"][0])
|
||||
elif invalid == "deadline":
|
||||
manifest["job_wall_time_us"] = 1000
|
||||
else:
|
||||
manifest["run_limits"]["cpu_time_us"] = policy.run_ceiling.cpu_time_us + 1
|
||||
(root / "problem.json").write_text(json.dumps(manifest))
|
||||
store = ArtifactStore(tmp_path / "artifacts")
|
||||
|
||||
async def run():
|
||||
engine = create_async_engine(postgres_url)
|
||||
platform_role(engine)
|
||||
sessions = async_sessionmaker(engine, expire_on_commit=False)
|
||||
try:
|
||||
service = ProblemService(sessions, ArtifactRepository(store), policy)
|
||||
with pytest.raises(ValueError):
|
||||
await service.import_directory(root)
|
||||
async with sessions() as session:
|
||||
assert await session.scalar(select(func.count()).select_from(ProblemVersion)) == 0
|
||||
finally:
|
||||
await engine.dispose()
|
||||
|
||||
try:
|
||||
asyncio.run(run())
|
||||
finally:
|
||||
store.close()
|
||||
|
||||
|
||||
def test_concurrent_import_revisions_pagination_and_storage_failure_rollback(
|
||||
postgres_url, tmp_path
|
||||
):
|
||||
root = tmp_path / "import"
|
||||
_, policy = import_fixture(root)
|
||||
store = ArtifactStore(tmp_path / "artifacts")
|
||||
|
||||
async def run():
|
||||
engine = create_async_engine(postgres_url)
|
||||
platform_role(engine)
|
||||
sessions = async_sessionmaker(engine, expire_on_commit=False)
|
||||
try:
|
||||
service = ProblemService(sessions, ArtifactRepository(store), policy)
|
||||
versions = await asyncio.gather(
|
||||
service.import_directory(root), service.import_directory(root)
|
||||
)
|
||||
assert versions[0] != versions[1]
|
||||
assert sorted([(await service.detail(v)).revision for v in versions]) == [1, 2]
|
||||
other = tmp_path / "other"
|
||||
import_fixture(other, slug="another-problem")
|
||||
await service.import_directory(other)
|
||||
page = await service.list(limit=1)
|
||||
assert page.next_after == "another-problem" and len(page.items) == 1
|
||||
next_page = await service.list(after=page.next_after, limit=1)
|
||||
assert next_page.next_after is None and next_page.items[0].slug == "fixture-problem"
|
||||
limited = ArtifactStore(tmp_path / "limited", max_bytes=1, max_files=5)
|
||||
try:
|
||||
with pytest.raises(ArtifactError):
|
||||
await ProblemService(
|
||||
sessions, ArtifactRepository(limited), policy
|
||||
).import_directory(root)
|
||||
finally:
|
||||
limited.close()
|
||||
async with sessions() as session:
|
||||
assert await session.scalar(select(func.count()).select_from(ProblemVersion)) == 3
|
||||
finally:
|
||||
await engine.dispose()
|
||||
|
||||
try:
|
||||
asyncio.run(run())
|
||||
finally:
|
||||
store.close()
|
||||
@@ -1,222 +0,0 @@
|
||||
import asyncio
|
||||
import json
|
||||
import secrets
|
||||
from uuid import uuid4
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from sqlalchemy import func, select, text
|
||||
from sqlalchemy.exc import DBAPIError
|
||||
from sqlalchemy.ext.asyncio import create_async_engine
|
||||
from test_problems import import_fixture, platform_role
|
||||
|
||||
from kf_oj.api.app import create_app
|
||||
from kf_oj.contracts.judging import MAX_SOURCE_BYTES, Language
|
||||
from kf_oj.contracts.submissions import CreateSubmission
|
||||
from kf_oj.core.settings import Settings
|
||||
from kf_oj.infra.artifacts.models import Artifact
|
||||
from kf_oj.infra.messaging.models import Outbox
|
||||
from kf_oj.modules.submissions.models import Evaluation, Submission
|
||||
from kf_oj.modules.submissions.public import IdempotencyConflict, SubmissionUnavailable
|
||||
|
||||
|
||||
def test_submission_input_utf8_and_unknown_fields():
|
||||
base = dict(problem_version_id=str(uuid4()), language="c17", idempotency_key=str(uuid4()))
|
||||
CreateSubmission(**base, source="é" * (MAX_SOURCE_BYTES // 2))
|
||||
for extra in [
|
||||
dict(source="é" * (MAX_SOURCE_BYTES // 2 + 1)),
|
||||
dict(source="x", argv=[]),
|
||||
dict(source="x", user_id=str(uuid4())),
|
||||
dict(source=""),
|
||||
dict(source="\ud800"),
|
||||
]:
|
||||
with pytest.raises(ValueError):
|
||||
CreateSubmission(**base, **extra)
|
||||
|
||||
|
||||
async def setup_app(postgres_url, tmp_path):
|
||||
_, policy = import_fixture(tmp_path / "import")
|
||||
app = create_app(
|
||||
Settings(
|
||||
_env_file=None,
|
||||
environment="test",
|
||||
database_url=postgres_url,
|
||||
artifact_root=tmp_path / "artifacts",
|
||||
judge_admission=policy,
|
||||
)
|
||||
)
|
||||
engine = create_async_engine(postgres_url)
|
||||
platform_role(engine)
|
||||
# Test creates separate permission-constrained platform connections used by application.
|
||||
app_engine_build = __import__("kf_oj.api.app", fromlist=["build_engine"])
|
||||
return app, engine, app_engine_build
|
||||
|
||||
|
||||
def test_actual_concurrent_idempotency_version_and_outbox(postgres_url, tmp_path, monkeypatch):
|
||||
async def run():
|
||||
app, engine, mod = await setup_app(postgres_url, tmp_path)
|
||||
monkeypatch.setattr(mod, "build_engine", lambda _: engine)
|
||||
async with app.router.lifespan_context(app):
|
||||
account = await app.state.identity.create_account(
|
||||
"sub_fixture", secrets.token_urlsafe(24)
|
||||
)
|
||||
version = await app.state.problems.import_directory(tmp_path / "import")
|
||||
request = CreateSubmission(
|
||||
problem_version_id=version,
|
||||
language="c17",
|
||||
idempotency_key=uuid4(),
|
||||
source="synthetic source, never executed",
|
||||
)
|
||||
results = await asyncio.gather(
|
||||
*(app.state.submissions.create(account, request) for _ in range(8))
|
||||
)
|
||||
assert len({r.submission_id for r in results}) == 1
|
||||
async with app.state.sessions() as s:
|
||||
for model in [Submission, Evaluation, Outbox]:
|
||||
assert await s.scalar(select(func.count()).select_from(model)) == 1
|
||||
stored = await s.get(Submission, results[0].submission_id)
|
||||
data = await app.state.problems.artifacts.read(
|
||||
s, app.state.problems.artifacts.store.reference(request.source.encode())
|
||||
)
|
||||
assert data == request.source.encode() and stored.language == "c17"
|
||||
with pytest.raises(IdempotencyConflict):
|
||||
await app.state.submissions.create(
|
||||
account, request.model_copy(update={"source": "different source"})
|
||||
)
|
||||
admin = create_async_engine(postgres_url)
|
||||
async with admin.begin() as c:
|
||||
await c.execute(text("SET LOCAL ROLE kf_oj_coordinator"))
|
||||
await c.execute(
|
||||
text("UPDATE evaluations SET state='finished',verdict='accepted' WHERE id=:id"),
|
||||
{"id": results[0].evaluation_id},
|
||||
)
|
||||
await admin.dispose()
|
||||
assert (await app.state.submissions.create(account, request)).state == "finished"
|
||||
cpp = await app.state.submissions.create(
|
||||
account,
|
||||
request.model_copy(update={"language": Language.CPP20, "idempotency_key": uuid4()}),
|
||||
)
|
||||
assert cpp.submission_id != results[0].submission_id
|
||||
async with app.state.sessions.begin() as s:
|
||||
await s.execute(text("UPDATE problems SET enabled=false"))
|
||||
assert (await app.state.submissions.create(account, request)).submission_id == results[
|
||||
0
|
||||
].submission_id
|
||||
with pytest.raises(SubmissionUnavailable):
|
||||
await app.state.submissions.create(
|
||||
account, request.model_copy(update={"idempotency_key": uuid4()})
|
||||
)
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_actual_database_failure_rolls_back_entire_creation(postgres_url, tmp_path, monkeypatch):
|
||||
async def run():
|
||||
app, engine, mod = await setup_app(postgres_url, tmp_path)
|
||||
monkeypatch.setattr(mod, "build_engine", lambda _: engine)
|
||||
async with app.router.lifespan_context(app):
|
||||
account = await app.state.identity.create_account(
|
||||
"rollback_fixture", secrets.token_urlsafe(24)
|
||||
)
|
||||
version = await app.state.problems.import_directory(tmp_path / "import")
|
||||
admin = create_async_engine(postgres_url)
|
||||
async with admin.begin() as c:
|
||||
await c.execute(
|
||||
text(
|
||||
"""CREATE FUNCTION test_reject_outbox() RETURNS trigger
|
||||
LANGUAGE plpgsql AS $$
|
||||
BEGIN RAISE EXCEPTION 'synthetic rollback'; END $$"""
|
||||
)
|
||||
)
|
||||
await c.execute(
|
||||
text(
|
||||
"CREATE TRIGGER test_reject_outbox BEFORE INSERT ON outbox "
|
||||
"FOR EACH ROW EXECUTE FUNCTION test_reject_outbox()"
|
||||
)
|
||||
)
|
||||
request = CreateSubmission(
|
||||
problem_version_id=version,
|
||||
language="c17",
|
||||
idempotency_key=uuid4(),
|
||||
source="rollback synthetic bytes",
|
||||
)
|
||||
with pytest.raises(DBAPIError):
|
||||
await app.state.submissions.create(account, request)
|
||||
async with app.state.sessions() as s:
|
||||
for model in [Submission, Evaluation, Outbox]:
|
||||
assert await s.scalar(select(func.count()).select_from(model)) == 0
|
||||
ref = app.state.problems.artifacts.store.reference(request.source.encode())
|
||||
assert await s.get(Artifact, ref.digest) is None
|
||||
# Bytes can survive rollback as a collectable orphan; no durable reference exists.
|
||||
assert app.state.problems.artifacts.store.read(ref) == request.source.encode()
|
||||
async with admin.begin() as c:
|
||||
await c.execute(text("DROP TRIGGER test_reject_outbox ON outbox"))
|
||||
await c.execute(text("DROP FUNCTION test_reject_outbox()"))
|
||||
assert (await app.state.submissions.create(account, request)).state == "queued"
|
||||
await admin.dispose()
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_actual_authenticated_creation_request_boundaries(postgres_url, tmp_path, monkeypatch):
|
||||
async def run():
|
||||
app, engine, mod = await setup_app(postgres_url, tmp_path)
|
||||
monkeypatch.setattr(mod, "build_engine", lambda _: engine)
|
||||
async with app.router.lifespan_context(app):
|
||||
password = secrets.token_urlsafe(24)
|
||||
await app.state.identity.create_account("api_submission", password)
|
||||
version = await app.state.problems.import_directory(tmp_path / "import")
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="http://localhost:8000"
|
||||
) as client:
|
||||
req = dict(
|
||||
problem_version_id=str(version),
|
||||
language="c17",
|
||||
idempotency_key=str(uuid4()),
|
||||
source="synthetic ordinary data",
|
||||
)
|
||||
assert (await client.post("/api/v1/submissions", json=req)).status_code == 403
|
||||
login = await client.post(
|
||||
"/api/v1/auth/login",
|
||||
json=dict(username="api_submission", password=password),
|
||||
headers={"Origin": "http://localhost:8000"},
|
||||
)
|
||||
assert login.status_code == 200
|
||||
headers = {
|
||||
"Origin": "http://localhost:8000",
|
||||
"X-CSRF-Token": login.json()["csrf_token"],
|
||||
}
|
||||
assert (
|
||||
await client.post("/api/v1/submissions", json=req, headers=headers)
|
||||
).status_code == 201
|
||||
assert (
|
||||
await client.post(
|
||||
"/api/v1/submissions", json=req | {"user_id": str(uuid4())}, headers=headers
|
||||
)
|
||||
).status_code == 422
|
||||
assert (
|
||||
await client.post(
|
||||
"/api/v1/submissions", json=req | {"language": "python"}, headers=headers
|
||||
)
|
||||
).status_code == 422
|
||||
assert (
|
||||
await client.post(
|
||||
"/api/v1/submissions", json=req | {"source": "changed"}, headers=headers
|
||||
)
|
||||
).status_code == 409
|
||||
assert (
|
||||
await client.post(
|
||||
"/api/v1/submissions",
|
||||
content=b"x" * (6 * MAX_SOURCE_BYTES + 8193),
|
||||
headers=headers | {"Content-Type": "application/json"},
|
||||
)
|
||||
).status_code == 413
|
||||
# JSON escaping can exceed old1MiB body ceiling while valid source remains256KiB.
|
||||
req["idempotency_key"] = str(uuid4())
|
||||
req["source"] = "\x01" * MAX_SOURCE_BYTES
|
||||
assert len(json.dumps(req)) > 1024 * 1024
|
||||
assert (
|
||||
await client.post("/api/v1/submissions", json=req, headers=headers)
|
||||
).status_code == 201
|
||||
|
||||
asyncio.run(run())
|
||||
@@ -1,163 +0,0 @@
|
||||
"""Real PostgreSQL owner projection/pagination acceptance; no program execution."""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import secrets
|
||||
from pathlib import Path
|
||||
from uuid import uuid4
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.ext.asyncio import create_async_engine
|
||||
from test_submission_create import setup_app
|
||||
|
||||
from kf_oj.contracts.submissions import CreateSubmission
|
||||
from kf_oj.core.settings import Settings
|
||||
from kf_oj.modules.submissions.queries import cursor_decode
|
||||
|
||||
|
||||
def test_cursor_and_control_configuration_fail_closed():
|
||||
for value in ["garbage", "_" * 129, "", "eHx4"]:
|
||||
with pytest.raises(ValueError):
|
||||
cursor_decode(value)
|
||||
for url in ["http://localhost:8100", "https://127.0.0.1:8100", "http://127.0.0.1:8100/path"]:
|
||||
with pytest.raises(ValueError):
|
||||
Settings(_env_file=None, coordinator_url=url, coordinator_token="a" * 64)
|
||||
Settings(_env_file=None, coordinator_url="http://127.0.0.1:8100", coordinator_token="a" * 64)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("wire_version", [1, 2])
|
||||
def test_actual_owned_history_source_result_and_cancel(
|
||||
postgres_url, tmp_path, monkeypatch, wire_version
|
||||
):
|
||||
async def run():
|
||||
app, engine, mod = await setup_app(postgres_url, tmp_path)
|
||||
monkeypatch.setattr(mod, "build_engine", lambda _: engine)
|
||||
async with app.router.lifespan_context(app):
|
||||
passwords = [secrets.token_urlsafe(24), secrets.token_urlsafe(24)]
|
||||
owners = [
|
||||
await app.state.identity.create_account(name, password)
|
||||
for name, password in zip(["reader_one", "reader_two"], passwords, strict=True)
|
||||
]
|
||||
version = await app.state.problems.import_directory(tmp_path / "import")
|
||||
records = []
|
||||
for i in range(4):
|
||||
records.append(
|
||||
await app.state.submissions.create(
|
||||
owners[0],
|
||||
CreateSubmission(
|
||||
problem_version_id=version,
|
||||
language="c17",
|
||||
idempotency_key=uuid4(),
|
||||
source="synthetic private source " + str(i),
|
||||
),
|
||||
)
|
||||
)
|
||||
# Synthetic contract projection only, never claimed as a real judgement.
|
||||
fixture = Path(__file__).parents[2] / (
|
||||
"tests/fixtures/contracts/judge-result.valid.json"
|
||||
if wire_version == 1
|
||||
else "tests/fixtures/contracts-v2/judge-result.valid-c17.json"
|
||||
)
|
||||
report = json.loads(fixture.read_text())
|
||||
report["binding"]["evaluation_id"] = str(records[0].evaluation_id)
|
||||
if wire_version == 2:
|
||||
report["compilation"]["program"]["binding"]["evaluation_id"] = str(
|
||||
records[0].evaluation_id
|
||||
)
|
||||
admin = create_async_engine(postgres_url)
|
||||
async with admin.begin() as conn:
|
||||
await conn.execute(text("SET LOCAL ROLE kf_oj_coordinator"))
|
||||
await conn.execute(
|
||||
text(
|
||||
"UPDATE evaluations SET state='finished',verdict='accepted',"
|
||||
"final_report=CAST(:report AS jsonb) WHERE id=:id"
|
||||
),
|
||||
{"id": records[0].evaluation_id, "report": json.dumps(report)},
|
||||
)
|
||||
await admin.dispose()
|
||||
|
||||
class CancelProbe:
|
||||
accepted = True
|
||||
seen = []
|
||||
|
||||
async def cancel(self, evaluation_id):
|
||||
self.seen.append(evaluation_id)
|
||||
return self.accepted
|
||||
|
||||
probe = CancelProbe()
|
||||
app.state.submission_queries.cancellation = probe
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="http://localhost:8000"
|
||||
) as client:
|
||||
first = await client.post(
|
||||
"/api/v1/auth/login",
|
||||
json={"username": "reader_one", "password": passwords[0]},
|
||||
headers={"Origin": "http://localhost:8000"},
|
||||
)
|
||||
assert first.status_code == 200
|
||||
csrf = {
|
||||
"Origin": "http://localhost:8000",
|
||||
"X-CSRF-Token": first.json()["csrf_token"],
|
||||
}
|
||||
page = await client.get("/api/v1/submissions?limit=2")
|
||||
assert page.status_code == 200 and page.headers["Cache-Control"] == "no-store"
|
||||
after = page.json()["next_after"]
|
||||
# Concurrent insertion must not shift the cursor into repeats or omit old rows.
|
||||
await app.state.submissions.create(
|
||||
owners[0],
|
||||
CreateSubmission(
|
||||
problem_version_id=version,
|
||||
language="cpp20",
|
||||
idempotency_key=uuid4(),
|
||||
source="new synthetic private source",
|
||||
),
|
||||
)
|
||||
tail = await client.get("/api/v1/submissions", params={"limit": 2, "after": after})
|
||||
seen = [i["submission_id"] for i in page.json()["items"] + tail.json()["items"]]
|
||||
assert len(set(seen)) == 4 and set(seen) == {str(r.submission_id) for r in records}
|
||||
assert tail.json()["next_after"] is None
|
||||
base = f"/api/v1/submissions/{records[0].submission_id}"
|
||||
public = await client.get(base)
|
||||
assert public.status_code == 200 and public.json()["verdict"] == "accepted"
|
||||
projection = public.json()["result"]
|
||||
assert projection["cpu_time_us"] == report["run_cpu_time_us"]
|
||||
assert projection["memory_peak_bytes"] == report["run_memory_peak_bytes"]
|
||||
assert all(
|
||||
key not in public.text
|
||||
for key in ["case_id", "digest", "lease", "expected", "manifest", "program"]
|
||||
)
|
||||
source = await client.get(base + "/source")
|
||||
assert source.json()["source"] == "synthetic private source 0"
|
||||
waiting = await client.get(f"/api/v1/submissions/{records[1].submission_id}")
|
||||
assert waiting.json()["result"] is None and waiting.json()["verdict"] is None
|
||||
cancel = f"/api/v1/submissions/{records[1].submission_id}/cancel"
|
||||
assert (await client.post(cancel)).status_code == 403
|
||||
assert (await client.post(cancel, headers=csrf)).status_code == 202
|
||||
assert probe.seen == [records[1].evaluation_id]
|
||||
assert (await client.get(cancel.removesuffix("/cancel"))).json()[
|
||||
"state"
|
||||
] == "queued"
|
||||
probe.accepted = False
|
||||
assert (await client.post(cancel, headers=csrf)).status_code == 503
|
||||
assert (await client.get("/api/v1/submissions?after=invalid")).status_code == 422
|
||||
second = await client.post(
|
||||
"/api/v1/auth/login",
|
||||
json={"username": "reader_two", "password": passwords[1]},
|
||||
headers={"Origin": "http://localhost:8000"},
|
||||
)
|
||||
csrf["X-CSRF-Token"] = second.json()["csrf_token"]
|
||||
assert (await client.get("/api/v1/submissions")).json()["items"] == []
|
||||
for suffix in ["", "/source"]:
|
||||
response = await client.get(base + suffix)
|
||||
assert (
|
||||
response.status_code == 404
|
||||
and response.headers["Cache-Control"] == "no-store"
|
||||
)
|
||||
assert (await client.post(cancel, headers=csrf)).status_code == 404
|
||||
assert len(probe.seen) == 2
|
||||
assert (await client.post("/api/v1/auth/logout", headers=csrf)).status_code == 204
|
||||
assert (await client.get(base)).status_code == 401
|
||||
|
||||
asyncio.run(run())
|
||||
@@ -1,343 +0,0 @@
|
||||
import asyncio
|
||||
import hashlib
|
||||
import secrets
|
||||
from datetime import timedelta
|
||||
from pathlib import Path
|
||||
from uuid import uuid4
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from sqlalchemy import func, select, text, update
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
|
||||
from test_problems import import_fixture, platform_role
|
||||
|
||||
from kf_oj.api.app import create_app
|
||||
from kf_oj.contracts.judging import (
|
||||
JobManifest,
|
||||
LeaseBinding,
|
||||
manifest_bytes,
|
||||
)
|
||||
from kf_oj.core.settings import Settings
|
||||
from kf_oj.modules.identity.service import token_digest
|
||||
from kf_oj.modules.submissions.models import Evaluation, JudgeAttempt, Submission
|
||||
from kf_oj.modules.submissions.public import SubmissionUnavailable
|
||||
from kf_oj.modules.submissions.queries import SubmissionQueries
|
||||
|
||||
|
||||
def test_canonical_manifest_snapshot():
|
||||
root = Path(__file__).resolve().parents[2] / "tests/fixtures/contracts"
|
||||
manifest = JobManifest.model_validate_json((root / "judge-manifest.valid.json").read_bytes())
|
||||
assert manifest_bytes(manifest) == (root / "canonical-manifest.v1.txt").read_bytes()
|
||||
|
||||
|
||||
async def install_attempt(
|
||||
sessions, app, account, version_id, *, source=b"synthetic source", language="c17"
|
||||
):
|
||||
# Explicit controlled DB fixture, not a simulated Coordinator API or claimed execution.
|
||||
# E implements actual transactional claim; here the real B01 schema protects associations.
|
||||
submission_id, evaluation_id, attempt_id, worker_id = (uuid4() for _ in range(4))
|
||||
lease_token = secrets.token_hex(32)
|
||||
async with app.state.sessions.begin() as session:
|
||||
source_ref = await app.state.problems.artifacts.put(session, source)
|
||||
projected = await app.state.problems.for_submission(session, version_id)
|
||||
session.add(
|
||||
Submission(
|
||||
id=submission_id,
|
||||
account_id=account.account_id,
|
||||
problem_id=projected.problem_id,
|
||||
problem_version_id=version_id,
|
||||
language=language,
|
||||
source_digest=source_ref.digest,
|
||||
idempotency_key=uuid4(),
|
||||
request_digest="e" * 64,
|
||||
)
|
||||
)
|
||||
await session.flush()
|
||||
# Use column-level INSERT. ORM inserts optional execution fields, which the platform
|
||||
# deliberately cannot assign; application creation must use the same column projection.
|
||||
await session.execute(
|
||||
Evaluation.__table__.insert().values(
|
||||
id=evaluation_id, submission_id=submission_id, problem_version_id=version_id
|
||||
)
|
||||
)
|
||||
manifest = projected.manifest(evaluation_id, submission_id, language, source_ref)
|
||||
async with sessions.begin() as session:
|
||||
await session.execute(text("SET LOCAL ROLE kf_oj_coordinator"))
|
||||
now = await session.scalar(select(func.now()))
|
||||
session.add(
|
||||
JudgeAttempt(
|
||||
id=attempt_id,
|
||||
evaluation_id=evaluation_id,
|
||||
worker_id=worker_id,
|
||||
number=1,
|
||||
fencing_version=1,
|
||||
lease_token_hash=token_digest(lease_token),
|
||||
expires_at=now + timedelta(minutes=10),
|
||||
manifest=manifest.model_dump(mode="json"),
|
||||
manifest_digest=hashlib.sha256(manifest_bytes(manifest)).hexdigest(),
|
||||
)
|
||||
)
|
||||
await session.flush()
|
||||
await session.execute(
|
||||
update(Evaluation)
|
||||
.where(Evaluation.id == evaluation_id)
|
||||
.values(
|
||||
current_attempt_id=attempt_id, state="claimed", attempts_count=1, fencing_version=1
|
||||
)
|
||||
)
|
||||
return (
|
||||
LeaseBinding(
|
||||
evaluation_id=evaluation_id,
|
||||
attempt_id=attempt_id,
|
||||
worker_id=worker_id,
|
||||
fencing_version=1,
|
||||
),
|
||||
lease_token,
|
||||
manifest,
|
||||
)
|
||||
|
||||
|
||||
def test_real_two_user_two_version_manifest_and_artifact_authorization(postgres_url, tmp_path):
|
||||
root = tmp_path / "import"
|
||||
_, policy = import_fixture(root)
|
||||
|
||||
async def run():
|
||||
owner = create_async_engine(postgres_url)
|
||||
owner_sessions = async_sessionmaker(owner, expire_on_commit=False)
|
||||
app = create_app(
|
||||
Settings(
|
||||
_env_file=None,
|
||||
environment="test",
|
||||
database_url=postgres_url,
|
||||
artifact_root=tmp_path / "artifacts",
|
||||
judge_admission=policy,
|
||||
)
|
||||
)
|
||||
try:
|
||||
async with app.router.lifespan_context(app):
|
||||
platform_role(app.state.sessions.kw["bind"])
|
||||
password = secrets.token_hex(16)
|
||||
first_user = await app.state.identity.create_account("first_user", password)
|
||||
second_password = secrets.token_hex(16)
|
||||
second_user = await app.state.identity.create_account(
|
||||
"second_user", second_password
|
||||
)
|
||||
old_version = await app.state.problems.import_directory(root)
|
||||
first, token, original = await install_attempt(
|
||||
owner_sessions, app, first_user, old_version
|
||||
)
|
||||
(root / "expected-1").write_bytes(b"different hidden version\x00")
|
||||
new_version = await app.state.problems.import_directory(root)
|
||||
second, other_token, other_manifest = await install_attempt(
|
||||
owner_sessions, app, second_user, new_version, source=b"other user's source"
|
||||
)
|
||||
queries = SubmissionQueries(app.state.sessions, app.state.problems.artifacts)
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="http://localhost:8000"
|
||||
) as client:
|
||||
login = await client.post(
|
||||
"/api/v1/auth/login",
|
||||
headers={"Origin": "http://localhost:8000"},
|
||||
json={"username": "first_user", "password": password},
|
||||
)
|
||||
assert login.status_code == 200
|
||||
authenticated = await app.state.identity.authenticate(
|
||||
client.cookies.get("kf_session")
|
||||
)
|
||||
assert (
|
||||
await queries.owned_source(authenticated, original.submission_id)
|
||||
== b"synthetic source"
|
||||
)
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="http://localhost:8000"
|
||||
) as other_client:
|
||||
other_login = await other_client.post(
|
||||
"/api/v1/auth/login",
|
||||
headers={"Origin": "http://localhost:8000"},
|
||||
json={"username": "second_user", "password": second_password},
|
||||
)
|
||||
assert other_login.status_code == 200
|
||||
other_principal = await app.state.identity.authenticate(
|
||||
other_client.cookies.get("kf_session")
|
||||
)
|
||||
assert other_principal.account_id == second_user.account_id
|
||||
with pytest.raises(SubmissionUnavailable):
|
||||
await queries.owned_source(other_principal, original.submission_id)
|
||||
path = (
|
||||
f"/api/v1/internal/evaluations/{first.evaluation_id}"
|
||||
f"/attempts/{first.attempt_id}"
|
||||
)
|
||||
headers = {
|
||||
"Authorization": "Bearer " + token,
|
||||
"X-Worker-ID": str(first.worker_id),
|
||||
"X-Fencing-Version": "1",
|
||||
}
|
||||
assert (
|
||||
await client.get(path + "/manifest", headers=headers)
|
||||
).content == manifest_bytes(original)
|
||||
# Changing the active problem version did not change an existing evaluation.
|
||||
async with owner_sessions() as s:
|
||||
evaluation = await s.get(Evaluation, first.evaluation_id)
|
||||
assert evaluation.problem_version_id == old_version
|
||||
assert (
|
||||
await client.get(
|
||||
path + "/manifest",
|
||||
headers={k: v for k, v in headers.items() if k != "Authorization"},
|
||||
)
|
||||
).status_code == 401
|
||||
assert (
|
||||
await client.get(
|
||||
path + "/manifest",
|
||||
headers=headers | {"Authorization": "Bearer " + other_token},
|
||||
)
|
||||
).status_code == 404
|
||||
assert (
|
||||
await client.get(
|
||||
path + "/manifest", headers=headers | {"X-Worker-ID": str(uuid4())}
|
||||
)
|
||||
).status_code == 404
|
||||
assert (
|
||||
await client.get(
|
||||
path + "/manifest", headers=headers | {"X-Fencing-Version": "2"}
|
||||
)
|
||||
).status_code == 404
|
||||
for reference in (
|
||||
original.source,
|
||||
original.cases[0].input,
|
||||
original.cases[0].expected,
|
||||
):
|
||||
response = await client.get(
|
||||
path + "/artifacts/" + reference.digest, headers=headers
|
||||
)
|
||||
assert response.status_code == 200
|
||||
assert len(response.content) == reference.bytes
|
||||
assert hashlib.sha256(response.content).hexdigest() == reference.digest
|
||||
assert response.headers["cache-control"] == "no-store"
|
||||
assert (
|
||||
await client.get(
|
||||
path + "/artifacts/" + other_manifest.source.digest, headers=headers
|
||||
)
|
||||
).status_code == 404
|
||||
assert (
|
||||
await client.get(
|
||||
path + "/artifacts/" + other_manifest.cases[0].expected.digest,
|
||||
headers=headers,
|
||||
)
|
||||
).status_code == 404
|
||||
assert (
|
||||
await client.get(path + "/artifacts/" + "f" * 64, headers=headers)
|
||||
).status_code == 404
|
||||
second_path = (
|
||||
f"/api/v1/internal/evaluations/{second.evaluation_id}"
|
||||
f"/attempts/{second.attempt_id}"
|
||||
)
|
||||
assert (
|
||||
await client.get(second_path + "/manifest", headers=headers)
|
||||
).status_code == 404
|
||||
assert (
|
||||
await client.get(
|
||||
path + "/manifest",
|
||||
headers={
|
||||
"Authorization": "Bearer " + client.cookies.get("kf_session"),
|
||||
"X-Worker-ID": str(first.worker_id),
|
||||
"X-Fencing-Version": "1",
|
||||
},
|
||||
)
|
||||
).status_code == 404
|
||||
finally:
|
||||
await owner.dispose()
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"invalid", ["expired", "revoked", "cancel_requested", "terminal", "old_attempt"]
|
||||
)
|
||||
def test_expired_revoked_cancelled_terminal_or_old_attempt_rejected(
|
||||
postgres_url, tmp_path, invalid
|
||||
):
|
||||
root = tmp_path / "import"
|
||||
_, policy = import_fixture(root)
|
||||
|
||||
async def run():
|
||||
owner = create_async_engine(postgres_url)
|
||||
sessions = async_sessionmaker(owner, expire_on_commit=False)
|
||||
app = create_app(
|
||||
Settings(
|
||||
_env_file=None,
|
||||
environment="test",
|
||||
database_url=postgres_url,
|
||||
artifact_root=tmp_path / "artifacts",
|
||||
judge_admission=policy,
|
||||
)
|
||||
)
|
||||
try:
|
||||
async with app.router.lifespan_context(app):
|
||||
platform_role(app.state.sessions.kw["bind"])
|
||||
account = await app.state.identity.create_account(
|
||||
"fixture_user", secrets.token_hex(16)
|
||||
)
|
||||
version = await app.state.problems.import_directory(root)
|
||||
binding, token, manifest = await install_attempt(sessions, app, account, version)
|
||||
async with sessions.begin() as s:
|
||||
await s.execute(text("SET LOCAL ROLE kf_oj_coordinator"))
|
||||
if invalid == "expired":
|
||||
await s.execute(
|
||||
update(JudgeAttempt).values(
|
||||
expires_at=func.now() - timedelta(seconds=1)
|
||||
)
|
||||
)
|
||||
elif invalid == "revoked":
|
||||
await s.execute(update(JudgeAttempt).values(revoked_at=func.now()))
|
||||
elif invalid == "cancel_requested":
|
||||
await s.execute(update(Evaluation).values(cancel_requested_at=func.now()))
|
||||
elif invalid == "terminal":
|
||||
await s.execute(update(Evaluation).values(state="finished"))
|
||||
else:
|
||||
new_attempt = uuid4()
|
||||
now = await s.scalar(select(func.now()))
|
||||
s.add(
|
||||
JudgeAttempt(
|
||||
id=new_attempt,
|
||||
evaluation_id=binding.evaluation_id,
|
||||
worker_id=uuid4(),
|
||||
number=2,
|
||||
fencing_version=2,
|
||||
lease_token_hash="e" * 64,
|
||||
expires_at=now + timedelta(minutes=1),
|
||||
manifest=manifest.model_dump(mode="json"),
|
||||
manifest_digest=hashlib.sha256(
|
||||
manifest_bytes(manifest)
|
||||
).hexdigest(),
|
||||
)
|
||||
)
|
||||
await s.flush()
|
||||
await s.execute(
|
||||
update(Evaluation).values(
|
||||
current_attempt_id=new_attempt, fencing_version=2, attempts_count=2
|
||||
)
|
||||
)
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="http://test"
|
||||
) as client:
|
||||
prefix = (
|
||||
f"/api/v1/internal/evaluations/{binding.evaluation_id}"
|
||||
f"/attempts/{binding.attempt_id}"
|
||||
)
|
||||
headers = {
|
||||
"Authorization": "Bearer " + token,
|
||||
"X-Worker-ID": str(binding.worker_id),
|
||||
"X-Fencing-Version": "1",
|
||||
}
|
||||
assert (
|
||||
await client.get(prefix + "/manifest", headers=headers)
|
||||
).status_code == 404
|
||||
assert (
|
||||
await client.get(
|
||||
prefix + "/artifacts/" + manifest.source.digest, headers=headers
|
||||
)
|
||||
).status_code == 404
|
||||
finally:
|
||||
await owner.dispose()
|
||||
|
||||
asyncio.run(run())
|
||||
Generated
+2
-72
@@ -73,40 +73,6 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/12/b8/4bd346e22b28902df4d651910f5242c28d84e4a5c2435ca5c3f797ed7e2e/anyio-4.15.1-py3-none-any.whl", hash = "sha256:6152fdbbf9a77fdec97731721bebf7c4c44f7c29b424b0065826173efc7ed101", size = 132079, upload-time = "2026-09-05T10:42:37.923Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "argon2-cffi"
|
||||
version = "25.1.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "argon2-cffi-bindings" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/0e/89/ce5af8a7d472a67cc819d5d998aa8c82c5d860608c4db9f46f1162d7dab9/argon2_cffi-25.1.0.tar.gz", hash = "sha256:694ae5cc8a42f4c4e2bf2ca0e64e51e23a040c6a517a85074683d3959e1346c1", size = 45706, upload-time = "2025-06-03T06:55:32.073Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/4f/d3/a8b22fa575b297cd6e3e3b0155c7e25db170edf1c74783d6a31a2490b8d9/argon2_cffi-25.1.0-py3-none-any.whl", hash = "sha256:fdc8b074db390fccb6eb4a3604ae7231f219aa669a2652e0f20e16ba513d5741", size = 14657, upload-time = "2025-06-03T06:55:30.804Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "argon2-cffi-bindings"
|
||||
version = "26.1.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "cffi" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/0b/43/bb8b6e8708d49a5ab36781333af092d9f483b198a2710d01281204640055/argon2_cffi_bindings-26.1.0.tar.gz", hash = "sha256:63505c71542a44b68b1e38060450fb006404170da375feb31af153e7f9c6205d", size = 1790807, upload-time = "2026-08-20T07:44:22.492Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/e7/d2/0ae991f1b2181e5be49007c574710a800ad36c2978683addb3e67c474e55/argon2_cffi_bindings-26.1.0-cp310-abi3-macosx_11_0_arm64.whl", hash = "sha256:21ca0396fe5ec995dd54431c32698189666f9224810acfa752e50d2bd94d9df2", size = 25521, upload-time = "2026-08-20T07:32:43.019Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7e/e4/ad91d8297638aa2258aad4501c306aca99480dfe76ccd638173fa3702db9/argon2_cffi_bindings-26.1.0-cp310-abi3-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:78de2d65e0b9ea7ce9d1b1c3e87297b2d7305a02c266ee2a2d6910daddd7ee69", size = 27177, upload-time = "2026-08-20T07:32:44.158Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/6f/86/5363df11b86d02cf3662208e7406496327649cc90eb365bf6f4e8a54a41f/argon2_cffi_bindings-26.1.0-cp310-abi3-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:27f1821903e2ceadcb88ec2b45ef190897b7682449c772f4d9b53e42c520cf29", size = 26597, upload-time = "2026-08-20T07:32:45.172Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f4/b5/a14dcc592652347dad23ee93b278a4da5d2a25c9ed3ebd10d68eea823a4f/argon2_cffi_bindings-26.1.0-cp310-abi3-manylinux_2_34_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:d88e5f7e60f28ae0b0cc6b2f16c43e87cd642a196a86f85e0d8bb6fe016fc16d", size = 27403, upload-time = "2026-08-20T07:32:46.13Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b3/81/b4a20d4902af7f796390bf9245ff83c5217dfa7367efa1d14986956c482b/argon2_cffi_bindings-26.1.0-cp310-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:34b7d9c24a4165a2c61cc8ae11d44d48c9ce2830fb536cb7914e11fdd9962728", size = 27132, upload-time = "2026-08-20T07:32:47.13Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7e/1b/c8de358af07b1c490e0fcb863ef98e46ddb486e45567aca5a60bd68d9daa/argon2_cffi_bindings-26.1.0-cp310-abi3-musllinux_1_2_riscv64.whl", hash = "sha256:224865cbbcb7a2bd1356741dff12b0134df726b6d44bb7b500df8e303cbd9e81", size = 27588, upload-time = "2026-08-20T07:32:48.087Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/48/2f/7ee62a6e79f9309f9d9982d301b22a00010adb580c05c8109b94d7b33de0/argon2_cffi_bindings-26.1.0-cp310-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:ffff613aaa9ce6236766e2fc6dc560bb5abde7a2e2416e3db1f9ae395a2b4dd4", size = 26785, upload-time = "2026-08-20T07:32:48.977Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e9/10/960d0ee93d4897741bcaf4799c697dae2d81499f66fd1ed042a7dd54c1f4/argon2_cffi_bindings-26.1.0-cp310-abi3-win32.whl", hash = "sha256:a86c069c91a747a2c4e5c51473590aeb48172fff9b2130d23729a42d98665ecb", size = 23898, upload-time = "2026-08-20T07:32:50.114Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/6d/3a/0cc14a05810e6add9bce5e87693334baa2222de5f647fa31781885b6573f/argon2_cffi_bindings-26.1.0-cp310-abi3-win_amd64.whl", hash = "sha256:2c36ff87b5dfaa477d0bd51e9d7f6abdae7c8955d2983c97419085d842154b3e", size = 25730, upload-time = "2026-08-20T07:32:51.091Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/4e/db/d83cf2af140547f0b9cdaece05b2dc2dcbf991be4667331d073eff771435/argon2_cffi_bindings-26.1.0-cp310-abi3-win_arm64.whl", hash = "sha256:f9c4420a7a864fe1b86ce35befc95b8e39fb852493b81cf798671ddc265de638", size = 24478, upload-time = "2026-08-20T07:32:52.111Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/bb/5f/f652055e18d2627e2eed94c7f31a792127cfe38df786635395d742321674/argon2_cffi_bindings-26.1.0-cp313-cp313-pyemscripten_2025_0_wasm32.whl", hash = "sha256:af11ac37a7c53dc16cb7950a6190851b0870fe218b6c60c0bb7ac355234e3083", size = 15434, upload-time = "2026-08-20T07:32:53.143Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "asyncpg"
|
||||
version = "0.31.0"
|
||||
@@ -141,31 +107,6 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/0b/a7/71ac2cff56fec219ed242bb11b8efb69fcc4bec75db06fb7bfe35de520e6/certifi-2026.7.22-py3-none-any.whl", hash = "sha256:62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775", size = 136983, upload-time = "2026-07-22T03:35:11.276Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "cffi"
|
||||
version = "2.1.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "pycparser", marker = "implementation_name != 'PyPy'" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/9e/ef/008a1939e372c06329a3fce4279c02f328488f3526744906eeec3da7ad5f/cffi-2.1.1.tar.gz", hash = "sha256:dd31f52ea1086513bb9df30f8fcee9b8918323ae067a3d5b78bc826a000712be", size = 530807, upload-time = "2026-08-03T21:21:18.939Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/9d/f4/035513d4117049066b4779dc3b7c0c0fdad175fa13731c9f4003f1cd1478/cffi-2.1.1-cp313-cp313-ios_13_0_arm64_iphoneos.whl", hash = "sha256:b5bdfd1c873d4e093aabc0ca84c4ca6dbc4f752afb5c86f146d9742580c9da2e", size = 194248, upload-time = "2026-08-03T21:19:59.399Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/76/af/2aeb4dbb5fc41a04161ae9ff1518de7cec08e164f44a8ce6a4cf7fd2cd1d/cffi-2.1.1-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:31348097ff5bbe827ccc41795d4dd099d9f0625e7def00ee653c137a490c2a6c", size = 196908, upload-time = "2026-08-03T21:20:00.746Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a7/46/2e5fdde8555706dd98139a910ca11be02809f3f605ce956f655d0214e100/cffi-2.1.1-cp313-cp313-macosx_10_15_x86_64.whl", hash = "sha256:9d2055050ea716bd38b7f7f1579c275386646b4894c155a3e2f3cd62ed41b7c6", size = 184805, upload-time = "2026-08-03T21:20:02.02Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/55/41/4c7042f317b9217502988f0873af87e16ad606dc20f84e546e3e6ce9764c/cffi-2.1.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:19ee6127ee34de7d83ce3d371ebc5ed91addbdcc39f9ab15ce4eb35a4e534971", size = 184764, upload-time = "2026-08-03T21:20:03.141Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/43/1f/1c3d90d91811c8f86ced9ed637956c54bfe5b79ca98fe976d7f8c8979f6b/cffi-2.1.1-cp313-cp313-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:6a8dddef476fab96d066d578fc88526767b836ab5ab21754e1d5bf3879c31c7c", size = 214722, upload-time = "2026-08-03T21:20:04.377Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/37/6f/3b5ce4c3b2192d250f04908f2bfd91ef34552ec8f7716a5d4abdb8d67bb2/cffi-2.1.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:f16c709686a78c727bbbf059f92b0bf41c6fc60deec706d2dc19f529175a6125", size = 222369, upload-time = "2026-08-03T21:20:05.544Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/02/10/4b3c75dde3d9663c9e02ba05c2668b954f671d4bbe346413ca8c696b295a/cffi-2.1.1-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:fcd22650c908d7b7da162bbfaab594a1227a15d1643a98c68b122ac642fa2264", size = 210175, upload-time = "2026-08-03T21:20:06.75Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/df/62/14f74b9543e605d17701dc797b815958b8bb70b7624ce1b832ddad48ed6c/cffi-2.1.1-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:aa9511c62d14da7aacc9b4bf51f3f697a621e83b2d6919008243c3aad168eea3", size = 208670, upload-time = "2026-08-03T21:20:08.04Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/95/95/86342356ff5953b3fb06f7ef7c5bee212d45e770abc7218d451b9148313c/cffi-2.1.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:a931079504ecc49efed7744c476a5c343a92fabf66dec2db95edb1b2fdc770e2", size = 221824, upload-time = "2026-08-03T21:20:09.274Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/eb/ff/7b3429ff53aafe931ed8a5fc69f481bbef7ba6de87ddcbb63d08f483f613/cffi-2.1.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a2d7755bef5a12ed488f4ef1f1b69ee9191d7396083b755a5d2295f6edb4768b", size = 225148, upload-time = "2026-08-03T21:20:10.7Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/34/34/a95870b9221e09cf4f2ce3178b1a210abdfe63a1bd357da940418d7b8d15/cffi-2.1.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:e0bcb7e0f677f543555d2adff3bf19c05f66cdb4796e5ff602442ab2fe3c4ef7", size = 223564, upload-time = "2026-08-03T21:20:12.165Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/70/ea/839b50531021a647fb5e929f72cf97bc1ff702b5472166164b5b6e76b851/cffi-2.1.1-cp313-cp313-win32.whl", hash = "sha256:334644fbac4eff73d985a17a91226df55d0f394160c4cfb880e084c8f7161cac", size = 175263, upload-time = "2026-08-03T21:20:13.559Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/60/a6/8b149b2c3f2e11aaa1618ef64500b45f50f22c57a977a4dff1aff1f91042/cffi-2.1.1-cp313-cp313-win_amd64.whl", hash = "sha256:1aa5645c30469b09530c4ebca77ebf8f17618293c58f8549cb1a543a50236e7d", size = 185688, upload-time = "2026-08-03T21:20:14.69Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/01/9a/11f687cb39d6a3504060d5242f04f48c735afb4d3d533958a20594890cb2/cffi-2.1.1-cp313-cp313-win_arm64.whl", hash = "sha256:63bbfd5ded17c4840ac07cd8f1c21ba9d9708141f840b324f422f41b207e3973", size = 180078, upload-time = "2026-08-03T21:20:15.917Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "click"
|
||||
version = "8.5.0"
|
||||
@@ -308,10 +249,8 @@ source = { editable = "." }
|
||||
dependencies = [
|
||||
{ name = "aio-pika" },
|
||||
{ name = "alembic" },
|
||||
{ name = "argon2-cffi" },
|
||||
{ name = "asyncpg" },
|
||||
{ name = "fastapi" },
|
||||
{ name = "httpx" },
|
||||
{ name = "pydantic" },
|
||||
{ name = "pydantic-settings" },
|
||||
{ name = "sqlalchemy", extra = ["asyncio"] },
|
||||
@@ -320,6 +259,7 @@ dependencies = [
|
||||
|
||||
[package.dev-dependencies]
|
||||
dev = [
|
||||
{ name = "httpx" },
|
||||
{ name = "jsonschema" },
|
||||
{ name = "pytest" },
|
||||
{ name = "ruff" },
|
||||
@@ -329,10 +269,8 @@ dev = [
|
||||
requires-dist = [
|
||||
{ name = "aio-pika", specifier = ">=9.5,<10" },
|
||||
{ name = "alembic", specifier = ">=1.15,<2" },
|
||||
{ name = "argon2-cffi", specifier = ">=25.1,<26" },
|
||||
{ name = "asyncpg", specifier = ">=0.30,<1" },
|
||||
{ name = "fastapi", specifier = ">=0.115,<1" },
|
||||
{ name = "httpx", specifier = ">=0.28,<1" },
|
||||
{ name = "pydantic", specifier = ">=2.10,<3" },
|
||||
{ name = "pydantic-settings", specifier = ">=2.7,<3" },
|
||||
{ name = "sqlalchemy", extras = ["asyncio"], specifier = ">=2.0,<3" },
|
||||
@@ -341,6 +279,7 @@ requires-dist = [
|
||||
|
||||
[package.metadata.requires-dev]
|
||||
dev = [
|
||||
{ name = "httpx", specifier = ">=0.28,<1" },
|
||||
{ name = "jsonschema", specifier = ">=4.23,<5" },
|
||||
{ name = "pytest", specifier = ">=8.3,<10" },
|
||||
{ name = "ruff", specifier = ">=0.11,<1" },
|
||||
@@ -483,15 +422,6 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/f5/cd/785c64ed382f3f04201870267b02783f63b4678c2acfddc177a3ebcc2727/propcache-0.5.4-py3-none-any.whl", hash = "sha256:62c60aec739ed00124573cce1178138fd690c7676352d67a37328c1cf51d7468", size = 16338, upload-time = "2026-09-16T00:17:13.106Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pycparser"
|
||||
version = "3.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/ac/d3/eb1d3bc30dda12f7e69640ae2ac8cb10240b71fb73024ad528b7d2ae73da/pycparser-3.1.tar.gz", hash = "sha256:b3fc6dec06a8b2fefa0ed4ff92285306a5e3be9987bc5603c9edbdc4e492418f", size = 105946, upload-time = "2026-10-08T12:26:34.349Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/99/ce/b3ae9ee0324d991c860187be2a6ee436d27a3d02397eaddbb101ec901f3d/pycparser-3.1-py3-none-any.whl", hash = "sha256:f09d358c840bd147b79e55f2bc494f18ea869dc897f5852a8f5766b74f787882", size = 48709, upload-time = "2026-10-08T12:26:32.97Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pydantic"
|
||||
version = "2.13.5"
|
||||
|
||||
+1
-27
@@ -9,32 +9,6 @@
|
||||
|
||||
`make contracts` 从 Python 模型生成 Schema/OpenAPI,并生成前端 API 类型。
|
||||
`make check` 检查快照未过期及两种语言对样例的接受/拒绝一致。
|
||||
执行器私有规格与 FD 协议属于 Sandbox;平台不接收 argv、宿主路径或执行镜像路径。
|
||||
|
||||
普通题提交协议现有 `judge-manifest`、`judge-lease`、`judge-stage`、`judge-result`
|
||||
和本地可信 `judge-admission` v1 模型。清单绑定源码、不可变题目版本、按序工件摘要、
|
||||
C17/C++20、固定 GNU 工具链版本及编译/运行预算;租约绑定 evaluation/attempt/Worker
|
||||
与 fencing 版本。Root 权限、执行和 DB 状态转换接口仍由各自服务实现,解析清单或租约
|
||||
不授予执行权。
|
||||
|
||||
Python/Rust `decode_wire` 在解析前限制 256 KiB UTF-8 JSON,拒绝重复字段与非法数据,
|
||||
传输边界错误不回显输入。令牌及编译诊断的默认调试输出隐藏内容。`serde_json` 的运行期
|
||||
直接依赖用于此有界解码,不引入 IO、网络或数据库。
|
||||
|
||||
Schema 描述结构;模型另外核对规范 UUID、严格整数(拒绝 `1.0`/布尔值)、工件总大小、
|
||||
重复测试点、预算相容、诊断 UTF-8 字节数、编译状态/产物、清理失败和实际结果汇总。
|
||||
JSON Schema 的 integer 按数值判断,允许 `1.0`;不能替代协议解码。共享样例中的
|
||||
`invalid-semantic-*` 结构合法但被两种语言模型拒绝,其他非法样例同时验证结构拒绝。
|
||||
样例全为合成元数据;其中令牌、摘要不是已签发凭证或实际隐藏测试数据。
|
||||
|
||||
Coordinator 必须在事务锁内核对当前租约及状态规则,并将结果和授权清单比较:
|
||||
完整用例次序、数量和清单摘要不能只靠报告自身校验。已提交报告的幂等确认与过期租约
|
||||
拒绝分别处理,旧 attempt 无权改写结果。`cancelling` 在实际停止确认前不得转为
|
||||
`cancelled`;清理失败必须保留独立故障,缺失计量始终为 null。
|
||||
执行请求/报告、评测状态机、租约与结果提交契约在 JS-01/DR-01 中逐项补充。
|
||||
|
||||
生成器 `export.py` 是开发工具,允许读取 API 组合层;运行期的契约模型禁止依赖业务模块。
|
||||
# 任务清单字节身份
|
||||
|
||||
Python/Rust 的 `manifest_bytes` 输出同一 UTF-8 JSON:递归排序对象键,无额外空白/换行。
|
||||
清单摘要是这些确切字节的 SHA-256,不能使用数据库 JSONB 的输出顺序或任意 prettify 后的字节。
|
||||
共享 `canonical-manifest.v1.txt` 为合成示例的字节验收,平台内部响应复用这些序列化字节。
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,169 +0,0 @@
|
||||
{
|
||||
"$comment": "Structural schema; also run versioned model validation for cross-field budgets, result consistency, UUID canonical form and UTF-8 byte limits.",
|
||||
"$defs": {
|
||||
"RunLimits": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"cpu_time_us": {
|
||||
"maximum": 60000000,
|
||||
"minimum": 1000,
|
||||
"title": "Cpu Time Us",
|
||||
"type": "integer"
|
||||
},
|
||||
"file_bytes": {
|
||||
"maximum": 67108864,
|
||||
"minimum": 1,
|
||||
"title": "File Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"memory_bytes": {
|
||||
"maximum": 2147483648,
|
||||
"minimum": 1,
|
||||
"title": "Memory Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"open_files": {
|
||||
"maximum": 1024,
|
||||
"minimum": 3,
|
||||
"title": "Open Files",
|
||||
"type": "integer"
|
||||
},
|
||||
"processes": {
|
||||
"maximum": 128,
|
||||
"minimum": 1,
|
||||
"title": "Processes",
|
||||
"type": "integer"
|
||||
},
|
||||
"stack_bytes": {
|
||||
"maximum": 268435456,
|
||||
"minimum": 1,
|
||||
"title": "Stack Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"stderr_bytes": {
|
||||
"maximum": 16777216,
|
||||
"minimum": 1,
|
||||
"title": "Stderr Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"stdout_bytes": {
|
||||
"maximum": 16777216,
|
||||
"minimum": 1,
|
||||
"title": "Stdout Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"wall_time_us": {
|
||||
"maximum": 120000000,
|
||||
"minimum": 1000,
|
||||
"title": "Wall Time Us",
|
||||
"type": "integer"
|
||||
},
|
||||
"workdir_bytes": {
|
||||
"maximum": 268435456,
|
||||
"minimum": 1,
|
||||
"title": "Workdir Bytes",
|
||||
"type": "integer"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"cpu_time_us",
|
||||
"wall_time_us",
|
||||
"memory_bytes",
|
||||
"stack_bytes",
|
||||
"processes",
|
||||
"open_files",
|
||||
"stdout_bytes",
|
||||
"stderr_bytes",
|
||||
"file_bytes",
|
||||
"workdir_bytes"
|
||||
],
|
||||
"title": "RunLimits",
|
||||
"type": "object"
|
||||
},
|
||||
"Toolchain": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"image_kind": {
|
||||
"const": "linux_gnu_cxx_x86_64_v1",
|
||||
"title": "Image Kind",
|
||||
"type": "string"
|
||||
},
|
||||
"image_revision": {
|
||||
"maximum": 4294967295,
|
||||
"minimum": 1,
|
||||
"title": "Image Revision",
|
||||
"type": "integer"
|
||||
},
|
||||
"toolchain_id": {
|
||||
"pattern": "^[a-z0-9][a-z0-9_-]{0,63}$",
|
||||
"title": "Toolchain Id",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"toolchain_id",
|
||||
"image_kind",
|
||||
"image_revision"
|
||||
],
|
||||
"title": "Toolchain",
|
||||
"type": "object"
|
||||
}
|
||||
},
|
||||
"$id": "urn:kf-oj:judge-admission:v1",
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"additionalProperties": false,
|
||||
"description": "Trusted node policy. A manifest never supplies or replaces this policy.",
|
||||
"properties": {
|
||||
"compile_ceiling": {
|
||||
"$ref": "#/$defs/RunLimits"
|
||||
},
|
||||
"max_concurrent_jobs": {
|
||||
"default": 1,
|
||||
"maximum": 64,
|
||||
"minimum": 1,
|
||||
"title": "Max Concurrent Jobs",
|
||||
"type": "integer"
|
||||
},
|
||||
"max_job_wall_time_us": {
|
||||
"maximum": 3600000000,
|
||||
"minimum": 1000,
|
||||
"title": "Max Job Wall Time Us",
|
||||
"type": "integer"
|
||||
},
|
||||
"max_package_bytes": {
|
||||
"maximum": 67108864,
|
||||
"minimum": 1,
|
||||
"title": "Max Package Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"max_retries": {
|
||||
"default": 2,
|
||||
"maximum": 4,
|
||||
"minimum": 0,
|
||||
"title": "Max Retries",
|
||||
"type": "integer"
|
||||
},
|
||||
"prefetch": {
|
||||
"default": 1,
|
||||
"maximum": 64,
|
||||
"minimum": 1,
|
||||
"title": "Prefetch",
|
||||
"type": "integer"
|
||||
},
|
||||
"run_ceiling": {
|
||||
"$ref": "#/$defs/RunLimits"
|
||||
},
|
||||
"toolchain": {
|
||||
"$ref": "#/$defs/Toolchain"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"max_job_wall_time_us",
|
||||
"max_package_bytes",
|
||||
"toolchain",
|
||||
"compile_ceiling",
|
||||
"run_ceiling"
|
||||
],
|
||||
"title": "AdmissionPolicy",
|
||||
"type": "object"
|
||||
}
|
||||
@@ -1,226 +0,0 @@
|
||||
{
|
||||
"$comment": "Multi-runtime protocol. Also validate runtime/source/attempt bindings, startup and node budgets, canonical bounded class sets and result consistency.",
|
||||
"$defs": {
|
||||
"Language": {
|
||||
"enum": [
|
||||
"c17",
|
||||
"cpp20",
|
||||
"python313",
|
||||
"java21"
|
||||
],
|
||||
"title": "Language",
|
||||
"type": "string"
|
||||
},
|
||||
"RunLimits": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"cpu_time_us": {
|
||||
"maximum": 60000000,
|
||||
"minimum": 1000,
|
||||
"title": "Cpu Time Us",
|
||||
"type": "integer"
|
||||
},
|
||||
"file_bytes": {
|
||||
"maximum": 67108864,
|
||||
"minimum": 1,
|
||||
"title": "File Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"memory_bytes": {
|
||||
"maximum": 2147483648,
|
||||
"minimum": 1,
|
||||
"title": "Memory Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"open_files": {
|
||||
"maximum": 1024,
|
||||
"minimum": 3,
|
||||
"title": "Open Files",
|
||||
"type": "integer"
|
||||
},
|
||||
"processes": {
|
||||
"maximum": 128,
|
||||
"minimum": 1,
|
||||
"title": "Processes",
|
||||
"type": "integer"
|
||||
},
|
||||
"stack_bytes": {
|
||||
"maximum": 268435456,
|
||||
"minimum": 1,
|
||||
"title": "Stack Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"stderr_bytes": {
|
||||
"maximum": 16777216,
|
||||
"minimum": 1,
|
||||
"title": "Stderr Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"stdout_bytes": {
|
||||
"maximum": 16777216,
|
||||
"minimum": 1,
|
||||
"title": "Stdout Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"wall_time_us": {
|
||||
"maximum": 120000000,
|
||||
"minimum": 1000,
|
||||
"title": "Wall Time Us",
|
||||
"type": "integer"
|
||||
},
|
||||
"workdir_bytes": {
|
||||
"maximum": 268435456,
|
||||
"minimum": 1,
|
||||
"title": "Workdir Bytes",
|
||||
"type": "integer"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"cpu_time_us",
|
||||
"wall_time_us",
|
||||
"memory_bytes",
|
||||
"stack_bytes",
|
||||
"processes",
|
||||
"open_files",
|
||||
"stdout_bytes",
|
||||
"stderr_bytes",
|
||||
"file_bytes",
|
||||
"workdir_bytes"
|
||||
],
|
||||
"title": "RunLimits",
|
||||
"type": "object"
|
||||
},
|
||||
"RuntimeCapability": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"compile_ceiling": {
|
||||
"$ref": "#/$defs/RunLimits"
|
||||
},
|
||||
"language": {
|
||||
"$ref": "#/$defs/Language"
|
||||
},
|
||||
"run_ceiling": {
|
||||
"$ref": "#/$defs/RunLimits"
|
||||
},
|
||||
"toolchain": {
|
||||
"$ref": "#/$defs/RuntimeIdentity"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"language",
|
||||
"toolchain",
|
||||
"compile_ceiling",
|
||||
"run_ceiling"
|
||||
],
|
||||
"title": "RuntimeCapability",
|
||||
"type": "object"
|
||||
},
|
||||
"RuntimeIdentity": {
|
||||
"additionalProperties": false,
|
||||
"description": "An exact identity matched against trusted configuration, never a host path.",
|
||||
"properties": {
|
||||
"image_digest": {
|
||||
"maxLength": 64,
|
||||
"minLength": 64,
|
||||
"pattern": "^[0-9a-f]{64}$",
|
||||
"title": "Image Digest",
|
||||
"type": "string"
|
||||
},
|
||||
"image_kind": {
|
||||
"enum": [
|
||||
"linux_gnu_cxx_x86_64_v1",
|
||||
"linux_cpython313_x86_64_v1",
|
||||
"linux_openjdk21_x86_64_v1"
|
||||
],
|
||||
"title": "Image Kind",
|
||||
"type": "string"
|
||||
},
|
||||
"image_revision": {
|
||||
"maximum": 4294967295,
|
||||
"minimum": 1,
|
||||
"title": "Image Revision",
|
||||
"type": "integer"
|
||||
},
|
||||
"runtime_version": {
|
||||
"pattern": "^[A-Za-z0-9][A-Za-z0-9_.+-]{0,63}$",
|
||||
"title": "Runtime Version",
|
||||
"type": "string"
|
||||
},
|
||||
"toolchain_id": {
|
||||
"pattern": "^[a-z0-9][a-z0-9_-]{0,63}$",
|
||||
"title": "Toolchain Id",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"toolchain_id",
|
||||
"image_kind",
|
||||
"image_revision",
|
||||
"image_digest",
|
||||
"runtime_version"
|
||||
],
|
||||
"title": "RuntimeIdentity",
|
||||
"type": "object"
|
||||
}
|
||||
},
|
||||
"$id": "urn:kf-oj:judge-admission:v2",
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"capabilities": {
|
||||
"items": {
|
||||
"$ref": "#/$defs/RuntimeCapability"
|
||||
},
|
||||
"maxItems": 4,
|
||||
"minItems": 1,
|
||||
"title": "Capabilities",
|
||||
"type": "array"
|
||||
},
|
||||
"max_concurrent_jobs": {
|
||||
"default": 1,
|
||||
"maximum": 64,
|
||||
"minimum": 1,
|
||||
"title": "Max Concurrent Jobs",
|
||||
"type": "integer"
|
||||
},
|
||||
"max_job_wall_time_us": {
|
||||
"maximum": 3600000000,
|
||||
"minimum": 1000,
|
||||
"title": "Max Job Wall Time Us",
|
||||
"type": "integer"
|
||||
},
|
||||
"max_package_bytes": {
|
||||
"maximum": 67108864,
|
||||
"minimum": 1,
|
||||
"title": "Max Package Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"max_retries": {
|
||||
"default": 2,
|
||||
"maximum": 4,
|
||||
"minimum": 0,
|
||||
"title": "Max Retries",
|
||||
"type": "integer"
|
||||
},
|
||||
"prefetch": {
|
||||
"default": 1,
|
||||
"maximum": 64,
|
||||
"minimum": 1,
|
||||
"title": "Prefetch",
|
||||
"type": "integer"
|
||||
},
|
||||
"schema_version": {
|
||||
"const": 2,
|
||||
"title": "Schema Version",
|
||||
"type": "integer"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"schema_version",
|
||||
"max_job_wall_time_us",
|
||||
"max_package_bytes",
|
||||
"capabilities"
|
||||
],
|
||||
"title": "AdmissionPolicy",
|
||||
"type": "object"
|
||||
}
|
||||
@@ -1,94 +0,0 @@
|
||||
{
|
||||
"$comment": "Structural schema; also run versioned model validation for cross-field budgets, result consistency, UUID canonical form and UTF-8 byte limits.",
|
||||
"$defs": {
|
||||
"LeaseBinding": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"attempt_id": {
|
||||
"format": "uuid",
|
||||
"title": "Attempt Id",
|
||||
"type": "string"
|
||||
},
|
||||
"evaluation_id": {
|
||||
"format": "uuid",
|
||||
"title": "Evaluation Id",
|
||||
"type": "string"
|
||||
},
|
||||
"fencing_version": {
|
||||
"maximum": 9223372036854775807,
|
||||
"minimum": 1,
|
||||
"title": "Fencing Version",
|
||||
"type": "integer"
|
||||
},
|
||||
"worker_id": {
|
||||
"format": "uuid",
|
||||
"title": "Worker Id",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"evaluation_id",
|
||||
"attempt_id",
|
||||
"worker_id",
|
||||
"fencing_version"
|
||||
],
|
||||
"title": "LeaseBinding",
|
||||
"type": "object"
|
||||
}
|
||||
},
|
||||
"$id": "urn:kf-oj:judge-lease:v1",
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"attempt_number": {
|
||||
"maximum": 5,
|
||||
"minimum": 1,
|
||||
"title": "Attempt Number",
|
||||
"type": "integer"
|
||||
},
|
||||
"binding": {
|
||||
"$ref": "#/$defs/LeaseBinding"
|
||||
},
|
||||
"expires_at_unix_ms": {
|
||||
"maximum": 9223372036854775807,
|
||||
"minimum": 1,
|
||||
"title": "Expires At Unix Ms",
|
||||
"type": "integer"
|
||||
},
|
||||
"kind": {
|
||||
"const": "judge.lease",
|
||||
"title": "Kind",
|
||||
"type": "string"
|
||||
},
|
||||
"lease_token": {
|
||||
"maxLength": 64,
|
||||
"minLength": 64,
|
||||
"pattern": "^[0-9a-f]{64}$",
|
||||
"title": "Lease Token",
|
||||
"type": "string"
|
||||
},
|
||||
"manifest_digest": {
|
||||
"maxLength": 64,
|
||||
"minLength": 64,
|
||||
"pattern": "^[0-9a-f]{64}$",
|
||||
"title": "Manifest Digest",
|
||||
"type": "string"
|
||||
},
|
||||
"schema_version": {
|
||||
"const": 1,
|
||||
"title": "Schema Version",
|
||||
"type": "integer"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"schema_version",
|
||||
"kind",
|
||||
"binding",
|
||||
"lease_token",
|
||||
"attempt_number",
|
||||
"expires_at_unix_ms",
|
||||
"manifest_digest"
|
||||
],
|
||||
"title": "LeaseGrant",
|
||||
"type": "object"
|
||||
}
|
||||
@@ -1,94 +0,0 @@
|
||||
{
|
||||
"$comment": "Multi-runtime protocol. Also validate runtime/source/attempt bindings, startup and node budgets, canonical bounded class sets and result consistency.",
|
||||
"$defs": {
|
||||
"LeaseBinding": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"attempt_id": {
|
||||
"format": "uuid",
|
||||
"title": "Attempt Id",
|
||||
"type": "string"
|
||||
},
|
||||
"evaluation_id": {
|
||||
"format": "uuid",
|
||||
"title": "Evaluation Id",
|
||||
"type": "string"
|
||||
},
|
||||
"fencing_version": {
|
||||
"maximum": 9223372036854775807,
|
||||
"minimum": 1,
|
||||
"title": "Fencing Version",
|
||||
"type": "integer"
|
||||
},
|
||||
"worker_id": {
|
||||
"format": "uuid",
|
||||
"title": "Worker Id",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"evaluation_id",
|
||||
"attempt_id",
|
||||
"worker_id",
|
||||
"fencing_version"
|
||||
],
|
||||
"title": "LeaseBinding",
|
||||
"type": "object"
|
||||
}
|
||||
},
|
||||
"$id": "urn:kf-oj:judge-lease:v2",
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"attempt_number": {
|
||||
"maximum": 5,
|
||||
"minimum": 1,
|
||||
"title": "Attempt Number",
|
||||
"type": "integer"
|
||||
},
|
||||
"binding": {
|
||||
"$ref": "#/$defs/LeaseBinding"
|
||||
},
|
||||
"expires_at_unix_ms": {
|
||||
"maximum": 9223372036854775807,
|
||||
"minimum": 1,
|
||||
"title": "Expires At Unix Ms",
|
||||
"type": "integer"
|
||||
},
|
||||
"kind": {
|
||||
"const": "judge.lease",
|
||||
"title": "Kind",
|
||||
"type": "string"
|
||||
},
|
||||
"lease_token": {
|
||||
"maxLength": 64,
|
||||
"minLength": 64,
|
||||
"pattern": "^[0-9a-f]{64}$",
|
||||
"title": "Lease Token",
|
||||
"type": "string"
|
||||
},
|
||||
"manifest_digest": {
|
||||
"maxLength": 64,
|
||||
"minLength": 64,
|
||||
"pattern": "^[0-9a-f]{64}$",
|
||||
"title": "Manifest Digest",
|
||||
"type": "string"
|
||||
},
|
||||
"schema_version": {
|
||||
"const": 2,
|
||||
"title": "Schema Version",
|
||||
"type": "integer"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"schema_version",
|
||||
"kind",
|
||||
"binding",
|
||||
"lease_token",
|
||||
"attempt_number",
|
||||
"expires_at_unix_ms",
|
||||
"manifest_digest"
|
||||
],
|
||||
"title": "LeaseGrant",
|
||||
"type": "object"
|
||||
}
|
||||
@@ -1,249 +0,0 @@
|
||||
{
|
||||
"$comment": "Structural schema; also run versioned model validation for cross-field budgets, result consistency, UUID canonical form and UTF-8 byte limits.",
|
||||
"$defs": {
|
||||
"ArtifactReference": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"bytes": {
|
||||
"maximum": 67108864,
|
||||
"minimum": 0,
|
||||
"title": "Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"digest": {
|
||||
"maxLength": 64,
|
||||
"minLength": 64,
|
||||
"pattern": "^[0-9a-f]{64}$",
|
||||
"title": "Digest",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"digest",
|
||||
"bytes"
|
||||
],
|
||||
"title": "ArtifactReference",
|
||||
"type": "object"
|
||||
},
|
||||
"Language": {
|
||||
"enum": [
|
||||
"c17",
|
||||
"cpp20"
|
||||
],
|
||||
"title": "Language",
|
||||
"type": "string"
|
||||
},
|
||||
"RunLimits": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"cpu_time_us": {
|
||||
"maximum": 60000000,
|
||||
"minimum": 1000,
|
||||
"title": "Cpu Time Us",
|
||||
"type": "integer"
|
||||
},
|
||||
"file_bytes": {
|
||||
"maximum": 67108864,
|
||||
"minimum": 1,
|
||||
"title": "File Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"memory_bytes": {
|
||||
"maximum": 2147483648,
|
||||
"minimum": 1,
|
||||
"title": "Memory Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"open_files": {
|
||||
"maximum": 1024,
|
||||
"minimum": 3,
|
||||
"title": "Open Files",
|
||||
"type": "integer"
|
||||
},
|
||||
"processes": {
|
||||
"maximum": 128,
|
||||
"minimum": 1,
|
||||
"title": "Processes",
|
||||
"type": "integer"
|
||||
},
|
||||
"stack_bytes": {
|
||||
"maximum": 268435456,
|
||||
"minimum": 1,
|
||||
"title": "Stack Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"stderr_bytes": {
|
||||
"maximum": 16777216,
|
||||
"minimum": 1,
|
||||
"title": "Stderr Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"stdout_bytes": {
|
||||
"maximum": 16777216,
|
||||
"minimum": 1,
|
||||
"title": "Stdout Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"wall_time_us": {
|
||||
"maximum": 120000000,
|
||||
"minimum": 1000,
|
||||
"title": "Wall Time Us",
|
||||
"type": "integer"
|
||||
},
|
||||
"workdir_bytes": {
|
||||
"maximum": 268435456,
|
||||
"minimum": 1,
|
||||
"title": "Workdir Bytes",
|
||||
"type": "integer"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"cpu_time_us",
|
||||
"wall_time_us",
|
||||
"memory_bytes",
|
||||
"stack_bytes",
|
||||
"processes",
|
||||
"open_files",
|
||||
"stdout_bytes",
|
||||
"stderr_bytes",
|
||||
"file_bytes",
|
||||
"workdir_bytes"
|
||||
],
|
||||
"title": "RunLimits",
|
||||
"type": "object"
|
||||
},
|
||||
"TestCase": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"case_id": {
|
||||
"format": "uuid",
|
||||
"title": "Case Id",
|
||||
"type": "string"
|
||||
},
|
||||
"expected": {
|
||||
"$ref": "#/$defs/ArtifactReference"
|
||||
},
|
||||
"input": {
|
||||
"$ref": "#/$defs/ArtifactReference"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"case_id",
|
||||
"input",
|
||||
"expected"
|
||||
],
|
||||
"title": "TestCase",
|
||||
"type": "object"
|
||||
},
|
||||
"Toolchain": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"image_kind": {
|
||||
"const": "linux_gnu_cxx_x86_64_v1",
|
||||
"title": "Image Kind",
|
||||
"type": "string"
|
||||
},
|
||||
"image_revision": {
|
||||
"maximum": 4294967295,
|
||||
"minimum": 1,
|
||||
"title": "Image Revision",
|
||||
"type": "integer"
|
||||
},
|
||||
"toolchain_id": {
|
||||
"pattern": "^[a-z0-9][a-z0-9_-]{0,63}$",
|
||||
"title": "Toolchain Id",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"toolchain_id",
|
||||
"image_kind",
|
||||
"image_revision"
|
||||
],
|
||||
"title": "Toolchain",
|
||||
"type": "object"
|
||||
}
|
||||
},
|
||||
"$id": "urn:kf-oj:judge-manifest:v1",
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"cases": {
|
||||
"items": {
|
||||
"$ref": "#/$defs/TestCase"
|
||||
},
|
||||
"maxItems": 256,
|
||||
"minItems": 1,
|
||||
"title": "Cases",
|
||||
"type": "array"
|
||||
},
|
||||
"comparison": {
|
||||
"const": "exact_bytes",
|
||||
"title": "Comparison",
|
||||
"type": "string"
|
||||
},
|
||||
"compile_limits": {
|
||||
"$ref": "#/$defs/RunLimits"
|
||||
},
|
||||
"evaluation_id": {
|
||||
"format": "uuid",
|
||||
"title": "Evaluation Id",
|
||||
"type": "string"
|
||||
},
|
||||
"job_wall_time_us": {
|
||||
"maximum": 3600000000,
|
||||
"minimum": 1000,
|
||||
"title": "Job Wall Time Us",
|
||||
"type": "integer"
|
||||
},
|
||||
"kind": {
|
||||
"const": "judge.manifest",
|
||||
"title": "Kind",
|
||||
"type": "string"
|
||||
},
|
||||
"language": {
|
||||
"$ref": "#/$defs/Language"
|
||||
},
|
||||
"problem_version_id": {
|
||||
"format": "uuid",
|
||||
"title": "Problem Version Id",
|
||||
"type": "string"
|
||||
},
|
||||
"run_limits": {
|
||||
"$ref": "#/$defs/RunLimits"
|
||||
},
|
||||
"schema_version": {
|
||||
"const": 1,
|
||||
"title": "Schema Version",
|
||||
"type": "integer"
|
||||
},
|
||||
"source": {
|
||||
"$ref": "#/$defs/ArtifactReference"
|
||||
},
|
||||
"submission_id": {
|
||||
"format": "uuid",
|
||||
"title": "Submission Id",
|
||||
"type": "string"
|
||||
},
|
||||
"toolchain": {
|
||||
"$ref": "#/$defs/Toolchain"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"schema_version",
|
||||
"kind",
|
||||
"evaluation_id",
|
||||
"submission_id",
|
||||
"problem_version_id",
|
||||
"language",
|
||||
"toolchain",
|
||||
"source",
|
||||
"cases",
|
||||
"comparison",
|
||||
"compile_limits",
|
||||
"run_limits",
|
||||
"job_wall_time_us"
|
||||
],
|
||||
"title": "JobManifest",
|
||||
"type": "object"
|
||||
}
|
||||
@@ -1,270 +0,0 @@
|
||||
{
|
||||
"$comment": "Multi-runtime protocol. Also validate runtime/source/attempt bindings, startup and node budgets, canonical bounded class sets and result consistency.",
|
||||
"$defs": {
|
||||
"ArtifactReference": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"bytes": {
|
||||
"maximum": 67108864,
|
||||
"minimum": 0,
|
||||
"title": "Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"digest": {
|
||||
"maxLength": 64,
|
||||
"minLength": 64,
|
||||
"pattern": "^[0-9a-f]{64}$",
|
||||
"title": "Digest",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"digest",
|
||||
"bytes"
|
||||
],
|
||||
"title": "ArtifactReference",
|
||||
"type": "object"
|
||||
},
|
||||
"Language": {
|
||||
"enum": [
|
||||
"c17",
|
||||
"cpp20",
|
||||
"python313",
|
||||
"java21"
|
||||
],
|
||||
"title": "Language",
|
||||
"type": "string"
|
||||
},
|
||||
"RunLimits": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"cpu_time_us": {
|
||||
"maximum": 60000000,
|
||||
"minimum": 1000,
|
||||
"title": "Cpu Time Us",
|
||||
"type": "integer"
|
||||
},
|
||||
"file_bytes": {
|
||||
"maximum": 67108864,
|
||||
"minimum": 1,
|
||||
"title": "File Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"memory_bytes": {
|
||||
"maximum": 2147483648,
|
||||
"minimum": 1,
|
||||
"title": "Memory Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"open_files": {
|
||||
"maximum": 1024,
|
||||
"minimum": 3,
|
||||
"title": "Open Files",
|
||||
"type": "integer"
|
||||
},
|
||||
"processes": {
|
||||
"maximum": 128,
|
||||
"minimum": 1,
|
||||
"title": "Processes",
|
||||
"type": "integer"
|
||||
},
|
||||
"stack_bytes": {
|
||||
"maximum": 268435456,
|
||||
"minimum": 1,
|
||||
"title": "Stack Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"stderr_bytes": {
|
||||
"maximum": 16777216,
|
||||
"minimum": 1,
|
||||
"title": "Stderr Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"stdout_bytes": {
|
||||
"maximum": 16777216,
|
||||
"minimum": 1,
|
||||
"title": "Stdout Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"wall_time_us": {
|
||||
"maximum": 120000000,
|
||||
"minimum": 1000,
|
||||
"title": "Wall Time Us",
|
||||
"type": "integer"
|
||||
},
|
||||
"workdir_bytes": {
|
||||
"maximum": 268435456,
|
||||
"minimum": 1,
|
||||
"title": "Workdir Bytes",
|
||||
"type": "integer"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"cpu_time_us",
|
||||
"wall_time_us",
|
||||
"memory_bytes",
|
||||
"stack_bytes",
|
||||
"processes",
|
||||
"open_files",
|
||||
"stdout_bytes",
|
||||
"stderr_bytes",
|
||||
"file_bytes",
|
||||
"workdir_bytes"
|
||||
],
|
||||
"title": "RunLimits",
|
||||
"type": "object"
|
||||
},
|
||||
"RuntimeIdentity": {
|
||||
"additionalProperties": false,
|
||||
"description": "An exact identity matched against trusted configuration, never a host path.",
|
||||
"properties": {
|
||||
"image_digest": {
|
||||
"maxLength": 64,
|
||||
"minLength": 64,
|
||||
"pattern": "^[0-9a-f]{64}$",
|
||||
"title": "Image Digest",
|
||||
"type": "string"
|
||||
},
|
||||
"image_kind": {
|
||||
"enum": [
|
||||
"linux_gnu_cxx_x86_64_v1",
|
||||
"linux_cpython313_x86_64_v1",
|
||||
"linux_openjdk21_x86_64_v1"
|
||||
],
|
||||
"title": "Image Kind",
|
||||
"type": "string"
|
||||
},
|
||||
"image_revision": {
|
||||
"maximum": 4294967295,
|
||||
"minimum": 1,
|
||||
"title": "Image Revision",
|
||||
"type": "integer"
|
||||
},
|
||||
"runtime_version": {
|
||||
"pattern": "^[A-Za-z0-9][A-Za-z0-9_.+-]{0,63}$",
|
||||
"title": "Runtime Version",
|
||||
"type": "string"
|
||||
},
|
||||
"toolchain_id": {
|
||||
"pattern": "^[a-z0-9][a-z0-9_-]{0,63}$",
|
||||
"title": "Toolchain Id",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"toolchain_id",
|
||||
"image_kind",
|
||||
"image_revision",
|
||||
"image_digest",
|
||||
"runtime_version"
|
||||
],
|
||||
"title": "RuntimeIdentity",
|
||||
"type": "object"
|
||||
},
|
||||
"TestCase": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"case_id": {
|
||||
"format": "uuid",
|
||||
"title": "Case Id",
|
||||
"type": "string"
|
||||
},
|
||||
"expected": {
|
||||
"$ref": "#/$defs/ArtifactReference"
|
||||
},
|
||||
"input": {
|
||||
"$ref": "#/$defs/ArtifactReference"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"case_id",
|
||||
"input",
|
||||
"expected"
|
||||
],
|
||||
"title": "TestCase",
|
||||
"type": "object"
|
||||
}
|
||||
},
|
||||
"$id": "urn:kf-oj:judge-manifest:v2",
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"cases": {
|
||||
"items": {
|
||||
"$ref": "#/$defs/TestCase"
|
||||
},
|
||||
"maxItems": 256,
|
||||
"minItems": 1,
|
||||
"title": "Cases",
|
||||
"type": "array"
|
||||
},
|
||||
"comparison": {
|
||||
"const": "exact_bytes",
|
||||
"title": "Comparison",
|
||||
"type": "string"
|
||||
},
|
||||
"compile_limits": {
|
||||
"$ref": "#/$defs/RunLimits"
|
||||
},
|
||||
"evaluation_id": {
|
||||
"format": "uuid",
|
||||
"title": "Evaluation Id",
|
||||
"type": "string"
|
||||
},
|
||||
"job_wall_time_us": {
|
||||
"maximum": 3600000000,
|
||||
"minimum": 1000,
|
||||
"title": "Job Wall Time Us",
|
||||
"type": "integer"
|
||||
},
|
||||
"kind": {
|
||||
"const": "judge.manifest",
|
||||
"title": "Kind",
|
||||
"type": "string"
|
||||
},
|
||||
"language": {
|
||||
"$ref": "#/$defs/Language"
|
||||
},
|
||||
"problem_version_id": {
|
||||
"format": "uuid",
|
||||
"title": "Problem Version Id",
|
||||
"type": "string"
|
||||
},
|
||||
"run_limits": {
|
||||
"$ref": "#/$defs/RunLimits"
|
||||
},
|
||||
"schema_version": {
|
||||
"const": 2,
|
||||
"title": "Schema Version",
|
||||
"type": "integer"
|
||||
},
|
||||
"source": {
|
||||
"$ref": "#/$defs/ArtifactReference"
|
||||
},
|
||||
"submission_id": {
|
||||
"format": "uuid",
|
||||
"title": "Submission Id",
|
||||
"type": "string"
|
||||
},
|
||||
"toolchain": {
|
||||
"$ref": "#/$defs/RuntimeIdentity"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"schema_version",
|
||||
"kind",
|
||||
"evaluation_id",
|
||||
"submission_id",
|
||||
"problem_version_id",
|
||||
"language",
|
||||
"toolchain",
|
||||
"source",
|
||||
"cases",
|
||||
"comparison",
|
||||
"compile_limits",
|
||||
"run_limits",
|
||||
"job_wall_time_us"
|
||||
],
|
||||
"title": "JobManifest",
|
||||
"type": "object"
|
||||
}
|
||||
@@ -1,349 +0,0 @@
|
||||
{
|
||||
"$comment": "Multi-runtime protocol. Also validate runtime/source/attempt bindings, startup and node budgets, canonical bounded class sets and result consistency.",
|
||||
"$defs": {
|
||||
"Language": {
|
||||
"enum": [
|
||||
"c17",
|
||||
"cpp20",
|
||||
"python313",
|
||||
"java21"
|
||||
],
|
||||
"title": "Language",
|
||||
"type": "string"
|
||||
},
|
||||
"RunLimits": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"cpu_time_us": {
|
||||
"maximum": 60000000,
|
||||
"minimum": 1000,
|
||||
"title": "Cpu Time Us",
|
||||
"type": "integer"
|
||||
},
|
||||
"file_bytes": {
|
||||
"maximum": 67108864,
|
||||
"minimum": 1,
|
||||
"title": "File Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"memory_bytes": {
|
||||
"maximum": 2147483648,
|
||||
"minimum": 1,
|
||||
"title": "Memory Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"open_files": {
|
||||
"maximum": 1024,
|
||||
"minimum": 3,
|
||||
"title": "Open Files",
|
||||
"type": "integer"
|
||||
},
|
||||
"processes": {
|
||||
"maximum": 128,
|
||||
"minimum": 1,
|
||||
"title": "Processes",
|
||||
"type": "integer"
|
||||
},
|
||||
"stack_bytes": {
|
||||
"maximum": 268435456,
|
||||
"minimum": 1,
|
||||
"title": "Stack Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"stderr_bytes": {
|
||||
"maximum": 16777216,
|
||||
"minimum": 1,
|
||||
"title": "Stderr Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"stdout_bytes": {
|
||||
"maximum": 16777216,
|
||||
"minimum": 1,
|
||||
"title": "Stdout Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"wall_time_us": {
|
||||
"maximum": 120000000,
|
||||
"minimum": 1000,
|
||||
"title": "Wall Time Us",
|
||||
"type": "integer"
|
||||
},
|
||||
"workdir_bytes": {
|
||||
"maximum": 268435456,
|
||||
"minimum": 1,
|
||||
"title": "Workdir Bytes",
|
||||
"type": "integer"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"cpu_time_us",
|
||||
"wall_time_us",
|
||||
"memory_bytes",
|
||||
"stack_bytes",
|
||||
"processes",
|
||||
"open_files",
|
||||
"stdout_bytes",
|
||||
"stderr_bytes",
|
||||
"file_bytes",
|
||||
"workdir_bytes"
|
||||
],
|
||||
"title": "RunLimits",
|
||||
"type": "object"
|
||||
},
|
||||
"RuntimeCapability": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"compile_ceiling": {
|
||||
"$ref": "#/$defs/RunLimits"
|
||||
},
|
||||
"language": {
|
||||
"$ref": "#/$defs/Language"
|
||||
},
|
||||
"run_ceiling": {
|
||||
"$ref": "#/$defs/RunLimits"
|
||||
},
|
||||
"toolchain": {
|
||||
"$ref": "#/$defs/RuntimeIdentity"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"language",
|
||||
"toolchain",
|
||||
"compile_ceiling",
|
||||
"run_ceiling"
|
||||
],
|
||||
"title": "RuntimeCapability",
|
||||
"type": "object"
|
||||
},
|
||||
"RuntimeIdentity": {
|
||||
"additionalProperties": false,
|
||||
"description": "An exact identity matched against trusted configuration, never a host path.",
|
||||
"properties": {
|
||||
"image_digest": {
|
||||
"maxLength": 64,
|
||||
"minLength": 64,
|
||||
"pattern": "^[0-9a-f]{64}$",
|
||||
"title": "Image Digest",
|
||||
"type": "string"
|
||||
},
|
||||
"image_kind": {
|
||||
"enum": [
|
||||
"linux_gnu_cxx_x86_64_v1",
|
||||
"linux_cpython313_x86_64_v1",
|
||||
"linux_openjdk21_x86_64_v1"
|
||||
],
|
||||
"title": "Image Kind",
|
||||
"type": "string"
|
||||
},
|
||||
"image_revision": {
|
||||
"maximum": 4294967295,
|
||||
"minimum": 1,
|
||||
"title": "Image Revision",
|
||||
"type": "integer"
|
||||
},
|
||||
"runtime_version": {
|
||||
"pattern": "^[A-Za-z0-9][A-Za-z0-9_.+-]{0,63}$",
|
||||
"title": "Runtime Version",
|
||||
"type": "string"
|
||||
},
|
||||
"toolchain_id": {
|
||||
"pattern": "^[a-z0-9][a-z0-9_-]{0,63}$",
|
||||
"title": "Toolchain Id",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"toolchain_id",
|
||||
"image_kind",
|
||||
"image_revision",
|
||||
"image_digest",
|
||||
"runtime_version"
|
||||
],
|
||||
"title": "RuntimeIdentity",
|
||||
"type": "object"
|
||||
},
|
||||
"Toolchain": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"image_kind": {
|
||||
"const": "linux_gnu_cxx_x86_64_v1",
|
||||
"title": "Image Kind",
|
||||
"type": "string"
|
||||
},
|
||||
"image_revision": {
|
||||
"maximum": 4294967295,
|
||||
"minimum": 1,
|
||||
"title": "Image Revision",
|
||||
"type": "integer"
|
||||
},
|
||||
"toolchain_id": {
|
||||
"pattern": "^[a-z0-9][a-z0-9_-]{0,63}$",
|
||||
"title": "Toolchain Id",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"toolchain_id",
|
||||
"image_kind",
|
||||
"image_revision"
|
||||
],
|
||||
"title": "Toolchain",
|
||||
"type": "object"
|
||||
},
|
||||
"kf_oj__contracts__judging__AdmissionPolicy": {
|
||||
"additionalProperties": false,
|
||||
"description": "Trusted node policy. A manifest never supplies or replaces this policy.",
|
||||
"properties": {
|
||||
"compile_ceiling": {
|
||||
"$ref": "#/$defs/RunLimits"
|
||||
},
|
||||
"max_concurrent_jobs": {
|
||||
"default": 1,
|
||||
"maximum": 64,
|
||||
"minimum": 1,
|
||||
"title": "Max Concurrent Jobs",
|
||||
"type": "integer"
|
||||
},
|
||||
"max_job_wall_time_us": {
|
||||
"maximum": 3600000000,
|
||||
"minimum": 1000,
|
||||
"title": "Max Job Wall Time Us",
|
||||
"type": "integer"
|
||||
},
|
||||
"max_package_bytes": {
|
||||
"maximum": 67108864,
|
||||
"minimum": 1,
|
||||
"title": "Max Package Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"max_retries": {
|
||||
"default": 2,
|
||||
"maximum": 4,
|
||||
"minimum": 0,
|
||||
"title": "Max Retries",
|
||||
"type": "integer"
|
||||
},
|
||||
"prefetch": {
|
||||
"default": 1,
|
||||
"maximum": 64,
|
||||
"minimum": 1,
|
||||
"title": "Prefetch",
|
||||
"type": "integer"
|
||||
},
|
||||
"run_ceiling": {
|
||||
"$ref": "#/$defs/RunLimits"
|
||||
},
|
||||
"toolchain": {
|
||||
"$ref": "#/$defs/Toolchain"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"max_job_wall_time_us",
|
||||
"max_package_bytes",
|
||||
"toolchain",
|
||||
"compile_ceiling",
|
||||
"run_ceiling"
|
||||
],
|
||||
"title": "AdmissionPolicy",
|
||||
"type": "object"
|
||||
},
|
||||
"kf_oj__contracts__judging_v2__AdmissionPolicy": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"capabilities": {
|
||||
"items": {
|
||||
"$ref": "#/$defs/RuntimeCapability"
|
||||
},
|
||||
"maxItems": 4,
|
||||
"minItems": 1,
|
||||
"title": "Capabilities",
|
||||
"type": "array"
|
||||
},
|
||||
"max_concurrent_jobs": {
|
||||
"default": 1,
|
||||
"maximum": 64,
|
||||
"minimum": 1,
|
||||
"title": "Max Concurrent Jobs",
|
||||
"type": "integer"
|
||||
},
|
||||
"max_job_wall_time_us": {
|
||||
"maximum": 3600000000,
|
||||
"minimum": 1000,
|
||||
"title": "Max Job Wall Time Us",
|
||||
"type": "integer"
|
||||
},
|
||||
"max_package_bytes": {
|
||||
"maximum": 67108864,
|
||||
"minimum": 1,
|
||||
"title": "Max Package Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"max_retries": {
|
||||
"default": 2,
|
||||
"maximum": 4,
|
||||
"minimum": 0,
|
||||
"title": "Max Retries",
|
||||
"type": "integer"
|
||||
},
|
||||
"prefetch": {
|
||||
"default": 1,
|
||||
"maximum": 64,
|
||||
"minimum": 1,
|
||||
"title": "Prefetch",
|
||||
"type": "integer"
|
||||
},
|
||||
"schema_version": {
|
||||
"const": 2,
|
||||
"title": "Schema Version",
|
||||
"type": "integer"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"schema_version",
|
||||
"max_job_wall_time_us",
|
||||
"max_package_bytes",
|
||||
"capabilities"
|
||||
],
|
||||
"title": "AdmissionPolicy",
|
||||
"type": "object"
|
||||
}
|
||||
},
|
||||
"$id": "urn:kf-oj:judge-registry:v2",
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"additionalProperties": false,
|
||||
"description": "Explicit operator opt-in for both protocol generations; no silent conversion.",
|
||||
"properties": {
|
||||
"legacy": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/$defs/kf_oj__contracts__judging__AdmissionPolicy"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"multi_runtime": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/$defs/kf_oj__contracts__judging_v2__AdmissionPolicy"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"schema_version": {
|
||||
"const": 2,
|
||||
"title": "Schema Version",
|
||||
"type": "integer"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"schema_version",
|
||||
"legacy",
|
||||
"multi_runtime"
|
||||
],
|
||||
"title": "AdmissionRegistry",
|
||||
"type": "object"
|
||||
}
|
||||
@@ -1,304 +0,0 @@
|
||||
{
|
||||
"$comment": "Structural schema; also run versioned model validation for cross-field budgets, result consistency, UUID canonical form and UTF-8 byte limits.",
|
||||
"$defs": {
|
||||
"ArtifactReference": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"bytes": {
|
||||
"maximum": 67108864,
|
||||
"minimum": 0,
|
||||
"title": "Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"digest": {
|
||||
"maxLength": 64,
|
||||
"minLength": 64,
|
||||
"pattern": "^[0-9a-f]{64}$",
|
||||
"title": "Digest",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"digest",
|
||||
"bytes"
|
||||
],
|
||||
"title": "ArtifactReference",
|
||||
"type": "object"
|
||||
},
|
||||
"CaseReport": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"case_id": {
|
||||
"format": "uuid",
|
||||
"title": "Case Id",
|
||||
"type": "string"
|
||||
},
|
||||
"cpu_time_us": {
|
||||
"anyOf": [
|
||||
{
|
||||
"maximum": 9223372036854775807,
|
||||
"minimum": 0,
|
||||
"type": "integer"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"title": "Cpu Time Us"
|
||||
},
|
||||
"memory_peak_bytes": {
|
||||
"anyOf": [
|
||||
{
|
||||
"maximum": 9223372036854775807,
|
||||
"minimum": 0,
|
||||
"type": "integer"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"title": "Memory Peak Bytes"
|
||||
},
|
||||
"verdict": {
|
||||
"$ref": "#/$defs/Verdict"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"case_id",
|
||||
"verdict",
|
||||
"cpu_time_us",
|
||||
"memory_peak_bytes"
|
||||
],
|
||||
"title": "CaseReport",
|
||||
"type": "object"
|
||||
},
|
||||
"CompilationReport": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"cpu_time_us": {
|
||||
"anyOf": [
|
||||
{
|
||||
"maximum": 9223372036854775807,
|
||||
"minimum": 0,
|
||||
"type": "integer"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"title": "Cpu Time Us"
|
||||
},
|
||||
"diagnostics": {
|
||||
"maxLength": 65536,
|
||||
"title": "Diagnostics",
|
||||
"type": "string"
|
||||
},
|
||||
"diagnostics_truncated": {
|
||||
"title": "Diagnostics Truncated",
|
||||
"type": "boolean"
|
||||
},
|
||||
"memory_peak_bytes": {
|
||||
"anyOf": [
|
||||
{
|
||||
"maximum": 9223372036854775807,
|
||||
"minimum": 0,
|
||||
"type": "integer"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"title": "Memory Peak Bytes"
|
||||
},
|
||||
"program": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/$defs/ArtifactReference"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"status": {
|
||||
"$ref": "#/$defs/CompilationStatus"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"status",
|
||||
"program",
|
||||
"diagnostics",
|
||||
"diagnostics_truncated",
|
||||
"cpu_time_us",
|
||||
"memory_peak_bytes"
|
||||
],
|
||||
"title": "CompilationReport",
|
||||
"type": "object"
|
||||
},
|
||||
"CompilationStatus": {
|
||||
"enum": [
|
||||
"succeeded",
|
||||
"compilation_error",
|
||||
"compilation_limit",
|
||||
"cancelled",
|
||||
"system_error"
|
||||
],
|
||||
"title": "CompilationStatus",
|
||||
"type": "string"
|
||||
},
|
||||
"FailureCode": {
|
||||
"enum": [
|
||||
"artifact_unavailable",
|
||||
"sandbox_unavailable",
|
||||
"cleanup_failed",
|
||||
"lease_lost",
|
||||
"job_deadline"
|
||||
],
|
||||
"title": "FailureCode",
|
||||
"type": "string"
|
||||
},
|
||||
"LeaseBinding": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"attempt_id": {
|
||||
"format": "uuid",
|
||||
"title": "Attempt Id",
|
||||
"type": "string"
|
||||
},
|
||||
"evaluation_id": {
|
||||
"format": "uuid",
|
||||
"title": "Evaluation Id",
|
||||
"type": "string"
|
||||
},
|
||||
"fencing_version": {
|
||||
"maximum": 9223372036854775807,
|
||||
"minimum": 1,
|
||||
"title": "Fencing Version",
|
||||
"type": "integer"
|
||||
},
|
||||
"worker_id": {
|
||||
"format": "uuid",
|
||||
"title": "Worker Id",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"evaluation_id",
|
||||
"attempt_id",
|
||||
"worker_id",
|
||||
"fencing_version"
|
||||
],
|
||||
"title": "LeaseBinding",
|
||||
"type": "object"
|
||||
},
|
||||
"Verdict": {
|
||||
"enum": [
|
||||
"accepted",
|
||||
"wrong_answer",
|
||||
"compilation_error",
|
||||
"runtime_error",
|
||||
"time_limit_exceeded",
|
||||
"memory_limit_exceeded",
|
||||
"output_limit_exceeded",
|
||||
"cancelled",
|
||||
"system_error"
|
||||
],
|
||||
"title": "Verdict",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"$id": "urn:kf-oj:judge-result:v1",
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"binding": {
|
||||
"$ref": "#/$defs/LeaseBinding"
|
||||
},
|
||||
"cases": {
|
||||
"items": {
|
||||
"$ref": "#/$defs/CaseReport"
|
||||
},
|
||||
"maxItems": 256,
|
||||
"title": "Cases",
|
||||
"type": "array"
|
||||
},
|
||||
"cleanup_complete": {
|
||||
"title": "Cleanup Complete",
|
||||
"type": "boolean"
|
||||
},
|
||||
"compilation": {
|
||||
"$ref": "#/$defs/CompilationReport"
|
||||
},
|
||||
"failure_code": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/$defs/FailureCode"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"kind": {
|
||||
"const": "judge.result",
|
||||
"title": "Kind",
|
||||
"type": "string"
|
||||
},
|
||||
"manifest_digest": {
|
||||
"maxLength": 64,
|
||||
"minLength": 64,
|
||||
"pattern": "^[0-9a-f]{64}$",
|
||||
"title": "Manifest Digest",
|
||||
"type": "string"
|
||||
},
|
||||
"run_cpu_time_us": {
|
||||
"anyOf": [
|
||||
{
|
||||
"maximum": 9223372036854775807,
|
||||
"minimum": 0,
|
||||
"type": "integer"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"title": "Run Cpu Time Us"
|
||||
},
|
||||
"run_memory_peak_bytes": {
|
||||
"anyOf": [
|
||||
{
|
||||
"maximum": 9223372036854775807,
|
||||
"minimum": 0,
|
||||
"type": "integer"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"title": "Run Memory Peak Bytes"
|
||||
},
|
||||
"schema_version": {
|
||||
"const": 1,
|
||||
"title": "Schema Version",
|
||||
"type": "integer"
|
||||
},
|
||||
"verdict": {
|
||||
"$ref": "#/$defs/Verdict"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"schema_version",
|
||||
"kind",
|
||||
"binding",
|
||||
"manifest_digest",
|
||||
"compilation",
|
||||
"cases",
|
||||
"verdict",
|
||||
"cleanup_complete",
|
||||
"failure_code",
|
||||
"run_cpu_time_us",
|
||||
"run_memory_peak_bytes"
|
||||
],
|
||||
"title": "JudgementReport",
|
||||
"type": "object"
|
||||
}
|
||||
@@ -1,416 +0,0 @@
|
||||
{
|
||||
"$comment": "Multi-runtime protocol. Also validate runtime/source/attempt bindings, startup and node budgets, canonical bounded class sets and result consistency.",
|
||||
"$defs": {
|
||||
"ArtifactReference": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"bytes": {
|
||||
"maximum": 67108864,
|
||||
"minimum": 0,
|
||||
"title": "Bytes",
|
||||
"type": "integer"
|
||||
},
|
||||
"digest": {
|
||||
"maxLength": 64,
|
||||
"minLength": 64,
|
||||
"pattern": "^[0-9a-f]{64}$",
|
||||
"title": "Digest",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"digest",
|
||||
"bytes"
|
||||
],
|
||||
"title": "ArtifactReference",
|
||||
"type": "object"
|
||||
},
|
||||
"CaseReport": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"case_id": {
|
||||
"format": "uuid",
|
||||
"title": "Case Id",
|
||||
"type": "string"
|
||||
},
|
||||
"cpu_time_us": {
|
||||
"anyOf": [
|
||||
{
|
||||
"maximum": 9223372036854775807,
|
||||
"minimum": 0,
|
||||
"type": "integer"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"title": "Cpu Time Us"
|
||||
},
|
||||
"memory_peak_bytes": {
|
||||
"anyOf": [
|
||||
{
|
||||
"maximum": 9223372036854775807,
|
||||
"minimum": 0,
|
||||
"type": "integer"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"title": "Memory Peak Bytes"
|
||||
},
|
||||
"verdict": {
|
||||
"$ref": "#/$defs/Verdict"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"case_id",
|
||||
"verdict",
|
||||
"cpu_time_us",
|
||||
"memory_peak_bytes"
|
||||
],
|
||||
"title": "CaseReport",
|
||||
"type": "object"
|
||||
},
|
||||
"CompilationReport": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"cpu_time_us": {
|
||||
"anyOf": [
|
||||
{
|
||||
"maximum": 9223372036854775807,
|
||||
"minimum": 0,
|
||||
"type": "integer"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"title": "Cpu Time Us"
|
||||
},
|
||||
"diagnostics": {
|
||||
"maxLength": 65536,
|
||||
"title": "Diagnostics",
|
||||
"type": "string"
|
||||
},
|
||||
"diagnostics_truncated": {
|
||||
"title": "Diagnostics Truncated",
|
||||
"type": "boolean"
|
||||
},
|
||||
"memory_peak_bytes": {
|
||||
"anyOf": [
|
||||
{
|
||||
"maximum": 9223372036854775807,
|
||||
"minimum": 0,
|
||||
"type": "integer"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"title": "Memory Peak Bytes"
|
||||
},
|
||||
"program": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/$defs/CompiledProgram"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"status": {
|
||||
"$ref": "#/$defs/CompilationStatus"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"status",
|
||||
"program",
|
||||
"diagnostics",
|
||||
"diagnostics_truncated",
|
||||
"cpu_time_us",
|
||||
"memory_peak_bytes"
|
||||
],
|
||||
"title": "CompilationReport",
|
||||
"type": "object"
|
||||
},
|
||||
"CompilationStatus": {
|
||||
"enum": [
|
||||
"succeeded",
|
||||
"compilation_error",
|
||||
"compilation_limit",
|
||||
"cancelled",
|
||||
"system_error"
|
||||
],
|
||||
"title": "CompilationStatus",
|
||||
"type": "string"
|
||||
},
|
||||
"CompiledProgram": {
|
||||
"additionalProperties": false,
|
||||
"description": "Sealed file identities bound to a particular source, runtime and lease attempt.",
|
||||
"properties": {
|
||||
"binding": {
|
||||
"$ref": "#/$defs/LeaseBinding"
|
||||
},
|
||||
"files": {
|
||||
"items": {
|
||||
"$ref": "#/$defs/ProgramFile"
|
||||
},
|
||||
"maxItems": 256,
|
||||
"minItems": 1,
|
||||
"title": "Files",
|
||||
"type": "array"
|
||||
},
|
||||
"kind": {
|
||||
"$ref": "#/$defs/ProgramKind"
|
||||
},
|
||||
"source": {
|
||||
"$ref": "#/$defs/ArtifactReference"
|
||||
},
|
||||
"toolchain": {
|
||||
"$ref": "#/$defs/RuntimeIdentity"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"kind",
|
||||
"binding",
|
||||
"source",
|
||||
"toolchain",
|
||||
"files"
|
||||
],
|
||||
"title": "CompiledProgram",
|
||||
"type": "object"
|
||||
},
|
||||
"FailureCode": {
|
||||
"enum": [
|
||||
"artifact_unavailable",
|
||||
"sandbox_unavailable",
|
||||
"cleanup_failed",
|
||||
"lease_lost",
|
||||
"job_deadline"
|
||||
],
|
||||
"title": "FailureCode",
|
||||
"type": "string"
|
||||
},
|
||||
"LeaseBinding": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"attempt_id": {
|
||||
"format": "uuid",
|
||||
"title": "Attempt Id",
|
||||
"type": "string"
|
||||
},
|
||||
"evaluation_id": {
|
||||
"format": "uuid",
|
||||
"title": "Evaluation Id",
|
||||
"type": "string"
|
||||
},
|
||||
"fencing_version": {
|
||||
"maximum": 9223372036854775807,
|
||||
"minimum": 1,
|
||||
"title": "Fencing Version",
|
||||
"type": "integer"
|
||||
},
|
||||
"worker_id": {
|
||||
"format": "uuid",
|
||||
"title": "Worker Id",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"evaluation_id",
|
||||
"attempt_id",
|
||||
"worker_id",
|
||||
"fencing_version"
|
||||
],
|
||||
"title": "LeaseBinding",
|
||||
"type": "object"
|
||||
},
|
||||
"ProgramFile": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"artifact": {
|
||||
"$ref": "#/$defs/ArtifactReference"
|
||||
},
|
||||
"path": {
|
||||
"maxLength": 128,
|
||||
"minLength": 1,
|
||||
"title": "Path",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"path",
|
||||
"artifact"
|
||||
],
|
||||
"title": "ProgramFile",
|
||||
"type": "object"
|
||||
},
|
||||
"ProgramKind": {
|
||||
"enum": [
|
||||
"native_executable",
|
||||
"python_source",
|
||||
"java_classes"
|
||||
],
|
||||
"title": "ProgramKind",
|
||||
"type": "string"
|
||||
},
|
||||
"RuntimeIdentity": {
|
||||
"additionalProperties": false,
|
||||
"description": "An exact identity matched against trusted configuration, never a host path.",
|
||||
"properties": {
|
||||
"image_digest": {
|
||||
"maxLength": 64,
|
||||
"minLength": 64,
|
||||
"pattern": "^[0-9a-f]{64}$",
|
||||
"title": "Image Digest",
|
||||
"type": "string"
|
||||
},
|
||||
"image_kind": {
|
||||
"enum": [
|
||||
"linux_gnu_cxx_x86_64_v1",
|
||||
"linux_cpython313_x86_64_v1",
|
||||
"linux_openjdk21_x86_64_v1"
|
||||
],
|
||||
"title": "Image Kind",
|
||||
"type": "string"
|
||||
},
|
||||
"image_revision": {
|
||||
"maximum": 4294967295,
|
||||
"minimum": 1,
|
||||
"title": "Image Revision",
|
||||
"type": "integer"
|
||||
},
|
||||
"runtime_version": {
|
||||
"pattern": "^[A-Za-z0-9][A-Za-z0-9_.+-]{0,63}$",
|
||||
"title": "Runtime Version",
|
||||
"type": "string"
|
||||
},
|
||||
"toolchain_id": {
|
||||
"pattern": "^[a-z0-9][a-z0-9_-]{0,63}$",
|
||||
"title": "Toolchain Id",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"toolchain_id",
|
||||
"image_kind",
|
||||
"image_revision",
|
||||
"image_digest",
|
||||
"runtime_version"
|
||||
],
|
||||
"title": "RuntimeIdentity",
|
||||
"type": "object"
|
||||
},
|
||||
"Verdict": {
|
||||
"enum": [
|
||||
"accepted",
|
||||
"wrong_answer",
|
||||
"compilation_error",
|
||||
"runtime_error",
|
||||
"time_limit_exceeded",
|
||||
"memory_limit_exceeded",
|
||||
"output_limit_exceeded",
|
||||
"cancelled",
|
||||
"system_error"
|
||||
],
|
||||
"title": "Verdict",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"$id": "urn:kf-oj:judge-result:v2",
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"binding": {
|
||||
"$ref": "#/$defs/LeaseBinding"
|
||||
},
|
||||
"cases": {
|
||||
"items": {
|
||||
"$ref": "#/$defs/CaseReport"
|
||||
},
|
||||
"maxItems": 256,
|
||||
"title": "Cases",
|
||||
"type": "array"
|
||||
},
|
||||
"cleanup_complete": {
|
||||
"title": "Cleanup Complete",
|
||||
"type": "boolean"
|
||||
},
|
||||
"compilation": {
|
||||
"$ref": "#/$defs/CompilationReport"
|
||||
},
|
||||
"failure_code": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/$defs/FailureCode"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"kind": {
|
||||
"const": "judge.result",
|
||||
"title": "Kind",
|
||||
"type": "string"
|
||||
},
|
||||
"manifest_digest": {
|
||||
"maxLength": 64,
|
||||
"minLength": 64,
|
||||
"pattern": "^[0-9a-f]{64}$",
|
||||
"title": "Manifest Digest",
|
||||
"type": "string"
|
||||
},
|
||||
"run_cpu_time_us": {
|
||||
"anyOf": [
|
||||
{
|
||||
"maximum": 9223372036854775807,
|
||||
"minimum": 0,
|
||||
"type": "integer"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"title": "Run Cpu Time Us"
|
||||
},
|
||||
"run_memory_peak_bytes": {
|
||||
"anyOf": [
|
||||
{
|
||||
"maximum": 9223372036854775807,
|
||||
"minimum": 0,
|
||||
"type": "integer"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"title": "Run Memory Peak Bytes"
|
||||
},
|
||||
"schema_version": {
|
||||
"const": 2,
|
||||
"title": "Schema Version",
|
||||
"type": "integer"
|
||||
},
|
||||
"verdict": {
|
||||
"$ref": "#/$defs/Verdict"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"schema_version",
|
||||
"kind",
|
||||
"binding",
|
||||
"manifest_digest",
|
||||
"compilation",
|
||||
"cases",
|
||||
"verdict",
|
||||
"cleanup_complete",
|
||||
"failure_code",
|
||||
"run_cpu_time_us",
|
||||
"run_memory_peak_bytes"
|
||||
],
|
||||
"title": "JudgementReport",
|
||||
"type": "object"
|
||||
}
|
||||
@@ -1,94 +0,0 @@
|
||||
{
|
||||
"$comment": "Structural schema; also run versioned model validation for cross-field budgets, result consistency, UUID canonical form and UTF-8 byte limits.",
|
||||
"$defs": {
|
||||
"LeaseBinding": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"attempt_id": {
|
||||
"format": "uuid",
|
||||
"title": "Attempt Id",
|
||||
"type": "string"
|
||||
},
|
||||
"evaluation_id": {
|
||||
"format": "uuid",
|
||||
"title": "Evaluation Id",
|
||||
"type": "string"
|
||||
},
|
||||
"fencing_version": {
|
||||
"maximum": 9223372036854775807,
|
||||
"minimum": 1,
|
||||
"title": "Fencing Version",
|
||||
"type": "integer"
|
||||
},
|
||||
"worker_id": {
|
||||
"format": "uuid",
|
||||
"title": "Worker Id",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"evaluation_id",
|
||||
"attempt_id",
|
||||
"worker_id",
|
||||
"fencing_version"
|
||||
],
|
||||
"title": "LeaseBinding",
|
||||
"type": "object"
|
||||
}
|
||||
},
|
||||
"$id": "urn:kf-oj:judge-stage:v1",
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"binding": {
|
||||
"$ref": "#/$defs/LeaseBinding"
|
||||
},
|
||||
"case_index": {
|
||||
"anyOf": [
|
||||
{
|
||||
"exclusiveMaximum": 256,
|
||||
"minimum": 0,
|
||||
"type": "integer"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"title": "Case Index"
|
||||
},
|
||||
"kind": {
|
||||
"const": "judge.stage",
|
||||
"title": "Kind",
|
||||
"type": "string"
|
||||
},
|
||||
"schema_version": {
|
||||
"const": 1,
|
||||
"title": "Schema Version",
|
||||
"type": "integer"
|
||||
},
|
||||
"sequence": {
|
||||
"maximum": 9223372036854775807,
|
||||
"minimum": 1,
|
||||
"title": "Sequence",
|
||||
"type": "integer"
|
||||
},
|
||||
"stage": {
|
||||
"enum": [
|
||||
"compiling",
|
||||
"running"
|
||||
],
|
||||
"title": "Stage",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"schema_version",
|
||||
"kind",
|
||||
"binding",
|
||||
"sequence",
|
||||
"stage",
|
||||
"case_index"
|
||||
],
|
||||
"title": "StageReport",
|
||||
"type": "object"
|
||||
}
|
||||
@@ -1,106 +0,0 @@
|
||||
{
|
||||
"$comment": "Multi-runtime protocol. Also validate runtime/source/attempt bindings, startup and node budgets, canonical bounded class sets and result consistency.",
|
||||
"$defs": {
|
||||
"LeaseBinding": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"attempt_id": {
|
||||
"format": "uuid",
|
||||
"title": "Attempt Id",
|
||||
"type": "string"
|
||||
},
|
||||
"evaluation_id": {
|
||||
"format": "uuid",
|
||||
"title": "Evaluation Id",
|
||||
"type": "string"
|
||||
},
|
||||
"fencing_version": {
|
||||
"maximum": 9223372036854775807,
|
||||
"minimum": 1,
|
||||
"title": "Fencing Version",
|
||||
"type": "integer"
|
||||
},
|
||||
"worker_id": {
|
||||
"format": "uuid",
|
||||
"title": "Worker Id",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"evaluation_id",
|
||||
"attempt_id",
|
||||
"worker_id",
|
||||
"fencing_version"
|
||||
],
|
||||
"title": "LeaseBinding",
|
||||
"type": "object"
|
||||
},
|
||||
"PreparationKind": {
|
||||
"enum": [
|
||||
"compile",
|
||||
"syntax_check"
|
||||
],
|
||||
"title": "PreparationKind",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"$id": "urn:kf-oj:judge-stage:v2",
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"binding": {
|
||||
"$ref": "#/$defs/LeaseBinding"
|
||||
},
|
||||
"case_index": {
|
||||
"anyOf": [
|
||||
{
|
||||
"exclusiveMaximum": 256,
|
||||
"minimum": 0,
|
||||
"type": "integer"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"title": "Case Index"
|
||||
},
|
||||
"kind": {
|
||||
"const": "judge.stage",
|
||||
"title": "Kind",
|
||||
"type": "string"
|
||||
},
|
||||
"preparation_kind": {
|
||||
"$ref": "#/$defs/PreparationKind"
|
||||
},
|
||||
"schema_version": {
|
||||
"const": 2,
|
||||
"title": "Schema Version",
|
||||
"type": "integer"
|
||||
},
|
||||
"sequence": {
|
||||
"maximum": 9223372036854775807,
|
||||
"minimum": 1,
|
||||
"title": "Sequence",
|
||||
"type": "integer"
|
||||
},
|
||||
"stage": {
|
||||
"enum": [
|
||||
"compiling",
|
||||
"running"
|
||||
],
|
||||
"title": "Stage",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"schema_version",
|
||||
"kind",
|
||||
"binding",
|
||||
"sequence",
|
||||
"stage",
|
||||
"case_index",
|
||||
"preparation_kind"
|
||||
],
|
||||
"title": "StageReport",
|
||||
"type": "object"
|
||||
}
|
||||
@@ -1,69 +0,0 @@
|
||||
{
|
||||
"$comment": "Structural schema; also run versioned model validation for cross-field budgets, result consistency, UUID canonical form and UTF-8 byte limits.",
|
||||
"$defs": {
|
||||
"LeaseBinding": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"attempt_id": {
|
||||
"format": "uuid",
|
||||
"title": "Attempt Id",
|
||||
"type": "string"
|
||||
},
|
||||
"evaluation_id": {
|
||||
"format": "uuid",
|
||||
"title": "Evaluation Id",
|
||||
"type": "string"
|
||||
},
|
||||
"fencing_version": {
|
||||
"maximum": 9223372036854775807,
|
||||
"minimum": 1,
|
||||
"title": "Fencing Version",
|
||||
"type": "integer"
|
||||
},
|
||||
"worker_id": {
|
||||
"format": "uuid",
|
||||
"title": "Worker Id",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"evaluation_id",
|
||||
"attempt_id",
|
||||
"worker_id",
|
||||
"fencing_version"
|
||||
],
|
||||
"title": "LeaseBinding",
|
||||
"type": "object"
|
||||
}
|
||||
},
|
||||
"$id": "urn:kf-oj:judge-stopped:v1",
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"binding": {
|
||||
"$ref": "#/$defs/LeaseBinding"
|
||||
},
|
||||
"cleanup_complete": {
|
||||
"title": "Cleanup Complete",
|
||||
"type": "boolean"
|
||||
},
|
||||
"kind": {
|
||||
"const": "judge.stopped",
|
||||
"title": "Kind",
|
||||
"type": "string"
|
||||
},
|
||||
"schema_version": {
|
||||
"const": 1,
|
||||
"title": "Schema Version",
|
||||
"type": "integer"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"schema_version",
|
||||
"kind",
|
||||
"binding",
|
||||
"cleanup_complete"
|
||||
],
|
||||
"title": "StoppedReport",
|
||||
"type": "object"
|
||||
}
|
||||
@@ -1,69 +0,0 @@
|
||||
{
|
||||
"$comment": "Multi-runtime protocol. Also validate runtime/source/attempt bindings, startup and node budgets, canonical bounded class sets and result consistency.",
|
||||
"$defs": {
|
||||
"LeaseBinding": {
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"attempt_id": {
|
||||
"format": "uuid",
|
||||
"title": "Attempt Id",
|
||||
"type": "string"
|
||||
},
|
||||
"evaluation_id": {
|
||||
"format": "uuid",
|
||||
"title": "Evaluation Id",
|
||||
"type": "string"
|
||||
},
|
||||
"fencing_version": {
|
||||
"maximum": 9223372036854775807,
|
||||
"minimum": 1,
|
||||
"title": "Fencing Version",
|
||||
"type": "integer"
|
||||
},
|
||||
"worker_id": {
|
||||
"format": "uuid",
|
||||
"title": "Worker Id",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"evaluation_id",
|
||||
"attempt_id",
|
||||
"worker_id",
|
||||
"fencing_version"
|
||||
],
|
||||
"title": "LeaseBinding",
|
||||
"type": "object"
|
||||
}
|
||||
},
|
||||
"$id": "urn:kf-oj:judge-stopped:v2",
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"binding": {
|
||||
"$ref": "#/$defs/LeaseBinding"
|
||||
},
|
||||
"cleanup_complete": {
|
||||
"title": "Cleanup Complete",
|
||||
"type": "boolean"
|
||||
},
|
||||
"kind": {
|
||||
"const": "judge.stopped",
|
||||
"title": "Kind",
|
||||
"type": "string"
|
||||
},
|
||||
"schema_version": {
|
||||
"const": 2,
|
||||
"title": "Schema Version",
|
||||
"type": "integer"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"schema_version",
|
||||
"kind",
|
||||
"binding",
|
||||
"cleanup_complete"
|
||||
],
|
||||
"title": "StoppedReport",
|
||||
"type": "object"
|
||||
}
|
||||
@@ -19,127 +19,3 @@ API 存活检查和前端预览不依赖这些容器,可以先独立启动。
|
||||
|
||||
官方镜像说明:[PostgreSQL](https://hub.docker.com/_/postgres)、
|
||||
[RabbitMQ](https://hub.docker.com/_/rabbitmq)。
|
||||
|
||||
### 单机 helper 工厂
|
||||
|
||||
`kf-sandbox factory --config /etc/kf-judge/runtime.json` 是 Root 服务入口;
|
||||
普通 Worker 使用配置中 `state_root/factory/control.sock` 的普通用户客户端。
|
||||
`deploy/kf-sandbox-factory.service` 给出有限服务预算示例,安装前由可信管理员
|
||||
核对固定二进制、不可写 GNU 镜像/配置、Worker UID/GID,以及**与服务自身 cgroup
|
||||
分离**的 Root 所有任务委派根。工厂实际读回服务 cgroup 限额,应用/读回任务总预算;
|
||||
每次编译和测试点仍须通过已有完整隔离门控。服务文件本身不证明这些条件成立。
|
||||
|
||||
工厂只接收有界 run ID,无排队:槽位满、重复 ID 或未知字段立即拒绝。
|
||||
每次申请使用全新单线程 helper,固定继承已认证的连接与工厂生命线;
|
||||
额外控制帧、断开或生命线消失会终止整棵任务树。ID 先写入有限持久记录
|
||||
(最多 4096 个,跨重启拒绝重放);容量用尽需要管理员在所有任务停止、已核对
|
||||
回收后安排新记录周期,不自动删除记录。清理失败保留身份租约并停止新分配;
|
||||
重启只根据已记录归属和 inode 回收,未知/外来内容保留并拒绝启动。
|
||||
|
||||
`kf-worker judge-submission` 是受控本地适配入口。常驻 `kf-worker serve --config`
|
||||
使用有界 AMQP 通知和经过服务身份认证的 Coordinator claim;配置与依赖见
|
||||
[判题工程说明](../judge/README.md#单机持久化协调与常驻-worker)。
|
||||
它验证不可变清单摘要,C17/C++20 仅映射到固定 GNU 命令,编译一次;
|
||||
成功产物经完整清理后封印并计算 SHA-256,每点分别申请 helper、stdin 与工作区。
|
||||
期望值逐点读取,留在比较端,不传给工厂/用户文件系统。
|
||||
新增锁定 `sha2 0.10.9` 仅供 Worker 验证输入及编译产物摘要;
|
||||
Sandbox 同版本仅为独立 VM 测试数据生成的开发依赖,Engine 不依赖加密或 OS 库。
|
||||
|
||||
### 单机提交服务的启动与停止
|
||||
|
||||
1. Backend Alembic 迁移;创建分别仅继承平台/协调器 NOLOGIN 角色的专用登录账号,
|
||||
平台角色只可插入初始评测,执行状态仅由 Coordinator 写入。
|
||||
2. 普通平台服务持有私有工件目录;使用应用服务导入不可变题目和创建测试账号。
|
||||
3. 验证固定 GNU 镜像、Root 配置及工厂服务/任务 cgroup 的真实上限,再启动工厂。
|
||||
4. 独立启动 API、Coordinator、outbox publisher 和普通 Worker;单机 HTTP/AMQP
|
||||
仅数值 loopback,PostgreSQL loopback/Unix socket。任何入口都不自动创建表。
|
||||
5. Coordinator readiness 验证数据库和权限;仍需独立 VM 验证真实执行门控。
|
||||
Root 工厂不可持有平台/MQ/数据库口令,用户运行镜像不含服务文件。
|
||||
6. 先向 Worker 发送 SIGTERM,使当前任务取消/清理,再停止 publisher/API/Coordinator,
|
||||
最后停止工厂和本地依赖。强制死亡后的旧 attempt 必须撤销并取得实际停止证明;
|
||||
`cleanup_failed` 不等于普通终态,先核对保留资源再由管理员恢复工厂/Worker。
|
||||
|
||||
平台配置私有 `KF_OJ_ARTIFACT_ROOT`、有界字节/文件容量以及
|
||||
`KF_OJ_JUDGE_ADMISSION`(完整可信 JSON);其工具链、编译/运行上限、总体作业期限、
|
||||
并发/prefetch/retry 必须与 Coordinator/Worker 一致。单机使用并发1、prefetch1,
|
||||
不能仅提高 Worker 数量绕过节点容量。源码最多256KiB,测试点最多256个;
|
||||
stdout/stderr 各不超过 file_bytes,合计不超过 workdir_bytes。
|
||||
|
||||
迁移账号与平台账号分开配置,命令均在 `backend/` 运行:
|
||||
|
||||
```sh
|
||||
uv run --locked alembic upgrade head # 私有迁移账号,仅执行受控 schema 操作
|
||||
uv run --locked kf-accounts create local_user # 换成平台角色;终端读取密码并二次确认
|
||||
uv run --locked kf-problems /srv/private-import
|
||||
uv run --locked uvicorn kf_oj.api.app:create_app --factory --host 127.0.0.1 --port 8000 \
|
||||
--no-proxy-headers --limit-concurrency 32 --backlog 64
|
||||
uv run --locked python -m kf_oj.infra.messaging.publisher
|
||||
```
|
||||
|
||||
每个账号操作/导入目录都由管理员准备,不从浏览器接受私有路径。导入格式见
|
||||
[题目模块](../backend/src/kf_oj/modules/problems/README.md),不可变版本以 UUID
|
||||
在题库/题面链接和提交请求中传递。更新题目需导入新版本,不覆盖已有提交的题目版本。
|
||||
临时验收使用合成数据和测试口令,实际业务数据与服务配置都保存在仓库外的私有目录。
|
||||
|
||||
取消另需 API 私有配置 `KF_OJ_COORDINATOR_URL=http://127.0.0.1:8100` 与
|
||||
`KF_OJ_COORDINATOR_TOKEN`(Coordinator 的 platform_token)。所有服务令牌私下生成,
|
||||
不可互用或放进前端环境。Coordinator/Worker 私有 JSON 为0600、单链接、有界文件;
|
||||
Root 镜像/配置/二进制由 root 持有,任务委派子树独立于监督器自身 cgroup。
|
||||
运行前实际读回整体节点、服务和任务 memory/swap/pids/CPU/文件/运行期限上限,
|
||||
并通过真实隔离门控;配置解析、readiness、存活探测都不能替代执行验收。
|
||||
|
||||
前端从 `frontend/` 启动 `pnpm dev`,默认127.0.0.1:5173;平台明确允许相同浏览器
|
||||
Origin,`/api` 同源代理。生产需另行配置 HTTPS、安全 cookie、SPA 回退与 `/api`
|
||||
反向代理,本仓库不自动部署。用户以受控账号登录,在题面选择支持语言、提交源码,
|
||||
随后查看同一 submission UUID 的持久化状态。等待/取消中/系统失败都不等于成功成绩。
|
||||
|
||||
broker 暂停时 API 仍能持久化 queued/outbox;恢复后 publisher confirm 与 Coordinator
|
||||
补偿重新投递。Worker/Coordinator 重启使用原私有配置、工件、日志与 DB,不删除
|
||||
attempt 或强改状态。旧租约不能覆盖新结果,有限基础设施重试最终收敛。
|
||||
容量满时返回429或503;先确认待执行/清理状态,恢复服务或由管理员扩充经过核对的
|
||||
存储容量,不删除仍被提交/题目引用的工件。`kf-artifacts --limit 100` 仅回收满足年龄
|
||||
要求的无引用工件和已标记删除项;它不会通过删除用户源码来腾出容量。
|
||||
恢复 Root 必须核对 journal/inode/实际排空;未知资源保留,管理员处理自己确认的
|
||||
故障来源后再重启。不要递归清空任务目录,也不要把 cleanup_failed 改成已成功。
|
||||
|
||||
这些入口和顺序不表示生产部署已完成。隔离/超限与用户程序仅在无 NIC、无宿主共享
|
||||
挂载、无业务凭据且整体上限已读回的独立测试 VM 内验收。开发 Compose 仍只提供可信
|
||||
数据库和 broker,不在宿主或业务服务器运行用户代码。workflow 如需启用由指定
|
||||
GitHub 私有仓库承载,并设置有限并发;Gitea 只保存逐阶段工程提交。
|
||||
|
||||
|
||||
## 四语言节点的受控升级
|
||||
|
||||
新增公开示例 `judge.admission.example.json` 为显式 v1/v2 注册表,
|
||||
`sandbox.managed.runtime.example.json` 为一槽位的共享 Root 配置;原 v1 示例保持原语义。
|
||||
所有 `aaaaaaaa…` 镜像摘要均是无权限占位值,不能当作已验镜像。管理员制作固定镜像后,
|
||||
对 manifest.json 的实际字节计算 SHA-256,逐文件验证版本/字节/权限,并将相同完整身份
|
||||
写入准入注册表与 Root runtime_images。Root 镜像路径仅在私有可信配置中保存。
|
||||
配置解析通过或 check-runtime 可见仍不允许跳过实际隔离门控。
|
||||
|
||||
1. 停止创建新提交/消费投递,向所有普通 Worker 发 SIGTERM;保留 DB、清单、工件和 journal。
|
||||
2. 取得工厂实际 stopped/cleanup_complete 证明,读回空任务 cgroup 和 owned 资源;
|
||||
未排空或存在未知资源时停止升级,保留证据,不递归清空任务目录。
|
||||
3. 使用独立迁移账号执行 Backend Alembic upgrade;新语言数据存在时旧迁移降级会拒绝,
|
||||
不删除数据以实现回退。平台与 Coordinator 继续使用各自最小权限角色,Worker 无 DB 账号。
|
||||
4. 安装固定版本的 Backend/Coordinator/Worker/Sandbox;同时更新三个普通服务的同一
|
||||
`judge_admission`/`admission` 注册表,Root 配置单独由管理员持有。旧消费者不混用新协议。
|
||||
5. 为整体节点、控制面各服务、工厂和任务设置有限 CPU/memory/swap/pids/文件/期限,
|
||||
读回内核值。节点至少预留一份512MiB编译任务以及独立控制面;Java运行示例总预算384MiB,
|
||||
Xmx不能代替整体预算。并发1/prefetch1,共享UID池,不因第二Worker存在而扩大节点槽位。
|
||||
6. 在独立无NIC、无宿主共享挂载、无业务凭据 VM 用合成题目验证四语言、故障与实际排空,
|
||||
再恢复工厂/Coordinator/API/publisher/Worker。生产启用需单独安排,仓库不自动部署。
|
||||
|
||||
请求不在线安装 Python/Java 依赖,不运行 pip/uvx,不接收自定义解释器参数/JAR/agents。
|
||||
Worker 私有配置可设置 `control_timeout_ms`(1000–5000,缺省2000)作为 Coordinator
|
||||
单次控制请求的期限。资源受限节点应依据实际领取/续租/停止耗时选择,并核对租约和
|
||||
停止恢复;这不改变学生阶段/作业期限、重试次数或工件下载的固定2秒期限,连接仍限1秒。
|
||||
Python main.py 仅标准库;Java Main.java 使用默认包 Main,最多256个有界编译class。
|
||||
题目语言与预算以公开版本配置为准,旧题目和草稿保留,Rust继续明确不支持。
|
||||
新版检查阶段CE/运行RE/真实限额与SystemError的区别见 [判题说明](../judge/README.md#python--java-与明确协议版本)。
|
||||
|
||||
停止/故障恢复沿原单机流程:停止Worker使任务全树排空,旧租约/report拒绝后恢复;
|
||||
publisher/broker中断保留outbox,有限重试而非清空记录。恢复旧软件前应先排空v2任务,
|
||||
保留不可变v2版本和报告供兼容消费者读取;不能把旧配置静默转换为宽松新能力。
|
||||
这些公开配置不含业务令牌。数据库/AMQP/服务令牌分别在仓库外私下生成,Root不持有它们。
|
||||
需要CI时只在GitHub私有仓库 `KiriAky107/KF-OnlineJudge` 运行并设置有限并发,Gitea保存源码。
|
||||
|
||||
@@ -1,185 +0,0 @@
|
||||
{
|
||||
"schema_version": 2,
|
||||
"legacy": {
|
||||
"max_concurrent_jobs": 1,
|
||||
"prefetch": 1,
|
||||
"max_retries": 2,
|
||||
"max_job_wall_time_us": 60000000,
|
||||
"max_package_bytes": 67108864,
|
||||
"compile_ceiling": {
|
||||
"cpu_time_us": 10000000,
|
||||
"wall_time_us": 30000000,
|
||||
"memory_bytes": 536870912,
|
||||
"stack_bytes": 16777216,
|
||||
"processes": 32,
|
||||
"open_files": 128,
|
||||
"stdout_bytes": 1048576,
|
||||
"stderr_bytes": 1048576,
|
||||
"file_bytes": 67108864,
|
||||
"workdir_bytes": 268435456
|
||||
},
|
||||
"run_ceiling": {
|
||||
"cpu_time_us": 1000000,
|
||||
"wall_time_us": 3000000,
|
||||
"memory_bytes": 67108864,
|
||||
"stack_bytes": 8388608,
|
||||
"processes": 8,
|
||||
"open_files": 64,
|
||||
"stdout_bytes": 1048576,
|
||||
"stderr_bytes": 1048576,
|
||||
"file_bytes": 16777216,
|
||||
"workdir_bytes": 67108864
|
||||
},
|
||||
"toolchain": {
|
||||
"toolchain_id": "gnu-cxx-v1",
|
||||
"image_kind": "linux_gnu_cxx_x86_64_v1",
|
||||
"image_revision": 1
|
||||
}
|
||||
},
|
||||
"multi_runtime": {
|
||||
"max_concurrent_jobs": 1,
|
||||
"prefetch": 1,
|
||||
"max_retries": 2,
|
||||
"max_job_wall_time_us": 60000000,
|
||||
"max_package_bytes": 67108864,
|
||||
"schema_version": 2,
|
||||
"capabilities": [
|
||||
{
|
||||
"language": "c17",
|
||||
"toolchain": {
|
||||
"toolchain_id": "gnu-cxx-v1",
|
||||
"image_kind": "linux_gnu_cxx_x86_64_v1",
|
||||
"image_revision": 1,
|
||||
"image_digest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"runtime_version": "gnu-cxx-v1"
|
||||
},
|
||||
"compile_ceiling": {
|
||||
"cpu_time_us": 10000000,
|
||||
"wall_time_us": 30000000,
|
||||
"memory_bytes": 536870912,
|
||||
"stack_bytes": 16777216,
|
||||
"processes": 32,
|
||||
"open_files": 128,
|
||||
"stdout_bytes": 1048576,
|
||||
"stderr_bytes": 1048576,
|
||||
"file_bytes": 67108864,
|
||||
"workdir_bytes": 268435456
|
||||
},
|
||||
"run_ceiling": {
|
||||
"cpu_time_us": 1000000,
|
||||
"wall_time_us": 3000000,
|
||||
"memory_bytes": 67108864,
|
||||
"stack_bytes": 8388608,
|
||||
"processes": 8,
|
||||
"open_files": 64,
|
||||
"stdout_bytes": 1048576,
|
||||
"stderr_bytes": 1048576,
|
||||
"file_bytes": 16777216,
|
||||
"workdir_bytes": 67108864
|
||||
}
|
||||
},
|
||||
{
|
||||
"language": "cpp20",
|
||||
"toolchain": {
|
||||
"toolchain_id": "gnu-cxx-v1",
|
||||
"image_kind": "linux_gnu_cxx_x86_64_v1",
|
||||
"image_revision": 1,
|
||||
"image_digest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"runtime_version": "gnu-cxx-v1"
|
||||
},
|
||||
"compile_ceiling": {
|
||||
"cpu_time_us": 10000000,
|
||||
"wall_time_us": 30000000,
|
||||
"memory_bytes": 536870912,
|
||||
"stack_bytes": 16777216,
|
||||
"processes": 32,
|
||||
"open_files": 128,
|
||||
"stdout_bytes": 1048576,
|
||||
"stderr_bytes": 1048576,
|
||||
"file_bytes": 67108864,
|
||||
"workdir_bytes": 268435456
|
||||
},
|
||||
"run_ceiling": {
|
||||
"cpu_time_us": 1000000,
|
||||
"wall_time_us": 3000000,
|
||||
"memory_bytes": 67108864,
|
||||
"stack_bytes": 8388608,
|
||||
"processes": 8,
|
||||
"open_files": 64,
|
||||
"stdout_bytes": 1048576,
|
||||
"stderr_bytes": 1048576,
|
||||
"file_bytes": 16777216,
|
||||
"workdir_bytes": 67108864
|
||||
}
|
||||
},
|
||||
{
|
||||
"language": "python313",
|
||||
"toolchain": {
|
||||
"toolchain_id": "cpython313-v1",
|
||||
"image_kind": "linux_cpython313_x86_64_v1",
|
||||
"image_revision": 1,
|
||||
"image_digest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"runtime_version": "3.13.16"
|
||||
},
|
||||
"compile_ceiling": {
|
||||
"cpu_time_us": 1000000,
|
||||
"wall_time_us": 3000000,
|
||||
"memory_bytes": 67108864,
|
||||
"stack_bytes": 8388608,
|
||||
"processes": 8,
|
||||
"open_files": 64,
|
||||
"stdout_bytes": 1048576,
|
||||
"stderr_bytes": 1048576,
|
||||
"file_bytes": 16777216,
|
||||
"workdir_bytes": 67108864
|
||||
},
|
||||
"run_ceiling": {
|
||||
"cpu_time_us": 1000000,
|
||||
"wall_time_us": 3000000,
|
||||
"memory_bytes": 67108864,
|
||||
"stack_bytes": 8388608,
|
||||
"processes": 8,
|
||||
"open_files": 64,
|
||||
"stdout_bytes": 1048576,
|
||||
"stderr_bytes": 1048576,
|
||||
"file_bytes": 16777216,
|
||||
"workdir_bytes": 67108864
|
||||
}
|
||||
},
|
||||
{
|
||||
"language": "java21",
|
||||
"toolchain": {
|
||||
"toolchain_id": "openjdk21-v1",
|
||||
"image_kind": "linux_openjdk21_x86_64_v1",
|
||||
"image_revision": 1,
|
||||
"image_digest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"runtime_version": "21.0.12.1+1-LTS"
|
||||
},
|
||||
"compile_ceiling": {
|
||||
"cpu_time_us": 10000000,
|
||||
"wall_time_us": 30000000,
|
||||
"memory_bytes": 536870912,
|
||||
"stack_bytes": 16777216,
|
||||
"processes": 32,
|
||||
"open_files": 128,
|
||||
"stdout_bytes": 1048576,
|
||||
"stderr_bytes": 1048576,
|
||||
"file_bytes": 67108864,
|
||||
"workdir_bytes": 268435456
|
||||
},
|
||||
"run_ceiling": {
|
||||
"cpu_time_us": 1000000,
|
||||
"wall_time_us": 3000000,
|
||||
"memory_bytes": 402653184,
|
||||
"stack_bytes": 8388608,
|
||||
"processes": 32,
|
||||
"open_files": 64,
|
||||
"stdout_bytes": 1048576,
|
||||
"stderr_bytes": 1048576,
|
||||
"file_bytes": 16777216,
|
||||
"workdir_bytes": 67108864
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -1,28 +0,0 @@
|
||||
[Unit]
|
||||
Description=KF bounded Linux helper factory
|
||||
After=local-fs.target
|
||||
|
||||
[Service]
|
||||
Type=exec
|
||||
User=root
|
||||
Group=root
|
||||
# Trusted operator installs this exact binary/config and separate delegated
|
||||
# task root. No caller-controlled executable, image or environment is accepted.
|
||||
ExecStart=/usr/bin/env -i PATH=/usr/bin:/bin LC_ALL=C /usr/lib/kf-judge/kf-sandbox factory --config /etc/kf-judge/runtime.json --lifetime-seconds 3600
|
||||
MemoryMax=256M
|
||||
MemorySwapMax=0
|
||||
TasksMax=32
|
||||
CPUQuota=50%
|
||||
LimitFSIZE=64M
|
||||
RuntimeMaxSec=3700
|
||||
TimeoutStopSec=30
|
||||
# First stop the factory; guardians cancel live helpers through their normal
|
||||
# cleanup path. Remaining processes are terminated at the finite stop deadline.
|
||||
KillMode=mixed
|
||||
Restart=no
|
||||
UMask=0077
|
||||
# Requires root namespace/mount/cgroup privileges. Do not set NoNewPrivileges or
|
||||
# an inherited seccomp filter: the existing execution gate intentionally refuses.
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -1,111 +0,0 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"state_root": "/var/lib/kf-judge",
|
||||
"tasks_root": "/var/lib/kf-judge/tasks",
|
||||
"image_root": "/var/lib/kf-judge/images/gnu-cxx-x86_64-v1",
|
||||
"image_kind": "linux_gnu_cxx_x86_64_v1",
|
||||
"delegated_cgroup_root": "/sys/fs/cgroup/kf-judge.service",
|
||||
"identities": {
|
||||
"worker_uid": 1000,
|
||||
"worker_gid": 1000,
|
||||
"first_task_uid": 100000,
|
||||
"first_task_gid": 100000,
|
||||
"count": 1
|
||||
},
|
||||
"max_concurrent_runs": 1,
|
||||
"per_task_ceiling": {
|
||||
"cpu_time_us": 10000000,
|
||||
"wall_time_us": 30000000,
|
||||
"memory_bytes": 536870912,
|
||||
"stack_bytes": 16777216,
|
||||
"processes": 32,
|
||||
"open_files": 128,
|
||||
"stdout_bytes": 1048576,
|
||||
"stderr_bytes": 1048576,
|
||||
"file_bytes": 67108864,
|
||||
"workdir_bytes": 268435456
|
||||
},
|
||||
"aggregate": {
|
||||
"memory_bytes": 805306368,
|
||||
"supervisor_memory_bytes": 268435456,
|
||||
"processes": 64,
|
||||
"supervisor_processes": 32,
|
||||
"workdir_bytes": 402653184,
|
||||
"supervisor_workdir_bytes": 67108864,
|
||||
"cpu_period_us": 100000,
|
||||
"cpu_quota_us": 150000,
|
||||
"task_cpu_quota_us": 100000,
|
||||
"supervisor_cpu_quota_us": 50000
|
||||
},
|
||||
"runtime_images": [
|
||||
{
|
||||
"image_root": "/var/lib/kf-judge/images/gnu-cxx-v2",
|
||||
"image_kind": "linux_gnu_cxx_x86_64_v1",
|
||||
"runtime_identity": {
|
||||
"toolchain_id": "gnu-cxx-v1",
|
||||
"image_kind": "linux_gnu_cxx_x86_64_v1",
|
||||
"image_revision": 1,
|
||||
"image_digest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"runtime_version": "gnu-cxx-v1"
|
||||
},
|
||||
"per_task_ceiling": {
|
||||
"cpu_time_us": 10000000,
|
||||
"wall_time_us": 30000000,
|
||||
"memory_bytes": 536870912,
|
||||
"stack_bytes": 16777216,
|
||||
"processes": 32,
|
||||
"open_files": 128,
|
||||
"stdout_bytes": 1048576,
|
||||
"stderr_bytes": 1048576,
|
||||
"file_bytes": 67108864,
|
||||
"workdir_bytes": 268435456
|
||||
}
|
||||
},
|
||||
{
|
||||
"image_root": "/var/lib/kf-judge/images/cpython313-v1",
|
||||
"image_kind": "linux_cpython313_x86_64_v1",
|
||||
"runtime_identity": {
|
||||
"toolchain_id": "cpython313-v1",
|
||||
"image_kind": "linux_cpython313_x86_64_v1",
|
||||
"image_revision": 1,
|
||||
"image_digest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"runtime_version": "3.13.16"
|
||||
},
|
||||
"per_task_ceiling": {
|
||||
"cpu_time_us": 10000000,
|
||||
"wall_time_us": 30000000,
|
||||
"memory_bytes": 536870912,
|
||||
"stack_bytes": 16777216,
|
||||
"processes": 32,
|
||||
"open_files": 128,
|
||||
"stdout_bytes": 1048576,
|
||||
"stderr_bytes": 1048576,
|
||||
"file_bytes": 67108864,
|
||||
"workdir_bytes": 268435456
|
||||
}
|
||||
},
|
||||
{
|
||||
"image_root": "/var/lib/kf-judge/images/openjdk21-v1",
|
||||
"image_kind": "linux_openjdk21_x86_64_v1",
|
||||
"runtime_identity": {
|
||||
"toolchain_id": "openjdk21-v1",
|
||||
"image_kind": "linux_openjdk21_x86_64_v1",
|
||||
"image_revision": 1,
|
||||
"image_digest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"runtime_version": "21.0.12.1+1-LTS"
|
||||
},
|
||||
"per_task_ceiling": {
|
||||
"cpu_time_us": 10000000,
|
||||
"wall_time_us": 30000000,
|
||||
"memory_bytes": 536870912,
|
||||
"stack_bytes": 16777216,
|
||||
"processes": 32,
|
||||
"open_files": 128,
|
||||
"stdout_bytes": 1048576,
|
||||
"stderr_bytes": 1048576,
|
||||
"file_bytes": 67108864,
|
||||
"workdir_bytes": 268435456
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,40 +0,0 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"state_root": "/var/lib/kf-judge",
|
||||
"tasks_root": "/var/lib/kf-judge/tasks",
|
||||
"image_root": "/var/lib/kf-judge/images/gnu-cxx-x86_64-v1",
|
||||
"image_kind": "linux_gnu_cxx_x86_64_v1",
|
||||
"delegated_cgroup_root": "/sys/fs/cgroup/kf-judge.service",
|
||||
"identities": {
|
||||
"worker_uid": 1000,
|
||||
"worker_gid": 1000,
|
||||
"first_task_uid": 100000,
|
||||
"first_task_gid": 100000,
|
||||
"count": 2
|
||||
},
|
||||
"max_concurrent_runs": 2,
|
||||
"per_task_ceiling": {
|
||||
"cpu_time_us": 10000000,
|
||||
"wall_time_us": 30000000,
|
||||
"memory_bytes": 536870912,
|
||||
"stack_bytes": 16777216,
|
||||
"processes": 32,
|
||||
"open_files": 128,
|
||||
"stdout_bytes": 1048576,
|
||||
"stderr_bytes": 1048576,
|
||||
"file_bytes": 67108864,
|
||||
"workdir_bytes": 268435456
|
||||
},
|
||||
"aggregate": {
|
||||
"memory_bytes": 1610612736,
|
||||
"supervisor_memory_bytes": 268435456,
|
||||
"processes": 128,
|
||||
"supervisor_processes": 32,
|
||||
"workdir_bytes": 1073741824,
|
||||
"supervisor_workdir_bytes": 67108864,
|
||||
"cpu_period_us": 100000,
|
||||
"cpu_quota_us": 250000,
|
||||
"task_cpu_quota_us": 100000,
|
||||
"supervisor_cpu_quota_us": 50000
|
||||
}
|
||||
}
|
||||
+2
-23
@@ -1,8 +1,7 @@
|
||||
# Vue 前端工程
|
||||
|
||||
Vue 3 + TypeScript + Vite + Element Plus,使用 Vue Router 与 Pinia。
|
||||
登录、已发布题库、题面、提交与个人成绩页面使用真实 API。训练、比赛、社区等
|
||||
本轮以外页面仍为明确的开发预览,不提供这些尚未实现的业务能力。
|
||||
当前页面是开发预览和实际 API 连接检查;没有模拟登录或模拟评测结果。
|
||||
|
||||
```bash
|
||||
pnpm install --frozen-lockfile
|
||||
@@ -25,24 +24,4 @@ pnpm build
|
||||
API 类型更新执行根目录 `make contracts`;前端通过 `pnpm check:api` 拒绝过期类型。
|
||||
登录凭据、模型密钥和数据库连接不能进入 `VITE_` 环境变量。
|
||||
|
||||
已有账号登录使用 HttpOnly/Strict cookie 与单独 CSRF cookie;401清除内存身份,
|
||||
保护页面重新登录后继续。自助注册暂不开放。密码/会话不存入 localStorage。
|
||||
浏览器与 API 使用同一 origin 的 `/api`,明确配置 `KF_OJ_BROWSER_ORIGINS`;
|
||||
生产必须使用 HTTPS、安全 cookie 与独立部署的反向代理。
|
||||
|
||||
编辑器按题目版本和语言保留本地草稿,新增 Java 并保留原 C17/C++/Python/Rust 草稿。
|
||||
可提交语言由服务端已发布版本的能力与语言配置决定,支持 C17/C++20、固定 CPython
|
||||
3.13.16 的单文件 main.py/标准库,以及固定 OpenJDK 21.0.12.1+1-LTS 的默认包
|
||||
Main.java/Main;Java 可包含受控的辅助、内部和匿名类。缺失或无效配置不能提交,
|
||||
Rust 当前保留草稿且不可提交。Python 准备阶段只检查语法,不执行顶层代码;
|
||||
Java 编译错误显示真实诊断,语言版本与预算取自服务端。请求不在线安装依赖。
|
||||
提交凭据按账号/版本/内容 SHA-256 绑定,
|
||||
POST前先保存 UUID,双击共用请求,网络超时/刷新重试复用原键;内容变化产生新键。
|
||||
草稿和已保存凭据不因失败/退出而清除,本地存储不可用时拒绝发起无法稳定重试的提交。
|
||||
|
||||
详情和列表查询每2秒一次,每轮最多240次/10分钟;终态、错误或离页停止,隐藏页面
|
||||
取消当前查询,恢复可见时继续原预算。路由变化/会话变化后的旧响应不能覆盖当前视图。
|
||||
耗尽后可手动刷新;不会用预览成绩填补错误。取消受理与最终停止分别展示。
|
||||
运行指标仅来自完整持久化报告,null显示“—”;诊断和源码按纯文本显示。
|
||||
`pnpm test` 覆盖草稿、幂等重试、轮询期限/隐藏/终态/失效响应和401处理,
|
||||
已纳入 `make check-frontend`;真实浏览器编译/判题另在受限独立 VM 验收。
|
||||
`pnpm test` 验证代码草稿在刷新、切到后台与页面离开时的保存行为,已纳入 `make check-frontend`。
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user