feat: 完成 OpenNexus 第三阶段核心功能与生产化基础 #45
@@ -519,3 +519,15 @@ Core 的独立数据目录目前不等于已授权 Vault。Python 旧笔记写
|
|||||||
- 单元测试覆盖预览对象被改写后实际参数仍冻结、重复确认、跨实例、目录代次、确认前/后过期、条数/字节上限及清理后复用。真实 AppContainer 测试核对预览来源/包/Vault,修改展示副本不会改变实际请求,旧目录凭据被拒绝;新增确认后锁定保险库导致 CREDENTIALS_LOCKED,重新解锁也不能恢复已失效会话。
|
- 单元测试覆盖预览对象被改写后实际参数仍冻结、重复确认、跨实例、目录代次、确认前/后过期、条数/字节上限及清理后复用。真实 AppContainer 测试核对预览来源/包/Vault,修改展示副本不会改变实际请求,旧目录凭据被拒绝;新增确认后锁定保险库导致 CREDENTIALS_LOCKED,重新解锁也不能恢复已失效会话。
|
||||||
- 64 项扩展回归通过,5 项 ignored 为三个父测试驱动 Job 辅助入口及两个显式长时验收,本轮未重跑长时项。desktop 全目标 Clippy -D warnings 通过。日志 `.build/extension-call-binding-tests.log`、`.build/extension-call-binding-clippy.log`。
|
- 64 项扩展回归通过,5 项 ignored 为三个父测试驱动 Job 辅助入口及两个显式长时验收,本轮未重跑长时项。desktop 全目标 Clippy -D warnings 通过。日志 `.build/extension-call-binding-tests.log`、`.build/extension-call-binding-clippy.log`。
|
||||||
- 当前 confirm_call 仍只是 Host 内部组件接口,未登记为 renderer/Core 命令,也没有自动批准策略或真实用户确认 UI。完整产品入口必须通过经过认证的 Host 路由确认用户批准,再与活动安装/在线信任、实例注册和 Core 路由组合;全部沙箱资源与其他生产化验收仍未完成,第三方执行能力保持禁用。
|
- 当前 confirm_call 仍只是 Host 内部组件接口,未登记为 renderer/Core 命令,也没有自动批准策略或真实用户确认 UI。完整产品入口必须通过经过认证的 Host 路由确认用户批准,再与活动安装/在线信任、实例注册和 Core 路由组合;全部沙箱资源与其他生产化验收仍未完成,第三方执行能力保持禁用。
|
||||||
|
|
||||||
|
|
||||||
|
## 增量:后台实例注册、串行派发与停止清理
|
||||||
|
|
||||||
|
- 新增 extension_instance::Registry。启动先用当前时间核验执行许可,再按执行类型、来源/命名空间/包和 Vault 限制同键单实例;最多 16 个注册项。每个工作线程在自身栈中创建并持有 Profile、PinnedPackage、BoundEntry、Running 与 Session,没有把带裸 SID 的 Profile 强行标为 Send。
|
||||||
|
- 工作线程完成包校验、AppContainer/Job 挂起创建后调用 before_resume 复核钩子,再执行带租约恢复和 MCP 初始化/目录发现。该 start API 为 unsafe 内部集成边界,调用者仍须建立完整资源、用户与活动安装授权;本轮没有将“钩子存在”当作在线信任或完整沙箱策略已接通,也未新增 renderer/Core 启动命令。
|
||||||
|
- 每实例四槽请求队列,预览参数入队前即执行 256 KiB/结构限制;预览、确认并调用、目录刷新由实例线程串行处理。确认和一次性凭据消费合在同一线程,ApprovedCall 不通过 IPC 返回或另行长期排队。回执等待限于后台,取消/丢弃 Ticket 会通知排队或进行中的工作;超时不证明远端工具尚未产生副作用,后续 Core 路由仍需正确呈现未知完成状态。
|
||||||
|
- stop 使用独立原子信号并取消当前请求,不会被满队列阻塞;后台循环持续处理空闲 MCP 通知和租约撤销。终态在原生栈所有者退出后发布,Profile 显式删除失败有稳定错误;Registry 只回收确认已完成的线程,清理失败或 worker panic 的注册项保留隔离,避免直接允许同键替换。Registry 析构会停止并 join 工作线程,因此其生命周期管理不能置于 UI 同步调用中。
|
||||||
|
- 真实后台 AppContainer 集成测试覆盖启动/目录就绪、预览与确认调用、重复确认拒绝、停止、线程回收后以新实例 ID 重建、活动工具取消、启动前策略拒绝及保险库锁定自动退出。测试同时确认旧 Endpoint 不能再提交请求,并通过保留的仅测试 Job 观察句柄确认最终 ActiveProcesses=0。
|
||||||
|
- 集成测试最初以固定 ActiveProcesses=2 判断工具子进程启动,本机在 Ready 时已计入 2 个进程,导致实际取消的是排队请求。已改为同时确认当前 Ticket 处于执行槽、且进程数超过该实例就绪基线,再触发取消;修正后得到进行中工具的 MCP_CANCELLED 并清空 Job。此前仅凭固定计数的证据不能单独用于识别某个具体后代进程。
|
||||||
|
- 新增队列满/输入超限/停止后队列不执行,以及已结束但清理失败的注册项不回收测试。67 项扩展回归通过,5 项 ignored 为三个父测试驱动 Job 辅助入口及两个显式长时验收,本轮未重跑长时项。desktop 全目标 Clippy -D warnings 通过,首次检查的回调类型复杂度提示已通过类型别名修正。日志 `.build/extension-instance-tests.log`、`.build/extension-instance-regression.log`、`.build/extension-instance-clippy.log`。
|
||||||
|
- 仍需将注册表纳入实际 Host 依赖与 UI/Core 的认证路由,完成活动安装/在线信任钩子、包对象权限回收/残留检查、全部 Host 创建路径继承审计、scratch/CPU 和完整攻击矩阵;其他同步/部署/安装包验收也继续推进。第三方执行能力保持禁用,完整生产化目标尚未完成。
|
||||||
|
|||||||
@@ -0,0 +1,815 @@
|
|||||||
|
//! Native instance workers. Construct/drop this manager off the UI thread.
|
||||||
|
//! Production callers must still satisfy the complete launch policy contract.
|
||||||
|
use crate::{
|
||||||
|
credentials::CredentialBroker,
|
||||||
|
extension_call_authorization::{Identity, Review},
|
||||||
|
extension_container::Profile,
|
||||||
|
extension_launch_authorization::Context,
|
||||||
|
extension_mcp::Session,
|
||||||
|
extension_mcp_tools::Description,
|
||||||
|
extension_package::Inventory,
|
||||||
|
extension_permit::{Authority, Claims, Permit},
|
||||||
|
extension_pinned::PinnedPackage,
|
||||||
|
workspace::{HostError, Result},
|
||||||
|
};
|
||||||
|
use serde::Serialize;
|
||||||
|
use serde_json::Value;
|
||||||
|
use sha2::{Digest, Sha256};
|
||||||
|
use std::{
|
||||||
|
collections::BTreeMap,
|
||||||
|
path::PathBuf,
|
||||||
|
sync::{
|
||||||
|
atomic::{AtomicBool, AtomicU8, Ordering},
|
||||||
|
mpsc::{self, Receiver, RecvTimeoutError, SyncSender},
|
||||||
|
Arc, Mutex,
|
||||||
|
},
|
||||||
|
thread::JoinHandle,
|
||||||
|
time::{Duration, SystemTime, UNIX_EPOCH},
|
||||||
|
};
|
||||||
|
fn now_ms() -> Result<u64> {
|
||||||
|
u64::try_from(
|
||||||
|
SystemTime::now()
|
||||||
|
.duration_since(UNIX_EPOCH)
|
||||||
|
.map_err(|_| HostError::new("EXTENSION_CLOCK_INVALID"))?
|
||||||
|
.as_millis(),
|
||||||
|
)
|
||||||
|
.map_err(|_| HostError::new("EXTENSION_CLOCK_INVALID"))
|
||||||
|
}
|
||||||
|
pub type ResumeCheck = Box<dyn FnOnce(&Claims) -> Result<()> + Send>;
|
||||||
|
pub struct LaunchSpec {
|
||||||
|
pub package: cap_std::fs::Dir,
|
||||||
|
pub inventory: Inventory,
|
||||||
|
pub claims: Claims,
|
||||||
|
pub permit: Permit,
|
||||||
|
pub authority: Arc<Authority>,
|
||||||
|
pub credentials: Arc<Mutex<CredentialBroker>>,
|
||||||
|
pub vault_id: String,
|
||||||
|
pub policy_version: String,
|
||||||
|
pub system_root: PathBuf,
|
||||||
|
/// Revalidate active install/current trust and all external policy immediately
|
||||||
|
/// before resume, after expensive package checks. Errors prohibit execution.
|
||||||
|
pub before_resume: ResumeCheck,
|
||||||
|
}
|
||||||
|
#[derive(Clone, Copy, Serialize, PartialEq, Eq, Debug)]
|
||||||
|
#[serde(rename_all = "snake_case")]
|
||||||
|
pub enum Status {
|
||||||
|
Starting,
|
||||||
|
Ready,
|
||||||
|
Stopping,
|
||||||
|
Stopped,
|
||||||
|
Failed,
|
||||||
|
}
|
||||||
|
#[derive(Serialize)]
|
||||||
|
pub struct Snapshot {
|
||||||
|
pub status: Status,
|
||||||
|
pub identity: Option<Identity>,
|
||||||
|
pub tool_count: usize,
|
||||||
|
pub error: Option<String>,
|
||||||
|
}
|
||||||
|
struct Control {
|
||||||
|
stop: AtomicBool,
|
||||||
|
#[cfg(test)]
|
||||||
|
job: Mutex<Option<crate::extension_job::Job>>,
|
||||||
|
active: Mutex<Option<Arc<AtomicBool>>>,
|
||||||
|
status: AtomicU8,
|
||||||
|
identity: Mutex<Option<Identity>>,
|
||||||
|
tools: Mutex<Vec<Description>>,
|
||||||
|
error: Mutex<Option<String>>,
|
||||||
|
}
|
||||||
|
impl Control {
|
||||||
|
fn status(&self) -> Status {
|
||||||
|
match self.status.load(Ordering::Acquire) {
|
||||||
|
0 => Status::Starting,
|
||||||
|
1 => Status::Ready,
|
||||||
|
2 => Status::Stopping,
|
||||||
|
3 => Status::Stopped,
|
||||||
|
_ => Status::Failed,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
fn stop(&self) {
|
||||||
|
self.stop.store(true, Ordering::Release);
|
||||||
|
if let Some(active) = &*self.active.lock().unwrap_or_else(|e| e.into_inner()) {
|
||||||
|
active.store(true, Ordering::Release);
|
||||||
|
}
|
||||||
|
let _ = self
|
||||||
|
.status
|
||||||
|
.fetch_update(Ordering::AcqRel, Ordering::Acquire, |value| {
|
||||||
|
(value < 2).then_some(2)
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
struct Active<'a>(&'a Control);
|
||||||
|
impl Drop for Active<'_> {
|
||||||
|
fn drop(&mut self) {
|
||||||
|
self.0
|
||||||
|
.active
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(|e| e.into_inner())
|
||||||
|
.take();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
struct Request<T> {
|
||||||
|
cancel: Arc<AtomicBool>,
|
||||||
|
reply: SyncSender<Result<T>>,
|
||||||
|
}
|
||||||
|
impl<T> Request<T> {
|
||||||
|
fn execute(
|
||||||
|
self,
|
||||||
|
control: &Control,
|
||||||
|
action: impl FnOnce(&AtomicBool) -> Result<T>,
|
||||||
|
) -> Option<String> {
|
||||||
|
let mut active = control.active.lock().unwrap_or_else(|e| e.into_inner());
|
||||||
|
if control.stop.load(Ordering::Acquire) || self.cancel.load(Ordering::Acquire) {
|
||||||
|
let _ = self
|
||||||
|
.reply
|
||||||
|
.try_send(Err(HostError::new("EXTENSION_INSTANCE_CANCELLED")));
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
*active = Some(Arc::clone(&self.cancel));
|
||||||
|
drop(active);
|
||||||
|
let _active = Active(control);
|
||||||
|
let result = action(&self.cancel);
|
||||||
|
let error = result.as_ref().err().map(|e| e.code.clone());
|
||||||
|
let _ = self.reply.try_send(result);
|
||||||
|
error
|
||||||
|
}
|
||||||
|
}
|
||||||
|
enum Command {
|
||||||
|
Review {
|
||||||
|
name: String,
|
||||||
|
arguments: Value,
|
||||||
|
request: Request<Review>,
|
||||||
|
},
|
||||||
|
Invoke {
|
||||||
|
review_id: String,
|
||||||
|
request: Request<Value>,
|
||||||
|
},
|
||||||
|
Refresh {
|
||||||
|
request: Request<Vec<Description>>,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
pub struct Ticket<T> {
|
||||||
|
receiver: Receiver<Result<T>>,
|
||||||
|
cancel: Arc<AtomicBool>,
|
||||||
|
}
|
||||||
|
impl<T> Ticket<T> {
|
||||||
|
/// Background wait only; dropping a ticket cancels its queued/in-flight work.
|
||||||
|
pub fn wait(self, timeout: Duration) -> Result<T> {
|
||||||
|
if timeout > Duration::from_secs(65) {
|
||||||
|
return Err(HostError::new("EXTENSION_INSTANCE_WAIT_INVALID"));
|
||||||
|
}
|
||||||
|
self.receiver.recv_timeout(timeout).map_err(|error| {
|
||||||
|
HostError::new(match error {
|
||||||
|
RecvTimeoutError::Timeout => "EXTENSION_INSTANCE_TIMEOUT",
|
||||||
|
RecvTimeoutError::Disconnected => "EXTENSION_INSTANCE_CLOSED",
|
||||||
|
})
|
||||||
|
})?
|
||||||
|
}
|
||||||
|
pub fn cancel(&self) {
|
||||||
|
self.cancel.store(true, Ordering::Release);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
impl<T> Drop for Ticket<T> {
|
||||||
|
fn drop(&mut self) {
|
||||||
|
self.cancel.store(true, Ordering::Release);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
#[derive(Clone)]
|
||||||
|
pub struct Endpoint {
|
||||||
|
control: Arc<Control>,
|
||||||
|
commands: SyncSender<Command>,
|
||||||
|
}
|
||||||
|
impl Endpoint {
|
||||||
|
fn submit<T>(&self, build: impl FnOnce(Request<T>) -> Command) -> Result<Ticket<T>> {
|
||||||
|
if self.control.status() != Status::Ready || self.control.stop.load(Ordering::Acquire) {
|
||||||
|
return Err(HostError::new("EXTENSION_INSTANCE_NOT_READY"));
|
||||||
|
}
|
||||||
|
let (reply, receiver) = mpsc::sync_channel(1);
|
||||||
|
let cancel = Arc::new(AtomicBool::new(false));
|
||||||
|
self.commands
|
||||||
|
.try_send(build(Request {
|
||||||
|
cancel: Arc::clone(&cancel),
|
||||||
|
reply,
|
||||||
|
}))
|
||||||
|
.map_err(|_| HostError::new("EXTENSION_INSTANCE_BACKPRESSURE"))?;
|
||||||
|
Ok(Ticket { receiver, cancel })
|
||||||
|
}
|
||||||
|
pub fn review(&self, name: String, arguments: Value) -> Result<Ticket<Review>> {
|
||||||
|
if name.is_empty() || name.len() > 128 {
|
||||||
|
return Err(HostError::new("EXTENSION_MCP_ARGUMENTS_INVALID"));
|
||||||
|
}
|
||||||
|
crate::extension_mcp_tools::argument_bounds(&arguments)?;
|
||||||
|
self.submit(|request| Command::Review {
|
||||||
|
name,
|
||||||
|
arguments,
|
||||||
|
request,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
/// Host route only: the user must have approved the exact saved review.
|
||||||
|
/// Confirmation and consumption happen together on the instance thread.
|
||||||
|
pub fn invoke_confirmed(&self, review_id: String) -> Result<Ticket<Value>> {
|
||||||
|
if uuid::Uuid::parse_str(&review_id).is_err() || review_id.len() != 36 {
|
||||||
|
return Err(HostError::new("EXTENSION_CALL_REVIEW_UNKNOWN"));
|
||||||
|
}
|
||||||
|
self.submit(|request| Command::Invoke { review_id, request })
|
||||||
|
}
|
||||||
|
pub fn refresh(&self) -> Result<Ticket<Vec<Description>>> {
|
||||||
|
self.submit(|request| Command::Refresh { request })
|
||||||
|
}
|
||||||
|
pub fn stop(&self) {
|
||||||
|
self.control.stop();
|
||||||
|
}
|
||||||
|
pub fn snapshot(&self) -> Snapshot {
|
||||||
|
Snapshot {
|
||||||
|
status: self.control.status(),
|
||||||
|
identity: self
|
||||||
|
.control
|
||||||
|
.identity
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(|e| e.into_inner())
|
||||||
|
.clone(),
|
||||||
|
tool_count: self
|
||||||
|
.control
|
||||||
|
.tools
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(|e| e.into_inner())
|
||||||
|
.len(),
|
||||||
|
error: self
|
||||||
|
.control
|
||||||
|
.error
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(|e| e.into_inner())
|
||||||
|
.clone(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
struct Entry {
|
||||||
|
endpoint: Endpoint,
|
||||||
|
worker: JoinHandle<()>,
|
||||||
|
}
|
||||||
|
#[derive(Default)]
|
||||||
|
pub struct Registry {
|
||||||
|
entries: BTreeMap<String, Entry>,
|
||||||
|
}
|
||||||
|
impl Registry {
|
||||||
|
/// # Safety
|
||||||
|
/// The caller must establish all sandbox limits and current install/user
|
||||||
|
/// authorization. before_resume must recheck live trust/active installation.
|
||||||
|
/// This API is not exposed to renderer/Core and does not enable extensions.
|
||||||
|
pub unsafe fn start(&mut self, spec: LaunchSpec) -> Result<Endpoint> {
|
||||||
|
self.reap();
|
||||||
|
spec.authority
|
||||||
|
.verify(&spec.permit, &spec.claims, now_ms()?)?;
|
||||||
|
let key = format!(
|
||||||
|
"{:x}",
|
||||||
|
Sha256::digest(
|
||||||
|
serde_json::to_vec(&(
|
||||||
|
spec.claims.kind,
|
||||||
|
&spec.claims.source,
|
||||||
|
&spec.claims.namespace,
|
||||||
|
&spec.claims.package_id,
|
||||||
|
&spec.claims.vault_id
|
||||||
|
))
|
||||||
|
.map_err(|_| HostError::new("EXTENSION_INSTANCE_INVALID"))?
|
||||||
|
)
|
||||||
|
);
|
||||||
|
if self.entries.contains_key(&key) {
|
||||||
|
return Err(HostError::new("EXTENSION_INSTANCE_ALREADY_RUNNING"));
|
||||||
|
}
|
||||||
|
if self.entries.len() >= 16 {
|
||||||
|
return Err(HostError::new("EXTENSION_INSTANCE_LIMIT"));
|
||||||
|
}
|
||||||
|
let control = Arc::new(Control {
|
||||||
|
stop: AtomicBool::new(false),
|
||||||
|
#[cfg(test)]
|
||||||
|
job: Mutex::new(None),
|
||||||
|
active: Mutex::new(None),
|
||||||
|
status: AtomicU8::new(0),
|
||||||
|
identity: Mutex::new(None),
|
||||||
|
tools: Mutex::new(Vec::new()),
|
||||||
|
error: Mutex::new(None),
|
||||||
|
});
|
||||||
|
let (commands, receiver) = mpsc::sync_channel(4);
|
||||||
|
let endpoint = Endpoint {
|
||||||
|
control: Arc::clone(&control),
|
||||||
|
commands,
|
||||||
|
};
|
||||||
|
let worker = std::thread::Builder::new()
|
||||||
|
.name("opennexus-instance".into())
|
||||||
|
.spawn(move || {
|
||||||
|
let result = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
|
||||||
|
run(spec, &control, receiver)
|
||||||
|
}));
|
||||||
|
// All native stack owners have dropped before publishing terminal state.
|
||||||
|
let error = match result {
|
||||||
|
Ok(Ok(())) => None,
|
||||||
|
Ok(Err(error))
|
||||||
|
if control.stop.load(Ordering::Acquire)
|
||||||
|
&& matches!(
|
||||||
|
error.code.as_str(),
|
||||||
|
"EXTENSION_MCP_CANCELLED" | "EXTENSION_INSTANCE_CANCELLED"
|
||||||
|
) =>
|
||||||
|
{
|
||||||
|
None
|
||||||
|
}
|
||||||
|
Ok(Err(error)) => Some(error.code),
|
||||||
|
Err(_) => Some("EXTENSION_INSTANCE_WORKER_FAILED".into()),
|
||||||
|
};
|
||||||
|
let failed = error.is_some();
|
||||||
|
*control.error.lock().unwrap_or_else(|e| e.into_inner()) = error;
|
||||||
|
control
|
||||||
|
.tools
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(|e| e.into_inner())
|
||||||
|
.clear();
|
||||||
|
control
|
||||||
|
.status
|
||||||
|
.store(if failed { 4 } else { 3 }, Ordering::Release);
|
||||||
|
})
|
||||||
|
.map_err(|_| HostError::new("EXTENSION_INSTANCE_WORKER_UNAVAILABLE"))?;
|
||||||
|
self.entries.insert(
|
||||||
|
key,
|
||||||
|
Entry {
|
||||||
|
endpoint: endpoint.clone(),
|
||||||
|
worker,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
Ok(endpoint)
|
||||||
|
}
|
||||||
|
/// Reap only threads confirmed finished, so an old generation cannot overlap
|
||||||
|
/// a replacement merely because stop was requested or status was changed.
|
||||||
|
pub fn reap(&mut self) {
|
||||||
|
let done: Vec<_> = self
|
||||||
|
.entries
|
||||||
|
.iter()
|
||||||
|
.filter(|(_, entry)| {
|
||||||
|
entry.worker.is_finished()
|
||||||
|
&& !entry
|
||||||
|
.endpoint
|
||||||
|
.control
|
||||||
|
.error
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(|e| e.into_inner())
|
||||||
|
.as_deref()
|
||||||
|
.is_some_and(|code| {
|
||||||
|
matches!(
|
||||||
|
code,
|
||||||
|
"EXTENSION_CONTAINER_CLEANUP_FAILED"
|
||||||
|
| "EXTENSION_RESOURCE_TERMINATE_FAILED"
|
||||||
|
| "EXTENSION_INSTANCE_WORKER_FAILED"
|
||||||
|
)
|
||||||
|
})
|
||||||
|
})
|
||||||
|
.map(|(key, _)| key.clone())
|
||||||
|
.collect();
|
||||||
|
for key in done {
|
||||||
|
if let Some(entry) = self.entries.remove(&key) {
|
||||||
|
let _ = entry.worker.join();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
pub fn stop_all(&self) {
|
||||||
|
for entry in self.entries.values() {
|
||||||
|
entry.endpoint.stop();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
impl Drop for Registry {
|
||||||
|
fn drop(&mut self) {
|
||||||
|
self.stop_all();
|
||||||
|
for (_, entry) in std::mem::take(&mut self.entries) {
|
||||||
|
let _ = entry.worker.join();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
fn run(spec: LaunchSpec, control: &Control, receiver: Receiver<Command>) -> Result<()> {
|
||||||
|
if control.stop.load(Ordering::Acquire) {
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
let profile = Profile::create()?;
|
||||||
|
let result = run_in_profile(spec, control, receiver, &profile);
|
||||||
|
profile.remove()?;
|
||||||
|
result
|
||||||
|
}
|
||||||
|
fn run_in_profile(
|
||||||
|
spec: LaunchSpec,
|
||||||
|
control: &Control,
|
||||||
|
receiver: Receiver<Command>,
|
||||||
|
profile: &Profile,
|
||||||
|
) -> Result<()> {
|
||||||
|
let pinned = PinnedPackage::open(&spec.package, &spec.inventory, &spec.claims.tree_sha256)?;
|
||||||
|
pinned.grant_read_execute(profile)?;
|
||||||
|
let entry = pinned.bind_entry(&spec.claims.entry)?;
|
||||||
|
let folder = profile.folder()?;
|
||||||
|
let scratch = folder.join("Temp");
|
||||||
|
let context = Context {
|
||||||
|
vault_id: &spec.vault_id,
|
||||||
|
policy_version: &spec.policy_version,
|
||||||
|
system_root: &spec.system_root,
|
||||||
|
container_data: &folder,
|
||||||
|
scratch: &scratch,
|
||||||
|
};
|
||||||
|
let prepared = {
|
||||||
|
let credentials = spec
|
||||||
|
.credentials
|
||||||
|
.lock()
|
||||||
|
.map_err(|_| HostError::new("CREDENTIALS_LOCKED"))?;
|
||||||
|
context.prepare(
|
||||||
|
&spec.authority,
|
||||||
|
&spec.permit,
|
||||||
|
&spec.claims,
|
||||||
|
&entry,
|
||||||
|
&credentials,
|
||||||
|
now_ms()?,
|
||||||
|
)?
|
||||||
|
};
|
||||||
|
let (suspended, io) = prepared.create_suspended_with_stdio(profile, &entry)?;
|
||||||
|
(spec.before_resume)(&spec.claims)?;
|
||||||
|
if control.stop.load(Ordering::Acquire) {
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
// Safety obligation belongs to Registry::start's caller, rechecked above.
|
||||||
|
let running = unsafe { suspended.resume()? };
|
||||||
|
#[cfg(test)]
|
||||||
|
{
|
||||||
|
*control.job.lock().unwrap() = Some(running.test_job()?);
|
||||||
|
}
|
||||||
|
let mut session = Session::new(&running, io)?;
|
||||||
|
session.initialize(&control.stop)?;
|
||||||
|
let tools = session.refresh_tools(&control.stop)?;
|
||||||
|
*control.identity.lock().unwrap_or_else(|e| e.into_inner()) = Some(running.call_identity()?);
|
||||||
|
*control.tools.lock().unwrap_or_else(|e| e.into_inner()) = tools;
|
||||||
|
if control.stop.load(Ordering::Acquire) {
|
||||||
|
return session.shutdown();
|
||||||
|
}
|
||||||
|
let _ = control
|
||||||
|
.status
|
||||||
|
.compare_exchange(0, 1, Ordering::AcqRel, Ordering::Acquire);
|
||||||
|
while !control.stop.load(Ordering::Acquire) {
|
||||||
|
session.drain_pending()?;
|
||||||
|
if session.take_tools_changed() {
|
||||||
|
control
|
||||||
|
.tools
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(|e| e.into_inner())
|
||||||
|
.clear();
|
||||||
|
}
|
||||||
|
let error = match receiver.recv_timeout(Duration::from_millis(20)) {
|
||||||
|
Ok(Command::Review {
|
||||||
|
name,
|
||||||
|
arguments,
|
||||||
|
request,
|
||||||
|
}) => request.execute(control, |_| session.review_call(&name, arguments)),
|
||||||
|
Ok(Command::Invoke { review_id, request }) => request.execute(control, |cancel| {
|
||||||
|
let approved = session.confirm_call(&review_id)?;
|
||||||
|
session.call_tool(approved, cancel)
|
||||||
|
}),
|
||||||
|
Ok(Command::Refresh { request }) => request.execute(control, |cancel| {
|
||||||
|
let tools = session.refresh_tools(cancel)?;
|
||||||
|
*control.tools.lock().unwrap_or_else(|e| e.into_inner()) = tools.clone();
|
||||||
|
Ok(tools)
|
||||||
|
}),
|
||||||
|
Err(RecvTimeoutError::Timeout) => None,
|
||||||
|
Err(RecvTimeoutError::Disconnected) => break,
|
||||||
|
};
|
||||||
|
if session.is_failed() {
|
||||||
|
if error.as_deref() == Some("EXTENSION_MCP_CANCELLED") {
|
||||||
|
return session.shutdown();
|
||||||
|
}
|
||||||
|
return Err(HostError::new(
|
||||||
|
error.as_deref().unwrap_or("EXTENSION_INSTANCE_FAILED"),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
session.shutdown()
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
use serde_json::json;
|
||||||
|
use std::time::Instant;
|
||||||
|
use zeroize::Zeroizing;
|
||||||
|
fn wait_for(mut condition: impl FnMut() -> bool) {
|
||||||
|
let started = Instant::now();
|
||||||
|
while !condition() {
|
||||||
|
assert!(started.elapsed() < Duration::from_secs(10));
|
||||||
|
std::thread::sleep(Duration::from_millis(5));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
#[test]
|
||||||
|
fn native_worker_routes_reviews_cancels_calls_and_reaps_generations() {
|
||||||
|
let temp = tempfile::tempdir().unwrap();
|
||||||
|
let package = temp.path().join("package");
|
||||||
|
std::fs::create_dir(&package).unwrap();
|
||||||
|
let executable = package.join("entry.exe");
|
||||||
|
let fixture = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
|
||||||
|
.join("tests/fixtures/sandbox_network_probe.rs");
|
||||||
|
let compiled = std::process::Command::new("rustc")
|
||||||
|
.arg("--edition=2021")
|
||||||
|
.arg(fixture)
|
||||||
|
.arg("-o")
|
||||||
|
.arg(&executable)
|
||||||
|
.output()
|
||||||
|
.unwrap();
|
||||||
|
assert!(
|
||||||
|
compiled.status.success(),
|
||||||
|
"{}",
|
||||||
|
String::from_utf8_lossy(&compiled.stderr)
|
||||||
|
);
|
||||||
|
let bytes = std::fs::read(&executable).unwrap();
|
||||||
|
let files: BTreeMap<String, String> =
|
||||||
|
[("entry.exe".into(), format!("{:x}", Sha256::digest(&bytes)))]
|
||||||
|
.into_iter()
|
||||||
|
.collect();
|
||||||
|
let dir =
|
||||||
|
cap_std::fs::Dir::open_ambient_dir(&package, cap_std::ambient_authority()).unwrap();
|
||||||
|
let inventory = || Inventory {
|
||||||
|
files: files.clone(),
|
||||||
|
expanded_size: bytes.len() as u64,
|
||||||
|
manifest: "entry.exe".into(),
|
||||||
|
};
|
||||||
|
let tree = crate::extension_unpack::verify_tree(&dir, &inventory()).unwrap();
|
||||||
|
let authority = Arc::new(Authority::default());
|
||||||
|
let credentials = Arc::new(Mutex::new(CredentialBroker::new(
|
||||||
|
temp.path().join("credentials.v1"),
|
||||||
|
)));
|
||||||
|
credentials
|
||||||
|
.lock()
|
||||||
|
.unwrap()
|
||||||
|
.unlock(Zeroizing::new(b"instance fixture password".to_vec()))
|
||||||
|
.unwrap();
|
||||||
|
let vault_id = uuid::Uuid::new_v4().to_string();
|
||||||
|
let make = |mode: &str| {
|
||||||
|
let claims = Claims {
|
||||||
|
kind: crate::extension_permit::ExecutionKind::Mcp,
|
||||||
|
source: "https://catalog.example/".into(),
|
||||||
|
namespace: "examples".into(),
|
||||||
|
package_id: "native-worker".into(),
|
||||||
|
version: "1.0.0".into(),
|
||||||
|
archive_sha256: "a".repeat(64),
|
||||||
|
tree_sha256: tree.clone(),
|
||||||
|
signer_sha256: "b".repeat(64),
|
||||||
|
entry: "entry.exe".into(),
|
||||||
|
arguments: vec![mode.into()],
|
||||||
|
environment: BTreeMap::new(),
|
||||||
|
permissions: Default::default(),
|
||||||
|
vault_id: vault_id.clone(),
|
||||||
|
platform: "windows".into(),
|
||||||
|
policy_version: "1".into(),
|
||||||
|
expires_at_ms: now_ms().unwrap() + 120_000,
|
||||||
|
};
|
||||||
|
LaunchSpec {
|
||||||
|
package: dir.try_clone().unwrap(),
|
||||||
|
inventory: inventory(),
|
||||||
|
permit: authority.issue(&claims, now_ms().unwrap()).unwrap(),
|
||||||
|
claims,
|
||||||
|
authority: Arc::clone(&authority),
|
||||||
|
credentials: Arc::clone(&credentials),
|
||||||
|
vault_id: vault_id.clone(),
|
||||||
|
policy_version: "1".into(),
|
||||||
|
system_root: PathBuf::from(std::env::var_os("SystemRoot").unwrap()),
|
||||||
|
before_resume: Box::new(|_| Ok(())),
|
||||||
|
}
|
||||||
|
};
|
||||||
|
let mut registry = Registry::default();
|
||||||
|
let mut invalid = make("mcp");
|
||||||
|
invalid.claims.arguments = vec!["changed".into()];
|
||||||
|
assert!(unsafe { registry.start(invalid) }.is_err());
|
||||||
|
assert!(registry.entries.is_empty());
|
||||||
|
let endpoint = unsafe { registry.start(make("mcp")) }.unwrap();
|
||||||
|
assert_eq!(
|
||||||
|
unsafe { registry.start(make("mcp")) }.err().unwrap().code,
|
||||||
|
"EXTENSION_INSTANCE_ALREADY_RUNNING"
|
||||||
|
);
|
||||||
|
wait_for(|| endpoint.snapshot().status != Status::Starting);
|
||||||
|
assert_eq!(
|
||||||
|
endpoint.snapshot().status,
|
||||||
|
Status::Ready,
|
||||||
|
"{:?}",
|
||||||
|
endpoint.snapshot().error
|
||||||
|
);
|
||||||
|
assert_eq!(endpoint.snapshot().tool_count, 1);
|
||||||
|
let first_identity = serde_json::to_value(endpoint.snapshot().identity.unwrap()).unwrap();
|
||||||
|
let review = endpoint
|
||||||
|
.review("echo".into(), json!({}))
|
||||||
|
.unwrap()
|
||||||
|
.wait(Duration::from_secs(5))
|
||||||
|
.unwrap();
|
||||||
|
let result = endpoint
|
||||||
|
.invoke_confirmed(review.review_id.clone())
|
||||||
|
.unwrap()
|
||||||
|
.wait(Duration::from_secs(5))
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(result["content"][0]["text"], "native MCP success");
|
||||||
|
assert_eq!(
|
||||||
|
endpoint
|
||||||
|
.invoke_confirmed(review.review_id)
|
||||||
|
.unwrap()
|
||||||
|
.wait(Duration::from_secs(5))
|
||||||
|
.unwrap_err()
|
||||||
|
.code,
|
||||||
|
"EXTENSION_CALL_REVIEW_UNKNOWN"
|
||||||
|
);
|
||||||
|
endpoint.stop();
|
||||||
|
wait_for(|| {
|
||||||
|
registry.reap();
|
||||||
|
registry.entries.is_empty()
|
||||||
|
});
|
||||||
|
assert_eq!(
|
||||||
|
endpoint.snapshot().status,
|
||||||
|
Status::Stopped,
|
||||||
|
"{:?}",
|
||||||
|
endpoint.snapshot().error
|
||||||
|
);
|
||||||
|
assert!(endpoint.review("echo".into(), json!({})).is_err());
|
||||||
|
assert_eq!(
|
||||||
|
endpoint
|
||||||
|
.control
|
||||||
|
.job
|
||||||
|
.lock()
|
||||||
|
.unwrap()
|
||||||
|
.as_ref()
|
||||||
|
.unwrap()
|
||||||
|
.active_processes()
|
||||||
|
.unwrap(),
|
||||||
|
0
|
||||||
|
);
|
||||||
|
let second = unsafe { registry.start(make("mcp_cancel")) }.unwrap();
|
||||||
|
wait_for(|| second.snapshot().status != Status::Starting);
|
||||||
|
assert_eq!(
|
||||||
|
second.snapshot().status,
|
||||||
|
Status::Ready,
|
||||||
|
"{:?}",
|
||||||
|
second.snapshot().error
|
||||||
|
);
|
||||||
|
assert_ne!(
|
||||||
|
serde_json::to_value(second.snapshot().identity.unwrap()).unwrap()["instance_id"],
|
||||||
|
first_identity["instance_id"]
|
||||||
|
);
|
||||||
|
let review = second
|
||||||
|
.review("echo".into(), json!({}))
|
||||||
|
.unwrap()
|
||||||
|
.wait(Duration::from_secs(5))
|
||||||
|
.unwrap();
|
||||||
|
let baseline = second
|
||||||
|
.control
|
||||||
|
.job
|
||||||
|
.lock()
|
||||||
|
.unwrap()
|
||||||
|
.as_ref()
|
||||||
|
.unwrap()
|
||||||
|
.active_processes()
|
||||||
|
.unwrap();
|
||||||
|
let ticket = second.invoke_confirmed(review.review_id).unwrap();
|
||||||
|
wait_for(|| {
|
||||||
|
let active = second
|
||||||
|
.control
|
||||||
|
.active
|
||||||
|
.lock()
|
||||||
|
.unwrap()
|
||||||
|
.as_ref()
|
||||||
|
.is_some_and(|active| Arc::ptr_eq(active, &ticket.cancel));
|
||||||
|
let processes = second
|
||||||
|
.control
|
||||||
|
.job
|
||||||
|
.lock()
|
||||||
|
.unwrap()
|
||||||
|
.as_ref()
|
||||||
|
.unwrap()
|
||||||
|
.active_processes()
|
||||||
|
.unwrap();
|
||||||
|
active && processes > baseline
|
||||||
|
});
|
||||||
|
ticket.cancel();
|
||||||
|
assert_eq!(
|
||||||
|
ticket.wait(Duration::from_secs(5)).unwrap_err().code,
|
||||||
|
"EXTENSION_MCP_CANCELLED"
|
||||||
|
);
|
||||||
|
wait_for(|| {
|
||||||
|
registry.reap();
|
||||||
|
registry.entries.is_empty()
|
||||||
|
});
|
||||||
|
assert_eq!(
|
||||||
|
second.snapshot().status,
|
||||||
|
Status::Stopped,
|
||||||
|
"{:?}",
|
||||||
|
second.snapshot().error
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
second
|
||||||
|
.control
|
||||||
|
.job
|
||||||
|
.lock()
|
||||||
|
.unwrap()
|
||||||
|
.as_ref()
|
||||||
|
.unwrap()
|
||||||
|
.active_processes()
|
||||||
|
.unwrap(),
|
||||||
|
0
|
||||||
|
);
|
||||||
|
let called = Arc::new(AtomicBool::new(false));
|
||||||
|
let marker = Arc::clone(&called);
|
||||||
|
let mut denied = make("mcp");
|
||||||
|
denied.before_resume = Box::new(move |_| {
|
||||||
|
marker.store(true, Ordering::Release);
|
||||||
|
Err(HostError::new("FIXTURE_POLICY_DENIED"))
|
||||||
|
});
|
||||||
|
let denied = unsafe { registry.start(denied) }.unwrap();
|
||||||
|
wait_for(|| {
|
||||||
|
registry.reap();
|
||||||
|
registry.entries.is_empty()
|
||||||
|
});
|
||||||
|
assert!(called.load(Ordering::Acquire));
|
||||||
|
assert_eq!(denied.snapshot().status, Status::Failed);
|
||||||
|
assert_eq!(
|
||||||
|
denied.snapshot().error.as_deref(),
|
||||||
|
Some("FIXTURE_POLICY_DENIED")
|
||||||
|
);
|
||||||
|
assert!(denied.control.job.lock().unwrap().is_none());
|
||||||
|
let locked = unsafe { registry.start(make("mcp")) }.unwrap();
|
||||||
|
wait_for(|| locked.snapshot().status != Status::Starting);
|
||||||
|
assert_eq!(locked.snapshot().status, Status::Ready);
|
||||||
|
credentials.lock().unwrap().lock();
|
||||||
|
wait_for(|| {
|
||||||
|
registry.reap();
|
||||||
|
registry.entries.is_empty()
|
||||||
|
});
|
||||||
|
assert_eq!(locked.snapshot().status, Status::Failed);
|
||||||
|
assert_eq!(
|
||||||
|
locked.snapshot().error.as_deref(),
|
||||||
|
Some("CREDENTIALS_LOCKED")
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
locked
|
||||||
|
.control
|
||||||
|
.job
|
||||||
|
.lock()
|
||||||
|
.unwrap()
|
||||||
|
.as_ref()
|
||||||
|
.unwrap()
|
||||||
|
.active_processes()
|
||||||
|
.unwrap(),
|
||||||
|
0
|
||||||
|
);
|
||||||
|
}
|
||||||
|
fn channel() -> (Endpoint, Receiver<Command>) {
|
||||||
|
let (commands, receiver) = mpsc::sync_channel(4);
|
||||||
|
let control = Arc::new(Control {
|
||||||
|
stop: AtomicBool::new(false),
|
||||||
|
job: Mutex::new(None),
|
||||||
|
active: Mutex::new(None),
|
||||||
|
status: AtomicU8::new(1),
|
||||||
|
identity: Mutex::new(None),
|
||||||
|
tools: Mutex::new(Vec::new()),
|
||||||
|
error: Mutex::new(None),
|
||||||
|
});
|
||||||
|
(Endpoint { control, commands }, receiver)
|
||||||
|
}
|
||||||
|
#[test]
|
||||||
|
fn bounded_queue_and_stop_prevent_queued_work_from_executing() {
|
||||||
|
let (endpoint, receiver) = channel();
|
||||||
|
let mut tickets = Vec::new();
|
||||||
|
for _ in 0..4 {
|
||||||
|
tickets.push(endpoint.review("echo".into(), json!({})).unwrap());
|
||||||
|
}
|
||||||
|
assert_eq!(
|
||||||
|
endpoint
|
||||||
|
.review("echo".into(), json!({}))
|
||||||
|
.err()
|
||||||
|
.unwrap()
|
||||||
|
.code,
|
||||||
|
"EXTENSION_INSTANCE_BACKPRESSURE"
|
||||||
|
);
|
||||||
|
assert!(endpoint
|
||||||
|
.review("echo".into(), json!({"large":"x".repeat(256 * 1024)}))
|
||||||
|
.is_err());
|
||||||
|
endpoint.stop();
|
||||||
|
assert_eq!(endpoint.snapshot().status, Status::Stopping);
|
||||||
|
for ticket in tickets {
|
||||||
|
let Command::Review { request, .. } = receiver.try_recv().unwrap() else {
|
||||||
|
panic!("wrong request")
|
||||||
|
};
|
||||||
|
request.execute(&endpoint.control, |_| panic!("stopped queue executed"));
|
||||||
|
assert_eq!(
|
||||||
|
ticket.wait(Duration::from_secs(1)).err().unwrap().code,
|
||||||
|
"EXTENSION_INSTANCE_CANCELLED"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
assert!(endpoint.refresh().is_err());
|
||||||
|
}
|
||||||
|
#[test]
|
||||||
|
fn cleanup_failures_remain_quarantined_even_after_worker_exit() {
|
||||||
|
let (endpoint, _receiver) = channel();
|
||||||
|
endpoint.control.status.store(4, Ordering::Release);
|
||||||
|
*endpoint.control.error.lock().unwrap() = Some("EXTENSION_CONTAINER_CLEANUP_FAILED".into());
|
||||||
|
let worker = std::thread::spawn(|| {});
|
||||||
|
wait_for(|| worker.is_finished());
|
||||||
|
let mut registry = Registry::default();
|
||||||
|
registry
|
||||||
|
.entries
|
||||||
|
.insert("quarantined".into(), Entry { endpoint, worker });
|
||||||
|
registry.reap();
|
||||||
|
assert_eq!(registry.entries.len(), 1);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -420,6 +420,9 @@ impl<'a, 'p> Session<'a, 'p> {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
pub(crate) fn is_failed(&self) -> bool {
|
||||||
|
self.failed
|
||||||
|
}
|
||||||
pub fn take_tools_changed(&mut self) -> bool {
|
pub fn take_tools_changed(&mut self) -> bool {
|
||||||
std::mem::take(&mut self.tools_changed)
|
std::mem::take(&mut self.tools_changed)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -44,6 +44,13 @@ fn bounded(value: &Value, bytes: usize, schema: bool) -> Result<()> {
|
|||||||
}
|
}
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
pub(crate) fn argument_bounds(value: &Value) -> Result<()> {
|
||||||
|
bounded(value, 256 * 1024, false)?;
|
||||||
|
if !value.is_object() {
|
||||||
|
return Err(HostError::new("EXTENSION_MCP_ARGUMENTS_INVALID"));
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
fn compile(schema: &Value) -> Result<jsonschema::Validator> {
|
fn compile(schema: &Value) -> Result<jsonschema::Validator> {
|
||||||
bounded(schema, 64 * 1024, true)?;
|
bounded(schema, 64 * 1024, true)?;
|
||||||
if !schema.is_object()
|
if !schema.is_object()
|
||||||
|
|||||||
@@ -298,6 +298,10 @@ impl Running<'_> {
|
|||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
|
pub(crate) fn test_job(&self) -> Result<crate::extension_job::Job> {
|
||||||
|
self.process.job.clone_for_deadline()
|
||||||
|
}
|
||||||
|
#[cfg(test)]
|
||||||
pub(crate) fn active_test_processes(&self) -> Result<u32> {
|
pub(crate) fn active_test_processes(&self) -> Result<u32> {
|
||||||
self.process.job.active_processes()
|
self.process.job.active_processes()
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -93,3 +93,6 @@ pub mod extension_mcp_tools;
|
|||||||
|
|
||||||
#[cfg(all(windows, feature = "desktop"))]
|
#[cfg(all(windows, feature = "desktop"))]
|
||||||
pub mod extension_call_authorization;
|
pub mod extension_call_authorization;
|
||||||
|
|
||||||
|
#[cfg(all(windows, feature = "desktop"))]
|
||||||
|
pub mod extension_instance;
|
||||||
|
|||||||
Reference in New Issue
Block a user