Eval (maiife-ai-pub) MCP Server
@maiife-ai-pub/eval
npm
v0.2.2
Published by @maiife-ai-pub — no publish provenance, so origin is unverified, but the source is public: the repository link below is self-declared yet readable, so you can inspect the code before adopting it.
LLM-as-judge evaluation engine — score agent outputs with structured rubrics
The grade answers one question — how safe is this server for you to adopt — so it is computed in two auditable stages. Nothing below is an opinion or an LLM's guess; every line is a real term the deterministic engine applied, and the same input always yields the same number.
1. Threat score — 100 − 0 = 100. What the published surface and source actually contain:
The deterministic scan raised no scored threat in the surface it inspected — the threat score stayed at 100. Capability observations and advisory notes are recorded but never lower it.
2. Client adoption risk — 100 − 7 = 93. Three small, subtract-only factors that reflect your risk in adopting it — a clean scan proves less on a powerful, unverified or barely-inspectable package, so the grade says so plainly:
| Points | Adoption-risk factor |
|---|---|
| −6 | capability blast radius (high) — client exposure if the model is manipulated |
| −1 | publisher verification (public source) — no provenance, but the source is public and inspectable |
Capability observations and info notes are shown under Findings but never scored.
Open any row's finding below for the file, line and evidence behind a deduction.
Findings 9
high Read-and-egress in one tool: "eval_score"MTC-FLOW-003
Tool "eval_score" can both read sensitive data and send data to an external destination. Even without an explicit untrusted-input leg, this is a single-call data-exfiltration path if the model is ever manipulated.
Fix: Separate reading from sending; require explicit user confirmation before egress of file/secret contents.
Location: flow eval_score
high Sensitive-source and external-sink co-existMTC-FLOW-004
Tools that read sensitive data ([eval_score]) and tools that can send data out ([eval_score, eval_batch, eval_compare]) are exposed together. An agent can move private data to the sink.
Evidence: sources [eval_score] → sinks [eval_score, eval_batch, eval_compare]
Fix: Keep secret-reading and egress capabilities on separate, separately-approved servers.
Location: flow eval_score → eval_batch
high Tool "eval_score" exposes command/code executionMTC-CAP-001
Tool "eval_score" appears to run shell commands or evaluate code (keyword "eval" in tool name). Arbitrary execution driven by model input is one of the most dangerous MCP capabilities; combined with any untrusted input it becomes RCE.
Fix: Sandbox execution, allowlist commands/arguments, and never pass model output to a shell unescaped.
Location: tool eval_score
high Tool "eval_batch" exposes command/code executionMTC-CAP-001
Tool "eval_batch" appears to run shell commands or evaluate code (keyword "eval" in tool name). Arbitrary execution driven by model input is one of the most dangerous MCP capabilities; combined with any untrusted input it becomes RCE.
Fix: Sandbox execution, allowlist commands/arguments, and never pass model output to a shell unescaped.
Location: tool eval_batch
high Tool "eval_compare" exposes command/code executionMTC-CAP-001
Tool "eval_compare" appears to run shell commands or evaluate code (keyword "eval" in tool name). Arbitrary execution driven by model input is one of the most dangerous MCP capabilities; combined with any untrusted input it becomes RCE.
Fix: Sandbox execution, allowlist commands/arguments, and never pass model output to a shell unescaped.
Location: tool eval_compare
medium Hardcoded egress to an external endpoint (dist/core/judge.js)MTC-SRC-003
In the server's implementation (`dist/core/judge.js:31`): A hardcoded outbound call to a fixed external host inside server code is a classic exfiltration/telemetry channel — especially paired with reads of local data. This is read from the code itself — not from the tool description — so a poisoned server cannot hide it behind honest-looking metadata.
Evidence: const res = await fetch("https://api.anthropic.com/v1/messages", { method: "POST", headers
Fix: Review this call path: confirm it never receives unsanitized tool input, constrain it, or remove it. Treat a server whose code reaches these sinks as high-capability regardless of what its tools claim.
Location: server dist/core/judge.js
low Mutating tool "eval_score" declares no destructiveHintMTC-CAP-005
Tool "eval_score" can mutate/egress but declares no destructiveHint. Clients that don't default to spec-safe behavior may not prompt before running it.
Fix: Declare accurate annotations, and gate destructive tools on user confirmation regardless.
Location: tool eval_score
low Mutating tool "eval_batch" declares no destructiveHintMTC-CAP-005
Tool "eval_batch" can mutate/egress but declares no destructiveHint. Clients that don't default to spec-safe behavior may not prompt before running it.
Fix: Declare accurate annotations, and gate destructive tools on user confirmation regardless.
Location: tool eval_batch
low Mutating tool "eval_compare" declares no destructiveHintMTC-CAP-005
Tool "eval_compare" can mutate/egress but declares no destructiveHint. Clients that don't default to spec-safe behavior may not prompt before running it.
Fix: Declare accurate annotations, and gate destructive tools on user confirmation regardless.
Location: tool eval_compare
Tools 3
Each tool and what it can reach — statically extracted from the published source.
eval_scorereads sensitive dataruns code / shelleval_batchruns code / shelleval_compareruns code / shell
Toxic flows 2
Cross-tool combinations that form a data-exfiltration primitive (untrusted input → sensitive source → external sink).
What this scan could not see
- Tools were statically extracted from the published source (3 recovered), not enumerated from a running server. Tool-poisoning, Unicode-smuggling, capability and toxic-flow analysis ran on this inferred surface, but a mis-parsed registration could be missed or mis-attributed, so tool-derived findings are capped below “confirmed”. To grade the real runtime surface, scan the running server: --command "npx -y <package>".
Versions 1
Scan history per published version. The engine is deterministic — the same version always yields the same score, so a changed score means the package itself changed.
| Version | Score | Findings | Engine | Scanned |
|---|---|---|---|---|
v0.2.2 latest |
A 93/100 | 9 | 1.13.0 | 2026-09-07 |
Embed this score
Show this server's live Trust Score in your README, docs or website. The badge is served straight from the registry and updates automatically after every rescan — no API key needed. It links back to this page, so anyone who sees the grade can also read the findings behind it instead of taking a number on faith.
Verify this score yourself
The score above is reproducible: the same package version always yields the same result. Run it locally or over the free API — no account, no LLM, fully deterministic.
npx mcptrustchecker scan @maiife-ai-pub/eval --onlineOther implementations of Eval 3
Independent packages implementing the same tool, scanned with the same engine. Compare all 4 side by side →
End to end vibe-testing framework for MCPs:
Security scan results for the Eval (npm) MCP server.
Pytest-style framework for evaluating Model Context Protocol (MCP) servers.
More in AI & Agents
Authenticated MCP transport with HTTP Signatures for AAuth agents
Local-first MCP server for parallel AI coding agents to claim file ownership before edits, preventing stomping on each other in the same worktree.
Agent-agnostic intercommunication system — sessions, messaging, channels, shared state, and real-time events
MCP server for AI agent task communication and delegation with diagnostic lifecycle visibility
Programmatic add/link/unlink for MCP servers across 23 AI coding agents (Claude Code, Claude Desktop, Cursor, VS Code, Codex, Gemini CLI, Zed, Cline, OpenCode, Goose, Kiro, Windsurf, and more). Functional API with dry-run support.
MCP server layer exposing agent-mesh orchestrator as an MCP agent