In short:
ak-deep-sweinstalls Pier and scores a coding model on the DeepSWE task set through OpenRouter: 1 task, then a fixed small sample, then the full set once you clearly agree. It has no fixed next skill; to optimize a metric in your own repo, useak-loop. It never submits results to a leaderboard on its own.
Disclosure: I maintain AgentKit and have an affiliate relationship with it: the AgentKit link in this post is an affiliate link; buying through it gets you 30% off.
Summary
Use it when you want to know how much real work a coding model can do, want a rough score on a small sample, want to compare two models on the same task set, or want to check a run you already have. You do not need it when you want to iterate on a metric in your own repo: that is the job of ak-loop.
flowchart LR
A["OpenRouter model slug"] --> B["ak-deep-swe"]
B --> C["Report: command, task count, score, cost"]
B --> D["Stop: missing key or auth error"]
E["Want to optimize a metric in your repo"] --> F["ak-loop"]It always settles 4 points:
| Point | Question | Example |
|---|---|---|
| Model slug | Exactly which model is being scored? | The slug exactly as openrouter.ai/models shows it, passed to Pier as openrouter/<vendor/model> |
| Scale ladder | How many tasks, in what order? | 1 task to check the environment, then a 10-task sample with seed 0, and only then the full set |
| Full-run approval gate | What do you see before it spends a lot of money and time? | The exact command, the likely cost, the stop condition. Until you say yes, it does not run. |
| Report with measurement conditions | Under what conditions was this number measured, and what can it be compared with? | Pier version, slug, prompt hash, tasks attempted, completed, passed. If Pier does not report cache numbers, it writes "unknown", not 0. |
Every task costs time and billed tokens through your OpenRouter key, and the skill gives no concrete cost figure: it only records cost when Pier has the numbers. OPENROUTER_API_KEY must already be set in the environment; the skill only checks whether the key exists and never prints or writes its value.
Sample prompt for a quick score on a small sample:
/ak-deep-swe <vendor/model>
Score this model on a fixed sample of 10 tasks, seed 0.
Do not run the full task set.
Done when: there is a score for the 10 tasks, plus cost if Pier reports it.This skill has no flags.
Below is the detailed version: the run flow, 5 situations with prompts, how to pass options when the skill has no flags, the change history by ak release, and common pitfalls. The detailed version and the interactive page are for subscribers. Your agent can also search them through the portal's MCP.
Opens when your account has subscriber access
This guide is for subscribers
Once your account has access, the full piece, the interactive visual and the agent call all open.
- Sign in or create a modernEngineer accountReading access belongs to your account email.
- Choose a plan on the pricing page1, 3 or 12 months, paid by VietQR bank transfer. Access opens about a minute after the money arrives.
- Or redeem a Facebook subscriber codePeople supporting the page on Facebook get an activation code by message each month.
Already a subscriber? Sign in to read.
Use this piece in your agent
Get an API keyCopy the text below and paste it into Claude Code, Codex or whichever agent you use. The agent reads the piece and checks it against the repo you have open.
Read "ak-deep-swe: score a coding model on DeepSWE" on modernEngineer and apply it to this repo.
1. Call the content_get tool of the modernengineer MCP server with locale "en", slug "ak-deep-swe". If the modernengineer MCP server is not connected, stop and walk me through connecting it from https://modernengineer.dev/en/mcp.
2. Summarize the main steps of the piece and point out which apply to this repo.
3. Propose concrete changes and ask me before editing any file.Connect MCP in Claude Code (once)
claude mcp add --transport http modernengineer \
https://api.modernengineer.dev/mcp \
--header "Authorization: Bearer <API key>"Reading the full piece over MCP needs an API key from a subscriber account or one that bought this guide. Without access, the agent only gets the opening.
- Version
- v1.0.0
- Updated
- 22/10/2026
- Licence
- All rights reserved
- Slug
- ak-deep-swe
