Harness Guide¶

This guide explains how to use the Harness as the middleware between solving agents and challenge platforms. It covers the web UI, CLI authentication, credential profiles, scoped run tokens, agent workflows, dashboards, logs, and security boundaries.

What Harness Does¶

Harness is a local-first challenge proxy and experiment ledger.

Agents submit candidates to Harness. Harness owns:

• connector credentials
• submission transport to challenge platforms
• immutable candidate bundles
• run and candidate numbering
• token and usage metadata
• raw connector responses
• normalized scores and statuses
• deterministic exports and strategy dashboards
• request traces and operational logs

Agents should not call Tensara, HighLoad, CPU.mode, GPU Mode, or other platform APIs directly when working under a Harness run. They should call the Harness CLI with a scoped run token.

Main Concepts¶

User¶

A Harness user logs in to the web UI with a username and password. The logged in user owns credential profiles and run tokens created from the UI.

The default user is admin. That account behaves like any other user for credentials, run tokens, dashboards, and sessions. Its only extra permission is creating other Harness users from the Account page.

Connector¶

A connector is the Harness integration for one platform or deployment.

Current connector examples:

• highload: HighLoad.fun through an authenticated cookie.
• local_tensara: local/private Tensara deployment.
• public_tensara: public Tensara at https://tensara.org.
• cpumode: CPU.mode submissions.
• gpumode: GPU Mode / Popcorn submissions.
• vliw: private ScoreBench VLIW judge (no venue credential, scored in cycles).
• github_pr: pull request backed workflows (config/CLI only; hidden from the web UI).

Credential Profile¶

A credential profile is a named secret for one connector.

Examples:

• skill-research
• skill-quant
• no-skill
• josusanmartin
• anboto1

The profile name is not itself secret. The secret value, such as a cookie or API key, is stored by Harness and is never handed to agents.

Exercise¶

An exercise is the platform problem being solved, such as:

• sum_of_prime_numbers
• large_matrix_multiplication
• leaky-relu
• matrix-multiplication

Run tokens are scoped to exactly one exercise.

Run¶

A run is one independent attempt under one user, connector, credential profile, and exercise. A run has a name such as run001, skill-claude, or no-skill-001.

Runs are what the strategy comparison dashboard compares.

Scoped Run Token¶

A scoped run token starts with hrun_. It is what you give to a solving agent.

The token binds:

• user
• connector
• credential profile
• exercise
• optional pre-bound run name

Agents cannot use a scoped run token to list other credentials, read sibling runs, change connectors, or submit to another exercise.

The full token is shown only in the creation or reissue handoff. Store the handoff securely; the server keeps only a hash and cannot reveal it later. Reissue an active token when the value is lost or exposed. Reissue preserves the scope and immediately revokes the previous token.

Public URL¶

The public Harness URL is:

https://scorebench.dev/

Browser UI:

https://scorebench.dev/ui/login

Required: Install the Scorebench Skill¶

Use the canonical public repository:

https://github.com/josusanmartin/scorebench-skill

The complete Codex and Claude Code installation, verification, and update commands are on the Install the Scorebench Skill page.

Do not hand an agent a SCOREBENCH_RUN_TOKEN until the agent confirms that its scorebench/SKILL.md is installed and loaded.

Install the CLI¶

This is a separate requirement. Installing the CLI does not install the Scorebench skill.

Install the scorebench CLI (with a legacy harness alias) straight from the deployment; no repository access is needed:

curl -fsSL https://scorebench.dev/install.sh | bash
export PATH="$HOME/.local/bin:$PATH"
scorebench --help

The installer pins the CLI download to the same origin that served the script and verifies its embedded SHA-256 digest before extraction. Existing SCOREBENCH_URL or legacy HARNESS_URL environment variables configure the installed CLI, but cannot redirect the installer payload.

The CLI reads SCOREBENCH_URL / SCOREBENCH_RUN_TOKEN, falling back to the legacy HARNESS_URL / HARNESS_RUN_TOKEN names, so existing handoff blocks keep working.

Run A Paradigm Puzzle With An Agent Or Coding Harness¶

Open /ui/paradigm/agent while signed in to ScoreBench. This is a handoff and monitoring tool; it does not run an agent in the browser or on the ScoreBench server.

The dashboard's Connector picker always includes Paradigm Puzzles and its active challenge catalog. Challenges with recorded runs open their strategy dashboard; challenges without runs open this handoff flow.

1. Select the Paradigm puzzle and agent or coding harness.
2. Select a model and effort level. Codex and Claude Code provide known defaults. Claude Code includes low, medium, high, extra high, and max; choose Other to enter any model or effort identifier.
3. Select the skills to use. ScoreBench is required; optional and user-saved skills are independent of the coding harness.
4. Set an arbitrary duration in minutes, hours, or days, or choose no limit. Add any other ideas or constraints as free-form instructions.
5. Create the handoff and copy the generated prompt once.
6. Paste the prompt into the selected agent or coding harness on your computer.
7. Watch the best-score improvement chart, or close the page and return from the recent handoffs list. Run status updates frequently; the trajectory refreshes every five minutes to keep monitoring lightweight.

The harness menu includes Codex and Claude Code. Select Other / add another to supply any name and optional HTTP(S) GitHub, project, or documentation URL. Selecting Save this option for next time stores that profile for the current ScoreBench user and adds it to future dropdowns. Profiles are not shared with other users. ScoreBench records the URL and places it in the generated prompt, but never fetches or executes it.

The separate Skills section always includes the required scorebench skill and offers Problem-Agnostic Optimization as an optional example. Users can add another skill by name and HTTP(S) GitHub, project, or documentation URL and save it privately for later runs. The prompt tells the local agent to install or make each selected skill available and read its SKILL.md or linked documentation. Skill URLs, harness parameters, and other ideas are prompt text; ScoreBench never fetches or executes them.

The prompt requires the agent to install the canonical scorebench skill, install or update the CLI from the same ScoreBench deployment, and run:

scorebench pair PZL-... --url https://scorebench.dev/

The pairing code expires after 30 minutes, works once, and is stored by the server only as a hash. Redeeming it creates the normal exercise- and run-scoped token and saves that token in the local CLI config with mode 0600, bound to the selected workspace. The prompt does not contain the Paradigm API key or an hrun_... token. After pairing, all normal ScoreBench skill, submission, cooldown, accounting, and run-progress rules apply.

The generated prompt makes progress reporting explicit. The agent submits a validated baseline early, then submits every material validated improvement as a separate candidate. It reads scorebench run progress before each new submission and obeys the server's allowance and retry timing. Active work should not go more than 20-30 minutes without a useful checkpoint when the venue and ScoreBench permit, but unchanged or unvalidated work is never submitted merely to satisfy that cadence. Each changed candidate gets a new idempotency key; an uncertain exact retry reuses its existing key, and pending candidates are refreshed rather than resubmitted. When an older server exposes no limit metadata, routine attempts stay at least five minutes apart while a newly validated best can still be submitted promptly.

Closing the browser does not stop a run. Closing the terminal, agent, or harness does. The browser monitor reports authoritative server-side run progress and can be revisited without exposing the worker token.

Public Access and Registration¶

The dashboard reports (/ui/reports/...) and this documentation (/ui/docs/) are viewable without logging in, so visitors can explore existing results. The root URL redirects to the featured exercise dashboard.

To use the harness (credentials, runs, account), create an account at /ui/register with a username and a password of at least 12 characters. Self-registered accounts get the operator role; only admins can manage other users or view logs.

Passwords are stored as salted PBKDF2-SHA256 hashes. Older SHA-256 password records are upgraded after the first successful login. Repeated login and registration attempts are throttled per client.

Browser Login¶

1. Open:

https://scorebench.dev/ui/login

1. Log in with your Harness username and password.

2. Open the Account page to inspect login sessions:

https://scorebench.dev/ui/account

The Account page shows browser and CLI sessions for the signed-in user. Session tokens are not displayed. Active sessions can be revoked individually or in groups.

Login sessions last one year unless revoked.

CLI Login¶

The CLI login command is for humans or coordinators. It is not for worker agents.

Log in:

harness admin login \
  --url https://scorebench.dev/ \
  --username admin

The command opens or prints a browser authorization link.

If you are already signed in to the browser UI, click Authorize CLI. If not, log in in the browser first, then authorize the CLI request.

Verify:

harness admin whoami

The CLI stores its web session in:

~/.config/harness/cli.json

That file is a user session credential. Do not give it to agents.

Log out:

harness admin logout

Use a named profile when you want multiple local admin contexts:

harness admin login \
  --profile prod \
  --url https://scorebench.dev/ \
  --username admin

harness admin whoami --profile prod
harness admin logout --profile prod

For SSH or headless shells, print the browser link instead of trying to open it:

harness admin login \
  --url https://scorebench.dev/ \
  --username admin \
  --no-browser

For supervised automation, avoid putting the password in shell history:

printf '%s\n' "$HARNESS_ADMIN_PASSWORD" | harness admin login \
  --url https://scorebench.dev/ \
  --username admin \
  --password-stdin

Web UI Pages¶

Dashboard¶

The chart dashboard opens at:

/ui/reports/

That route redirects to:

/ui/reports/strategy-compare.html

Use it to compare runs for one venue and exercise. It is the page with the interactive charts and selectors.

Useful controls:

• custom connector selector
• custom exercise selector with logical run counts
• user filter
• run filter
• collapsed original prompts for the runs in the current scope
• candidate hover and pinned details with the server submission timestamp
• x-axis selector: active (idle auto-removed), elapsed (true clock time), tokens, API-equivalent cost, candidate
• x-axis truncation by hours
• Y range selector: 90% keeps every best-side score visible and clips only statistically unusual worse-side values while retaining at least 90% of visible scored candidates; full restores every value
• initial-outlier selector: hide removes only the contiguous leading scores of a run that are worse-side statistical outliers, and only when that run later reaches a non-outlier score; show restores those initial scores
• best-only toggle

Runs¶

Dashboard management opens at:

/ui/runs

Use it to search personal runs, open an isolated dashboard, and control whether each run appears in comparison dashboards. Hiding a run removes it from that user's comparison dashboards without deleting submissions, candidate bundles, raw evidence, or audit logs.

Credentials¶

Use Credentials to save named connector credential profiles.

Each connector defines its own fields. Examples:

• HighLoad requires an authenticated cookie.
• Public Tensara requires a Tensara API key.
• Local Tensara requires a Tensara API key for the local deployment.
• Paradigm Puzzles requires a personal pp_... API key from its API page.
• CPU.mode accepts an API token or authenticated session cookie.
• GPU Mode stores the Popcorn CLI ID created by the browser auth flow.
• VLIW requires no venue credential. Select ScoreBench main; the scoped ScoreBench run token is the only worker credential.

Credential profile names are used in dashboards and run-token creation.

Secrets are stored locally by Harness. They are not shown again after save.

Exercise API Keys¶

Use Exercise API Keys under Account to create scoped run tokens.

The normal flow:

1. Select connector.
2. Select credential profile.
3. Select exercise.
4. Optionally pre-bind a run name.
5. Create the token.
6. Copy the handoff block to the agent.

If the run name is left blank, the agent must choose a run name with:

harness run start --id run001 \
  --skills scorebench \
  --model gpt-5-codex \
  --coding-harness Codex \
  --effort high \
  --autonomy autonomous

If the run name is pre-bound in the UI, the agent should inspect it with:

harness run current

Account¶

Use Account to:

• change your password
• add or manage users
• inspect login sessions
• revoke browser or CLI sessions

Creating Agent Tokens From The CLI¶

After admin CLI login, create a scoped token:

harness admin create-run-token \
  --connector public_tensara \
  --credential skill-research \
  --exercise leaky-relu \
  --run-id run001 \
  --prompt-file prompt.md \
  --skills scorebench \
  --model gpt-5-codex \
  --coding-harness Codex \
  --effort high \
  --autonomy autonomous

The output includes a handoff block for the agent. --prompt or --prompt-file is required so the complete assignment is retained with the run and available in the dashboard.

For multiple parallel workers:

harness admin launch \
  --connector local_tensara \
  --credential skill-research \
  --exercise leaky-relu \
  --count 4 \
  --run-prefix no-skill- \
  --skills scorebench \
  --model gpt-5-codex \
  --coding-harness Codex \
  --effort high \
  --autonomy autonomous \
  --goal 'Use the scorebench skill. Solve leaky-relu for 3 hours. Submit only through Harness. Do not use exploits.' \
  --agent-command codex \
  --dry-run \
  --json

Use --dry-run --json first. Then run without --dry-run when the launch shape is correct.

Agent Handoff¶

Agents should receive only:

export HARNESS_URL=https://scorebench.dev/
export HARNESS_RUN_TOKEN=hrun_...

Then the agent should run:

harness context
harness exercise
harness run current
harness run progress

If harness context says needs_run_name: true, the agent should start or continue one run:

harness run start \
  --id run001 \
  --strategy "short description of what this run is testing" \
  --hypothesis "why this strategy should improve the score" \
  --skills scorebench,problem-agnostic-optimization \
  --model gpt-5-codex \
  --coding-harness Codex \
  --effort high \
  --autonomy autonomous

--model and --effort are required: harness run start is rejected without them so every run can be attributed in the strategy reports. They may be supplied on the command line or inherited from a pre-bound exercise API key that was created with --model/--effort (either satisfies the requirement). --coding-harness records the agent interface independently of the model. Use Claude Code for Claude, Codex for GPT/OpenAI, Deep Code for DeepSeek, Grok Build for Grok, Kimi Code for Kimi (including model ID k3), ZCode for GLM, and the actual native harness for other models. Known families are inferred for legacy clients, but crossed or custom setups must set it explicitly. The historical claude-codex-* runs used Claude inside Codex and therefore report Codex. --skills and --autonomy remain optional but strongly encouraged; missing values there are returned as non-fatal warnings.

Record the instructions that created the run with --prompt or --prompt-file, for example harness run start ... --prompt-file prompt.md. The Account run form requires the complete Original run prompt, and harness admin launch records its --goal plus --prompt automatically. Reports store this text once per run and show it in a collapsed Original prompts section. Existing runs without this metadata are labeled as not recorded; report generation does not invent or backfill prompt text.

On GPU-backed connectors (tensara, local tensara, GPU Mode) a run can also declare the GPU it targets with --gpu, for example --gpu H100. This pins the run: submissions inherit the GPU automatically and a conflicting harness submit --gpu is rejected, so one run never mixes results from two GPUs. The same --gpu flag exists on harness admin create-run-token and harness admin launch so pre-bound tokens carry the GPU to workers. Dashboards record the GPU per candidate and filter strategy comparisons to one GPU by default; mixing GPUs in one view is an explicit opt-in there.

Before the first submission, and after resuming work, the agent must ping the run:

harness run ping --event start --note "starting work"
harness run ping --event resume --note "resuming work"
harness run ping --event activity --note "actively optimizing" # every <=5m

Each ping gives Harness a server timestamp. Start/resume establishes a session boundary; periodic activity prevents long, genuinely active tool calls from being mistaken for idle time. Do not send activity pings while idle or complete.

Read the run's canonical trusted accounting progress with:

harness run progress

This run-token-scoped read returns active time, elapsed time, working tokens, their sources, and measurement timestamps. It advances through trusted candidate and ping timestamps, uses the same active-time heuristic as reports, and does not count the read itself as activity. It is not a replacement for required start/resume and periodic activity pings. Supervisors should use this command instead of parsing dashboard HTML or inferring latest progress from harness best; the best candidate can be older than the latest measurement.

Agent Submission Workflow¶

A normal solving agent loop is:

harness context
harness exercise
harness run current
harness run ping --event start --note "starting work"
harness run progress

harness submit path/to/solution \
  --label c001-baseline \
  --notes "baseline candidate" \
  --idempotency-key c001-baseline \
  --total-tokens 123456 \
  --usage-source codex_usage \
  --usage-confidence exact \
  --tokens-total-source codex_goal

harness best
harness history
harness refresh

For Paradigm Puzzles, submit the file shape returned by harness exercise. The twelve active API exercises use Solidity, Rust, Python, plain text, packing JSON, or ONNX depending on the scoped exercise. --label is also used as the upstream strategy/model name when the API accepts one. ScoreBench validates AMM, Prop AMM, QEC, and Packing before consuming a cooldown; it checks venue cooldowns before all SSE submissions; and it keeps Lean Semantics pending until harness refresh observes terminal adjudication. Never give a worker the Paradigm pp_... key or call the Paradigm submit endpoint directly.

For connectors that expose website data through Harness, inspect it through the CLI instead of using connector credentials directly:

harness leaderboard
harness solutions --lang <LANG>
harness inspect-solution <solution_id>
harness solve-form --language <LANG>

HighLoad agents should use harness solve-form and harness solutions to see available compiler defaults and competitor compiler choices. Compiler and flag changes are part of the optimization search; record them in --notes and use a new --idempotency-key when they change. Use the language for the current candidate, such as CPP, RUST, GO, CSHARP, or ZIG; do not assume C++. harness exercise returns HighLoad language metadata including languages, defaultLanguage, and defaultFilenames. harness solve-form --language RUST fetches the upstream /solve/RUST form and returns availableLanguages, defaultFilenames, compiler, and compilerArgs with CSRF/default source redacted. Additional safe language ids may be enabled by connector config, so agents should prefer the metadata returned by Harness over hard-coded language lists.

Connector-specific options can be added when required:

harness submit sol.cu \
  --label leaky-relu-v1 \
  --notes "simple CUDA kernel" \
  --idempotency-key leaky-relu-v1 \
  --solution-file sol.cu \
  --language cuda \
  --gpu T4 \
  --total-tokens 123456 \
  --usage-source codex_usage \
  --usage-confidence exact \
  --tokens-total-source codex_goal

Do not resubmit just to check status. Use:

harness refresh

Some connectors take several minutes. Keep refreshing until the candidate is terminally scored or failed.

Scoped Submission Invalidation¶

If a candidate later turns out to be invalid, exploity, or based on a false assumption, mark it invalid instead of hiding it or rewriting history:

harness invalidate <candidate_id> \
  --reason "exploit: memoizes exact matrix inputs instead of general multiplication" \
  --meta class=exploit

Omit <candidate_id> only when invalidating the latest candidate visible to the current run token:

harness invalidate --reason "bug: latest candidate used an invalid assumption"

Invalidation is scoped. An agent can invalidate only candidates visible to its current run token. It does not delete evidence: immutable bundles, raw connector payloads, scores, logs, token data, and history remain intact. The candidate status becomes invalidated.

harness history includes an audit object with the invalidation reason, actor, timestamp, metadata, and any later reinstatement. Read that reason before changing or resubmitting descendants of an invalidated candidate.

Invalidated candidates remain visible in history and exports for auditability, but Harness excludes them from:

• harness best
• best-so-far curves
• promotion decisions
• dashboard winner calculations

If a contract review proves the invalidation was incorrect, append a reinstatement instead of editing or deleting the old event:

harness reinstate <candidate_id> \
  --reason "contract correction: the documented input domain permits this specialization"

For new invalidations Harness restores the exact status captured at invalidation time. Legacy rows without that field infer scored, failed, or submitted from preserved score evidence; --restore-status is available for an audited operator correction. Reinstatement is scoped to candidates visible to the current run token.

Recent example:

harness invalidate profile_local_tensara_josu_cand_0533 \
  --reason "device-side exact input comparison and cached output reuse is not a valid general square matrix multiplication submission"

After regenerating the local_tensara / square-matmul dashboard, that candidate is marked INVALIDATED. The affected run falls back to profile_local_tensara_josu_cand_0529 at 8078.938681941922 us.

Token Accounting¶

Every submission must include a cumulative, run-relative token snapshot.

Required field:

--total-tokens <integer>

Recommended provenance fields:

--usage-source codex_usage
--usage-confidence exact
--tokens-total-source codex_goal

Agents must not invent token counts. If no exact source is available, the agent should stop before submitting and ask for a supervised runner or visible exact usage counter.

At the end of a run, record final run usage:

harness run usage \
  --total-tokens 10643192 \
  --usage-source codex_usage \
  --usage-confidence exact \
  --tokens-total-source final_goal_usage

If exact input/output breakdowns are available, include them. Do not invent breakdown fields. Grok's native aggregate includes cache reads, so Grok runs must use the installed ScoreBench skill's token_usage.py --grok-jsonl parser; the server rejects aggregate-only Grok usage instead of recording an inflated working-token total.

The dashboard derives API-equivalent cost from these counters and its versioned public list-price table. Estimated values use a ~ prefix. See API Cost Accounting for the price table, fallback rules, and exclusions.

Idempotency¶

Use --idempotency-key for each candidate.

Retry with the same key only when retrying the exact same candidate after a network error, timeout, or uncertain response. If the source code, compiler, GPU, exercise, or submission semantics change, use a new idempotency key.

Harness rejects reuse of an idempotency key with different content.

Submission Limits¶

harness context reports the effective per-run submission limits. By default a run can create at most 1,000 candidates, retain 2 GiB of compressed candidate bundles, report at most 100 million normalized working tokens, and make 30 new submissions per rolling minute. Experiments can override these values.

Exhausted candidate, artifact, or token budgets return HTTP 409. Submission throttling returns HTTP 429 with Retry-After. Retrying an already accepted idempotency key remains valid and does not call the connector again. Token totals that are unusual for elapsed run time but below the hard ceiling are kept with a visible suspect trust warning.

Dashboards And Reports¶

Harness writes reports from the SQLite ledger and immutable artifacts.

The web UI separates the chart view from management controls:

• /ui/reports/: chart dashboard, redirected to /ui/reports/strategy-compare.html.
• /ui/runs: personal Runs page with search and isolated dashboard links. Hide locally is reversible and affects only your normal comparisons; Publish publicly is explicit and off by default; Delete is permanent.
• /ui/keys: create and manage scoped exercise API keys under Account.
• /ui/docs/: searchable MkDocs documentation.

Important report files:

• report.json: structured data.
• report.csv: candidate table.
• progress.tsv: canonical deterministic progress log.
• progress-details.tsv: debug and provenance log.
• strategy-compare.html: interactive comparison dashboard.

The strategy comparison dashboard is the main view for evaluating approaches. It compares runs by:

• best score trajectory
• first accepted candidate
• time to threshold
• token spend
• API-equivalent model cost
• model and coding-harness attribution
• failures and rejections
• promoted candidates
• final best score

The dashboard can compare independent runs across time, filter by model or coding harness, and include external player rows when connector data supports it. Only best run by can retain the strongest scored run per model, coding harness, experimental skill set, or any combination of those dimensions. Basic infrastructure skills such as scorebench do not count as experimental skills. The winner is computed inside the selected time window, after the current scope, Relevant/All mode, and manual variant selection; exercises use their configured lower- or higher-is-better direction. Candidate details and progress-details.tsv report the coding harness; Export Studio receives only the runs visible in the dashboard and can group or split comparisons further.

Logs And Debugging¶

Every HTTP response includes:

X-Harness-Trace-Id: trc_...

When a command fails, preserve the exact error and trace ID.

Search logs:

TRACE_ID=trc_...
rg "$TRACE_ID" /home/josu/dev/harness/runs/highload_sum_of_prime_numbers/logs

Log files:

runs/highload_sum_of_prime_numbers/logs/harness.jsonl
runs/highload_sum_of_prime_numbers/logs/trace.jsonl
runs/highload_sum_of_prime_numbers/logs/errors.jsonl

Logs redact cookies, bearer tokens, passwords, CSRF values, submitted bundle bodies, and other secret-bearing fields.

Security Rules¶

Do not give agents:

• Harness user password
• browser cookies
• ~/.config/harness/cli.json
• connector API keys
• connector cookies
• run_state.json
• credential env files
• another run's token

Give agents only:

export HARNESS_URL=https://scorebench.dev/
export HARNESS_RUN_TOKEN=hrun_...

The CLI user profile can create and manage run tokens for that user. A worker run token can only operate inside its own scope.

Session Lifetime And Revocation¶

Browser and CLI sessions last one year unless revoked.

Use the Account page to revoke sessions. Revoking the current browser session signs that browser out. Revoking a CLI session invalidates the local CLI profile until it logs in again.

Expired sessions stay expired. Harness does not reactivate old expired sessions when the TTL policy changes.

Connector Notes¶

Use docs/connectors.md as the connector reference. It has a section for each current connector:

• fake: deterministic local test connector.
• local_tensara: private Tensara deployment.
• public_tensara: public Tensara at https://tensara.org.
• highload: HighLoad.fun cookie-authenticated compute submissions.
• cpumode: CPU.mode API submissions and job polling.
• gpumode: GPU Mode / Popcorn proxy submissions.
• vliw: queued private VLIW judging with local instruction extraction.
• paradigm_puzzles: the twelve active Paradigm Puzzles API submission contracts.
• github_pr: deterministic dry-run or real PR transport.

The short rule for every connector is the same: agents submit only through Harness, while Harness owns the connector credential, platform request, raw response, normalized score, refresh behavior, and report evidence.

Operational Commands¶

Check service status:

sudo systemctl status harnessd-highload-sum.service

Restart service:

sudo systemctl restart harnessd-highload-sum.service

Tail service logs:

sudo journalctl -u harnessd-highload-sum.service -f

Check the public login page:

curl -sS https://scorebench.dev/ui/login -D - -o /dev/null

Common Problems¶

HARNESS_RUN_TOKEN is required¶

The agent does not have a scoped run token in the environment. Create one from Account > Exercise API Keys or with harness admin create-run-token, then pass it to the agent.

run needs a name¶

The token was created without a pre-bound run name. The agent must run:

harness run start --id run001 \
  --skills scorebench \
  --model gpt-5-codex \
  --coding-harness Codex \
  --effort high \
  --autonomy autonomous

submit requires a token snapshot¶

The agent tried to submit without --total-tokens. The agent must provide an exact run-relative token count.

scoped to exercise¶

The run token is bound to a different exercise. Create a new run token for the desired exercise.

CLI login works but agents cannot submit¶

CLI login is not the same as an agent run token. Create a scoped run token and pass HARNESS_RUN_TOKEN to the agent.

Dashboard shows no candidates¶

Check:

harness context
harness run current
harness history

Make sure the agent is using the expected HARNESS_RUN_TOKEN and that it has actually submitted candidates through Harness.

Related Docs¶

• docs/middleware-protocol.md: HTTP schema and exact agent contract.
• docs/connectors.md: connector-by-connector operator and maintainer reference.
• docs/architecture.md: system architecture and data model.
• docs/public-deployment.md: Nginx and systemd deployment notes.
• https://github.com/josusanmartin/scorebench-skill: agent-facing Codex skill.