Casper skills hub

Updated: 2026-10-17. The strategy doc (thesis, moat, phases, distribution, stack) lives in Claude Docs as "Casper plan"; this file is the engineering view.

What it is

The merchant-side operating layer for ecommerce: AI skills that run across a store and its ad and email channels, loyal to the merchant's margin rather than to any platform. The asset is a portable skill, not a model. The moat is the open skill format, the connector standard with write safety, and the outcome data behind a public leaderboard.

This repo holds two things that share the Shopify plumbing:

Layout

A skill

skills/restock-tags/
  skill.yaml   name, version, summary, owner, connectors + scopes, inputs, outputs, writes.mode, runtime budget, eval file
  SKILL.md     the prompt
  eval.yaml    cases: inputs, source, expect { mentions, not_mentions, proposals {min,max,kinds}, max_cost_usd, max_tool_calls, max_duration_s }

Scopes are <connector>.<resource>:<read|write|generate>. A tool outside the skill's scopes is never offered to the model. A write tool never executes: in propose mode it records a Proposal in the run's journal, and apply mode is refused until the apply path ships. A generate tool spends money to create an asset Casper keeps (an image, a clip); it touches no platform and runs under a per-run spend cap. A skill without an eval file does not validate.

Datasets

A dataset is a folder: dataset.yaml (name, version, summary, owner, license, kind, files with schemas, source) and data/*.jsonl or data/*.csv. Provenance is required: source.origin is synthetic, export, collected or licensed, and source.consent is not_applicable, recorded or pending. Collected or licensed data without recorded consent fails validation, and a pending dataset is never listed. Validation checks every row against its file's schema.

pnpm casper dataset init datasets/my-data
pnpm casper dataset validate datasets/my-data
pnpm casper dataset build-demo datasets      # regenerates the three demo datasets

Two rules for creator content: a kind: text dataset's files are JSONL and declare at least id, text, posted_at and channel; and source.origin: licensed requires a valid consent.yaml next to the manifest (creator name and handle, what is licensed, which is text only and nothing else, permitted uses, revocation terms, revenue share, signed date, signer). starterConsent() in @casper/dataset-spec is the template.

The hub lists datasets at /datasets with schema, sample rows, provenance and a download per file; when a consent record exists it is summarised in the dataset's sidebar. demo-creator-maya is the fourth demo dataset: sixty synthetic posts by an invented creator (packages/cli/src/demo-creator.ts), origin synthetic, consent not applicable, standing in for a real creator's export. See docs/CREATOR_MODELS.md.

Voice skills

pnpm casper voice build datasets/demo-creator-maya skills/voice-maya   # writes skill.yaml, SKILL.md, eval.yaml, creator.yaml
pnpm casper eval skills/voice-maya                                      # the mechanics: brief followed, hard no's hold, nothing proposed
pnpm casper voice judge skills/voice-maya --seed 7                      # five held-out posts, real beside generated in random order
pnpm casper voice score reports/voice-judge/voice-maya-7/key.json --pick 1=A --pick 2=B ...

voice build is deterministic and uses no model: it measures the posts (length by channel, sentence length, how often a post carries a number, speaks to the reader, asks a question; recurring phrases and cast; what never appears) and writes a style card whose every figure comes from the dataset and whose every quote is cited by post id. The skill takes a brief, channel, variants and max_words, reads posts through search_posts, and outputs text. It declares only creator: [posts:read]; writes.mode is propose and nothing can be written. creator.yaml in the skill folder points the CLI at the dataset so run, eval and judge can build the connector.

voice judge holds posts out of retrieval, has the skill answer each post's brief (the brief column, else the topic), and writes sheet.md (real and generated side by side, A/B and item order random) and key.json. A person marks which is real and how close the other came; voice score counts the picks. There is no LLM judge in Phase 0 by design: the pass bar is the creator's.

Measured 2026-10-17 on claude-opus-5-5, synthetic creator data: voice build $0 (no model call); voice-maya eval 3/3 cases, $0.10 total, mean 11.7s (reports/voice-maya.json; an earlier run during an API slowdown had one request time out and one case take 250s, both infrastructure, not content). Blind sheet: seed 7, five held-out posts, five generated pieces, $0.12, in reports/voice-judge/voice-maya-7/. Nobody has ranked that sheet; the number that matters comes from a creator, and there is no real creator yet. Reading it unblinded: the generated pieces keep the length, the numbers and the short last line, and run slightly more elaborate than the originals.

Run it

pnpm install
pnpm casper validate skills/restock-tags
pnpm casper run skills/restock-tags --input threshold=3     # needs ANTHROPIC_API_KEY
pnpm casper eval skills/weekly-brief --out reports/weekly-brief.json
pnpm casper run skills/budget-by-margin --input target_roas=3 --input max_change_pct=10
pnpm casper init skills/my-skill
pnpm typecheck && pnpm test

Sources: demo (default: a deterministic fake store and a fake ad account whose campaigns sell its products) or live, which builds each connector whose settings are present: Shopify from SHOPIFY_STORE_DOMAIN + SHOPIFY_ADMIN_TOKEN (read-only Admin GraphQL), Meta from META_ACCESS_TOKEN + META_AD_ACCOUNT_ID (read-only Marketing API; META_CAMPAIGN_PRODUCTS maps campaigns to product titles). Writes are proposals on either.

Measured 2026-10-16 on claude-opus-5-5, demo data: restock-tags 3/3 cases, $0.04 total, mean 4.9s; weekly-brief 2/2, $0.04, mean 6.7s; budget-by-margin 4/4, $0.43, mean 25.7s (two runs, both 4/4; report in reports/budget-by-margin.json).

The Meta Marketing API source has only been exercised against the demo source and a fake fetch in its tests; no real ad account has been connected yet.

The wedge skill

budget-by-margin reads Shopify sales by product and product margins (price and unit cost), then Meta spend and attributed revenue by campaign, and asks the Meta connector for each campaign's margin-led return: margin_roas is the contribution margin earned per unit of spend (1.0 is break-even), break_even_roas the revenue ROAS at which the campaign stops losing money after cost of goods. A campaign below break-even is cut whatever its ROAS; one well above the merchant's target_roas is raised; the rest hold. Every move is exactly max_change_pct of the current daily budget, and the write tool refuses anything larger. On demo data the Hammock campaign is the point of the skill: 2.8x ROAS clears a 2.5x target, and at a 30% margin it returns $0.85 per dollar.

Studio: generated media

pnpm casper run skills/ad-creative --input "product=Camp Mug" --input variants=3        # demo source: SVG stand-ins, $0
FAL_KEY=... pnpm casper run skills/ad-creative --input "product=Camp Mug" --source live   # fal.ai: real renders, real money
pnpm casper eval skills/ad-creative
pnpm casper run skills/ad-creative --input "product=Camp Mug" --out reports/runs/camp-mug.json   # the run's proposal shows on /studio

The media connector is the generation half of a creative suite, kept inside Casper's rules: models sit behind a MediaSource the way stores sit behind ShopifySource, every call is priced from a hand-checked table before it runs (FAL_MODELS in packages/connectors/src/media/catalog.ts, prices as of 2026-10-17: FLUX.1 dev $0.025 per megapixel, Nano Banana $0.0398 per image, Kling 2.5 Turbo Pro $0.07 per second, MiniMax H3 Max $0.05 per second), and MEDIA_MAX_USD_PER_RUN (default $1) refuses a call that would cross the cap. Generated files and their sidecars go to studio/<date>/; the hub lists them at /studio with prompt, model, cost, skill and a demo badge for stand-ins. Only models whose input schema, output schema and price were all verified are in the catalogue. Faces, likeness and face swap are out of scope, as in docs/CREATOR_MODELS.md.

Pick-and-approve, Phase 0 form: after generating, ad-creative calls propose_asset_use once for the variant it would run (use ad_image, target the product, with that variant's caption and a reason). The policy layer records it as a media.asset.use proposal like any other write; nothing is uploaded or attached, and the summary says when the asset is a demo stand-in. Eval reports now carry each case's proposals (cases[].proposed), and /studio reads them, plus any run saved with casper run --out reports/runs/<name>.json, to show a "Proposed for" line on the asset's card with the use, the target, which skill and case recorded it, and "not applied". The page's count line says how many assets were proposed and that 0 were applied. A person choosing on the page itself is the apply path's review UI and is not built.

Model token cost and generation cost are tracked separately: RunResult.costUsd and an eval's max_cost_usd cover tokens; generation spend is on each asset's sidecar and in the tool results, capped per run by the connector.

Measured 2026-10-17 on claude-opus-5-5, demo source, with the pick step: ad-creative 3/3 cases, $0.14 in tokens, mean 14.9s, $0.475 of catalogue-priced generation across the three cases (six SVG stand-ins; no model was called and nothing was charged), one media.asset.use proposal in each generating case and none in the catalogue-miss case. Report in reports/ad-creative.json, assets in studio/2026-10-17/. The fal.ai source has only run against a fake fetch in its tests; no key has been used. A first live run (MEDIA_MAX_USD_PER_RUN=0.20 … --source live, one Camp Mug variant) was attempted the same day and skipped because FAL_KEY was not in .env; no real model has been called, so there is no real cost or time to record yet.

Rules

1. Writes are proposals. No connector has an apply path yet. When one lands it goes behind the policy layer with a rollback journal, never in a tool the model calls directly.
2. Numbers come from tools. The runtime prompt says so; the evals check mentions against tool-derived facts.
3. Every skill ships an eval. validateSkill refuses a skill without one. The leaderboard is built from eval results, so this is the product, not hygiene.
4. Connectors are runtime-agnostic. @casper/connectors imports nothing from the Anthropic SDK. The same package runs under the CLI today and under a hosted runtime later.
5. Model choice is evidence-based. Default is Opus 5.5. A cheaper model is used for a skill only when its evals pass at the same rate; record the comparison in the skill's folder.
6. pnpm typecheck and pnpm test pass before a task is done. Say plainly what has only run on demo data.

Next

• Hosted runtime: Claude Managed Agents session per run with vault-held merchant credentials, Railway workers as the fallback. Connectors stay as they are.
• Share ShopifySource between src/store/ and packages/connectors (the texting app still has its own copy).
• Google Ads read connector, then budget-by-margin across both channels. Connect a real Meta ad account and run the skill against it.
• Outcome metrics beyond deterministic grading: time saved, edit distance to what the merchant published, and later lift. These feed the leaderboard.
• Creator models (design in docs/CREATOR_MODELS.md): Phase 0 steps 1 to 3 are built (consent record, voice build, blind fidelity sheet). Next: the approval loop (approved outputs append to the dataset with provenance), a creator-written voice.yaml of stated hard no's the card merges in, and the pilot.
• Studio: run ad-creative against fal.ai with a real key (only the demo source and a fake fetch have been exercised; the key was not present when the first live run was attempted on 2026-10-17), let a person pick on /studio (the skill's own pick is a media.asset.use proposal today; a person's pick is the apply path's review UI), generation spend in RunResult, and a hosted asset store instead of the repo folder.
• Apply path: proposal review UI, per-proposal approval, inverse ops in the journal, spend caps enforced server-side.
• Supabase project for skills, versions, runs, proposals and eval results; the hub web app on top.