Cost Control

Stage: all · What it does: keep every run and endpoint on a predictable infrastructure cost instead of a per-token bill — with cost estimates before every run, an optional GPU smoke test, and a run-confidence cache that tells you how likely a config is to succeed before you spend.

The model

Training/fine-tuning: you pay for the instance time (e.g. an hourly rate for a g5 GPU), not per token. Estimates are shown before launch. Spot instances can cut fine-tune costs ~60–70%.
Serving: a fixed hourly cost for your endpoint's instance, with autoscaling capped by your grant.
No per-token meter. Usage grows without the bill growing linearly per request.

The three safety rails

1. Cost estimate before every run

Every pipeline/fine-tune/post-train shows an estimate (instance × time). Budget owners get a number and a green check, not a surprise.

2. GPU smoke test (checkbox)

"Run GPU smoke before launch" runs a short GPU job that validates your data + config before the full run. A malformed dataset or a bad format fails in minutes at pennies — instead of hours into a full run.

3. Run-confidence cache

The platform keeps a cache of past run outcomes (config fingerprint + near-config keys). Before you launch, it tells you: - 🟢 High confidence — an identical config completed successfully before - 🟡 Medium — a near-identical config (same data class) completed - 🔴 Low — identical/near configs failed before, or none seen

This is informational (you can still launch), but it surfaces "this exact recipe has failed twice already" before you spend.

How to use

Before launching any run, read the estimate and, if shown, the confidence indicator.
Tick GPU smoke on anything non-trivial — it's the cheapest insurance you'll buy.
For serving, right-size the instance: the UI shows VRAM fit, hourly cost, and autoscaling caps from your grant.
When done with an endpoint, delete it (or keep-for-later at ≈$0 + small S3) so idle endpoints don't silently burn.

Real-world examples

🧑‍💼 SaaS — finance can sign off

A startup's finance lead approves a fine-tune because the estimate says "$12 for this run on a spot g5" and the GPU smoke passed. No surprise line item at month end.

🏦 Financial services — no idle spend

A bank's ML team reviews idle endpoints weekly using the platform's idle-burn report (which bounds estimates by creation time, so fresh endpoints aren't misreported). Stale dev endpoints get deleted and their GPU jobs stopped.

🛒 Retail — confidence saves a bad retrain

Before a weekly retrain, the confidence indicator shows 🔴 — the exact config failed last week. The team fixes the data issue before spending the run, not after.

🏭 Manufacturing — smoke catches a schema change

A pipeline's raw sensor files change column order. The GPU smoke fails in 3 minutes with the real error ("missing column"), saving a 2-hour full run that would have failed anyway.

Gotchas

Estimates are based on instance × expected time; spot can drop fine-tune cost but is less predictable in availability.
The confidence cache is informational — a 🔴 doesn't block you, but read the reason (data format? quota? instance fit?).
Deployed endpoints bill by uptime: delete or stage what you're not using.