Ship & Deploy
Stage: Ship · What it does: deploy a trained or tuned model as a real-time endpoint you own — with canaries, LoRA adapters, autoscaling, and automatic serving-artifact building for raw fine-tunes.
Why it matters
- You own the endpoint and the weights. No vendor per-token fee, no vendor roadmap for updates.
- Guided end-to-end. Pick a completed run, the URL auto-fills, deploy — the platform handles the rest.
- Safe by default. Raw adapter artifacts are automatically exported to a serving tarball; broken artifacts fail in minutes with the real reason, not after an expensive dead endpoint.
How to use
- Ship → Deploy model.
- Pick the completed run in the Model dropdown — the Model S3 URL auto-fills (prefers a completed export, falls back to the run's output).
- Choose instance + count (estimate shows hourly cost).
- Deploy. Status flows automatically:
Exporting → Verifying → Creating → InService
- Exporting = auto-building the serving export for a raw fine-tune (behind the scenes, no manual step).
- Verifying = artifact is tested on the serving image before any endpoint spend.
- Creating = SageMaker provisions your instance.
- InService = live. Test inference, run as an agent, resize, attach adapters.
Optional: GPU smoke and self-heal are available at launch; self-heal retries deploy-stage failures automatically.
Deploy features
| Feature | What it does |
|---|---|
| Canary | Route a % of traffic to a new variant before full cutover |
| LoRA adapters | Attach a tuned adapter to a deployed base model without retraining |
| Adapter merge | Merge multiple LoRA adapters into one serving model |
| Autoscaling | Scale instance count with traffic (capped by your grant) |
| Data capture | Log invocations for audit/replay |
| Instance fallback | A scarce instance falls back to a sibling instead of failing |
| Hosted | k3ld runs your endpoint on its managed infrastructure — zero AWS to set up (default) |
| BYO AWS | You bring the account; k3ld deploys and operates the endpoint into it |
| Self-hosted on AWS | The full platform runs in your environment, operated by you |
| Deleting | Any deploy — including stuck/Staged ones — can be deleted; teardown stops in-flight GPU jobs |
Real-world examples
🏦 Financial services — guarded rollout
A bank deploys a credit-risk scoring model, runs a canary at 10% traffic against the incumbent, and only cuts over when the error rate and score distribution match. Data capture is on for audit. The model lives on the bank's own endpoint with a fixed hourly cost.
🧑💼 SaaS — hot-swap support model without downtime
A SaaS company deploys a new support-assistant fine-tune, attaches it as a canary on the existing endpoint, and rolls traffic gradually. A regression in tone is caught at 10% traffic — rolled back in one click, zero downtime.
🛒 Retail — one base, many adapters
An e-commerce platform keeps one deployed base model and attaches adapter variants per brand/region (US, EU, APAC listing styles). Tuning a new brand = training a small adapter + attach, instead of a full redeploy.
🏥 Healthcare — your account, your rules
A health system starts on the hosted option (zero infra), uses BYO AWS so the endpoint runs in their own account, or fully self-hosts on AWS — every option covered by their existing security approvals and cloud procurement, with full version control over updates.
🧑💼 Any team — the "deploy didn't finish" case
A deploy stalls in Exporting or Staged. Instead of being stuck, the user opens the deploy and Deletes it — in-flight export/verify GPU jobs are stopped, models/configs/artifacts are torn down, and nothing keeps billing.
Gotchas
- Verify your instance fits the model. The platform VRAM-checks before creating an endpoint (the "Qwen3-8B on g4dn" lesson) — never create an endpoint the model can't fit.
- Deleting with "Keep model + config for later" stops billing but keeps a one-click resume (≈$0 + small S3 storage).
- Raw fine-tunes auto-export on deploy; if the export fails you'll see the real reason (not a cryptic error) and can fix the artifact.