Serverless Inference
Rate Limits
Rate limits protect shared infrastructure and ensure fair access across all users. Limits are applied per API key and vary by plan.
How rate limits work
Rate limits on Kimchi Serverless Endpoints are dynamic. They adjust based on current system load and available capacity rather than being fixed numbers. This means your effective throughput may vary, but the system is designed to maximise what each tier can use at any given moment.
When a rate limit is exceeded, the API returns a 429 Too Many Requests response with a Retry-After header indicating how long to wait before retrying.
Limits by plan
| Free | Starter | Enterprise | |
|---|---|---|---|
| Rate limits | Dynamic (infrastructure protection) | Dynamic (higher capacity) | Custom |
| Primary gating | Credit balance | PAYG after credits | Committed usage |
| When credits run out | Requests rejected | Billing kicks in | Custom |
Best practices
- Implement exponential backoff — When you receive a 429 response, wait and retry with increasing delays.
- Batch requests where possible — Combine multiple small prompts into fewer, larger requests to reduce overhead.
- Monitor your usage — Track token consumption in the Kimchi console to anticipate when you might approach limits.
- Upgrade for higher capacity — Paid tiers receive dynamically higher effective throughput.
📘
Rate limits exist across all tiers for infrastructure protection. The primary gating mechanism for the Free tier is credit balance, not rate limits. Paid tiers enjoy higher effective capacity.