Serverless Inference

    Continue

    This tutorial configures Continue, an agentic coding tool available as a CLI and IDE extension, to use Large Language Models hosted by Kimchi.

    This tutorial focuses on the Continue CLI. IDE extensions automatically pick up the same configuration file — see IDE integration after completing the CLI setup.

    Overview

    By the end of this tutorial, you'll be able to use GLM 5 and MiniMax 2.5 with Continue in your terminal, VS Code, and JetBrains IDEs.

    This tutorial is intended for developers familiar with command-line tools and YAML configuration files.

    Prerequisites

    Before starting, ensure you have:

    1. A Kimchi API key — Go to Kimchi → API Keys and click Create API key. Copy the key and store it securely.
    2. Continue CLI installed — Follow the official install instructions.
    📘
    Kimchi base URL is OpenAI-compatible. Although this tutorial covers Continue, information explained here should be transferable to other tools and IDEs, assuming they offer integration with third-party model providers who expose an OpenAI-compatible API.

    Step 1: Create the Continue configuration file

    Create the configuration file for your OS:

    text
    ~/.continue/config.yaml

    Add the following content:

    yaml
    name: Kimchi Config
version: 1.0.0
schema: v1
model_defaults: &model_defaults
  provider: "openai"
  apiBase: "https://llm.kimchi.dev/openai/v1"
  apiKey: "YOUR_CASTAI_API_KEY"
models:
  - name: "glm-5-fp8"
    model: "glm-5-fp8"
    <<: *model_defaults
  - name: "minimax-m2.5"
    model: "minimax-m2.5"
    <<: *model_defaults

    Replace YOUR_CASTAI_API_KEY with the API key you generated from the Kimchi console.

    The model_defaults anchor (&model_defaults) avoids repeating the provider, base URL, and API key for each model. Both GLM 5 and MiniMax 2.5 inherit these defaults.

    Step 2: Verify your configuration

    Launch Continue with the --config flag:

    bash
    cn --config ~/.continue/config.yaml

    Send a test prompt like "Hello" to confirm the connection.

    IDE integration

    The configuration file is automatically recognized by Continue extensions for VS Code and JetBrains. No additional setup is needed — install the extension and it picks up the same config.

    If the extension doesn't load the config, confirm that ~/.continue/config.yaml (macOS/Linux) or %USERPROFILE%\.continue\config.yaml (Windows) exists and contains valid YAML.

    1. Install the Continue extension from the VS Code Marketplace.
    2. Open the Continue panel from the sidebar and send a test prompt to verify the connection.

    The Continue icon appears in the sidebar. Clicking it opens the Continue panel. Confirm that Local Config is loaded and the model selector points to a model from the config file.

    Next steps

    Rate limits

    Understand request and token limits by plan before building for production.