LiteLLM routing

Keep Kimi, GLM and DeepSeek separate in LiteLLM even when the app wants one tidy interface.

LiteLLM is good at orchestration. The failure mode shows up when routing, access and billing all get mashed into one config layer. For Chinese providers, the useful fix is usually to make route identity explicit and keep usage visible per upstream.

Buyer-intent signal: if someone is already wiring Kimi, GLM or DeepSeek into LiteLLM, they have a real workload. The fastest commercial step is one verified route and one tiny paid test, not a wholesale migration.

1. Split orchestration from access

Use LiteLLM for model selection, fallback groups and application-facing compatibility. Use a separate route identity for the upstream account, region and payment rail.

route identity = provider adapter + baseUrl + api key slot + model id + billing account

If two routes share the same visible model name but come from different vendors or regions, treat them as different routes even if the app only sees one OpenAI-style interface.

2. Keep one explicit alias per real upstream

The cleanest debug pattern is to give each upstream a unique alias, then verify the final host and model id on the first request.

model_list:
  - model_name: kimi-chat
    litellm_params:
      model: kimi-k2
      api_base: https://example-router/v1
      api_key: os.environ/KIMI_API_KEY

  - model_name: glm-chat
    litellm_params:
      model: glm-5
      api_base: https://example-router/v1
      api_key: os.environ/GLM_API_KEY

Do not trust a config entry just because it exists in a dashboard. Trust the route only after one raw request succeeds against the exact base URL and model id you expect.

3. Debug order that saves time

  1. Prove one raw chat completion for each upstream.
  2. Attach a unique alias to each route.
  3. Log provider, base URL host, model id and usage together.
  4. Only then add fallback groups, retries and automatic merge logic.

4. Common LiteLLM failure patterns

Symptom Likely cause Useful next check
One model overwrites another Model name is being used as the only unique key. Separate alias from provider identity.
Usage totals look wrong Multiple usage events are being merged without route context. Log upstream host, model id and usage once per request.
Model works in a dashboard but not in LiteLLM Base URL or key slot is wrong for the live route. Run one raw request before changing LiteLLM config.

5. Where Black Eagle AI fits

Black Eagle AI is the narrow path for teams that are blocked by China phone verification, local payment or a need to test one prepaid route before investing more time in provider setup.

Run a $5 endpoint test Pay $5 Request access

6. Independent gateway pattern

When the upstream is an independent OpenAI-compatible gateway with its own model catalog and prepaid buyer path, treat the gateway as the source of truth for pricing and model identity. In LiteLLM, that usually means keeping the alias explicit and using the OpenAI-compatible route directly instead of inventing a provider-specific wrapper that hides the real host.

model_list:
  - model_name: gpt-5.5
    litellm_params:
      model: openai/gpt-5.5
      api_base: https://your-gateway.example.com/v1
      api_key: os.environ/GATEWAY_API_KEY

If the gateway also exposes public pricing or model metadata, the useful documentation pattern is to link that page beside the LiteLLM config note, then tell the user to verify one raw request before expanding the routing setup.

Commercial shortcut: if the blocker is just access, payment or provider setup, start with one small paid request and only then decide whether you need a more elaborate LiteLLM router.