Skip to main content

Guide

Configure Onyx to use models exposed by your LiteLLM Proxy server.
Onyx reads the model list from your LiteLLM Proxy. This is a good option when you want one gateway in front of multiple providers or self-hosted backends.
1

Set Up LiteLLM Proxy

Deploy LiteLLM Proxy and make sure it is reachable from the Onyx server.You need the LiteLLM Proxy API Base URL and an API Key (a LiteLLM virtual key or the master key).The local default is commonly http://localhost:4000.
2

Navigate to Language Models

Open the Admin Panel from your user profile icon, then open Language Models.
3

Connect LiteLLM Proxy

Under Gateways & Routers, click Connect on the LiteLLM Proxy card.Enter the proxy API Base URL and paste your API Key.
4

Load Your Models

In the Models section, click Refresh models. Onyx lists the models that your LiteLLM Proxy currently routes, including their context size when the proxy reports it.
5

Choose Visible Models

In the Models section, select the models that users can pick in chat. If a model is missing, open More Models or type its name and click Add Model.Limiting visible models helps when a provider publishes many models and versions of the same model.
6

Set Models Access

Under Models Access, choose All Users & Agents or Named Groups & Agents.With Named Groups & Agents, only Admins and the User Groups and Agents you name can use the models. Named groups need a Business plan or higher.
7

Connect

Click Connect. Onyx tests the proxy with one of the selected models before it saves the provider.