Guide
Configure Onyx to use models exposed by your LiteLLM Proxy server.Onyx reads the available models from the LiteLLM Proxy
/v1/models endpoint.
This is a good option when you want one gateway in front of multiple providers or self-hosted backends.Set Up LiteLLM Proxy
Deploy LiteLLM Proxy and make sure it is reachable from the Onyx server.You will need the LiteLLM Proxy API Base URL and API Key.The local default is commonly
http://localhost:4000.Navigate to Language Models
Access the Admin Panel from your user profile icon, then navigate to Configuration → Language Models.
Configure LiteLLM Proxy
Select LiteLLM Proxy from the available providers.Give your provider a Display Name.Enter the proxy API Base URL and API Key.Click Fetch Available Models to load the models currently exposed by your LiteLLM Proxy instance.
Review the Imported Models
Onyx stores the model IDs returned by LiteLLM Proxy and makes those models available to your users.This is useful when LiteLLM is routing requests to multiple providers through a single gateway.
Configure Default and Fast Models
The Default Model is selected automatically for new custom Agents and Chat sessions.Designating a Fast Model is optional.
This Fast Model is used behind the scenes for quick operations such as evaluating the type of message,
generating different queries (query expansion), and naming the chat session.
Choose Visible Models
In the Advanced Options, you will see a list of all models available from this provider.
You may choose which models are visible to your users in Onyx.Setting visible models is useful when a provider publishes multiple models and versions of the same model.