Skip to main content
The Braintrust connector indexes the prompts, datasets and experiments of your Braintrust organization, so an evaluation result can be found from Onyx search.

​
How it works

Onyx indexes prompts first, then datasets, then experiments, and keeps its place between runs so a large organization is indexed across several attempts. A dataset or experiment table is truncated to its 10,000 most recent rows. When that happens, Onyx adds a line to the document saying the table was truncated, so the limit is visible in search rather than silent.

​
Before you begin

You need:
  • A Braintrust account with access to the organization you want to index
  • A Braintrust API key
  • An Onyx administrator account
The connector reads the Braintrust API at https://api.braintrust.dev and indexes only what the API key can read.

​
Configure Onyx

1

Open the Braintrust connector

In Onyx, go to Admin Panel → Add Connector and select Braintrust.
2

Create a credential

Paste the key into Braintrust API Key.Create a Braintrust credential with a name and the Braintrust API key
3

Choose the scope

Give the connector a name. In Project Name, enter a single Braintrust project to index only its prompts, datasets and experiments. Leave it empty to index the whole organization.Onyx checks the name when you save the connector and refuses a project it cannot find.
4

Choose the access type

Public makes all indexed Braintrust content visible to every Onyx user. Private limits the connector to selected Onyx user groups, and is a paid feature: the Business and Enterprise tiers on Onyx Cloud, and the Enterprise Edition when self-hosted.
5

Connect and verify

Select Create Connector. Then open Admin Panel → Existing Connectors, select the connector, and confirm its first indexing attempt completes.

​
Advanced settings

Raise the lookback, or set it to 0, when you need to search the rows of older experiments. Both make each indexing run read more from Braintrust.

​
Limits

The connector downloads no files, so it has no file-size cap.

​
Troubleshooting

Braintrust API rejected the API key with HTTP 401 or 403 means the key is wrong, revoked, or belongs to an organization you did not intend. Create a new key in Braintrust and update the Onyx credential.
Braintrust project '<name>' was not found means Project Name does not match a project the key can read. The name must match exactly. Leave the field empty to index the whole organization.
Failed to reach the Braintrust API means the request did not complete. Check that the Onyx server can reach https://api.braintrust.dev, then retry.
Only experiments created within Experiment Row Lookback (days) have their rows indexed; older experiments keep their summary only. Raise the lookback, or set it to 0, then re-index.
A table stops at its 10,000 most recent rows, and the document says so. This limit is not configurable.