Skip to main content
The ingestion API is a lightweight way to index documents. If you need more control, consider creating a Connector in the Admin Panel or following the Create a Connector API guide.

When to use the Ingestion API?

Use the Ingestion API for:
  • Unsupported sources: Add content from systems that do not have built-in Connectors
  • Supplemental data: Add extra context to existing Connector data (like README files for GitLab)
  • Editing documents: Change documents in Onyx when the Admin cannot update the original document in the source
  • Programmatic workflows: Add document indexing to your existing data pipelines

Guide

Skip to the Full Code section if you don’t want the step-by-step guide. In this example, we’ll index local files with the Ingestion API.
The Ingestion API base URL is https://cloud.onyx.app/api/onyx-api/ingestion or your own domain.
Required permission: Manage Connectors & Document Sets (manage:connectors), or Admin (admin). A Group Manager can ingest into connector-credential pairs in the groups they manage, but not into the default pair, which is public.Use an API key or a full-access Personal Access Token. A limited-access token cannot call the Ingestion API, because no token scope grants manage:connectors.
Writes and deletes need a vector database. On deployments running with DISABLE_VECTOR_DB set (Onyx Lite), POST and DELETE /onyx-api/ingestion answer with 501.
1

Prepare your Connector

Since we want these files to appear in the Connectors page, we’ll need to first create a Connector. For this example, we’ll create a File Connector with a dummy text file.If you already have a Connector you would like to associate your Ingestion files with, open Admin Panel > Connectors, click the Connector, and copy the cc_pair_id from the URL.https://cloud.onyx.app/admin/connector/243 -> cc_pair_id is 243If you omit cc_pair_id, Onyx attaches the document to the default connector-credential pair (ID 1), which is public to every user.
2

Prepare your request

3

Process your documents into the proper JSON object

See Core Concepts: Documents for details on the IngestionDocument object.Image sections reference a file you uploaded first with POST /user/projects/file/upload. The upload must belong to the same user that calls the Ingestion API. Tabular sections and the file_id field are not accepted on ingested documents.
JSON
4

Make the request to the /ingestion endpoint

The request returns after Onyx has indexed the document, so the document is searchable immediately. The response is {"document_id": "...", "already_existed": false}. already_existed is true when a document with the same id was already indexed and Onyx updated it in place.

Manage ingested documents

The same router lists and deletes documents that came in through the Ingestion API. All three calls need manage:connectors. To update a document, send POST /onyx-api/ingestion again with the same id. Onyx replaces the content and skips re-embedding when the content has not changed.

Full Code

Python

Extra Example

This script crawls a specified directory for JSON files and sends the contents to the Ingestion API.
Python