Skip to main content

​
Overview

The Onyx application is a set of Docker Containers which can be deployed on any infra and for any scale.
As an open source project, Onyx is designed with security and data privacy as a foundational principle so that teams can trust the software even without needing partnership guarantees.

​
System Architecture

Onyx Architecture

​
Core Components

​
Application Layer

  • Frontend web client: Next.js-based web application providing the user interface for the Onyx platform
  • API Server: Python FastAPI web server to handle business logic and requests
  • Background Workers: Python workers to handle async jobs like fetching document updates, etc.

​
Model Servers

  • Inference Model Server: Serves the embedding and reranking models used at query time to retrieve and order context
  • Indexing Model Server: A dedicated model server that handles embedding of documents during indexing, isolating heavy indexing workloads from live query traffic

​
Data Layer

  • Relational Database: Postgres database to store application data, user sessions, and system state
  • Search Engine: OpenSearch providing keyword (BM25) search + vector store for retrieving context for queries
  • In Memory Cache: Redis, used to optimize performance
  • File Store: the object store (SeaweedFS) for user uploaded files and documents pulled from connectors

​
Infrastructure Layer

  • Request Router: Nginx reverse proxy for load balancing and request routing

​
Component Replacements

Some components of the Onyx stack are easily replaceable while others are not. The object store can be replaced with S3, Google Cloud Storage, Azure Blob Storage, or any S3-compatible object storage service. The file store interface is abstracted to make this straightforward. Redis can be replaced with a managed Redis service like AWS ElastiCache or Google Cloud Memorystore for Redis. Other in-memory caches could work but would require code changes. Postgres can be replaced with a managed Postgres service such as AWS RDS. It is much more challenging and not advised to switch to a different relational DB due to language specific features and optimizations. OpenSearch can be replaced with a multi-node OpenSearch cluster (typically not necessary) or a managed OpenSearch service such as AWS OpenSearch Service. OpenSearch is tightly integrated with the context retrieval functionalities and switching to a different search engine would require significant development effort to maintain the same functionality. Nginx can be removed and replaced with your own routing proxy.