Terraform

Check out the Resourcing Guide before getting started.

Onyx ships a set of Terraform modules that provision the AWS infrastructure Onyx needs: VPC, EKS, RDS for PostgreSQL, ElastiCache for Redis, S3, and (optionally) OpenSearch and WAF. Once the infra is up, you install the Onyx workloads with the Helm chart.

These modules are intended as a reference implementation. They reflect how we deploy Onyx and encode sensible defaults, but every environment is different — you should treat them as a starting point to fork and adapt to your account, networking, and compliance requirements rather than a black box to consume as-is.

What gets provisioned

The composed onyx module wires together the building blocks below. You can also use them individually if you need more granular control.

Module	Resource
`vpc`	VPC, public/private subnets across AZs, NAT, and an S3 gateway endpoint
`eks`	EKS cluster, managed node groups (main + dedicated Vespa), addons, IRSA
`postgres`	RDS for PostgreSQL with backups and CloudWatch alarms
`redis`	ElastiCache for Redis replication group with TLS in transit
`s3`	S3 bucket for the Onyx file store, locked to the VPC’s S3 endpoint
`opensearch`	(Optional) Amazon OpenSearch domain inside the VPC
`waf`	(Optional, recommended) AWS WAFv2 web ACL with managed rule sets, rate limits, and geo controls
`onyx`	Top-level composition that wires the modules above together

Prerequisites

Install tooling

Terraform >= 1.12.0
AWS CLI configured with credentials that can create VPCs, EKS clusters, RDS, ElastiCache, S3 buckets, IAM roles, and (optionally) WAF and OpenSearch
kubectl and helm for the post-apply Helm install

Clone the Onyx repo

git clone --depth 1 https://github.com/onyx-dot-app/onyx.git
cd onyx/deployment/terraform

The modules live under modules/aws/. You will create a small root module that calls them.

Pick an AWS region and a name

The name variable is used as a prefix for everything the modules create (VPC, cluster, RDS, S3 bucket, etc.) and is combined with the current Terraform workspace, so the same code can produce isolated dev, staging, and prod environments.

Quickstart

The snippet below is a minimal root module that provisions a complete Onyx stack on AWS using the composed onyx module.

main.tf

locals {
  region = "us-west-2"
}

terraform {
  required_version = ">= 1.12.0"

  required_providers {
    aws        = { source = "hashicorp/aws",        version = "~> 5.100" }
    helm       = { source = "hashicorp/helm",       version = "~> 2.16"  }
    kubernetes = { source = "hashicorp/kubernetes", version = "~> 2.37"  }
  }
}

provider "aws" {
  region = local.region
}

module "onyx" {
  # Adjust the source path to wherever you have the modules checked out.
  source = "./modules/aws/onyx"

  region            = local.region
  name              = "onyx"
  postgres_username = "onyx"
  postgres_password = var.postgres_password # pass via TF_VAR_postgres_password
}

# Wait for the EKS control plane before configuring the k8s/helm providers.
resource "null_resource" "wait_for_cluster" {
  provisioner "local-exec" {
    command = "aws eks wait cluster-active --name ${module.onyx.cluster_name} --region ${local.region}"
  }
}

data "aws_eks_cluster" "eks" {
  name       = module.onyx.cluster_name
  depends_on = [null_resource.wait_for_cluster]
}

data "aws_eks_cluster_auth" "eks" {
  name       = module.onyx.cluster_name
  depends_on = [null_resource.wait_for_cluster]
}

provider "kubernetes" {
  host                   = data.aws_eks_cluster.eks.endpoint
  cluster_ca_certificate = base64decode(data.aws_eks_cluster.eks.certificate_authority[0].data)
  token                  = data.aws_eks_cluster_auth.eks.token
}

provider "helm" {
  kubernetes {
    host                   = data.aws_eks_cluster.eks.endpoint
    cluster_ca_certificate = base64decode(data.aws_eks_cluster.eks.certificate_authority[0].data)
    token                  = data.aws_eks_cluster_auth.eks.token
  }
}

output "cluster_name" {
  value = module.onyx.cluster_name
}

output "postgres_endpoint" {
  value = module.onyx.postgres_endpoint
}

output "redis_connection_url" {
  value     = module.onyx.redis_connection_url
  sensitive = true
}

Apply it:

export TF_VAR_postgres_password='change-me'

terraform init
terraform plan
terraform apply

The full apply takes ~20 minutes — most of the time is spent waiting on EKS, RDS, and ElastiCache.

Common configuration

The composed onyx module exposes the inputs you’ll most often want to tune.

Core

name

string

default:"onyx"

Prefix for every resource created by the module. Combined with the active Terraform workspace, so the same code base can manage multiple environments.

region

string

default:"us-west-2"

AWS region for all resources.

postgres_username

string

required

Master username for the RDS Postgres instance.

postgres_password

string

required

Master password for the RDS Postgres instance. Marked sensitive — pass via TF_VAR_postgres_password or your secrets manager rather than hard-coding.

Networking

create_vpc

bool

default:"true"

When true, the module creates a new VPC sized for EKS. Set to false to reuse an existing VPC — see Using an existing VPC.

public_cluster_enabled

bool

default:"true"

Whether the EKS API endpoint is reachable from the public internet. Combine with cluster_endpoint_public_access_cidrs to lock it down to specific IPs.

private_cluster_enabled

bool

default:"false"

Enable the private EKS API endpoint. Recommended for production. You can enable both public and private together, or only private.

cluster_endpoint_public_access_cidrs

list(string)

default:"[]"

CIDR blocks allowed to reach the public EKS API endpoint when it is enabled.

Database & cache

enable_iam_auth

bool

default:"false"

Enables RDS IAM authentication and wires an IRSA role into the EKS module so Onyx workloads can connect to Postgres without a static password. Requires rds_db_connect_arn to be set.

postgres_backup_retention_period

number

default:"7"

Days to retain automated RDS backups. Set to 0 to disable.

redis_auth_token

string

Optional auth token for Redis. Marked sensitive.

OpenSearch (optional)

OpenSearch is off by default. Enable it if you want a managed search backend instead of the in-cluster Vespa node group.

enable_opensearch

bool

default:"false"

Provision an Amazon OpenSearch domain inside the VPC.

opensearch_instance_type

string

default:"r8g.large.search"

opensearch_instance_count

number

default:"3"

opensearch_master_user_name

string

opensearch_master_user_password

string

WAF

The waf module is optional but strongly recommended for any internet-facing deployment. It provisions an AWS WAFv2 web ACL with the common managed rule sets, rate limits, and optional geo blocking that you can attach to the load balancer fronting Onyx. Tune the inputs below to fit your traffic profile.

waf_allowed_ip_cidrs

list(string)

default:"[]"

Optional IP allowlist. Leave empty to allow all source IPs subject to the managed rule sets and rate limits.

waf_rate_limit_requests_per_5_minutes

number

default:"2000"

waf_api_rate_limit_requests_per_5_minutes

number

default:"1000"

waf_geo_restriction_countries

list(string)

default:"[]"

Country codes to block. Leave empty to disable geo restrictions.

For the full list of inputs, see modules/aws/onyx/variables.tf.

Using an existing VPC

If you already have a VPC you want to deploy into, set create_vpc = false and pass in the VPC details, including the ID of an existing S3 gateway VPC endpoint that the bucket policy will reference.

module "onyx" {
  source = "./modules/aws/onyx"

  region            = "us-west-2"
  name              = "onyx"
  postgres_username = "onyx"
  postgres_password = var.postgres_password

  create_vpc         = false
  vpc_id             = "vpc-0123456789abcdef0"
  vpc_cidr_block     = "10.0.0.0/16"
  private_subnets    = ["subnet-aaa", "subnet-bbb", "subnet-ccc"]
  public_subnets     = ["subnet-ddd", "subnet-eee", "subnet-fff"]
  s3_vpc_endpoint_id = "vpce-0123456789abcdef0"
}

The S3 bucket policy locks bucket access to the provided s3_vpc_endpoint_id. Make sure your worker nodes route S3 traffic through that endpoint, otherwise S3 calls from the cluster will be denied.

Production hardening

For production environments we recommend the following deltas from the quickstart:

module "onyx" {
  source = "./modules/aws/onyx"

  region            = "us-west-2"
  name              = "onyx"
  postgres_username = "onyx"
  postgres_password = var.postgres_password

  # Restrict the EKS control plane
  private_cluster_enabled              = true
  public_cluster_enabled               = true
  cluster_endpoint_public_access_cidrs = ["203.0.113.0/24"] # your office/VPN

  # Use IAM auth instead of a static Postgres password in cluster secrets
  enable_iam_auth    = true
  rds_db_connect_arn = "arn:aws:rds-db:us-west-2:123456789012:dbuser:*/onyx"

  # Tighten WAF
  waf_rate_limit_requests_per_5_minutes     = 1000
  waf_api_rate_limit_requests_per_5_minutes = 500
  waf_geo_restriction_countries             = ["RU", "KP"]

  # Backups
  postgres_backup_retention_period = 30

  # Optional: managed search backend
  enable_opensearch               = true
  opensearch_master_user_name     = "onyx-admin"
  opensearch_master_user_password = var.opensearch_password
}

Outputs

Once terraform apply finishes, the onyx module exposes the values you’ll need to configure the Helm chart.

Output	Description
`cluster_name`	EKS cluster name — pass to `aws eks update-kubeconfig`
`postgres_endpoint`	RDS hostname
`postgres_port`	RDS port
`postgres_db_name`	Database name (defaults to `postgres`)
`postgres_username`	Master username (sensitive)
`redis_connection_url`	Redis primary endpoint (sensitive)
`opensearch_endpoint`	OpenSearch domain endpoint, when `enable_opensearch = true`
`opensearch_dashboard_endpoint`	OpenSearch Dashboards endpoint, when enabled

Read them with terraform output:

terraform output cluster_name
terraform output -raw postgres_endpoint
terraform output -raw redis_connection_url

Install Onyx with Helm

Terraform only stands up the infrastructure — Onyx itself is installed via the Helm chart.

Update kubeconfig

aws eks update-kubeconfig \
  --name $(terraform output -raw cluster_name) \
  --region us-west-2

Create the namespace

kubectl create namespace onyx

Install the chart

The EKS module creates an IRSA-backed ServiceAccount named onyx-workload-access in the onyx namespace, which has access to the S3 bucket the module created. Point the chart at it and disable the in-cluster MinIO so file storage uses the real S3 bucket.

helm repo add onyx https://onyx-dot-app.github.io/onyx/
helm repo update

helm upgrade --install onyx onyx/onyx \
  --namespace onyx \
  --set minio.enabled=false \
  --set serviceAccount.create=false \
  --set serviceAccount.name=onyx-workload-access

You’ll also want to wire the chart’s database, Redis, and (optionally) OpenSearch settings to the Terraform outputs via your own values.yaml. See the Kubernetes deployment guide for the full set of values.

Verify

kubectl get pods -n onyx

Wait until all pods report Running before accessing Onyx.

Workspaces and multiple environments

Every resource the onyx module creates is named with the active Terraform workspace, so the same root module can manage isolated environments without collisions:

terraform workspace new dev
terraform workspace new staging
terraform workspace new prod

terraform workspace select prod
terraform apply

A name = "onyx" module call in workspace prod will produce onyx-prod-prefixed resources, while the same call in dev produces onyx-dev resources.

Notes & gotchas

Sensitive outputs. postgres_username, redis_connection_url, and the EKS CA data are marked sensitive. Hand them off to your secrets manager or Helm values file rather than echoing them to logs.
State storage. The quickstart uses local state for brevity. For shared or production use, configure an S3 backend with DynamoDB locking.
First apply is infra-only. EKS takes several minutes to become active. The null_resource.wait_for_cluster block in the quickstart blocks the Kubernetes/Helm providers until the API server is reachable.
The Vespa node group is tainted. The eks module provisions a dedicated node group for Vespa with a vespa-dedicated=true:NoSchedule taint. The Helm chart’s Vespa pods tolerate it; everything else lands on the main node group.
Reference, not a product. The modules encode the choices we make for our own deployments. Read them, copy what’s useful, and replace what isn’t.

Getting Started

Deploy Locally

Onyx Anywhere

Deploy on Cloud

Authentication

Miscellaneous

What gets provisioned

Prerequisites

Quickstart

Common configuration

Core

Networking

Database & cache

OpenSearch (optional)

WAF

Using an existing VPC

Production hardening

Outputs

Install Onyx with Helm

Workspaces and multiple environments

Notes & gotchas

Next Steps

Configure Authentication

More Onyx Configuration Options

Getting Started

Deploy Locally

Onyx Anywhere

Deploy on Cloud

Authentication

Miscellaneous

​What gets provisioned

​Prerequisites

​Quickstart

​Common configuration

​Core

​Networking

​Database & cache

​OpenSearch (optional)

​WAF

​Using an existing VPC

​Production hardening

​Outputs

​Install Onyx with Helm

​Workspaces and multiple environments

​Notes & gotchas

​Next Steps

Configure Authentication

More Onyx Configuration Options

What gets provisioned

Prerequisites

Quickstart

Common configuration

Core

Networking

Database & cache

OpenSearch (optional)

WAF

Using an existing VPC

Production hardening

Outputs

Install Onyx with Helm

Workspaces and multiple environments

Notes & gotchas

Next Steps