​
Prerequisites

  • Download and install the AWS CLI. This is required for creating and accessing the cluster.
  • Download and install kubectl CLI. This is needed for cluster access through the cli.

​
Cluster Setup and Configuration

​
Create the Cluster

Navigate to Elastic Kubernetes Service (EKS) and create a new cluster. For Cluster service role, create a new IAM role accepting all the defaults for the user. It’s name can be something like onyx-eks-cluster-role. Make sure to click the refresh button if you don’t see it as an option! For the Kubernetes version, select a version with standard support still offered and under Upgrade policy, select Standard. In the add-ons section, make sure to add the Amazon EBS CSI Driver add-on, as this is needed for the Persistent Volume Claims (PVCs) to be fulfilled. Also make sure to keep the other default add-ons! Cluster creation screen Select defaults for the rest of the forms. Finally, review the complete cluster setup and click Create when satisfied. The cluster may take several minutes to become ready.

​
Adding Nodes

Next we need to add worker nodes to the cluster. This is where the Onyx services will reside. When the cluster is Active and while viewing the cluster, select the Compute tab and then Add node group. Adding a node group First, provide a Name for the group (something like onyx-node-group). For Node IAM role, either select an existing role that your organization has setup or create a new role. Whichever route we choose, we have to make sure that the AmazonEBSCSIDriverPolicy is attached to the role, as this will be needed for our Persistent Volume Claims (PVCs) to be fulfilled. If creating a role, all the other defaults provided by AWS should work (with the addition of the AmazonEBSCSIDriverPolicy). Give the role a name like onyx-eks-nodegroup-role. Configure group node Replace the Instance Types with c5.2xlarge machines (or c5.4xlarge if you are planning on scaling up beyond 100k documents). For disk, we recommend setting Volume size somewhere in the 200GB - 800GB range depending on how many documents you plan on indexing (storage is cheap). For most setups, we recommend setting the Desired size and Minimum size to 1, although you can increase this if needed to scale the cluster up once traffic picks up. Maximum unavailable can generally be left as default. After reviewing, keep the default for the Networking section, and then proceed through to Create! Setting compute and scaling This may take up to 15 minutes for the compute nodes to come online.

​
Create and Connect a User

You will need to create an IAM user that will have access AWS and the cluster from the command line. Navigate to the IAM Dashboard found here. Select Users on the left sidebar and then Create user. Create user Give the user a name like onyx-eks-user. For the user permissions, click Attach policies directly and provide the following permissions:
  • AmazonEKSClusterPolicy
  • AmazonEKSServicePolicy
Finishing reviewing and creating the user. Back on the user’s page, click into the newly created user, and then select Create access key. Create access key Follow the process for creating the access key and secret. Select the Command Line Interferace (CLI) option during creation. Be sure to save the Access key and Secret access key for later. Navigate back to the EKS cluster and select Access and then Create access entry. EKS access entry In the IAM principal select the IAM ARN that you just created. Then click next. EKS IAM ARN For the Access policies, ensure that the Policy name is set to AmazonEKSClusterAdminPolicy and then click next. Review and then create the Access Policy. EKS policy

​
Fetching the kubeconfig file

Log into the AWS CLI with aws configure and provide the access key and secret key from the IAM you created. Then configure aws to connect with the cluster filling in the region-code and cluster-name in the following command 
aws eks update-kubeconfig --region region-code --name cluster-name
Additional material found on the AWS EKS documentation.

​
Installing the Services

You will want to create a namespace for this to install. Namespaces are logical separations of grouped services for resource and permission managament. 
kubectl create ns onyx
Next, we need to make sure that the gp2 storage class is set to the default storage class. This can be done by running the following command: 
kubectl patch storageclass gp2 -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}' -n onyx
Navigate to the onyx/deployment/helm/charts/onyx directory in the repository that you have cloned down. Make sure you have Helm installed. Before deploying, take a read through the values.yaml file. Adjust as necessary. Afterwards, you can run the installation of the Onyx helm chart. 
helm install onyx . --create-namespace --namespace onyx
It may take a few minutes for the all the services to come online. To monitor the progress, run the following command: 
kubectl -n onyx get pods
To check the status of the API server (usually the last to come online), run the following command: 
kubectl -n onyx get pods | grep api-server | awk '{print $1}' | xargs -I {} kubectl -n onyx logs {} -f
You can navigate find the nginx load balancer by running the following command and then navigating to the url provided. 
kubectl get svc -n onyx | grep nginx | awk '{print $4}'

​
Upgrading

To upgrade Onyx services, we can adjust the global.version field in values.yaml to the specific version (e.g. something like nightly-latest-20250616) we want to upgrade to. Then, we can run: 
helm upgrade onyx . -n onyx

​
Uninstalling

Run: 
helm uninstall onyx -n onyx
Vespa, Postgres, and MinIO leave behind a PVC. Delete it if you are completely done. 
kubectl -n onyx get pvc
kubectl -n onyx delete pvc vespa-storage-da-vespa-0
kubectl -n onyx delete pvc onyx-minio
kubectl -n onyx delete pvc data-onyx-postgresql-0