Skip to main content

Install on Kubernetes

OSIE runs on Kubernetes as three components — the API, the client portal and the admin portal — backed by MongoDB and RabbitMQ. In production the databases are run by Kubernetes operators and OSIE itself by the official Helm chart.

An operator is a controller that knows how to run a stateful workload: it initialises the MongoDB replica set, generates the credentials, provisions the storage, takes scheduled backups with point-in-time recovery, and performs rolling upgrades. RabbitMQ cluster formation, plugin configuration and restarts are handled the same way. Without operators, every one of those procedures is yours to script and to get right during an incident.

All manifests and example values used on this page are maintained in the public osie/deployment repository, under kubernetes/. Clone it and adapt the files to your environment rather than copying snippets from this page:

git clone https://github.com/osie/deployment.git
cd deployment/kubernetes
ComponentKindManaged by
osie-apiStatefulSetOSIE Helm chart
osie-ui — client portalDeploymentOSIE Helm chart
osie-admin — admin portalDeploymentOSIE Helm chart
MongoDB replica setPerconaServerMongoDBPercona Operator for MongoDB
RabbitMQRabbitmqClusterRabbitMQ Cluster Operator
Single machine

For an evaluation or a small installation on one machine, use the Docker Compose installer, or the MicroK8S tutorial if you want Kubernetes without a cluster.

Prerequisites​

  1. A Kubernetes cluster (1.30 or newer) with:
    • a default StorageClass with dynamic provisioning
    • an ingress controller (ingress-nginx, Traefik, …) exposed on a public IP address
    • cert-manager with a ClusterIssuer named letsencrypt, unless you provide your own TLS secret
  2. A DNS record for the hostname OSIE is served on — osie.mycompany.com — pointing at the ingress IP address. All three components share that single hostname.
  3. kubectl and Helm 3 on your workstation.

The single-node profile below needs about 2 vCPU and 4 GB of RAM and claims 35 GB of storage; the high-availability profile needs about 6 vCPU and 20 GB of RAM and claims 170 GB. Provide headroom on top of those figures.

Install the operators​

Both operators are cluster-wide and installed once, however many OSIE installations the cluster later hosts.

# Percona Operator for MongoDB
helm repo add percona https://percona.github.io/percona-helm-charts
helm repo update
helm install percona-mongodb-operator percona/psmdb-operator \
  --namespace percona-mongodb \
  --create-namespace \
  --version 1.22.0 \
  --set watchAllNamespaces=true

# RabbitMQ Cluster Operator
kubectl apply -f https://github.com/rabbitmq/cluster-operator/releases/download/v2.20.0/cluster-operator.yml

Wait until both are available:

kubectl wait deployment percona-mongodb-operator-psmdb-operator \
  -n percona-mongodb --for=condition=Available --timeout=300s
kubectl wait deployment rabbitmq-cluster-operator \
  -n rabbitmq-system --for=condition=Available --timeout=300s

Deploy the databases​

Create the namespace, then apply one MongoDB and one RabbitMQ resource. Each profile is a ready-made manifest in the deployment repository:

ProfileMongoDBRabbitMQ
Single nodepsmdb.single-node.yml — 3-member replica set co-located on one node, 10 GB, minimal requestsrabbitmq.single-node.yml — 1 node, 5 GB
High availabilitypsmdb.ha.yml — 3-member replica set spread across nodes, 50 GB, scheduled S3 backups with point-in-time recoveryrabbitmq.ha.yml — 3 nodes, 20 GB
kubectl apply -f infrastructure/namespace.yml

# MongoDB — pick one
kubectl apply -f infrastructure/percona-mongodb/psmdb.single-node.yml
#kubectl apply -f infrastructure/percona-mongodb/psmdb.ha.yml

# RabbitMQ — pick one
kubectl apply -f infrastructure/rabbitmq/rabbitmq.single-node.yml
#kubectl apply -f infrastructure/rabbitmq/rabbitmq.ha.yml

The manifests deploy into the osie namespace and leave storageClassName commented out, so the cluster default applies. To install into a different namespace, change metadata.namespace in all three files. Before applying psmdb.ha.yml, set your bucket in its backup.storages block and your credentials in the mongodb-backup-s3 secret it defines.

Wait for both operators to report their cluster healthy. Until they do, the databases are scheduled but not accepting connections, and the API fails to start:

kubectl wait --for=jsonpath='{.status.state}'=ready psmdb/mongodb -n osie --timeout=600s
kubectl wait rabbitmqcluster/rabbitmq -n osie --for=condition=AllReplicasReady --timeout=600s

Each operator writes its generated credentials to a secret in the same namespace — percona-server-mongodb-users and rabbitmq-default-user. The Helm chart reads them directly, so no database password is ever copied into your values file.

Configure the values file​

Start from values.example.yaml in the deployment repository:

values.yaml
global:
  ingress:
    enabled: true
    hostname: "osie.mycompany.com"
    ingressClassName: "nginx"
    annotations:
      cert-manager.io/cluster-issuer: letsencrypt
    tls: true

# MongoDB managed by the Percona operator
externalMongodb:
  hosts: "mongodb-rs0"
  port: 27017
  database: osie
  existingSecret: percona-server-mongodb-users
  existingSecretUsernameKey: MONGODB_DATABASE_ADMIN_USER
  existingSecretPasswordKey: MONGODB_DATABASE_ADMIN_PASSWORD

# RabbitMQ managed by the RabbitMQ operator
externalRabbitmq:
  host: rabbitmq
  port: 5672
  existingSecret: rabbitmq-default-user
  existingSecretUsernameKey: username
  existingSecretPasswordKey: password

# The chart can also deploy its own databases. Disable them — the operators own that job.
mongodb:
  enabled: false
rabbitmq:
  enabled: false
keycloak:
  enabled: false
  • externalMongodb.hosts is the service the Percona operator creates for the replica set, named <cluster-name>-<replset-name> — mongodb-rs0 with these manifests. The connection string is completed with the chart default extraQueryParams: "authSource=admin&replicaSet=rs0", which matches them.
  • mongodb, rabbitmq and keycloak must be disabled explicitly. They default to enabled, so that installations created before operator support keep working across chart upgrades.
  • No identity settings belong here. A new installation authenticates against OSIE's own identity service, and customer sign-in is configured afterwards per realm — see Identity management.
  • Add license.key if you were given one, and the smtp block if OSIE should send email — invoices, invitations and password recovery all depend on it. Every setting the chart accepts is listed in its values.yaml.

High availability​

With the high-availability database manifests applied, run the three components with more than one replica:

values.yaml
api:
  replicaCount: 3
ui:
  replicaCount: 3
admin:
  replicaCount: 3

Scheduled work inside the API — billing runs, metric collection, invoice generation — is guarded by a lock held in MongoDB, so exactly one replica executes each job however many are running.

Install the chart​

helm repo add osie https://helm.osie.io
helm repo update osie

helm --namespace osie upgrade --install osie osie/osie -f values.yaml

The API applies its database migrations at startup and reports ready only afterwards, so the first install takes a few minutes:

$ kubectl -n osie get pods

NAME                          READY   STATUS    RESTARTS   AGE
mongodb-rs0-0                 1/1     Running   0          8m
mongodb-rs0-1                 1/1     Running   0          7m
mongodb-rs0-2                 1/1     Running   0          7m
osie-admin-7c5fb75c85-28zjc   1/1     Running   0          3m
osie-api-0                    1/1     Running   0          3m
osie-ui-5648b4b458-trl4d      1/1     Running   0          3m
rabbitmq-server-0             1/1     Running   0          8m

If a pod stays unready, its logs say why:

kubectl -n osie logs -f osie-api-0

Three ingress objects share the hostname — the client portal on /, the admin portal on /osie_admin, and the API on its own paths:

$ kubectl -n osie get ingress

NAME         CLASS   HOSTS                ADDRESS       PORTS     AGE
osie-admin   nginx   osie.mycompany.com   12.34.56.78   80, 443   3m
osie-api     nginx   osie.mycompany.com   12.34.56.78   80, 443   3m
osie-ui      nginx   osie.mycompany.com   12.34.56.78   80, 443   3m

First sign-in​

Open https://osie.mycompany.com/osie_admin. On first access you are guided through onboarding, where you create the administrator account and set its password. The client portal is served from https://osie.mycompany.com/.

Save the encryption key​

OSIE encrypts sensitive values in the database — OpenStack passwords, access keys and similar — with a symmetric key passed to the API as OSIE_ENCRYPTION_DEFAULT_KEY. The chart generates one on first install and stores it in the <release-name>-bcrypt secret. The same key is required to decrypt that data: without a copy of it, a restore into a fresh installation cannot read what it restored.

kubectl -n osie get secret osie-bcrypt -o json | jq -r '.data."bcrypt-password"' | base64 -d

Keep the value outside the cluster. To reuse an existing key on a new installation, set encryption.bcrypt.password, or point encryption.bcrypt.existingPasswordSecret at a secret you manage yourself.

Upgrading​

Upgrades use the same command as the install. Database migrations run when the new API pod starts, so watch its logs before declaring the upgrade done.

helm repo update osie
helm --namespace osie upgrade osie osie/osie -f values.yaml
kubectl -n osie logs -f osie-api-0

Changes to values.yaml are applied the same way. Take a backup first.

The databases are upgraded independently of OSIE: change crVersion and image in the MongoDB manifest, or the operator version itself, and re-apply. The operators perform the rolling restart.

Next steps​