Network flows and ports
osie-api opens almost every connection itself. It connects to MongoDB and RabbitMQ, to the public
endpoints of each OpenStack cloud, and to the third-party services you enable. Users reach OSIE on one
hostname over HTTPS. Their browsers connect to OpenStack directly in three cases: the server console,
single sign-on into Keystone, and temporary object storage links. On the OpenStack side, only the
notifier and a federated Keystone open connections to OSIE.
OSIE has no setting for an outbound HTTP proxy. Allow the destinations of osie-api at your firewall.
Overview
Each arrow starts at the side that opens the connection. Dashed arrows exist only with Keystone federation.
Connection matrix
Every installation needs these connections:
| From | To | Port | Protocol | Used for |
|---|---|---|---|---|
| Users' browsers, API clients | The OSIE hostname | 443 | HTTPS | Everything users and integrations do in OSIE |
| Gateway | osie-api, osie-ui, osie-admin | 8080, 80, 80 | HTTP | Routing inside the deployment |
osie-api | MongoDB | 27017 | MongoDB, optionally TLS | All persistent state |
osie-api | RabbitMQ | 5672 | AMQP, optionally TLS | Scheduled work and live updates |
osie-api | The Keystone URL of each cloud, and the public endpoints in its catalog | The catalog's ports | HTTPS | Everything OSIE does in OpenStack |
osie-api | c1.osie.io, then c2.osie.io | 443 | HTTPS | Licence check, at start and every hour |
| Users' browsers | Nova console proxy | 6080 for noVNC by default | HTTPS, WebSocket | Server console |
An air-gapped installation uses a signed licence key. osie-api verifies it locally and makes no licence
check.
These connections exist only when the feature that needs them is in use:
| From | To | Port | Protocol | When |
|---|---|---|---|---|
| osie-notifier | OpenStack's RabbitMQ | 5672 | AMQP | OpenStack notifications are installed |
| osie-notifier | The OSIE hostname | 443 | HTTPS | The same |
| Keystone | The OSIE hostname | 443 | HTTPS | Keystone federation |
| Users' browsers | Keystone public endpoint | 5000 by default | HTTPS | Keystone federation |
| Users' browsers | The cloud's S3 endpoint | The endpoint's port | HTTPS | Users open temporary object links |
osie-api | A region's Prometheus | 9090 by default | HTTP or HTTPS | Instance metrics come from Prometheus |
osie-api | Your mail server | 25 or 587, as configured | SMTP, optionally STARTTLS | Email is sent over SMTP |
osie-api | Third-party services | 443 for most | HTTPS | Each integration you enable, see Third-party services |
| ACME certificate authority | The OSIE hostname | 80, 443 | HTTP, TLS | The gateway obtains a Let's Encrypt certificate |
Inside the deployment
| Component | Listens on | Reached by |
|---|---|---|
| Gateway: Caddy with the Docker Compose installer, your ingress controller or Gateway on Kubernetes | 443, and 80 for redirects and certificate validation | Users, API clients, the notifier, a federated Keystone |
osie-api | 8080, HTTP | The gateway |
osie-api management | 8081, HTTP | Kubernetes probes and your Prometheus, which reads /actuator/prometheus |
osie-ui, the client portal | 80, HTTP | The gateway |
osie-admin, the admin dashboard | 80, HTTP | The gateway |
| MongoDB | 27017 | osie-api |
| RabbitMQ | 5672, AMQP | osie-api |
TLS ends at the gateway, which reaches the containers over plain HTTP. osie-ui and osie-admin
serve static files only. The client portal and the admin dashboard call osie-api from the user's
browser, through the OSIE hostname, so neither container connects to osie-api inside the deployment.
Keep port 8081 inside the deployment. Its endpoints require no authentication, so never route it through the gateway or publish it. The Helm chart exposes only port 8080 through its Service.
With the Docker Compose installer, the gateway publishes 80 and 443 on the host and nothing else is
published. On Kubernetes, the Helm chart can reach an external MongoDB or RabbitMQ over TLS:
set externalMongodb.tls.enabled or externalRabbitmq.tls.enabled, and give the CA in caSecret or
caConfigMap. RabbitMQ over TLS usually listens on 5671, so set externalRabbitmq.port as well.
Paths on the public hostname
The gateway routes by path on the one hostname:
| Path | Served by |
|---|---|
/api-v1/, /admin_api/, /oauth2/, /realms/, /.well-known/, /v3/api-docs | osie-api |
/osie_admin/ | osie-admin |
| Every other path | osie-ui |
A reverse proxy or web application firewall in front of OSIE must pass these paths, and two kinds of
request on /api-v1/:
-
Image uploads of up to 10 GB, sent as one request body. ingress-nginx accepts 1 MB by default. Raise the limit, and stream the body to
osie-apiwithout buffering it on the controller's disk. Add these annotations next to the ones you already set, such as the cert-manager issuer:values.yamlglobal:
ingress:
annotations:
nginx.ingress.kubernetes.io/proxy-body-size: "11g"
nginx.ingress.kubernetes.io/proxy-request-buffering: "off" -
Live updates at
/api-v1/streaming, a server-sent events response that stays open while the portal is open.osie-apiwrites a heartbeat every 5 seconds, so any idle timeout longer than 5 seconds keeps it open.
OpenStack endpoints
osie-api reaches each cloud through the Keystone URL registered on the cloud provider. For every
other service, it reads the Keystone catalog and uses the public endpoint in the region of the
resource. It uses no internal or admin endpoints, so osie-api needs a route to the public endpoint of
every service it uses, in every region. The ports are the ones your catalog lists:
openstack endpoint list --interface public
Two kinds of credentials cross these connections. Administrative work, such as creating projects and users, granting roles, setting quotas and polling resources, uses the cloud provider's administrator credentials: a password or an application credential. Work inside a project uses a token for that user, scoped to that project. See Identity and access model.
| Service | Catalog type | Used for | Upstream default port |
|---|---|---|---|
| Keystone | identity | Tokens, projects, users, role assignments, application and EC2 credentials | 5000 |
| Nova | compute | Servers, bare metal servers, server snapshots, consoles, server groups, quotas | 8774 |
| Neutron | network | Networks, subnets, routers, ports, floating IPs, security groups, QoS, VPN | 9696 |
| Cinder | volumev3 or block-storage | Volumes, volume snapshots, volume backups and restores | 8776 |
| Glance | image | Images, image upload and download | 9292 |
| Swift or Ceph RADOS Gateway | object-store | Buckets and objects | 8080 |
| Octavia | load-balancer | Load balancers | 9876 |
| Designate | dns | DNS zones and records | 9001 |
| Heat | orchestration | Stacks | 8004 |
| Magnum | container-infra | Kubernetes clusters and their kubeconfig certificates | 9511 |
| Manila | sharev2 | File shares | 8786 |
| Barbican | key-manager | Secrets | 9311 |
| Trove | database | Databases | 8779 |
| Gnocchi | metric | Instance metrics, when Gnocchi is the source | 8041 |
| Trilio | workloads | Trilio backups | 8780 |
| Placement | placement | Quotas on Virtuozzo Hybrid Infrastructure | 8778 |
Bare metal servers are managed through Nova, so osie-api does not call the Ironic API. Two destinations
come from the cloud provider's settings instead of the catalog: the Prometheus URL of a region, when
it is the metrics source, and on Virtuozzo Hybrid Infrastructure the S3 endpoint and the object storage
administration endpoint (Configure VHI S3).
Apart from the notifier, nothing OSIE runs needs a route to OpenStack's database, message bus, hypervisors or storage network. Users who drive OpenStack with the OpenStack CLI, using credentials issued in the client portal, connect to the same public endpoints from their own machines.
Flows by feature
Sign-in and API calls
Users sign in to the OSIE hostname, at the realm's login page or through the realm's identity provider.
With an external identity provider, the browser is redirected to it, and osie-api then calls the
provider directly:
Without an external identity provider, the realm's own login page replaces steps 2 to 5. Every call
the client portal and the admin dashboard make then goes to /api-v1 on the same hostname. osie-api
checks the user's membership and calls OpenStack with a token for that user, scoped to the project.
Integrations call the Admin API at /admin_api/v1, with an HMAC-signed request or an OAuth2 token
from a trusted issuer (Authentication). The first time osie-api
receives a token from a trusted issuer, it reads that issuer's discovery document and keys, so
osie-api needs HTTPS to every issuer you trust.
Image upload and download
Image files pass through the gateway and osie-api, which stream them to Glance without storing them.
The browser never connects to Glance.
A download follows the same path in the other direction. The browser opens a single-use link under
/api-v1/download/, and osie-api streams the file from Glance's /v2/images/ID/file. OSIE does not
import images from a URL, so Glance opens no outbound connection on its behalf. Allow for the upload
size at your gateway, as described in
Paths on the public hostname.
Volumes, snapshots and backups
osie-api sends requests to Cinder and Nova, and the data moves inside OpenStack. No volume, snapshot or
backup data passes through OSIE.
- A volume created from an image is copied by Cinder from Glance.
- A server snapshot is written by Nova to Glance as an image.
- A volume backup is written by the Cinder backup service to its own backend, and restored from there.
Backup products connect as follows:
| Product | Connection | Notes |
|---|---|---|
| Cinder backups | osie-api to Cinder's public endpoint | Volume backups and restores |
| Trilio | osie-api to the workloads endpoint in the catalog | Uses the user's project-scoped token. No separate URL or credential |
| Storware Backup and Recovery | osie-api to the Storware server URL you register in the admin dashboard, on that URL's port | osie-api signs in with the username and password you register. The client portal's backup screens go through osie-api, so browsers never connect to Storware |
| Ferry | None from OSIE | Runs inside the cloud. See its network connections |
Storware's own connections, to OpenStack, to the hypervisors or storage and to its backup destinations, are described in Storware's documentation.
Object storage
Uploads and downloads in the client portal pass through osie-api, as images do. osie-api connects to
Swift through the catalog, or to the S3 endpoint on Virtuozzo Hybrid Infrastructure.
A temporary link is a presigned S3 URL that points at the cloud's S3 endpoint. Whoever opens it connects to that endpoint directly, so it must be reachable from where the link is used.
Server console
The console connects the browser to Nova's console proxy. osie-api only asks Nova for the URL.
The URL is the one Nova is configured to hand out, such as novncproxy_base_url for noVNC. It must
resolve and be reachable from users' networks. The console type is set per cloud provider: noVNC by
default, SPICE, serial or WebMKS.
OpenStack notifications
The notifier is a separate container that runs near OpenStack's RabbitMQ. It turns OpenStack's notifications into HTTPS requests to OSIE, so changes made outside the portal appear at once.
osie-api never connects to OpenStack's RabbitMQ, and OpenStack never connects to OSIE's. Without the
notifier, osie-api picks up changes made outside the portal when it polls each project, every 15 minutes.
See OpenStack notifications to install it.
Instance metrics
osie-api reads instance metrics per region from Gnocchi, through the catalog's metric endpoint, or
from the Prometheus URL set for the region. It polls every 15 minutes, and again when a user opens a
server's charts. See Instance metrics.
Keystone federation
With Keystone federation, a user's browser signs in to Keystone through OSIE, and Keystone calls OSIE's realm as an OpenID Connect provider:
Keystone therefore needs outbound HTTPS to the OSIE hostname, and users' browsers need Keystone's public endpoint. See Keystone federation for the Keystone-side configuration.
Third-party services
Each integration is optional, configured in the admin dashboard, and connects only once it is enabled.
Unless the table says otherwise, every connection is HTTPS on port 443, opened by osie-api.
| Integration | osie-api connects to | Users' browsers connect to | Inbound to OSIE |
|---|---|---|---|
| Email over SMTP | Your mail server, on the port you set, in plain SMTP or with STARTTLS. Implicit TLS on port 465 is not supported | ||
| Email over Mailgun | api.mailgun.net or api.eu.mailgun.net | ||
| OpenID Connect identity providers | The provider's discovery, token, keys and user info endpoints | The provider's login page | |
| Google, Facebook, GitHub sign-in | The provider's OAuth endpoints | The provider's login page | |
| Stripe | api.stripe.com | js.stripe.com | |
| PayPal | PayPal's REST API | www.paypal.com | |
| Razorpay | api.razorpay.com | api.razorpay.com | |
| Xendit | api.xendit.co | js.xendit.co, Xendit's invoice page | |
| HyperPay | eu-prod.oppwa.com | eu-prod.oppwa.com | |
| iPay (Banca Transilvania) | ecclients.btrl.ro | ecclients.btrl.ro | |
| BTCPay Server | Your BTCPay server | Your BTCPay server | |
| SmartBill | ws.smartbill.ro, port 8183 | ||
| WHMCS | Your WHMCS includes/api.php, and its OAuth endpoints when WHMCS is a sign-in provider | Your WHMCS invoice and login pages | |
| iDenfy | ivs.idenfy.com | iDenfy's verification page | Webhook under /api-v1/webhooks/ |
| Digio | api.digio.in | Digio's SDK | Webhook under /api-v1/digio/webhooks/ |
| Crisp, Intercom, Chatwoot, SupportPal | api.crisp.chat, api.intercom.io or its regional host, or your Chatwoot or SupportPal server | The chat widget from the provider | |
| Severalnines CCX | Your CCX URL | Your CCX URL, embedded in the portal | |
| Billing Resource Exporter | Each exporter's URL, with SigV4-signed requests (API) | ||
| Webhooks | Each receiver URL (Webhooks) | ||
| Billing profile validation | ipinfo.io | ||
| Error reporting | The Sentry DSN set in the Helm value sentry.dsn |
Payment gateways do not call OSIE. After a payment, the customer's browser returns to OSIE under
/api-v1/callbacks/payment/ or /api-v1/payments/. Bank transfers make no connection.
osie-api also downloads the logo set in the branding settings when it renders invoices, from the URL
you give. Webhook receivers must use HTTPS, and osie-api refuses private addresses unless you allow
them (Sending to an internal address).
Users' browsers load fonts, icons and scripts from fonts.googleapis.com, fonts.gstatic.com,
use.fontawesome.com, api.fontshare.com and cdn.jsdelivr.net, and avatars from gravatar.com in
the admin dashboard. When a user adds a security group rule for their own address, the client portal
reads that address from api.ipify.org.
If an identity provider or another service osie-api calls presents a certificate from a private
certificate authority, add that authority as described in
Custom CA certificates.
Installation and upgrades
Installing and upgrading pull from these sources. No registry credentials are needed.
| Source | Contents |
|---|---|
ghcr.io | OSIE's images: osie/osie-api, osie/osie-ui, osie/osie-admin and osie-io/osie-notifier |
Docker Hub (registry-1.docker.io) | MongoDB, RabbitMQ and Caddy for Docker Compose, the operators' images, and the images of the databases the Helm chart can deploy |
helm.osie.io | The OSIE Helm chart |
percona.github.io, github.com | The charts and manifests of the MongoDB and RabbitMQ operators |
github.com, raw.githubusercontent.com, download.docker.com, your OS package mirrors | The Docker Compose installer and Docker itself |
With a domain name, the Docker Compose installer's Caddy obtains a certificate from Let's Encrypt. It needs outbound HTTPS to the certificate authority, and inbound 80 and 443 from the internet for validation. With an IP address, Caddy issues its own certificate and needs neither. On Kubernetes, cert-manager has the same needs when it validates over HTTP.
Database backups need outbound access to their object storage: the S3 endpoint of the Percona operator's backups, and Velero's bucket (Backup).