Quick Start

Get your AetherLake instance up and running locally in minutes.

Prerequisites

• Kubernetes cluster (v1.26+) — local: Docker Desktop, minikube, or kind
• Helm v3.12+
• kubectl
• Docker — the installer builds the Flink SQL runner image locally
• An NGINX ingress controller — the installer installs one automatically if missing

Step-by-Step Installation

1. Clone the repository

git clone https://github.com/mrtozkl/AetherLake.git
cd AetherLake

2. Run the installer

The installation script automates the whole deployment:

./install.sh

What it does, in order:

1. Creates the aetherlake namespace and generates random credentials into the aetherlake-credentials / open-lake-credentials secrets (re-runs keep existing values and backfill only new keys).
2. Deploys the security-stack (Keycloak + its own PostgreSQL) and waits for it.
3. Adds a CoreDNS rewrite so keycloak.aetherlake.local resolves inside the cluster (required for server-side OIDC discovery).
4. Installs missing prerequisites: MinIO Operator, cert-manager + the self-signed AetherLake CA, and ingress-nginx.
5. Deploys the core-data-stack — MinIO, Trino, Polaris, Spark, Airflow, Superset, Kafka (Strimzi), Flink (operator), Milvus, oauth2-proxy and the shared PostgreSQL/Redis.
6. Builds the Flink SQL runner image (aetherlake/flink-sql-runner:flink-2.1) used by SQL jobs submitted from the Control Panel.
7. Applies the *.aetherlake.local ingress rules (including the Keycloak SSO gate for the Trino UI and Milvus Attu).

3. Configure local DNS

Add the following to your /etc/hosts (or use a local DNS resolver):

127.0.0.1  minio.aetherlake.local
127.0.0.1  trino.aetherlake.local
127.0.0.1  polaris.aetherlake.local
127.0.0.1  keycloak.aetherlake.local
127.0.0.1  airflow.aetherlake.local
127.0.0.1  milvus.aetherlake.local
127.0.0.1  superset.aetherlake.local
127.0.0.1  oauth2.aetherlake.local

oauth2.aetherlake.local hosts the SSO gate's login/callback endpoint; it must resolve for the gated UIs below.

4. Access the platform

Service	URL	Auth
Control Panel	http://localhost:3000	dev login admin/admin (local dev only)
Trino web UI	http://trino.aetherlake.local	Keycloak SSO (oauth2-proxy gate)
Milvus (Attu)	http://milvus.aetherlake.local	Keycloak SSO (oauth2-proxy gate)
MinIO Console	http://minio.aetherlake.local	Keycloak OIDC
Airflow	http://airflow.aetherlake.local	Keycloak OIDC
Superset	http://superset.aetherlake.local	Keycloak OIDC
Polaris	http://polaris.aetherlake.local	API (OAuth2 client credentials)
Keycloak	http://keycloak.aetherlake.local	admin console password

Credentials: all service passwords are randomly generated by install.sh and stored in the aetherlake-credentials secret. The SSO login is admin with the realm-admin-password key (a password change is forced on first login):

kubectl get secret aetherlake-credentials -n aetherlake \
  -o jsonpath='{.data.realm-admin-password}' | base64 -d

The Superset admin password is under superset-admin-password, the Keycloak admin-console password under keycloak-admin-password. The MCP server's Trino service user (mcp:<password>) is under trino-mcp-password — see MCP Server.

Two demo users ship with the realm to try per-role access in the SQL IDE (password for both: aetherlake-demo):

• elif — data-engineer (read-write, no system catalog)
• deniz — data-scientist (read-only)

Start the Control Panel locally with:

cd control-panel && npm install && npm run dev   # → http://localhost:3000

5. (Optional) TLS without browser warnings

Every host is also served over HTTPS with certificates from a self-signed AetherLake CA (cert-manager). Plain HTTP stays on in parallel because the SSO issuer URLs are http://. To trust the CA locally:

kubectl get secret aetherlake-root-ca -n cert-manager \
  -o jsonpath='{.data.ca\.crt}' | base64 -d > aetherlake-ca.crt
# macOS:
security add-trusted-cert -d -r trustRoot \
  -k ~/Library/Keychains/login.keychain-db aetherlake-ca.crt

6. (Optional) Enable resource metrics

The Control Panel's Observability page shows per-pod CPU/RAM when metrics-server is installed. On Docker Desktop install it with --kubelet-insecure-tls:

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
kubectl patch deployment metrics-server -n kube-system --type=json \
  -p='[{"op":"add","path":"/spec/template/spec/containers/0/args/-","value":"--kubelet-insecure-tls"}]'

Next steps

• Kafka — Streaming — topics, the Trino kafka catalog, and producing from outside the cluster (TLS + SCRAM).
• Flink — Stream Processing — submit SQL jobs from the Control Panel.
• Data Pipelines — ready-to-run streaming jobs including the Kafka-to-Iceberg lakehouse bridge (kafka-to-iceberg.sql).
• Control Panel — the Kafka view, Flink SQL workspace, SQL IDE and observability.
• Architecture — how the pieces fit together.