Components

The platform consists of several curated open-source projects. Each has a detailed reference page (settings, architecture, gotchas) under Component Reference in the sidebar.

Component	Role	Version	Reference
Keycloak	Identity & SSO (OIDC)	26.3.3	Keycloak — SSO
MinIO	S3-compatible object storage	RELEASE.2025-04-08	MinIO — Storage
Trino	Distributed SQL query engine	480 (chart 1.42.2)	Trino — Query
Apache Polaris	Iceberg REST catalog + vending	latest	Polaris — Catalog
Apache Airflow	Workflow orchestration	2.10.5 (chart 1.16.0)	Airflow — Orchestration
Apache Superset	BI & dashboards	3.1.2 (chart 0.12.8)	Superset — BI
Apache Spark	Distributed data processing	operator 1.1.27	Spark — Processing
Apache Kafka	Distributed streaming platform	4.3.0 (Strimzi 1.1.0)	Kafka — Streaming
Apache Flink	Stream processing (Flink SQL jobs)	2.1 (operator 1.15.0)	Flink — Stream Processing
Milvus	Vector similarity search	chart 5.0.14	Milvus — Vector DB
PostgreSQL	Metadata datastore (shared + Keycloak)	16-alpine	PostgreSQL — Datastores
dbt	SQL-based data transformation	—	Data Pipelines
Control Panel	Web UI for platform management	—	Control Panel
MCP Server	Agent tooling (K8s/Trino/Polaris/Airflow)	—	MCP Server

Helm chart layout

Chart	Components
security-stack	Keycloak + keycloak-postgres
core-data-stack	MinIO, Trino, Polaris, Spark, Airflow, Superset, Kafka (Strimzi), Flink (operator), Milvus, shared postgres/redis

Shared infrastructure

• aetherlake-postgres — one shared PostgreSQL hosting the airflow, superset and polaris databases. Keycloak deliberately keeps its ownkeycloak-postgres for security isolation — see PostgreSQL — Datastores for both instances and the rationale.
• aetherlake-redis — shared Redis (broker/cache for Airflow and Superset).
• aetherlake-credentials / open-lake-credentials — the Kubernetes Secret(s) holding all randomized credentials (see Configuration Guide).