Nessie and Trino on in-cluster MySQL

Iceberg is tables on object storage. A query engine cannot find those tables by listing S3: it needs a catalog that names each table and points at the current snapshot. On an EKS cluster that already existed, we stood up that first query path: Nessie as the catalog, one Trino, in-cluster MySQL on a PVC, and a single S3 iceberg/ prefix. This post is that path. Not Spark, not Argo, not a second Trino, not Aurora, and not the outages that came later.

A catalog, an engine, a warehouse

Four pieces, and they are not the same thing.

• Warehouse. Parquet files and Iceberg metadata.json under one S3 prefix. This is the data.
• Catalog. Nessie (Project Nessie). Git-like branches over that warehouse, REST to the engine. The catalog names tables. It does not store rows.
• Engine. One Trino cluster. It asks Nessie where the table is, then reads S3.
• Version store. Nessie's JDBC2 backend on in-cluster MySQL with a PersistentVolumeClaim (PVC): tables refs2 and objs2. Nessie's own refs and objects. Not the warehouse.

AWS objects (the bucket, IRSA service accounts) live in CDK. Pods (Nessie, Trino) live in Helmfile. The EKS cluster was created by hand. CDK imported it by name and never took over the control plane.

Figure 1. First query path: one Trino, Nessie, in-cluster MySQL, S3 iceberg/ on imported EKS.

Figure 1 is that first path, in layers. Grey dashed is the cluster we imported by name. Sky dashed is CDK: the warehouse bucket and IRSA, AWS objects, not pods. Solid blue is Helmfile: Nessie and one Trino, in data-lakehouse. Orange is data: S3 iceberg/ and MySQL on the PVC. Spark, a second Trino, Argo, Aurora, DataHub, and Superset are not on this map. They were not there yet.

Why CDK does not install Helm

CDK imported the cluster and attached the AWS objects the lakehouse needed. It did not install Nessie or Trino. If a change is an IAM policy or an S3 lifecycle rule, it is CDK. If a change is a JVM heap or a catalog URI, it is Helmfile. Mixing the two (installing Helm releases from CDK, or creating IAM roles by hand to match a values file) is how this architecture rots.

Import by name, apply with Helmfile

prod-eks is the existing cluster name. vpc is the VPC that cluster already sat in. The kubectlRole is mapped into aws-auth by hand; CDK cannot do that mapping for a cluster it did not create. Project stacks use the role to create service accounts.

import * as eks from "aws-cdk-lib/aws-eks";
import * as iam from "aws-cdk-lib/aws-iam";
 
const kubectlRole = new iam.Role(this, "KubectlRole", {
  roleName: "kubectlRole",
  assumedBy: new iam.AccountRootPrincipal(),
});
 
const cluster = eks.Cluster.fromClusterAttributes(this, "ExistingCluster", {
  clusterName: "prod-eks",
  vpc,
  kubectlRoleArn: kubectlRole.roleArn,
});

Helmfile owned the pods. One chart directory per component, dev.yaml / prod.yaml, applied with helmfile -e prod apply. Charts do not create namespaces, VPCs, or buckets. We created data-lakehouse ourselves, then applied Nessie from its chart directory.

aws eks update-kubeconfig --name prod-eks --region "$AWS_REGION"
 
kubectl create namespace data-lakehouse
 
cd charts/data-lakehouse/nessie
helmfile -e prod apply

$AWS_REGION is the region of prod-eks. Trino was the second chart, same pattern: in-tree chart, env overlay, helmfile -e prod apply. createNamespace: false is deliberate. The namespace is a cluster convention, not a side effect of the first chart you happen to apply.

environments:
  prod:
    values:
      - prod.yaml
  dev:
    values:
      - dev.yaml
---
releases:
  - name: nessie
    namespace: data-lakehouse
    chart: ./nessie_chart
    values:
      - "{{ .Environment.Name }}.yaml"
    createNamespace: false

JDBC2 on MySQL

Iceberg needs a catalog. We used Nessie: git-like branches over the same warehouse, REST to the engine. Branching is real; we mostly stay on main and keep a staging ref for experiments.

Nessie itself needs a version store. The first deploy used JDBC2 against in-cluster MySQL with a PVC. It worked. It also had no TLS, no HA, and no backup story. SSL was off on that MySQL.

Trino reached Nessie over in-cluster HTTP. Every table load goes through that hop. If the PVC-backed store is down, there is no catalog and there is no query.

One warehouse prefix

One bucket per environment: {org}-data-lakehouse-{env}. Versioned. RemovalPolicy.RETAIN. Iceberg tables live under a single prefix, iceberg/. Other objects in the same bucket stay off that prefix.

s3://{org}-data-lakehouse-{env}/
  iceberg/          # table data + metadata.json

{org} and {env} are the bucket name parts. Trino talks to the warehouse as s3://. Mixing prefixes is how you spend a day staring at empty schemas.

IAM is IRSA (IAM Roles for Service Accounts) for in-cluster engines, not long-lived keys on the coordinator. CDK creates the service accounts in data-lakehouse and attaches object and list permissions on that one bucket. Helm must use the same account names. A values-file typo looks like a random AccessDenied.

The Trino catalog pointed at Nessie and that prefix:

catalog:
  lakehouse: |
    connector.name=iceberg
    iceberg.catalog.type=nessie
    iceberg.nessie-catalog.uri=http://nessie.data-lakehouse.svc.cluster.local:19120/api/v2
    iceberg.nessie-catalog.default-warehouse-dir=s3://{org}-data-lakehouse-{env}/iceberg
    iceberg.nessie-catalog.ref=main
    iceberg.allowed-extra-properties=*

iceberg.allowed-extra-properties=* lets you ALTER TABLE … SET PROPERTIES without a redeploy. Prefer an allowlist over * once you know which properties you actually set.

When this applies

Use this if you already have an EKS cluster, you will not let CDK install Helm, and you need a catalog plus one query engine on one warehouse prefix before you have a writer. Skip it if the catalog store has to survive a node loss on day one, or if JDBC must use TLS from the start: in-cluster MySQL on a PVC gives you neither.

The production limit of this era is the store: no TLS, no replica, no backup. The next piece is standing up a writer: Argo Workflows and Spark as the writer.