materialized='table' rewrites metadata.json with the full snapshot list. The coordinator parses that list into heap. File version in the 700s; JVM 2 GiB then 8 GiB. Expiry runs on Spark, not on Trino. Nessie GC is the wrong tool.
6 min read0 views0 likes
Iceberg needs a catalog; in-cluster MySQL had no TLS, no HA, and no backup. RocksDB is single-node, DocumentDB is not Mongo; we kept JDBC2, required TLS, and left the PVC until Aurora was trusted.
5 min read0 views0 likes
One coordinator for dashboards and dbt full-refresh is a bad idea. trino-main is humans; trino-etl is batch with task retry. dbt stays a container in an Argo workflow, not a Helm release.
5 min read1 view0 likes
Argo Workflows was the data scheduler before Spark. The first job plane is Workflows + a standing Spark cluster: IRSA on the cluster, static keys on submit from `argo`, not the Spark Operator, not Argo CD, not Events yet.
6 min read2 views0 likes
First query path on an EKS cluster that already existed: Nessie + one Trino, JDBC2 on a PVC, CDK for the bucket, Helmfile for the two charts. It worked. No TLS, no HA, no Spark, no Argo jobs, no dbt.
5 min read0 views0 likes