At ~1,400 Workflows, EventSources at 512 Mi were OOMKilled; at ~2,000, with nodeStatusOffLoad false, the workflow controller at 2 GiB joined them. Label filters, CronWorkflow history limits, offload, and archive bounded the cache, not a message bus.
5 min read1 view0 likes
Nessie was medium-priority (600000); Trino coordinators were high-priority (1000000). The scheduler killed Nessie to place them. PDB is best-effort. Stock Nessie 0.103.3 could not set a PriorityClass. That is why the chart patch exists.
6 min read0 views0 likes
Events exists so raw ingestion is not the same graph as semantic-layer dbt. Sensors on completion; producers stay a CronWorkflow, a label, a Sensor. The informer-cache OOM is a later story.
6 min read0 views0 likes
Argo Workflows was the data scheduler before Spark. The first job plane is Workflows + a standing Spark cluster: IRSA on the cluster, static keys on submit from `argo`, not the Spark Operator, not Argo CD, not Events yet.
6 min read2 views0 likes
First query path on an EKS cluster that already existed: Nessie + one Trino, JDBC2 on a PVC, CDK for the bucket, Helmfile for the two charts. It worked. No TLS, no HA, no Spark, no Argo jobs, no dbt.
5 min read0 views0 likes