Cost War-Room Optimization Strategy

At $20M+/year spend, cost governance becomes a continuous process.

A “Cost War Room” is:

A standing cross-functional team monitoring compute economics weekly.

? War-Room Dashboard Metrics

Track weekly:

• Slot utilization %

• Shuffle bytes per TB scanned

• Cost per dashboard query

• Cost per ML training run

• Cost per 1M events processed

• Region egress charges

• Slot contention %

? War-Room Playbook

Step 1: Identify Top 5 Most Expensive Queries

Often:

• One ML feature job

• One poorly designed BI dashboard

• One massive DISTINCT

Fixing 3 queries can save millions.

Step 2: Enforce Guardrails

• Max slots per query

• Query time caps

• Raw dataset access restrictions

• Forced partition filters

Step 3: Rewrite Heavy Workloads

Common wins:

• Replace COUNT(DISTINCT) with HLL

• Break 1 huge query into 3 staged ones

• Pre-aggregate at ingestion

• Convert snowflake schema → wide tables

? Cultural Insight

Cost optimization is not technical.

It’s behavioral.

War-room exists to drive engineering discipline.