Chapter 8.2: ROI Tracking Dashboard

“In God we trust; all others must bring data.” — W. Edwards Deming

The ROI dashboard is how you demonstrate MLOps value to executives and secure continued investment. This chapter provides templates and best practices for building an effective dashboard.

8.2.1. Dashboard Design Principles

Know Your Audience

Design Principles

8.2.2. The Executive Dashboard

One page that tells the MLOps story.

Template: Executive Summary Dashboard

┌─────────────────────────────────────────────────────────────────────┐
│               MLOPS PLATFORM - EXECUTIVE DASHBOARD                  │
│                        as of [Date]                                 │
├─────────────────────────────────────────────────────────────────────┤
│                                                                     │
│  ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐   │
│  │  ROI YTD    │ │ Time Saved  │ │ Models in   │ │ Incidents   │   │
│  │   $8.2M     │ │   12,500    │ │ Production  │ │  Avoided    │   │
│  │   ▲ 145%    │ │   hours     │ │     34      │ │     12      │   │
│  │ vs target   │ │   ▲ 2x      │ │   ▲ 25%     │ │  ▼ from 16  │   │
│  └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘   │
│                                                                     │
│  ┌───────────────────────────────────────────────────────────────┐ │
│  │                  12-MONTH ROI TREND                           │ │
│  │                                                               │ │
│  │   $10M ├─────────────────────────────────────────────*       │ │
│  │        │                                         *           │ │
│  │    $5M ├───────────────────────────────*                     │ │
│  │        │                           *                         │ │
│  │    $0  ├───────*───*───*───*───*                             │ │
│  │        └───┬───┬───┬───┬───┬───┬───┬───┬───┬───┬───┬───     │ │
│  │           J   F   M   A   M   J   J   A   S   O   N   D      │ │
│  └───────────────────────────────────────────────────────────────┘ │
│                                                                     │
│  KEY HIGHLIGHTS THIS QUARTER:                                       │
│  ✓ Deployment velocity improved 4x (6 months → 6 weeks)            │
│  ✓ Zero production incidents in last 90 days                       │
│  ✓ 85% of ML team actively using platform                          │
│                                                                     │
│  NEXT QUARTER PRIORITIES:                                          │
│  → Complete Feature Store rollout                                   │
│  → Add A/B testing capability                                       │
│  → Onboard remaining 3 teams                                        │
└─────────────────────────────────────────────────────────────────────┘

8.2.3. ROI Calculation Methodology

Value Categories

Monthly ROI Calculation Template

def calculate_monthly_roi(month_data: dict) -> dict:
    # Productivity savings
    hours_saved = month_data['hours_saved_model_dev'] + \
                  month_data['hours_saved_deployment'] + \
                  month_data['hours_saved_debugging']
    hourly_rate = 150  # Fully loaded cost
    productivity_value = hours_saved * hourly_rate
    
    # Incident avoidance
    incidents_prevented = month_data['baseline_incidents'] - \
                          month_data['actual_incidents']
    avg_incident_cost = 100_000
    incident_value = max(0, incidents_prevented) * avg_incident_cost
    
    # Revenue acceleration
    models_deployed_early = month_data['models_deployed']
    months_saved_per_model = month_data['avg_months_saved']
    monthly_model_value = 50_000
    acceleration_value = models_deployed_early * months_saved_per_model * monthly_model_value
    
    # Infrastructure savings
    infra_savings = month_data['baseline_cloud_cost'] - \
                    month_data['actual_cloud_cost']
    
    # Total
    total_value = productivity_value + incident_value + \
                  acceleration_value + max(0, infra_savings)
    
    return {
        'productivity_value': productivity_value,
        'incident_value': incident_value,
        'acceleration_value': acceleration_value,
        'infra_savings': max(0, infra_savings),
        'total_monthly_value': total_value,
        'investment': month_data['platform_cost'],
        'net_value': total_value - month_data['platform_cost'],
        'roi_percent': (total_value / month_data['platform_cost'] - 1) * 100
    }

8.2.4. Dashboard Metrics by Category

Financial Metrics

Velocity Metrics

Quality Metrics

Adoption Metrics

8.2.5. Visualization Best Practices

Choose the Right Chart

Color Coding

Layout Hierarchy

┌─────────────────────────────────────────────────────────────┐
│  1. TOP: Most important KPIs (ROI, key health)              │
├─────────────────────────────────────────────────────────────┤
│  2. MIDDLE: Trends and breakdowns                           │
├─────────────────────────────────────────────────────────────┤
│  3. BOTTOM: Supporting detail and drill-downs               │
└─────────────────────────────────────────────────────────────┘

8.2.6. Building in Grafana

Sample Grafana Dashboard JSON Snippet

{
  "panels": [
    {
      "title": "Monthly ROI ($)",
      "type": "stat",
      "datasource": "prometheus",
      "targets": [
        {
          "expr": "sum(mlops_roi_value_monthly)",
          "legendFormat": "ROI"
        }
      ],
      "options": {
        "graphMode": "area",
        "colorMode": "value",
        "textMode": "auto"
      },
      "fieldConfig": {
        "defaults": {
          "unit": "currencyUSD",
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {"color": "red", "value": 0},
              {"color": "yellow", "value": 100000},
              {"color": "green", "value": 500000}
            ]
          }
        }
      }
    },
    {
      "title": "Time-to-Production (days)",
      "type": "timeseries",
      "datasource": "prometheus",
      "targets": [
        {
          "expr": "avg(mlops_deployment_time_days)",
          "legendFormat": "Avg Days"
        }
      ]
    }
  ]
}

Key Metrics to Expose

Export these metrics from your MLOps platform:

from prometheus_client import Gauge, Counter

# Business metrics
roi_monthly = Gauge('mlops_roi_value_monthly', 'Monthly ROI in dollars')
models_in_production = Gauge('mlops_models_production', 'Models in production')

# Velocity metrics  
deployment_time = Gauge('mlops_deployment_time_days', 'Days to deploy model')
deployments_total = Counter('mlops_deployments_total', 'Total deployments')

# Quality metrics
model_accuracy = Gauge('mlops_model_accuracy', 'Model accuracy in production', ['model_name'])
incidents_total = Counter('mlops_incidents_total', 'Total production incidents')

# Adoption metrics
active_users = Gauge('mlops_active_users', 'Weekly active users')
platform_nps = Gauge('mlops_platform_nps', 'Platform NPS score')

8.2.7. Reporting Cadence

Monthly Executive Summary Template

# MLOps Platform - Monthly Report
## [Month Year]

### Executive Summary
[2-3 sentences on overall health and key developments]

### Financial Performance
| Metric | Target | Actual | Status |
|--------|--------|--------|--------|
| Monthly Value | $600K | $720K | ✅ |
| Cumulative ROI | $3M | $3.5M | ✅ |
| Platform Cost | $150K | $140K | ✅ |

### Key Metrics
- Time-to-Production: 18 days (target: 14) ⚠️
- Models in Production: 28 (up from 24)
- Platform Satisfaction: 4.2/5

### Highlights
- Completed Feature Store rollout to Marketing team
- Zero production incidents this month

### Concerns
- Deployment time slightly above target due to compliance queue
- Action: Streamlining approval process (ETA: end of month)

### Next Month Focus
- Scale A/B testing capability
- Onboard Finance team

8.2.8. Key Takeaways

1. Design for your audience: Executives need different views than operators.

2. Lead with outcomes: ROI and business value first.

3. Show trends, not just snapshots: Direction matters.

4. Automate data collection: Manual dashboards become stale.

5. Use consistent methodology: ROI must be repeatable and auditable.

6. Report at the right cadence: Too much is as bad as too little.

7. Connect to decisions: Dashboards should drive action.

Next: 8.3 Continuous Improvement — Using data to get better over time.