AI Alert Triage: What It Actually Helps With (and What It Doesn't)
AI-assisted triage tools promise faster incident response. Here's a grounded look at where they help, where they fall short, and how to evaluate one.
Product updates, on-call war stories, and what we've learned building alert routing from scratch.
49 posts
AI-assisted triage tools promise faster incident response. Here's a grounded look at where they help, where they fall short, and how to evaluate one.
How to start injecting failure into your systems on purpose, without taking down production on day one.
MTTR alone won't reveal alert noise, responder overload, or repeat incidents. Here are the metrics that actually tell you whether your on-call program is healthy.
A practical guide to managing incidents caused by AWS, Stripe, or any external dependency your team doesn't control.
Stop paging your team five times for the same root cause. Here's how alert correlation works and how to implement it.
Most teams discover their alerts are broken during real incidents. Here's how to validate your full alerting pipeline before that happens.
How distributed tracing helps you find root cause faster during incidents, and how to build it into your on-call workflow.
Database outages are high-stakes and high-pressure. Here's how to investigate and stabilize without making things worse.
A practical guide to diagnosing and resolving incidents in Kubernetes, from pod crashes to node pressure and networking failures.