Posts

The Hidden Cost of Over-Centralized Incident Response: When Stability and Reliability Are Not the Same Thing

-
There can be appeal to a heavily centralized incident response approach. A dedicated team, highly trained in triage and response. A team accountable for resolving most incidents. System owners' involvement is minimal. The central team handles detection, triage, mitigation, and recovery. Metrics improve. Time to detect drops. Recovery speeds accelerate. For months, maybe years, everything feels stable. But here's the thing — stability isn't the same as reliability. A few months ago I wrote about a different road to this same failure — when AI absorbs too much of the operational thinking, and engineers slowly lose the muscle that hard incidents demand. This post is about the same atrophy, but the cause here isn't AI. It's organizational. It's what happens when incident response gets so centralized that the people who built the systems are removed from the loop entirely. A system can appear stable under routine conditions while being fundamentally fragile when thin...
Post a Comment
Read more

AI, Toil, and the SRE Feedback Loops We Can’t Afford to Break

-
There’s a lot of energy right now around AI in incident management. Automating toil. Improving signal-to-noise. Self-healing systems. Agents that detect deviations, mitigate issues, and even resolve incidents before humans wake up. And honestly, I’m excited about it. There are real opportunities here to improve detection, triage, operational efficiency, and recovery speed. AI has the potential to meaningfully elevate how we run distributed platforms at scale. It’s also entirely possible that AI will transform the SDLC so profoundly that many of today’s assumptions will evolve. But in the world we still operate in today, there are a few important principles we need to keep top of mind as we adopt these capabilities. Feedback Loops Are How Systems and Engineers Learn If you go back to the DevOps movement, especially The Phoenix Project and the Three Ways, the second Way emphasizes fast, tight feedback loops. Engineers need to see the consequences of the systems they design. ...
2 comments
Read more

Leading Through Change: Lessons on People, Systems, and Growth

-
If there’s one constant in technology, it’s  change . New architectures, new priorities, new expectations — the landscape never stops evolving. Over the years, I’ve learned that leading through change isn’t about control; it’s about  clarity, accountability, and empathy . It’s about guiding people through change, staying anchored to purpose, and adapting while keeping a long-term vision in focus. This post is a reflection on the principles that have shaped how I lead and think about change — in systems, in teams, and in myself. ⸻ 1. Start with the End in Mind Stephen Covey’s  The 7 Habits of Highly Effective People  begins with one of my favorite ideas: “Begin with the end in mind.” Every successful transformation I’ve been part of starts there — by defining what success actually looks like. What will be different when we’ve achieved it? That clarity keeps everyone aligned and helps avoid confusing activity with progress. And once you have clarity on the end goal, t...
Post a Comment
Read more