Mauricio's Tech Space

There can be appeal to a heavily centralized incident response approach. A dedicated team, highly trained in triage and response. A team accountable for resolving most incidents. System owners' involvement is minimal. The central team handles detection, triage, mitigation, and recovery. Metrics improve. Time to detect drops. Recovery speeds accelerate. For months, maybe years, everything feels stable. But here's the thing — stability isn't the same as reliability. A few months ago I wrote about a different road to this same failure — when AI absorbs too much of the operational thinking, and engineers slowly lose the muscle that hard incidents demand. This post is about the same atrophy, but the cause here isn't AI. It's organizational. It's what happens when incident response gets so centralized that the people who built the systems are removed from the loop entirely. A system can appear stable under routine conditions while being fundamentally fragile when thin...

There’s a lot of energy right now around AI in incident management. Automating toil. Improving signal-to-noise. Self-healing systems. Agents that detect deviations, mitigate issues, and even resolve incidents before humans wake up. And honestly, I’m excited about it. There are real opportunities here to improve detection, triage, operational efficiency, and recovery speed. AI has the potential to meaningfully elevate how we run distributed platforms at scale. It’s also entirely possible that AI will transform the SDLC so profoundly that many of today’s assumptions will evolve. But in the world we still operate in today, there are a few important principles we need to keep top of mind as we adopt these capabilities. Feedback Loops Are How Systems and Engineers Learn If you go back to the DevOps movement, especially The Phoenix Project and the Three Ways, the second Way emphasizes fast, tight feedback loops. Engineers need to see the consequences of the systems they design. ...