We'd Better Build Some Damn Good Brakes

Waymo is objectively better than an average human driver.

Data shows it reduces injury-causing crashes by over 80%. Yet, if an autonomous vehicle makes a single mistake, it's front-page news. We demand near-perfection from machines while accepting massive error rates from humans. And rightfully so.

We should apply that exact same zero-tolerance policy to "AI Twins" in the professional world.

Lately, I've been building a "Data Scientist twin" to handle chat requests on routine experimentation analysis in my domain. In testing, it's brilliant, it's fast, it never complains about an ad-hoc request.

And yet, I haven't deployed it widely.

Why? Because when I make a mistake, I understand the organizational fallout. I know when a quick estimate is "good enough," and when the stakes are high, I double-check the code, get it peer-reviewed and am on my toes till a decision is made and the consequences are known.

This is exactly why my DS twin, despite working well in tests, stays in its sandbox. I'm not convinced you can prompt a 'spidey sense' into the skill files for spotting a bad result. And even if you could inject enough context to get close, an AI isn't on the line when a decision goes sideways. Maybe it's the Google in me, but I will always lean toward "be right and useful" over "move fast and hallucinate." Being an AI-First Data Scientist means recognizing that the higher an AI's capability, the more rigorous our safety guardrails must be.

I'd love an AI Gilfoyle in my life. But until we can code true accountability into our AI doubles, I'm keeping mine in a heavily audited sandbox.

If we want AI to drive our workflows, we'd better build some damn good brakes.

Originally posted on LinkedIn.