AI policy is not enough once AI starts shaping decisions. Organizations need operating systems that make AI-assisted work visible, owned, and reviewable.
"Shadow decision-making" is the framing the governance conversation has been missing. Shadow AI as a tool problem is manageable you can audit tool usage. Shadow decision-making is structural the influence is invisible by design and doesn't show up in any dashboard until something expensive breaks.
The five diagnostic questions are the right unit of analysis. Moving from "was AI used?" to "what did AI change?" shifts governance from permission management to operational control. Those are completely different problems requiring completely different infrastructure.
I write about production AI systems and distributed backends the technical layer where these governance requirements have to actually be enforced at runtime. Worth a subscribe here too.
Thanks for your sharing your thoughts, Alireza. We want to believe that if we can’t see something it doesn’t exist. But decisions are all around us, and when we give AI more and more power it happens more and more often that it influence decision-making process or make decisions on our behalf without us even knowing and being able to trace them and understand the process and rationale behind it.
Really curious on your perspective on AI systems, from more technical perspective. Subscribed!
Very, very important perspectives here. I hope the right people pay attention.
"The failure may appear as a hidden assumption..." -- if the road to hell is paved with good intentions, the road to organizational calamity is going to be paved with hidden assumptions.
You *assume* because the LLM says "Of course" that it understands you and is going to carry out what you intended. Then these assumptions get built into the infrastructure.
My zeroth rule in editing is "Do no harm". My first rule is "Never, ever, assume". And I don't. When I was on the newsdesk, I had all the journalists' names in my spell check. The number of reporters who spelled their name wrong was quite scary. I always pointed this out to them.
Thanks Fred! I share the same rule, and I'm most strict at work. We tend to assume many things by ourselves at work, at home but still have more specific context than generic large language models. AI doesn't have that and it's much harder to spot it and make a recalibration.
Really well written! I echo what others have said - the concept of shadow decision making is something we're most likely already seeing, but not recognising. People passing off AI advice as their own without thinking through the second and third order consequences
"Shadow decision-making" is the framing the governance conversation has been missing. Shadow AI as a tool problem is manageable you can audit tool usage. Shadow decision-making is structural the influence is invisible by design and doesn't show up in any dashboard until something expensive breaks.
The five diagnostic questions are the right unit of analysis. Moving from "was AI used?" to "what did AI change?" shifts governance from permission management to operational control. Those are completely different problems requiring completely different infrastructure.
I write about production AI systems and distributed backends the technical layer where these governance requirements have to actually be enforced at runtime. Worth a subscribe here too.
Thanks for your sharing your thoughts, Alireza. We want to believe that if we can’t see something it doesn’t exist. But decisions are all around us, and when we give AI more and more power it happens more and more often that it influence decision-making process or make decisions on our behalf without us even knowing and being able to trace them and understand the process and rationale behind it.
Really curious on your perspective on AI systems, from more technical perspective. Subscribed!
Very, very important perspectives here. I hope the right people pay attention.
"The failure may appear as a hidden assumption..." -- if the road to hell is paved with good intentions, the road to organizational calamity is going to be paved with hidden assumptions.
You *assume* because the LLM says "Of course" that it understands you and is going to carry out what you intended. Then these assumptions get built into the infrastructure.
My zeroth rule in editing is "Do no harm". My first rule is "Never, ever, assume". And I don't. When I was on the newsdesk, I had all the journalists' names in my spell check. The number of reporters who spelled their name wrong was quite scary. I always pointed this out to them.
Thanks Fred! I share the same rule, and I'm most strict at work. We tend to assume many things by ourselves at work, at home but still have more specific context than generic large language models. AI doesn't have that and it's much harder to spot it and make a recalibration.
Really well written! I echo what others have said - the concept of shadow decision making is something we're most likely already seeing, but not recognising. People passing off AI advice as their own without thinking through the second and third order consequences