AI Transformation: Dealing with Uncertainty

One of the biggest hurdles I’ve observed when it comes to AI transformation, particularly for technical teams, is how do they deal with uncertainty. When we work with deterministic code it brings certainty, in that the same input gives you the same output every time. It can be tested until everyone knows it works as expected. But AI brings a level of uncertainty that teams are not accustomed to. Then add to that the uncertainty around the constant change happening at speed in the tech industry, and the change happening at the organization level to keep up.

Many organizations try to compensate for that uncertainty with process (reviews, approvals, libraries and so on), but at the end of the day, the results from an LLM are non-deterministic: ask the same question twice and you can get two different answers. Yes, you can run evals, which means testing the model against a set of example inputs to see how it does, and you can add guardrails and give it skills to follow, but you still can’t know for certain how the LLM will behave. And how the LLM behaves will differ between you and your teammate.

In some ways, it reminds me of training my puppy. You can teach the puppy how to behave in all kinds of circumstances, but when you come across an unknown situation, you’re still never sure how your puppy will behave.

So how do organizations teach people to better deal with uncertainty?

In his book “How to Not Know: The Value of Uncertainty in a World that Demands Answers”, Simone Stolzoff describes many life situations where people had to deal with uncertainty, and how they do so. At the end of the book, he provides four pillars that he believes help people deal with uncertainty. I’m going to take all four and apply them to AI transformation.

“Being certain in some aspects of our lives allows us to hold uncertainty in others.” Simone Stolzoff

When you look around your organization, yes, many things are changing, but many things are still the same. If you are lucky enough to work with great people, find security in those people. That developer or QA engineer that you work with daily is still great to work with.

Lock into the areas that remain largely the same, or similar. If you enjoy getting user feedback, well, that process still must happen with AI-driven products. If learning is your thing, keep watching those YouTubers that you follow, or listening to podcasts.

In his book, Simone also mentions the benefits of ritual and routine. If you spend every Tuesday morning doing competitor research, keep doing it.

This pillar works on both a work level and on a higher personal level. If your work environment is changing, well stick to rituals and routines outside of work e.g. hobbies, sport, etc.

“Curiosity transforms fear into possibility” Simone Stolzoff

When you work for many years in a job, you get into a comfort zone. And once you’re in that zone, it’s very difficult to then step outside that and deal with uncertainty. Organizations need to get better at recognizing when people are in that comfort zone and actively pushing them to do more curious work. Having people that have an expertise in their job is important, but so is having adaptable people who are comfortable being outside of their comfort zone and not necessarily “the expert” in a particular domain.

It’s not surprising that founders and technical business owners are faring better with working with AI and embracing this new technology quickly, because, in the bigger picture of life, they are already used to dealing with uncertainty, the uncertainty of owning a startup, running a business, or wondering where your next client will come from.

I think this is the most important of the four, even though it may sound a little corny. When you’re building with AI, you don’t know what future problems you’ll have with this build. You can’t mitigate against them all. You have to accept the uncertainty. But what you can do is know that you’ll be able to handle whatever those future problems are.

The same goes for change generally, and for AI transformation. You can’t know what impact the change will have on you, your team, or your organization, and you can’t fully know. But you can trust yourself to be able to deal with it, and to get to the place where you can deal with it, for example, if you have to learn new skills to do it.

“For every decision we must relinquish a fleet of other possibilities that may never come our way.” Simone Stolzoff

I apply this pillar in two ways. Firstly, we all have made decisions that haven’t turned out quite the way we wanted and learned lessons along the way. And typically, in life, we carry these lessons from our choices with us to learn from. In tech, we might have released a product too early, not done enough testing, not listened to the customer, and so on. But some of those lessons learned may not apply in this new AI world, and in holding onto them, we may be trying to grasp onto a certainty that doesn’t exist.

Secondly, we all have a sense and image of what our role is in an organization. And when change comes, it can disrupt that image. In an AI transformation, many roles are disrupted, modified or altered. And it can be hard to let go of that original image, our original choice, and to accept the fact that we may never get back that role, or that image, or that version of ourselves, again.

Working in uncertainty is a skill, and like any skill it needs practice. Saying “I don’t know” is valuable, as is knowing you don’t have all the answers right now and pushing forward anyway. That is what these four pillars are really asking of the teams going through an AI transformation.

“The best way out is always through.” Robert Frost