.jpg)
When people talk about bringing AI into their work, one of the first benefits they point to is time. If you ask someone how a new tool is working, you’ll often hear some version of, “We’re saving a lot of time.”
Time saved is a good place to start, but it only tells us part of the story.
If someone gets five hours back every week because AI helped them work faster, what happens to those five hours? Do they spend more time talking to customers and understanding their needs? Do they solve a problem they haven't had the capacity for? Or do those five hours simply get absorbed by more of the same work?
How your team uses that extra capacity tells you a lot more than the number of hours they saved. And that's why, as we bring AI into our companies, we need to ask ourselves:
Is AI making my company better?
AI makes it easier to produce more, and it's easy to mistake that for progress. But more output doesn’t necessarily mean we’re creating more value. Sometimes it just means we’re producing more of the same thing faster.
Throughout this AI Leadership series, I’ve focused on how leaders can bring AI into their companies thoughtfully. Now I want to look at how we know whether that work is actually paying off.
When we bring AI into our organizations, we want it to help us do better work, not just more of it. That means looking beyond time saved and asking whether AI is helping us serve our customers better, strengthen our processes, or create more capacity for higher-impact work. That’s a higher standard than simply asking how many hours someone got back.
Look at the Result, Not the Activity
When you try to measure the impact of AI on your company, it’s easy to look at whatever is right in front of you. Your Sales team may be creating twice as many account briefs, your Marketing team may be getting drafts done in half the time, or your Customer Support team may be closing more tickets each day. All of that can look like progress, but what you really need to know is what happened because of it.
Did the salesperson walk into the meeting better prepared? How did the marketing content perform once it was published? Did customers get better answers, or did the team just move through more tickets? Those are the things you need to be measuring.
Before you look at what AI helped your team produce, go back to the original purpose of the work by asking:
Before AI became part of this process, what was the work supposed to accomplish?
If the purpose of a support response is to solve a customer’s problem, then look at whether the issue stays solved or if the customer needs to come back again. If the purpose of a proposal is to help win more customers, look at whether those proposals are leading to more of the kinds of customers you want to serve. Once you're clear on what the work is there to accomplish, you have a much stronger way to judge whether AI is actually improving it.
This is one of the reasons I believe every growing company needs to have a strong business operating system. When your priorities, responsibilities, accountabilities, processes, and performance information are visible and connected, you have a baseline for judging what AI is improving. You can see whether it’s helping the company make progress on the things you’ve already decided are important rather than simply helping people produce more of the same thing.
Review the Whole Process
The next step is to look at your processes from beginning to end. If you only measure the step AI is used for, you can miss what it changes for everyone else involved in the process.
Say someone on your team used to spend two hours preparing a customer proposal, and AI can now produce a first draft in 20 minutes. That may sound like a big improvement, but you still need to know what happens after the draft is created.
If another person spends 90 minutes revising and reviewing the draft before the proposal can go to the customer, the improvement isn’t nearly as significant as it looked at first. AI may have made one part of the process faster, but did the process as a whole get better or faster?
To measure whether AI is really improving a process, you have to look at what happens before AI gets involved and what happens after it finishes its part. That means asking:
- How much work has to happen before AI can do its part?
- Did the output make the next step clearer or easier for the team to execute?
- How much review is required?
- How often does the work have to come back for corrections?
- Are people spending less time on the process overall, or has some of that work simply moved to a different part of the process?
- Did AI remove steps from the process, or add new ones?
Those questions can give you a very different picture from the one you get by looking only at the task AI completed. Sometimes AI creates meaningful capacity across an entire process. Other times, it saves time in one part while requiring more time and energy somewhere else.
This becomes even more important as AI agents begin handling several steps in a process. You may be able to see how much work an agent completed, but I’d want to know whether the people connected to that process are getting better results and spending less time fixing, checking, or redoing the work.
In the end, the number of tasks AI completes tells you far less than whether the process is producing better results.
Look for Consistency
A process can work well the first few times you use an AI tool and still not be something your team can depend on. You may find that a tool is dependable in some situations and needs a lot more help in others. Or the output might be strong one week but need a lot of correction the next. Before you expand how you’re using an AI tool, you want to understand where those differences come from.
Sometimes the issue is the information going into the tool. In other cases, the work may depend on context or judgment that AI doesn’t handle as well. The more clearly you understand those conditions, the easier it is to decide where AI belongs in a process and where your people still need to stay closely involved.
Before relying on a tool, look at the quality of the finished work over time and whether the process continues to produce strong results consistently. If the quality varies widely from one use to the next, figure out what’s driving that variation before expanding where you use it.
The person accountable for the process should be able to explain when AI tends to perform well and when it doesn’t. If your team can only say, “Sometimes it works great,” you’re not at the point that you can rely on it yet.
The more consistent the results become, the easier it is for your team to know how much of the work AI can take on and where human judgment still needs to stay involved. As you use a tool more, you want to see fewer corrections, more dependable results, and greater confidence from the people who are accountable for the outcome. That tells you a lot more than a single impressive result can.
There will be two types of companies in the future: Those that are great at AI, and those that used to be in business.
Use What You Learn to Make Decisions
The point of measuring AI use in your company is to help you make a better decision about what to do next. If an AI agent is helping a process produce stronger results and continues to become more consistent, you may decide to expand how that agent is being used. That could mean letting it handle another part of the process, applying the same approach somewhere else in the company, or redesigning the process now that you better understand where it performs well.
You may also learn that the agent isn’t ready to expand. The process itself could need more structure, the information going into the tool may not be dependable enough, or the team might need a clearer agreement around review or ownership.
In some cases, you may decide that AI simply doesn’t belong in that part of the work right now. That doesn’t make the experiment a failure. You learned something about the process, the technology, or both, and that gives you better information for the next decision.
We’re all still figuring out how to use AI inside our companies, and most of us aren’t going to get it right the first time. That’s why feedback loops are so important. You try something, look closely at the result, make an adjustment, and keep learning from what happens. AI should work the same way.
But those feedback loops only work if people have enough room to experiment in the first place. We should be building cultures where people feel encouraged to experiment with AI, try new approaches, and share what they learn. That means giving people enough room to test ideas without expecting it to work perfectly the first time. Some experiments will be worth expanding. Others will show us where the technology still needs more structure or review.
It’s up to us to make sure the learning doesn’t stay with one person or one team. The more openly those lessons get shared, the faster the company can get better at using AI. One person’s experiment can help another team avoid the same mistake, improve an existing process, or spot an opportunity they hadn’t considered before.
Keep Learning What Works
We’re all going to make some good calls with AI and some bad ones. That’s inevitable when the technology is moving this quickly. You won’t have every answer right away, but you can get better at recognizing what’s working, what isn’t, and why.
That means staying close enough to understand the difference between a tool that looks impressive and one that’s genuinely helping your company perform better. That discipline will separate the companies that use AI to level up from the ones that simply use a lot of it.
The next challenge is helping your people adapt as AI takes on more of the work. Roles will develop, responsibilities and accountabilities will change, and your team will need strong leadership through that transition. That’s what I’ll cover in the final article in the AI Leadership series: how to lead your people through the changes that come with bringing AI into your company.
Remember, using AI everywhere isn’t going to automatically make you better than a company that doesn’t. The companies that win the long game will be the ones that know where it belongs and how it’s helping them do better work. Don’t measure AI by how much you use it or how many hours it saves. Measure it by what it's helping your company become.
