What If Experience Isn't What We Think It Is?

I've been sitting with something I haven't quite figured out yet.

A few months ago I read a study by researchers from Harvard, MIT and BCG that evaluated hundreds of consultants doing their actual work, with and without AI. Those who used AI finished tasks faster, completed more of them and produced higher-quality results. So far, more or less expected.

But there was one detail I couldn't shake: the ones who improved the most were the ones who had performed worst before. The biggest beneficiaries of AI weren't the high performers. They were the weakest ones.

I'm not sure exactly what to do with that. But since I read it, I can't help applying it to conversations I have all the time.

The Uncomfortable Question

When I speak at companies about talent and new generations, the same topic almost always comes up: how do you evaluate someone who uses AI to do their work? If they deliver something good but did it with the help of a tool, is it as valuable as someone who did it alone?

My honest answer is that I don't know. And I don't think anyone does yet.

What I do notice is that most organisations are still answering that question with the same old criteria (years of experience, degrees, ways of working that haven't changed in decades) without questioning whether those criteria still measure what they think they measure.

I'm not saying they're wrong. I'm saying the question deserves to be asked out loud.

What I See in Gen Z

Gen Z, broadly speaking, isn't having this conversation. Not because they don't care, but because for many of them AI is already part of how they naturally work. It's not an external aid they declare or hide. It's simply how they do things.

And that creates a strange situation: young people are producing very high-quality work that traditional evaluation systems don't quite know how to classify. Is it their work or the AI's? Does the distinction matter if the result is good?

I think it does matter, but not in the way it's usually framed. It's not about whether they used a tool: it's about whether they understand what they're doing. Knowing when AI is wrong, when to correct it, when it's not fit for something: that's the hard part to learn, and it's what separates someone who truly knows from someone who just knows which button to press. That judgement is what the tool doesn't give you.

The Speed of Change

The World Economic Forum published this year that 170 million new roles will be created over the next five years, but 92 million will disappear. I'm not saying this to alarm anyone (big numbers always make for easy alarm), but because I think the speed of that change forces us to ask questions we could previously ignore comfortably.

How do you measure experience when tools change this fast? What's worth more: knowing a process really well, or knowing how to adapt when that process no longer exists? What do we say to someone who is 45 with 20 years of experience and suddenly finds themselves competing with someone who is 23 and produces the same output in half the time?

I don't have clean answers. But I think these questions matter, and they're still not being taken seriously enough.

Why This Topic Matters to Me

I talk about generations and AI because we're at a point where if we don't ask the uncomfortable questions in time, the answers will arrive anyway, just without us having prepared for them.

And in my experience, that tends to be expensive.

How are you experiencing this in your company or your world?

---

Want to dig into this further? Write to me.

---

Sources

• Dell'Acqua, F. et al.: Navigating the Jagged Technological Frontier: Field Experimental Evidence on the Effects of AI on Knowledge Worker Productivity and Quality, Harvard Business School Working Paper, September 2023
• World Economic Forum: Future of Jobs Report 2025, January 2025
• Microsoft & LinkedIn: Work Trend Index 2024: AI at Work is Here. Now Comes the Hard Part, May 2024