The modern workforce (and the world for that fact) is a huge puzzle of diverse skills.

Yet many find themselves on the outskirts of opportunity. Trying to figure out ‘how do I get there from here?’. The answer isn’t always so obvious or even accessible to those who do have it.

The skill gap is a topic we always talk about as an industry.

That daunting chasm between current abilities and the demands of tomorrow’s jobs. It can feel like a foe you’ll never quite conquer, but I believe we’ve found a new weapon to help us slay some of those demons.

You know what I’m going to say… Gen AI.

Yes. I’m a huge fanboy.

However, I’ve spent the last two years analysing every bit of research I can get on how the new wave of AI tools will impact workplace performance.

One finding is absolutely clear now after seeing it replicated again and again.

Conversational AI tools (LLMs or large language models) are bridging the skills gap at speed for the least experienced employees.

And I think this is wonderful.

Move from AI gimmicks to enablement

Let’s be straight, most of the AI use cases you’ve seen on social media are gimmicks.

While AI-generated images of cute cats doing random things might feel warm and fuzzy. It has no use in improving your performance or skills. Unless you’re so motivated by cute cats that it’s the driver of your performance.

In that case, I take back what I said!

Outside of my work (which is odd to explain because I work for myself), I’ve found daily use cases with LLMs (ChatGPT, Perplexity) in:

• Making sense of the complicated UK tax system
• Unpacking concepts from books and reports
• Advising on proposals
• Installing hardware on my gaming PC
• Brainstorming ideas

Of course, you can translate much of these into 9-5 work.

What we’re now seeing is the conversational AI tools, that have been shoved in our faces for nearly two years, are moving from a bit of fun to workplace productivity tools.

How Generative AI accelerates the performance of novice employees

I like to believe we all got into the L&D industry to help others.

I’m here because I want to help industry pros (like you) understand how to improve performance and skills with modern technology. As you know, conversational AI tools fall under this category.

What most excites me so far is 3 things:

• They are true ‘learning in the flow of work’ tools
• They’re the personalised experiences the industry has craved but failed to deliver
• They’re proven to enhance the skills and performance of employees

Obviously, I’m not going to say all this without dishing out some data.

Fret not, I have plenty for you to munch on.

🔮 Microsoft and the Future of Work

Those smart, and I now see very wealthy, folks at Microsoft released a fantastic new FOW report.

I know the term ‘future of work’ is overused. Everything beyond today is the future. No one actually knows what will happen, but it’s fun to indulge in some dubious speculation. Anyhoo, the below visual on page 8 of the report caught my eye.

My spidey sense (or should I say, AI sense!) was tingling when my eyes landed here.

The TL;DR is much like me, they’ve reviewed lots of reports and found the same conclusion.

LLMs help the least experienced the most.

Let’s dive into 3 case studies to illustrate this.

Boston Consulting Group + Harvard: Assessing ChatGPTs performance impact with Consultants

I talk about this one a lot.

You no doubt recognise the names. As will your teams and business leaders. so, it’s always a good one to roll out when making a business case (more on crafting your own later).

Here’s what you need to know:

Background: The study explored how Large Language Models (LLMs) like GPT-4 impact the way humans perform tasks, focusing on complex, knowledge-intensive tasks.

Study Design:

• Participants: Involved 758 consultants, making up approximately 7% of the individual contributor-level consultants at the firm.
  
• Experiment Setup: After setting a performance baseline on a task, participants were divided into three groups:

1. No AI access
2. GPT-4 AI access
3. GPT-4 AI access with a prompt engineering overview

Findings:

• AI excels at certain tasks but struggles with others that seem similarly complex.
  
• Productivity and Quality: Consultants using AI were notably more productive, completing 12.2% more tasks and 25.1% faster, with a 40% higher quality of results than the control group.
  
• Skill-Level Benefits: AI augmentation benefited consultants across all skill levels, with below-average performers improving by 43% and above-average by 17%.
  
• Limitations: For tasks deemed beyond AI’s current capabilities, consultants using AI were 19% less likely to find correct solutions.

Impact on low-skilled vs high-skilled consultants with AI

Here’s where it gets really interesting.

Low-Skilled Consultants:

• Improvement in Performance: The study found that consultants below the average performance threshold experienced a significant improvement in their task performance with AI augmentation, with their efficiency increasing by 43% compared to their baseline scores.
  
  
• Benefit from AI: This group benefitted from AI by being able to complete tasks more quickly and with higher quality. This suggests that AI tools can compensate for gaps in knowledge or experience, effectively elevating the performance of less experienced or skilled consultants to closer match their more skilled counterparts.

High-Skilled Consultants:

• Enhanced Productivity and Quality: Although the improvement was less pronounced than for their lower-skilled peers. high-skilled consultants still saw a notable increase in performance, with a 17% improvement over their baseline scores.
  
  
• Utilisation of AI: For high-skilled consultants, AI likely served as an enhancement tool that complemented their existing skills.

In sum:

Both skill levels benefited, yet it was the low-skilled who reaped the most rewards.

This is good news for humans.

It shows that skill and capability levels are a big factor in how AI can enhance work from a knowledge standpoint. It can’t know how to do everything!

National Bureau Of Economic Research: AI Performance Impact on Customer Agents

Here’s the TL;DR from NBER’s research:

This study focused on evaluating the impacts of a generative AI-based conversational assistant on job performance across 5,179 customer support agents.

The introduction of this AI tool boosted performance by 14%, as measured by the number of issues resolved per hour.

Interestingly, the productivity gains varied significantly across different skill levels. Novice and low-skilled workers experienced a substantial 34% improvement, whereas experienced and highly skilled workers saw minimal impact.

This disparity continues to underscore the nuanced effects of AI tools in the workplace.

1️⃣ Generative AI tools enhance the performance of low-skilled workers

The report unveils a significant improvement in productivity among novice and low-skilled agents working with a GEN AI conversational assistant (ChatGPT).

These agents experienced a 34% increase in the number of customer issues resolved per hour.

The research team attributed this to the AI tool’s capability to disseminate best practices and effectively guide less experienced workers down the learning curve more rapidly than traditional methods.

It enables the surfacing of the right information at the right time for the right level of expertise.

2️⃣ But it didn’t do much for experienced agents

A confirmation of the results from the BCG and Harvard report.

Like with this research we’re highlighting now. They discovered it was incredibly powerful for low-skilled consultants but the more experienced only found a 5% performance improvement.

The NBER team observed that while AI can replicate and transfer the knowledge and best practices of skilled workers, it offers less in terms of enhancing the performance of those who already operate at a high level.

See, we still need humans to grow together.

In sum:

What I found promising from these results was AI’s role in facilitating worker learning and improving customer sentiment.

The NBER report suggests interacting with AI allows workers to better internalise best practices, leading to durable gains in productivity, even in the absence of future AI assistance.

Nielsen Norman Group: AI improves performance by 66%

This is one of the first reports released in mid-2023 which drew attention to AI’s potential performance impact.

The 66% figure was calculated across 3 case studies the group worked with:

🤳 Customer service agents resolving customer inquiries in an enterprise software company.

✍️ Experienced business professionals (e.g., marketers, HR professionals) writing routine business documents (such as press releases) that take about half an hour to write

👩‍💻 Programmers coding a small software project that took about three hours to complete without AI assistance

The BIG takeaway: Users were much more efficient at performing their jobs with AI assistance than without AI tools.

Quick results overview

• Study 1: Support agents who used AI could handle 13.8% more customer inquiries per hour.

• Study 2: Business professionals who used AI could write 59% more business documents per hour.

• Study 3: Programmers who used AI could code 126% more projects per week.

Narrowing the skills gap for customer agents with AI

Yet more evidence for customer agent performance.

“In study 1 (customer support), the lowest-performing 20% of the agents (the bottom quintile) improved their task throughput by 35% — two and a half times as much as the average agent. In contrast, the best-performing 20% of the agents (top quintile) only improved their task throughput by a few per cent.”

Additionally, the speed and quality of learning increased too.

“An experienced agent can complete 2.5 inquiries per hour. This level of productivity is normally reached in 8 months of work (without using the AI tool). In contrast, the agents who started using the AI tool right off the bat reached this level of performance in only two months. In other words, AI used expedited learning (to this level of performance) by a factor of 4.”

What this means for the future of workplace learning

Clearly, in the right context, conversational AI tools are a performance enhancer.

We have enough data to say this comfortably. I feel right in saying we can’t discount LLMs in our learning performance strategies. Why shoot ourselves in the foot?

The next challenge will be how you work with and integrate these tools into your company’s workflow. We’re still early in this cycle, and this is not all on you. It’s an entire company effort. al though, I imagine most of it will land at your doorstep.

There are two things you can do today:

1. Get upskilled with conversational AI tools (guess who has a course on that for L&D pros 😉)
   
   
2. Craft a business case for utilising GEN AI tools to improve and enhance performance of teams

Let’s take a stab at number 2.

How to build a business case for Generative AI at your workplace

A lot of companies are running around like headless chickens trying to figure this out.

That’s usually what brings them to someone like me. Lucky for you, as a member of the cult of thoughts. You can get some of that juicy advice here.

I touched on this in a previous newsletter.

Let’s expand on that ↓

💼 The business case for AI

As part of your strategy proposal to senior teams, you will be asked why Gen AI.

You’ve got that answer with the above data already.

As a controversial idea, you could compare the findings above to those of your current learning tech. How do they match up? Share this too.

Here’s something to try for a proposal

1. Share this post with them as a reference point
   
   
2. Craft your case around the data shared here. Leaders like nothing more than seeing real case studies and getting FOMO. I don’t think you can get much bigger than Harvard.
   
   
3. Connect this to the state of performance in your company today and moving with the pace of modern technology (more fomo).
   
   
4. Recommend an experiment. Perhaps with the L&D and HR function, or if you’re a small company, identify a team of low-skilled workers who can benefit.

Final thoughts

In the right context, conversational AI tools can enhance performance and bridge the skill gap of low-skilled and novice employees.

Use this data to build your proposal for performance and shape the future of learning and performance in your work.

Before you go… 👋

If you like my writing and think “Hey, I’d like to hear more of what this guy has to say” then you’re in luck.

You can join me every Tuesday morning for more tools, templates and insights for the modern L&D pro in my weekly newsletter.