Generative AI Is Already Transforming Work—but Is It Helping Teams Deliver Results?

Soon after ChatGPT made its public debut in November 2022, the Internet exploded with hilarious examples of what the tech could accomplish. But it wasn’t long before professionals started realizing that the AI chatbot was not just a toy, and it quickly gained momentum as a tool for a variety of serious work tasks, including drafting emails, brainstorming, and scheduling.

By late January 2023, 43% of professionals were using AI tools such as ChatGPT for work-related purposes, according to a Fishbowl survey. Earlier that month, only 27% of professionals were using generative AI for work.

While ChatGPT has received a lot of media attention, other generative AI tools like Dall-E 2 and Synthesia are also helping people do more at work—all with simple text-based natural language prompts. Unlike traditional AI, which is focused on classifying data and detecting patterns for decision-making support, generative AI creates new content, such as videos, music, and business pitch decks. 

Tech enthusiasts are anticipating that the rise of generative AI will usher in an era of unprecedented productivity, and early research backs this up. According to a survey conducted by TalentLMS, 57% of employees who used ChatGPT at work reported that it boosted their productivity. 

Goldman Sachs predicts that generative AI could “lift productivity growth by 1.5 percentage points over a 10-year period.” Christopher Prissarides, who won a Nobel prize for work on labor market frictions, is optimistic that AI won’t necessarily lead to widespread job loss and could increase productivity to the point where people can easily move to a four-day work week. 

It’s easy, however, to get swept up in the AI excitement and lose sight of goals at the organizational level. To generate positive outcomes for the business, leaders need to dig a little deeper and understand how the technology is supporting people at work—and how it might be holding them back.

Generative AI: Benefits and challenges in the workplace

The speed at which generative AI can generate prose, code, and video is astonishing, but there’s more to it than velocity. For Victor Boutté, a software engineer who uses AI tools like GitHub Copilot to complete coding tasks, AI is a lot like a smart coworker who is always available. As he explained in an interview with Vox.com, the tech gives him the energy he needs to focus on more creative tasks. 

There is also the example of Vanessa Bowen, another professional interviewed by Vox, who uses AI to generate placeholder text for app prototypes. She says AI helps her focus on what she does best: product design.

Generative AI is helping people work smarter, not harder, which makes the tech a huge draw for many professionals. But it’s not perfect, and positive business outcomes are not a given. 

To start, AI can hallucinate, which means it can present false information with an air of authority. Large language model bias can lead to outputs that are harmful to people from underrepresented groups. The question of who owns the rights to AI-generated content is also unclear, creating intellectual property concerns. All of these issues can impact brand reputation.

Yes, developers are hard at work fixing these problems. Writer, an enterprise-grade text-generating app, is designed to help organizations enforce editorial rules and keep messages on-brand. As more and more vendors enter the AI productivity app race, the trial and error involved in delivering good prompts (instructions for the AI), will be addressed.

But there is also the social dimension to consider: research has shown that people evaluate messages more negatively if they suspect they have been generated by an AI. Maybe certain customer emails will always need that human touch.

What’s critical for business leaders, then, is determining whether the tech is helping employees generate value through enhanced customer satisfaction, higher revenue, or innovation. 

From outputs to outcomes: Rethinking productivity metrics in the age of generative AI

Now that generative AI can produce text, images, audio, and video in the blink of an eye, tracking productivity as output volume is becoming even less relevant. To measure business value, leaders need to look at productivity holistically. This way, they can gauge whether their workforce is well-positioned to hit organizational goals with generative AI. 

But many leaders are still focused on outputs, an approach that made sense in the industrial age, when work was pre-engineered as a linear process optimized for speed. In a modern context, this creates a gap between work activities and organizational goals.

As the authors of this McKinsey blog post observed in October 2022, just before ChatGPT usage exploded, many organizations are stuck in the old-fashioned mindset. 

By focusing on a cluster of key metrics, businesses can move away from an output-focused approach and identify areas where generative AI is most effective—and where it is not. People analytics can be used to measure productivity holistically by linking people to outcomes, supporting this new way of measuring value. 

3 ways to measure the effectiveness of generative AI in the workplace: 

1. Performance: When measuring AI-supported productivity, start by establishing goals as they relate to business outcomes. Examples include number of sales closed, amount of tickets processed, or largest contributions to revenue. Tracking measures—such as Net Promoter Scores (NPS)—that paint a clear picture of customer satisfaction and loyalty over time can help leaders gain insight into whether generative AI is helping their teams perform well. Don’t get sidetracked by word counts, number of videos produced, or volume of business pitch decks created. 

2. Skills: Only 14% of employees have any training on AI tools, according to TalentLMS. Lack of training can create a gap between activities and performance. Talk with AI champions internally and determine which skills make a difference for the business. Metrics like employee skills Average Employee Acquired Skill Level Versus Average Employee Desired Skill Level show where the current skill set among current employees is and where learning investments might need to be made.

3. Sentiment: Understanding how employees feel about their AI-supported productivity is critical to nurturing effectiveness. As the writer of this GitHub blog post observes, “we found that developers’ own view of productivity has a twist—it’s more akin to having a good day.” Open the lines of communication about generative AI, and then use pulse surveys to understand what people like and do not like about working with the tech. Using people analytics, leaders can overlay this information with demographic data to determine where they can make targeted investments as part of an effort to prevent turnover or drops in performance. 

Let go of the industrial mindset to create business value 

Generative AI is already changing the way people work, and the potential for increased productivity and business growth is enormous. However, the technology can be problematic, and it needs to be approached with cautious enthusiasm. 

To measure the effectiveness of generative AI in the workplace, businesses need to look beyond output-driven metrics and instead focus on whether it helps employees generate value that aligns with organizational goals. By letting go of the traditional ways of measuring productivity, business leaders can focus on the metrics that matter in a time of intense transformation.

On the Outsmart blog, we write about workforce-related topics like what makes a good manager, how to reduce employee turnover, and reskilling employees. We also report on trending topics like ESG and EU CSRD requirements and preparing for a recession, and advise on HR best practices like how to create a strategic compensation strategy, metrics every CHRO should track, and connecting people data to business data. But if you really want to know the bread and butter of Visier, read our post about the benefits of people analytics