In an automated future, what does productivity really mean?

In March, Ethan Mollick, an associate professor at The Wharton School of the University of Pennsylvania, conducted an experiment. Mollick, who teaches about innovation and entrepreneurship, wondered how much of a business project he could accomplish in half an hour. The goal: use artificial intelligence tools to generate promotional material for the launch of a new educational game. AI would generate all the assets; Mollick would be the guide.

He set a timer and got to work. 

Bing, Microsoft’s GPT-4 model, gave Mollick information on his product and what his market looked like. It also generated an email and social media campaign for the launch and outlined a webpage to help promote the game. Mollick opened up a separate tab, and GPT-4 helped him generate the HTML to build the actual webpage. MidJourney, an AI program that produces images from written prompts, gave Mollick a large, attention-grabbing visual to welcome visitors to the page. Finally, he generated a promotional video. Bing, once again, produced a script. Eleven Labs, a program that helps develop natural-sounding speech synthesis, generated a realistic voice. D-id, a program that generates photorealistic videos, took all the components and made it into a video.

Time was up. Mollick also had enough content to go live.

At its core, productivity is somewhat of a banal mathematical equation that compares a worker’s output over some unit of time. The more items a worker can complete during, say, an hour, the more productive they are considered to be. 

By this definition, AI absolutely made Mollick more productive. After all, how many companies can build out a promotional campaign in thirty minutes by sheer human and brainpower? How many hours’ worth of meetings did these AI technologies save? How many jobs of knowledge workers did it complete? These questions around knowledge worker productivity are now top of mind for economists, companies, researchers, and managers who are thinking about the impact of AI.

If productivity is equivalent to economic growth, it seems indisputable that certain AI tools will increase productivity. A recent report by the global consulting firm McKinsey predicted that AI can infuse $4.4 trillion US dollars into the global economy every year. Banks would generate an additional $200 to $340 billion from AI aiding in customer service, decision making, and tracking fraud. Medical and pharmaceutical companies could see 25% increases in profit, the report estimated, if AI were to help develop new drugs and medical materials. 

And studies by the Capgemini Research Institute, a think-tank that focuses on the impact of digital technologies, suggests that generative AI will also boost productivity in sectors like IT, sales, and marketing.

While the accessibility and prevalence of generative AI tools will overall boost productivity for companies, research has also found AI and automation tools are having an impact on how knowledge workers stay focused and get work done. In a Dropbox-sponsored study, Economist Impact found that, of those who report using automation tools—including AI—in their work, 79% are more productive, while nearly 70% say they’re more organized. Another report by McKinsey estimates that AI can free up 30% of US work hours for knowledge workers by helping code, answer emails, and automate other routine tasks. 

But Bhaskar Chakravorti, an economics scholar and consultant at The Fletcher School at Tufts University, who so far uses AI to help auto-complete emails, questions how much a single tool can do. “As the emails start piling on, I do see myself hitting the tab button and completing the sentence suggested to me by Outlook,” he said. “Does that extra few seconds I’ve shaved off the last email really add up and make me a whole lot more productive? I don’t think so.”

In other words, just because we can use AI tools to do more in less time doesn’t mean that the results will necessarily be better. Experts argue that the question isn’t about how quickly AI can take us to the finish line, but how it can be used to enable more critical thinking, creativity, and focus for knowledge workers. If we’re able to offload the rote tasks on our to-do lists to AI, how does this impact the way we spend our working hours and ultimately, measure human productivity?

How AI can impact productivity as we know it

The question of AI and its impact on knowledge work isn’t a new one. Researchers are already exploring the implications of this technology and so far, the results from early studies seem promising. 

In a 2022 study, Github recruited 95 software developers to see how long it would take for them to write an HTTP service in Javascript. (All the participants were professionals, so this would be a task that all of them could complete without help from software.) Half of them were allowed to use GitHub Copilot, an AI-based tool that could autocomplete code. The 45 programmers who ended up using Copilot completed the task 55% faster than the others who didn’t. 

More recently, researchers at MIT quantified how ChatGPT could affect productivity on writing tasks. 444 college educated professionals were asked to complete writing tasks with and without the help of ChatGPT, with the final reports judged by professionals. ChatGPT reduced the time participants spent on the assignment by improving the quality of their work. 

These early findings suggest that a lot of the time-consuming grunt work on our to-do lists could be offloaded by AI. So what does that leave for the rest of the list? Authors of the MIT study suggest that tools like ChatGPT could help writers focus more on generating ideas and editing. The Github Copilot research found that with more free time, developers could focus on more satisfying work and ultimately find more fun in the coding they do. The throughline being that AI could help support the average knowledge worker tasks—like calendar scheduling and organizing email. 

Still, correlating more free time with being more productive is a challenging line to draw. While these studies reflect the potential for AI’s positive influence on productivity, critics have pointed out that the canonical definition of productivity doesn’t quite do the trick when assessing the productivity of knowledge workers. That’s because knowledge workers create value for their respective companies and institutions by what they produce, rather than how fast they produce it. 

Results on a greater scale depend much more on the sum of an organization’s workers, rather than an individual. Not to mention, companies are not necessarily tracking hours for salaried employees, which means a worker’s productivity is not as simple as solving an equation. Sometimes, the outcomes of a company could be directly measurable in terms of productivity, Chakravorti said. “But in some cases, it’s just not.”

Ultimately, preliminary research is finding that it’s easy to measure AI’s efficiency in taking over rote tasks, but the larger implications for knowledge workers—more free time to generate ideas, more space to focus on satisfying work—is trickier to track and define. 

Finding the right metrics

For some, when it comes to scaling the future of knowledge work, productivity as we know it isn’t the right metric at all. “For knowledge workers who work in an inherently creative industry, productivity and creativity are at odds with each other,” said Dan Shipper, the CEO of Every, a newsletter about technology, AI, and productivity. 

At Every, Shipper is a lot more concerned about the metrics of its newsletter at a larger scale. “How many page views did we generate? How many emails did we grow by? What’s our revenue? We’re not looking at hours worked or number of words typed,” he said. These kinds of metrics don’t necessarily correlate with simply being productive and they’re not problems that can be easily offloaded to AI. What they demand is a mix of something else entirely—complex thinking, critical problem solving, and the time and mental space required for creative output. 

“Creativity is this magical, unpredictable process where you’re trying to pull something literally out of your soul,” he said. “Productivity demands that it be on a schedule and reliable. That’s actually quite difficult.” 

If what early AI tools provide for us is the ability to do more of what only humans can do, thinking purely in terms of productivity when it comes to this technology is not fully capturing the bigger picture. “Not only are these studies limited in the endpoints that they measure, but they’re often limited by industry,” says Chakravorti. More research would have to get done on workers in different industries before we have enough data to say, precisely, how AI can influence work across different fields.

The changing shape of productivity

At present, nobody really knows how to measure the true impact AI will have on knowledge workers, whether by productivity or something else entirely. 

With tools like GPT, the effect on traditional productivity—simply getting tasks done—is pretty clear cut. “People are certainly putting white papers, essays, marketing, visuals and graphics faster than they would have in the past,” says Chakravarti. Even so, the outputs of something like generative AI seem relatively limited when it comes to the quality of creative output. (As a journalist, I have at times used GPT-4 for research, and while it cuts down on my time wading through Google search results, I tend to spend more time thinking critically about the suggestions and sources that the software suggests to me.)

When looking at how AI tools could free up more time for knowledge workers to do more creative and complex tasks, however, the way we think about human productivity—and work in general—can take on a different slant. 

“It may turn a lot of knowledge work into work that is much more like what managers do, rather than what the lower-level employee is doing. Everyone moves up the chain, to some extent,” Shipper said. 

While conventional definitions of productivity still linger, companies will need to think about other ways to measure the performance of their workers, or focus on bigger-picture metrics, like Shipper does at Every. It's easy to imagine a future where a team of analysts at a consulting firm might resort to using GPT to put together a report. Work that previously required a manager and two analysts could potentially be require no workers—just an automated tool. Those individuals who are freed up from that initial report might then go off and do something else entirely and have more time and energy for higher-level tasks. 

“When you can accomplish everything that [Mollick] can do in thirty minutes, these kinds of skills are going to be quite important: vision, taste, and ability to prioritize,” he says.