AI Hype or Reality? Shifts in Corporate Investment after ChatGPT

Following the release of ChatGPT in November 2022, discussion about artificial intelligence (AI) increased not only on social media but also in the boardroom. In our previous blog post, we used text-as-data methods to document patterns in executive discussions of AI during quarterly earnings conference calls. We found a more than fivefold increase in the number of sentences mentioning AI, which were largely positive in sentiment, after the release of ChatGPT.

Does this surge in positive chatter by corporate executives about AI reflect material implementation of this new technology at the firm-level, or is it merely “cheap talk”? In this blog post, we examine this question by analyzing the relationship between AI discussions in earnings calls and firms’ actual investment decisions as reported in their 10-K filings, which are annual regulatory filings with the U.S. Securities and Exchange Commission.

Using Text-as-Data Analysis in Earnings Calls

Earnings calls are quarterly meetings in which executives of publicly traded companies discuss the financials and current operations of their firms and factors that may impact future performance. For our analysis, we parsed transcripts of 185,999 earnings calls from 7,047 U.S. firms between the second quarter of 2008 and the first quarter of 2024. To learn more about how we conducted our text-as-data analysis, refer to our previous post. In brief, however, we measured AI conversation in earnings calls by identifying sentences that contained keywords such as “machine learning” or “chatgpt.” We then identified sentiment in sentences that mentioned AI from positive- or negative-tone keywords, such as “advantageous” on one hand and “dangers” on the other. Finally, we computed net AI sentiment by subtracting the number of negative-sentiment sentences from that of positive-sentiment sentences for each earnings call.

AI Chatter and Firms’ Investment Outcomes

To test whether corporate executives’ positive sentiment about AI in discussions during earnings calls is associated with higher overall capital spending and more total research and development (R&D) investment, we estimated a linear regression model. Our model compares investment outcomes among firms showing different AI-related sentiment within the same industry and quarter, controlling for firm size.

The figures below display our results for two time periods, one before the introduction of ChatGPT (the first quarter of 2012 to the third quarter of 2022) and one after it (the fourth quarter of 2022 to the first quarter of 2024). Each column shows the regression coefficient for positive sentiment on firms’ capital expenditure and R&D investment outcomes along with 90% confidence intervals (illustrated by the dashed vertical lines).

We found a stark difference between the two periods. Prior to the release of ChatGPT, firms with positive AI sentiment spent more on capital expenditures and R&D than their peers. For every net positive-sentiment sentence in a firm’s earnings call, we observed on average a 3.1% increase in capital expenditures and 9.7% increase in R&D expenditures by the company. However, these patterns have changed significantly since the release of ChatGPT. AI sentiment no longer has a significant association with capital expenditures. The association between AI sentiment and R&D expenditures is still positive, but it is half of its value before 2023.

Capital Expenditure Growth by Positive AI Sentiment

R&D Investment Growth by Positive AI Sentiment

SOURCES FOR BOTH FIGURES: S&P Global, Compustat and authors’ calculations.

These results suggest one of two possible conclusions. First, a large chunk of the rise in corporate chatter around AI and large language models (LLMs) like ChatGPT may be corporate hype that does not lead to any significant increase in capital expenditures. Second, there may be key differences in implementing traditional AI and AI systems based on LLMs. While traditional AI models were developed by capital-intensive firms that spent heavily on R&D, recently these estimated LLMs have been implemented by partnering with outside firms and without the need for such large investments. Therefore, this outsourcing may not be captured in capital and R&D expenditure accounts. We need more data and observations over time to fully understand the effect of LLMs on firms’ decisions and outcomes.

Conclusion

Before the announcement of ChatGPT, AI discourse in earnings conference calls was positively associated with capital expenditures and R&D investment. Since its announcement, these relationships have become significantly weaker. While some of this weakening might be attributable to cheap talk and corporate hype around AI, time will tell whether these changes are driven by fundamental differences in the implementation of traditional AI and new LLMs.