China's Economic Data: An Accurate Reflection, or Just Smoke and Mirrors?

Since China became more open in 1978, its gross domestic product (GDP) is reported to have risen from 2.3 percent of the world’s economy to nearly 18 percent.¹ (See Figure 1.) Because of this rising share, the Chinese economy influences a myriad of economic outcomes, including, for example, global prices of oil and food, as well as U.S. imports and exports. But are these data on China’s economy accurate? Some people remain skeptical about the official statistics released by the Chinese government.

Reliable economic statistics are important for analysts who look at the performance of the Chinese economy to assess, for example, the demand for oil and other commodities. Thus, we explored some of the challenges to the Chinese data gathering/reporting process and put China’s data quality within the context of other developing nations’. We found that the Chinese National Bureau of Statistics has improved its source data and its collection practices, making its final official statistics higher quality than those of many counterparts in the developing world. However, due to the country’s complex economy and challenges posed by the transition from a command economy to a market economy, China’s economic statistics remain unreliable.

These issues with official Chinese government statistics have fostered attempts to obtain better estimates of Chinese GDP, using methods that vary widely. Some methods are simply corrections to the official Chinese GDP numbers, while others use alternative variables like energy imports that are correlated with output. Alternative data series are particularly useful if they are not compiled by the Chinese government.

From Command Economy to Market Economy

China’s National Bureau of Statistics (NBS) was created to track agriculture and production in the state-owned enterprises. In a command economy, the statistics bureau’s primary purpose is tracking physical output to ensure that economic activity meets preset production goals; this allows the state to allocate raw materials. Consequently, rather than tracking the output contribution of each sector, the NBS focused more narrowly on final physical production.² Because the means of production are owned and operated by the state, tracking exact economic activity—such as physical inputs, outputs and technology levels—is more straightforward in a command economy.

In a market economy, the statistics bureau tracks economic activity more broadly—focusing on the concept of variables like GDP, employment and unemployment—to obtain an economy-wide measure of macro growth.

In the late 1970s, China began a major economic transformation. The country allowed individuals to own companies and opened four coastal cities to foreign investment in special economic zones. These steps resulted in a new private service sector, which grew faster than the NBS was prepared for. According to economist Carsten A. Holz, many of these private service-sector businesses created a major measurement challenge because they did not report directly to the NBS until the 1990s.³

In 1993, the country transitioned to the United Nations’ System of National Accounts, which uses the more conventional value-added approach to GDP. China retroactively published GDP data applying these methods. Still, concepts like value-added were relatively new to the individual statisticians and government bureaucrats behind the national numbers: Understanding and adopting these new concepts takes time.

Measurement errors are inevitable in an economy as large and complex as China’s. The additional challenge of overhauling the country’s statistical system to measure market economy variables makes accurately measuring growth during the transition period unlikely.

Cooking the Books?

Some critics of official Chinese data cite falsification at the provincial and individual levels as the biggest source of unreliable GDP statistics. Holz explained that data falsification is thought to occur in rural areas, where leaders tend to only want good news because they are evaluated by the economic performance of their locality. After fabricating one report, leaders struggle to go back to accurate numbers because they would have to report lower than actual growth to rebalance the level of output. Hence, GDP statistics at the provincial level remain inflated.

However, the NBS is aware of the tendency for provincial officials to overstate GDP, and the bureau makes corrections for this behavior. In 1994, the country introduced census surveys to bypass lower-tier statistical departments and check the quality of the data collection. Four years later, the NBS took action against data falsification by issuing a reform that allowed for statistical breaks in provincial numbers to relieve past exaggeration. In 2015, the NBS reported national GDP of $10.4 trillion, which was about 7 percent less than the sum of the provincial numbers.

Former Fed Chairman Ben S. Bernanke and research analyst Peter Olson have argued that the Chinese NBS’ lack of transparency may be more of a factor in the unreliability of the statistics than its lack of political independence.⁴ For example, the NBS produces data series that are less volatile than those from other countries, making China’s time series statistics seem unreliable or manipulated. Bernanke and Olson have pointed out that this smoothness is more likely a result of technical issues rather than political manipulation.

Better Data than Others?

The degree of unreliability of China’s official statistics may be less egregious if the country is compared with other developing countries. The World Bank, which classifies China as a middle-income country, ranks low- and middle-income countries with populations greater than 1 million by a statistical capacity score, reflecting the country’s ability to produce and disseminate high-quality aggregate data. The statistical capacity score aggregates 25 individual variables that measure aspects of a country’s statistical methodology, source data, periodicity and timeliness.

In the past, China’s score has been at or below the median (38th percentile of low- and middle-income countries scored in 2004 and 52nd percentile in 2015). However, in the 2016 rankings, China earned a score of 83.3 out of 100, putting it in the 83rd percentile.

This score means China is actually on the upper end of the distribution for statistical capacity compared with similar countries. China’s score improvement comes mostly from better methodology, improving timeliness and periodicity of data releases, and joining the International Monetary Fund’s Special Data Dissemination Standard, a voluntary program that evaluates a country on criteria important for international capital markets.

Alternative Methods to Track GDP

Energy Consumption

Without more transparency from the NBS, the academic community has been forced to rely on alternative measures to track Chinese GDP growth. One alternative measure is the change in energy consumption. As an emerging economy with a large manufacturing sector, China consumes a lot of energy. Changes in energy consumption may be a good proxy for changes in output because energy usage typically correlates with output and can be verified by data sources outside the Chinese government; as an input to manufacturing, energy also is a variable that China’s command economy statisticians were well-equipped to measure.

Economist Thomas Rawski studies Chinese GDP through the lens of energy use. He points out that between 1997 and 2000, official figures reported that Chinese real GDP grew 24.7 percent, yet energy consumption decreased 12.8 percent.⁵ The difference implies a 30 percent reduction in energy use during those years, which seems unlikely for an industrializing economy. Rawski bolsters this argument by comparing energy use in other Asian countries during their respective episodes of growth. Figure 2 highlights his results. In each case, even that of China during an earlier growth period, a double-digit increase in GDP is related with a double-digit increase in energy consumption. But for the 1997-2001 period, Chinese energy consumption declined despite GDP growing at a faster pace than in the 1987-91 period.

Rawski argues this result is evidence of overestimation of Chinese GDP growth during that period. He suggests cumulative growth was more likely somewhere between 0.4 and 11.4 percent during those five years. Energy consumption, however, is an imperfect proxy of economic growth. A country’s energy consumption could be impacted by several factors external to economic output. Increased efficiency, a shift from an industrial to a service economy or, similarly, a shift from a production to a consumption economy could all result in lower energy consumption. For this reason, several individuals have come up with broader GDP proxies that include more variables.

Indexed GDP Proxies

Several private-sector research firms have developed their own measures of Chinese GDP growth based on a wide array of indicators, including freight volume, passenger travel, electricity output, construction indicators, purchasing managers indexes, financial indicators like money supply and the stock market, alternative GDP deflation measures, and alternative measures of production.⁶

These indexes focus on measuring the quarter-to-quarter growth rather than the level of output, but all of them suggest that there has been overstating of growth during downturns and in recent years. Consistent overestimation of quarterly growth could lead to an exaggerated GDP level, an issue we address below.

All the indexes suggest China’s GDP growth is lower than the official estimates. Lombard Street Research’s measure, based on searching for a more accurate way to deflate nominal GDP, estimated a 2.9 percent growth rate in the third quarter of 2015, while Bloomberg’s model—which includes more data on industrial output and retail sales—estimated growth of 6.6 percent in the same period; the official estimate was 6.9 percent for the third quarter of 2015. The problem with these measures is that a lot of them are black boxes, leaving one to wonder if they give adequate weight to the many complex facets of the Chinese economy.⁷

Perhaps the most popular index for Chinese GDP is the one suggested by and named after Li Keqiang, then China’s vice premier and now premier. In 2007, Li was quoted in a U.S. diplomatic cable later released by WikiLeaks as saying GDP figures are “man-made” and therefore unreliable. He went on to say that instead of looking at official figures, he uses electricity production, rail cargo shipments and loan disbursements.

This index is easily constructed by anyone with access to these three data series. Researchers John Fernald, Israel Malkin and Mark Spiegel designed an index based on the three variables and made the data and methodology available in an online appendix.⁸ The researchers fitted a regression of the index on real GDP growth from 2000 to 2009 and then used the fitted values to predict real GDP growth from 2009 through 2012 (referred to as out-of-sample in the accompanying graph).

Their results indicate that the relationship between GDP and the Li index during the 2000-2009 period continued to hold in the 2009-2012 period. In other words, the changes in official GDP statistics during the 2009-2012 slowdown were consistent with Li’s index. This analysis offers some validation that the quarter-to-quarter percent changes in Chinese official statistics are not overstated.

These methodologies, however, would not detect whether the level of Chinese GDP has been consistently overestimated for a long period of time. Moreover, many of the GDP proxies do not include any measure of China’s growing service sector or agricultural production. By some estimates, services now make up about 49 percent of Chinese output, compared with physical production, which makes up 42 percent.⁹

Luminosity

Another alternative method uses satellite data that measures the intensity of man-made night lights (luminosity). Unlike indexes of human-produced economic data, these data are immune to falsification or misreporting. The night-lights data we examine are gathered by Air Force satellites circling the earth 14 times a day since the 1970s. The satellites measure the light intensity emanating from specific geographic pixels, which can be aggregated to subnational, national and supranational levels. In 2012, economists J. Vernon Henderson, Adam Storeygard and David N. Weil created a dataset using information from night-lights satellites and applied it to estimate GDP in countries with low-quality data.¹⁰

The three researchers identified several reasons why night-lights data are a good proxy for economic activity. First, they argued that night-lights data track GDP because consumption of all goods in the evening requires light. To verify this claim, they confirmed that variation in pixels lit across countries is positively correlated with income (controlling for population density).¹¹

Henderson, Storeygard and Weil estimated a 14-year change and annual changes in economic activity for a panel of 188 countries between 1992 and 2008, including many low- and middle-income countries.¹² One way to assess the quality of Chinese economic data is to look at the difference between the growth rate of real GDP reported by the government and the estimated growth from 1992 to 2006 using the night-lights data. Reported real GDP growth in China over this period is about 122 percent, while predicted growth using the night-lights data is only 57 percent.

This sizable gap suggests cumulative Chinese growth over the years could be overstated by as much as 65 percent. Compared with other countries in the sample, the difference between the official and estimated numbers for China is large. In fact, the only country with a larger gap than China is Myanmar. India also has a large gap between actual and estimated, about 39 percent, although this gap is still notably smaller than China’s. Other emerging countries, like Brazil and Russia, have significantly smaller gaps between actual and estimated.

Figure 4 shows Henderson, Storeygard and Weil’s estimated annual GDP growth using night-lights data. The purple line represents estimated real GDP growth using night-lights data.¹³ The green line is estimated real GDP growth using night-lights data in conjunction with the country’s long-term growth path, calculated using official data. The inclusion of this trend essentially forces the estimated values to follow the same growth path as the official data so that the night-lights data are only informing annual fluctuations from the trend.

The purple line shows that real GDP growth is consistently overstated, particularly in the years before 1996. The green line (with the included growth trend) shows overstated growth before 1996; it also is much more volatile from year to year, moving more as one would expect real GDP growth to move. After 1996, however, the green line tracks the black line (the official growth rate) closely; this supports the other indexes’ conclusions that quarter-to-quarter fluctuations in Chinese real GDP growth are smoothed, but likely move in the correct direction.

Conclusion

Skepticism for Chinese official economic data is widespread, and it should be. Even if every Chinese economic number were reported truthfully and accurately to the best of an individual’s understanding, the official numbers would still fail to fully capture the evolution of an economy growing and changing so quickly.

China’s economic data system is a work in progress and a hurdle that statisticians have yet to overcome. The Chinese NBS could improve its system by offering greater transparency behind the data-gathering process and statistical procedures, allowing data users to better identify weaknesses in the official numbers. But the heavy criticism of Chinese officials and accusations of intentional falsification or manipulation are likely misplaced. The truth is more likely that economic growth in China is too challenging to capture as effectively as growth in developed countries.

Alternative measures of growth can offer useful insight into the accuracy of official statistics. Chinese growth was likely overstated during the transition period from command to market economy, possibly leading to an exaggerated level of output in the recent data. An exaggerated level of output could mean that the Chinese share of world GDP is overstated.

However, while the level of Chinese GDP may remain overstated, both the Li index and estimates from the night-lights data suggest that the recent growth rate numbers for Chinese official data are more reliable. They may be subject to collection error and smoothing, but appear to be moving in the correct direction.

Michael T. Owyang is an economist, and Hannah G. Shell is a senior research associate, both at the Federal Reserve Bank of St. Louis. For more on Owyang’s work, see https://research.stlouisfed.org/econ/owyang.