The Role of Anecdotal Information in Fed Policymaking

William Poole
President, Federal Reserve Bank of St. Louis

Century Club Breakfast
Olin School of Business
Washington University
St. Louis
Feb. 13, 2002

*I appreciate comments provided by my colleagues at the Federal Reserve Bank of St. Louis, especially Howard Wall, Research Officer. I take full responsibility for errors. The views expressed are mine and do not necessarily reflect official positions of the Federal Reserve System

There was a time when the Federal Reserve encouraged a public perception of the mystique of policymaking. That is not the Fed's view today, but my reading of press commentary suggests that the old perception has not disappeared. There is, however, a difference. Today, rather than referring to the mystique of policy, people are more likely to refer to policy as "obtuse," "incomprehensible" or "deliberately confusing." Chairman Greenspan sometimes takes delight in saying such things as "Senator, if you understand what I just said I must have made a mistake."

Although mystique has turned into fodder for cartoonists, and it is healthy that we smile at ourselves from time to time, there is a serious issue involved. Beginning about 35 years ago, developments in macroeconomic theory began to make clear that the performance of the economy depends critically on market expectations about how economic policymakers will act in the future. As our understanding of these issues has deepened, it has become clear that one of the key dimensions of a successful monetary policy is that the policymakers need to have well-defined goals and a clear plan as to how they will go about achieving the goals. Both of these need to be understood in the markets; otherwise, the economy faces the equivalent of a broken play in football, where some members of a team think one play is being run and others think another play is being run.

Despite the jokes about Fed policy, I think few people today have any doubt about the Fed's objectives. Policymakers emphasize and reemphasize the importance of achieving low and stable inflation and that the Fed will act to the full extent of its powers, consistent with that objective, to cushion fluctuations in income and employment. Financial markets understand that the Fed is prepared to act decisively in times of national emergency and financial market distress, as evidenced by Fed actions in response to the terrorist attacks last September 11, to the Russian default and near collapse of Long Term Capital Management in the late summer and fall 1998 and the stock market panic in 1987.

My aim this morning is to illuminate a part of the policy process that is, I believe, not very well understood-the use of informal, or anecdotal, information in the policy process. I'll discuss the nature of this information and how we use it. I'll start, though, by outlining the role of formal data.

Before proceeding, I want to emphasize that the views I express here are mine and do not necessarily reflect official positions of the Federal Reserve System. I thank my colleagues at the Federal Reserve Bank of St. Louis for their assistance and comments, especially Howard Wall, Research Officer, but I retain full responsibility for errors.

The Role of Formal Data and its Pitfalls

To avoid misunderstanding, I must begin by emphasizing that the basic picture of the economy comes from the formal data published by statistical agencies. Among other things, the data include the national income accounts; labor market statistics on employment, unemployment and related measures; price and wage statistics; industrial production and capacity utilization; financial market statistics on monetary variables, interest rates, security prices and banking markets; international trade and capital flows. We have available similar data on economies around the world.

Our basic knowledge of the economy depends on the formal data. We use these data in our econometric models. Formal data have many advantages. We know the statistical procedures by which the data are created and have historical records, well back into the 19th century in many cases, from which we can study regularities of economic behavior.

I like to think of informal data as providing insight into the formal data. The insight runs across several dimensions, including timeliness and potential measurement and even conceptual errors in the formal data.

Consider timeliness. When making monetary policy decisions, members of the Federal Open Market Committee (FOMC) need to know as much as possible about current and future economic conditions. Unfortunately, the formal data on which we rely lag current economic conditions. For example, the Fed has an incomplete picture of the economy's two most important indicators, growth and inflation. The Bureau of Economic Analysis releases formal estimates of Gross Domestic Product (GDP) with lags of a month or more. Moreover, the data are subject to frequent and major revisions. Price indexes also are produced with a lag and are sensitive to factors that may be temporary, such as fluctuating energy prices, and measurement error. Given these problems, it is easy to see how monetary policymaking has often been likened to driving a car with a blacked-out windshield and fogged-up side windows.

To get an idea of the scale of the difficulty of using formal data, let's take a look at the timeliness and uncertainty of estimates of GDP, the most comprehensive measure of economic conditions. Just over two weeks ago, the advance GDP estimate was released, indicating that real GDP had risen at an annual rate of 0.2 percent in the fourth quarter of 2001. About two weeks from now we will see preliminary GDP estimates for the fourth quarter and a month after that we will see final estimates.

Since 1978, two-thirds of the revisions between the advance estimate and the final estimate of real quarterly growth were between -0.6 and 0.9 percentage points. This means that the likely range of the final estimate of fourth-quarter real GDP growth-which we won't see until the end of this quarter-is between minus 0.4 and plus 1.1 percent. To compound the problem, the so-called final estimate isn't the last estimate. Every summer, in July or August, the final estimates are revised and every three years we get major revisions. Since 1978, latest estimates have differed from final estimates by an average of 1.2 percentage points in either direction. Thus, the latest estimate for 1980, say, changes over time. As a consequence, economists like to say that history is never what it used to be! In principle, the estimates keep getting better as the statisticians find improved source data, refine estimation methods, and improve underlying concepts.

When one considers the enormous task of estimating the size of the U.S. economy, these problems might seem small. But, for making monetary policy decisions, they can make a critical difference. In fact, the range of uncertainty over growth rates can imply opposite short-run monetary policy responses. Given the uncertainty, it is often best for policymakers to sit tight, waiting for the uncertainty to be resolved by new information and revised data. What this means, obviously, is that sometimes it is clear in hindsight that policy action should have come sooner, or even in a different direction.

Of course, official growth and inflation data are not all that we have to go on. Most financial data are very up-to-date, and futures markets allow us to peer into the future-or at least into markets' expectations of the future. In addition, some real economic data, such as initial unemployment claims, auto and steel production, and electricity consumption, are available every week. Because these data are used to construct the official GDP estimates, they can provide partial pictures of current-quarter GDP.

Other data are used as leading or coincident indicators. Economists and analysts rely on past patterns of these indicators to provide insight into the current business cycle phase. Average weekly hours in manufacturing are a leading indicator because firms tend to adjust work hours before increasing or decreasing their workforce. Nonagricultural employment is a coincident indicator because it tends to rise and fall with GDP. In addition, some data are used to identify turning points in the economy. For example, many analysts follow the ratio of inventories to sales because it has tended to peak at the same time that the economy is in a trough.

While the Fed relies heavily on formal data and sophisticated statistical methods for analyzing the data, staff and policymakers alike spend a lot of time collecting and using anecdotal information that we gather from an extensive network of contacts. This anecdotal information helps us to see what is going on in the economy almost as it is happening. Also, because it is collected from the people who are actually making day-to-day business decisions, it helps us to understand why trends in the data are occurring.

For the rest of this talk I am going to discuss the role that anecdotal information plays in Fed policymaking. I will outline the ways in which we gather this information and then describe the various ways that we use it. I will also discuss briefly some recent evidence that anecdotal information adds value beyond what we get from other sources. And finally, I will touch on some of the dangers and pitfalls of relying too heavily on anecdotal information.

How We Obtain Anecdotal Information

The Fed gathers its anecdotal information from a wide range of sources. Directors of the Federal Reserve Banks and their branches provide written economic reports of conditions in their regions. Reserve Bank presidents and economists travel around their Districts meeting with business people and bankers discussing conditions in their industries. Reserve Banks maintain a network of industry contacts who are contacted on a regular basis in advance of FOMC meetings.

We make additional effort to maintain contacts in bellwether industries, such as freight and transport, whose activity is closely related to total economic activity. We also pay close attention to the real estate industry, where the level of activity might be a good indicator of the confidence that people have in the future. After all, for most people the purchase of a home is the largest financial commitment that they will ever make. If they are willing to continue buying homes when the economy is slowing, as has been true recently, they must be reasonably confident about their personal economic outlook.

In addition, our eyes and ears are always open, looking for emerging economic trends. A well-known example of this hands-on approach is that the president of the Minneapolis Fed has been known to make regular visits to local shopping malls to count the cars in the parking lots. I routinely make a number of phone calls to business contacts before FOMC meetings. I seek specialized information highly dependent on current circumstances. For example, during one of my trips after September 11 I struck up a conversation with a Southwest flight attendant and learned that the airline was continuing to hire and train new flight attendants. That information reinforced what I knew from press reports, that Southwest was not cutting flights and had an optimistic view of the future.

How We Use Anecdotal Information

The vast amount of anecdotal information collected throughout the Federal Reserve System is used for a variety of purposes. Most systematically, it is used to produce the "Summary of Commentary on Current Economic Conditions"-commonly known as the Beige Book-which is published two weeks before every FOMC meeting. To produce the Beige Book, each Federal Reserve Bank gathers information about its District through a network of contacts. The 12 District reports are collected together by an assembling Bank, whose staff prepares the national summary of economic conditions. The Beige Book, by the way, is available on the web site maintained by the Board of Governors.

The anecdotal information collected makes its way into FOMC meetings, where Fed governors and Reserve Bank presidents present their views on the economic outlook. In addition to their use in assessing the state of the economy, the anecdotes might be used to illustrate a point, thus adding impact to the comments. For example, a Bank president could say that "the market for construction material in my District continues to be tight and prices are rising." Or, he could say the same thing and add, "The situation is so tight that we have had reports of truckloads of drywall being hijacked." The addition of the anecdote (which happens to be an actual one from another District) adds more to the report than could several charts or tables.

Anecdotal information also can be used to confirm or to help understand ongoing trends that arise from the formal data. For example, during the late 1990s, the unemployment rate fell well below what many people thought was the level where inflation would start to take off. If we had relied only on the formal data, the Fed might have overlooked what firms and workers were doing to drive down the unemployment rate and how they were responding to tight labor markets. For example, we learned from our contacts in businesses that companies were willing to leave positions unfilled rather than bid aggressively for labor. My interpretation of this information was that firms were convinced that inflation would remain low and that they dared not let wage costs increase, because they were unlikely to be able to recover those costs in higher prices.

For a specific example, the owner of a manufacturing firm in Louisville told us how he was able to expand employment and production even though most of his traditional workforce-prime aged men-were already employed. This challenge led him to rethink his production process to make it a better match for the workers that were available to him. The result was that, whenever possible, his production methods were changed to reduce the requirement for physical strength.

We heard many similar stories about how firms were providing basic skills, making their work schedules more flexible, providing transportation for their workers, and so forth, to cope with the rapidly changing nature of their workforce. Without this first-hand knowledge of firms' ability to respond to competitive challenges and new environments, the Fed might not have known that unemployment could keep falling, at least for a time, without inflation being ignited.

Our network of contacts is also useful for identify emerging trends. For example, well in advance of it actually occurring, we had a good idea that firms' health insurance premiums would increase at double-digit rates in 2001. We knew this increase was coming before it appeared in official data because our contacts told us in mid 2000 about the health insurance contracts they were signing for 2001.

Another good example of anecdotal information came from one of our Branch Directors who noted in the summer of 2000 that loan demand at his bank was falling and that other firms in his area were beginning to experience problems. This information was important because, at the time, the economy was growing rapidly and nearly all forecasts indicated that rapid growth would continue. Nevertheless, reports of this sort continued to surface throughout the rest of 2000 and into 2001, helping the Fed to get ahead of the recession by starting to lower its federal funds rate target early last year, even though official GDP data available at the time suggested that the economy was still going strong.

While most of the anecdotal information collected by the Fed is used to supplement other information at our disposal, for some one-time events the anecdotal reports become the primary source of information. In these instances, standard data are not reliable guides because history has not recorded a pattern for how the economy is likely to respond. An example such an event was the series of terrorist attacks on September 11th, which had immediate and dramatic economic consequences, although we had no history to use to predict what these consequences might be. Nonetheless, we were able to use our network of contacts to get a good idea of the sectors that were affected the most, weeks before any formal data were available.

We found out very quickly that the Fed's injection of liquidity into the banking system had been successful, in that few banks reported having liquidity problems despite the near-complete shutdown of financial markets. We also found that retail sales came to a halt in the two to three days after the attacks but surged back to near-normal levels by the weekend, that manufacturers in the District were anticipating that they would be reducing their output by an average of 10 percent, and that auto sales for the period might be down by as much as 50 percent. Within a few weeks, our contacts told us that auto sales in October were in fact strong, in response to the zero-interest financing incentive offered by auto manufacturers. All of this information was vital in the weeks immediately following the attacks when the Fed had to react very quickly while navigating the uncharted waters of September and October. Indeed, based on anecdotal reports and experience, but without any substantial amount of formal data applying to the period after September 11, the FOMC cut the intended federal funds rate on September 17 and again on October 2.

Value Added

I have already mentioned some of the ways that anecdotal information adds value to Fed policymaking, but my comments themselves are only anecdotal. Recently, though, economists within the Federal Reserve System have tried to use more technical methods to evaluate the Beige Book as an indicator of present and future economic activity. The first such study was done at the Minneapolis Fed and found that the Beige Book has been an accurate predictor of real growth in the current quarter. They also found, however, that the Beige Book did not improve upon private sector forecasts of real growth. The study concluded that the Beige Book's value is not in forecasting economic activity, but in reflecting the economy. In other words, the Beige Book was found to add value by providing insight and context not found in formal forecasting models, while not improving on the performance of these models.

More recently, research from the Dallas Fed has found stronger empirical support for the Beige Book as a predictive tool. This study found that the national summary of the Beige Book has significant predictive content for current and future quarterly growth. Further, it found that the Beige Book has predictive content beyond what is provided by private forecasts. Of particular interest is that, according to this study, the Beige Book appears to have been better than alternative methods at identifying turning points in the economy.

Another potential source of added value is from the Beige Book's 12 regional summaries. The decentralized nature of the Beige Book means that the Fed has an instrument for detecting regional differences in the business cycle. Business cycle fluctuations are now thought to be more heterogeneous across regions and sectors than they used to be. Hence, one hears references to a "rolling recession" that affects different regions with greatest severity at different times. State and regional data, however, are much less complete than national data-for example, Gross State Product data are produced with a two-year lag. In this void, the Beige Book can help pinpoint focal points of such a rolling downturn or a rolling recovery. In fact, the Dallas Fed study suggests that, taken as a whole, the regional sections of the Beige Book add predictive power beyond what the national summary provides. Further, some of the regional sections-including that produced by the St. Louis Fed-have been useful on their own in predicting GDP growth one quarter ahead.

Dangers and Pitfalls

There are a number of dangers and pitfalls inherent with anecdotal information, so a great deal of care should be taken in using it. For one thing, despite the effort that the Fed puts into it, the number of contacts is small from a statistical standpoint, and they are not selected randomly. They tend to be in businesses that are familiar to a director of a Fed Bank, who have voluntarily agreed to serve as a contact, or whose manager or owner has been asked to serve on a Fed Bank's advisory board. Because of this selection process, numerous biases can arise. For example, perhaps the type of person who would serve as a contact or be a member of an advisory board would also tend to be more successful than the average businessperson. If so, then the information that the Fed receives would tend to underrepresent firms that are more likely to be experiencing difficulties.

Also, the responses might reflect the biases of the contacts rather than be accurate representations of conditions. This bias would not arise through any conscious misrepresentation, but perhaps through the tendency for successful business people to be more optimistic than the average person. An example of this occurred during a recent lunch at our Bank when we hosted a group of residential housing developers. The first time I went around the table asking them for their outlook on future conditions in their industry, nearly every one of them was quite upbeat. This near unanimity was surprising because this was not long after September 11th and most of them also felt that the overall economy was not in the best of shape. I then asked them if their answers were really what they thought would happen or if they instead reflected what they hoped would happen. Several of them then admitted that their outlook was probably more hope than expectations, and adjusted their answers accordingly.

The biases of the economist collecting and analyzing the anecdotal information may also mean that it is not representative of general economic conditions. For example, the economist might tend to pay more attention to anecdotes that fit his or her previously held beliefs. As a consequence, the overall impression that is conveyed from the anecdotes in, for example, the Beige Book, might tend to reflect the economist's personal views. It might also be that the odd or quirky anecdotes are the ones that have the most influence because they are the most interesting, even though they might not be representative of general trends.

Summary and Conclusions

Because anecdotal information is inherently unscientific, the Fed will continue to rely most heavily on formal methods when making monetary policy decisions. Use of formal statistics is an important discipline. Nonetheless, because these methods provide a far from perfect picture of the economy, the Fed should continue to use anecdotal information to help fill the gaps. Anecdotal information improves upon our understanding of where the economy is and where it might be going, most notably by providing information ahead of formal data. The process of gathering the information puts us in direct contact with people actually making day-to-day economic decisions. The information forces us to question the formal data and provides a view of the economy that formal methods simply miss.

This constant process of testing the formal data against the anecdotal reports, and vice versa, strengthens our understanding of both types of information. I know that I did not understand the scale and importance of the effort when I came to the Fed about four years ago, and I suspect that few observers outside the Fed appreciate the role of anecdotal information in the monetary policy process. That is why I thought this topic deserves some attention, and I hope I've been successful in explaining to you how the process works.

Back to top