Tracking the state of the overall U.S. economy requires accurate employment measurements. However, the two primary measures of employment statistics—the payroll survey and the household survey—have shown differing trends and levels in employment since the recession began in March 2001. Some differences between the payroll survey and the household survey are detailed below:

• The payroll survey provides a more accurate picture of employment trends in the U.S. economy. In addition to being significantly larger (with a sample size 600 times greater than that of the household survey), it is also benchmarked annually to unemployment insurance tax records and less likely to be subject to large revisions or misreporting.
• According to the payroll survey, employment has fallen by 726,000 jobs since the end of the recession in November 2001 and employment has fallen by 2.4 million since the start of the recession in March 2001. In contrast, the household survey indicates that employment has risen by 2.0 million since the recovery began and by 600,000 since the start of the recession.
• Adjustments for differences between the two surveys (e.g., to account for self-employment or multiple job holding) do not affect the difference in the trends of the two surveys since 2001.

Nonpartisan government experts agree that the payroll survey employment numbers are more reliable than those from the household survey, despite Secretary of Labor Elaine Chao’s erroneous claim that experts do not know which survey is better (Utgoff 2003; Congressional Budget Office 2003).¹ However, some analysts continue to mistakenly use the household survey to measure employment.² Others incorrectly report trends in the household survey, while ignoring the discontinuity in the series that results from the January 2003 population adjustment.³ The payroll survey’s more precise measure of employment trends provide a clear advantage to the more volatile and less accurate household survey numbers.

What surveys provide employment statistics for the United States?
Employment statistics for the United States come from both the Current Population Survey (CPS)—also known as the household survey—and the Current Employment Statistics survey (CES), also known as the payroll survey. The household survey is a sample survey of about 60,000 households conducted by the U.S. Census Bureau for the Bureau of Labor Statistics (BLS). Its sample, based primarily on the U.S. Census, is designed to reflect the entire civilian noninstitutional population.

The payroll data are collected from employers’ payroll records of about 400,000 individual worksites. This information is gathered by the BLS from a sample based on the unemployment insurance tax records. Both the household survey and payroll survey data are collected for the week of each month containing the 12th of that month.

Why is the payroll survey more accurate than the household survey?

• The payroll survey samples 400,000 business establishments. This represents an average of 40 million jobs each month; in September 2003, 40.5 million jobs were sampled (Getz 2003). In contrast, the household survey samples only 60,000 households, representing fewer than 70,000 workers. In September 2003, employment estimates were based on a sample of 67,804 workers. Thus, the payroll survey sample covers 600 times as many workers as the household survey.
• The payroll survey employment estimates are benchmarked to the unemployment insurance tax records. This yearly process anchors the payroll employment numbers to the comprehensive count of all nonfarm payroll employment. The household survey, on the other hand, is benchmarked only once a decade to the decennial census, resulting in a less precise employment measurement than the payroll survey.
• Large revisions and misreporting are also less likely for the payroll than for the household employment numbers. In recent years, the household survey has undergone far more extensive revisions than the payroll survey, particularly with respect to population estimates. In January 2003, an additional 576,000 jobs were added.
• The household survey’s smaller sample size contributes to the increased variability in its employment estimates. Figure 1 displays the employment estimates for the household survey and the payroll survey. The household survey is extremely volatile, indicating its inadequacy for analyses of month-to-month employment trends.
• Statistical agencies use the payroll survey for measuring employment trends and for other analyses of economic conditions. For instance, the Bureau of Economic Analysis (BEA) uses employment, hours, and wages from the payroll survey to estimate gross domestic product (GDP) for service industries, and the BLS relies on payroll employment and hours (supplemented with self-employment from the household survey) to estimate productivity. The strengths of the household survey are in measurements that the payroll survey is not set up to do, such as the unemployment rate, self-employment, the employment-to-population ratio, occupations, and breakdowns by demographic. While the household survey is useful for measuring this type of economic information, the payroll survey is a much better tool for measuring employment levels and trends.

Government experts agree that the payroll survey is more accurate
Both the Congressional Budget Office and the Bureau of Labor Statistics have commented on their preference for the payroll survey numbers over the household survey numbers:

“The establishment [i.e., payroll] survey better reflects the state of labor markets, the Congressional Budge Office believes, not only because other indicators also imply rather weak labor market conditions but because large revisions or misreporting appears less likely for the establishment than for the household data. Data on tax withholding conform better to the establishment survey’s results than to the household survey’s; in addition, both the share of employed people who are working part time for economic reasons and the still-low labor force participation rate indicate weaker labor markets than those existing at the trough. Three other measures suggest the same conclusion: during the first half of the year, the unemployment rate rose, both initial and continuing claims for unemployment insurance remained elevated, and the help-wanted index fell.” (emphasis added)

—Congressional Budget Office 2003, p. 11

“It is our judgment that the payroll survey provides more reliable information on the current trend in wage and salary employment. The payroll survey has a larger sample than the household survey—400,000 business establishments covering about one-third of total nonfarm payroll employment. Moreover, the payroll survey estimates are regularly anchored to the comprehensive count of nonfarm payroll employment derived from the unemployment tax records.” (emphasis added)

—Bureau of Labor Statistics 2003, pp. 4-5

A response to critics of the payroll survey
Some have speculated that the household survey provides a better indication of the trend in employment at and around turning points in the business cycle. These critics question whether the payroll survey accurately and fully picks up new businesses, known as “firm births.” This pro
blem may be especially exacerbated when measuring employment in a recovery.

In its estimates of employment, the BLS addresses the problem of firm births and deaths using past history and various estimation techniques to provide an adjustment factor to the current series. In addition, updates to the payroll survey are conducted annually to adjust for any discrepancies.⁴

In September 2003, Allan Meltzer speculated in The Wall Street Journal that the BLS may have been undercounting post-recession firm births. Meltzer was expecting the revised numbers to show an increase in employment because generally revisions are upwards in a recovery; that is, revised employment numbers add to the ranks of the employed. However, the BLS announced in October that its analysis of detailed tax records through March 2003 would result in a downward revision of total nonfarm payroll employment by approximately 145,000 for the March 2003 reference month (BLS 2003b).

A second critique of the payroll survey is that it leaves out self-employment. However, because the household survey employment reports do not distinguish between the self-employed who are gainfully employed and those who are searching for work—and because the numbers of self-employed nonearners would be expected to increase during tough economic times—the omission of self-employment numbers from the payroll survey may more accurately reflect overall employment trends.

Population adjustments to the household survey
The BLS periodically revises the household survey to account for new Census Bureau population estimates. In the last four years, there have been two population adjustments: one in January 2000 and one in January 2003. The shift in January 2000 was based on the new population estimates from the decennial Census and added about 1.5 million persons employed. The shift in January 2003, based on new estimates of faster than expected population growth since the 2000 Census, added another 576,000. At each shift, a discontinuity occurs in the series, reflective of only new population estimates and not an actual jump in employment.

To make valid comparisons with the numbers since January 2003, previous employment numbers must be adjusted upward to account for new population estimates. The BLS warns that use of the household survey employment numbers without making these adjustments makes any estimates of trends since January 2003 not comparable with those for earlier months (Bowler et al. 2003). The household employment estimates in Table 1 reflect these population adjustments.

One of the most egregious mistakes made by some analysts reporting employment trends is to omit these population adjustments in their estimates. One such omission was in a Heritage Foundation report, based on the household survey numbers, which claimed that more than one million new jobs had been created between October 2002 and October 2003 (Beach and Hederman 2003). This report improperly includes the 576,000 jobs added in January 2003 due to the upward revision to population that month. Additionally, the payroll survey, a better indicator of employment trends, indicates a loss of 291,000 jobs during the same time period (see Table 1).

Unfortunately, because BLS publications do not highlight the break in series caused by the increase in population in January 2003, this is a relatively common mistake in the media. Robert Samuelson, columnist for The Washington Post, and Floyd Norris, reporter for The New York Times, left out the BLS updates to the household survey data in their reporting of employment trends (Norris 2003; Samuelson 2003a). To his credit, Samuelson promptly posted a correction to his employment numbers (Samuelson 2003b). Use of the payroll survey, which is less susceptible to large revisions and more accurately measures employment, would have avoided these and other similar miscalculations in employment numbers.

What are the trends in employment?
The National Bureau of Economic Research determined that the trough in business activity occurred in November 2001 for the recession beginning in March 2001. Therefore, we examine trends since the beginning of the recession and since the beginning of the expansion—March 2001 and November 2001, respectively.

Since the beginning of the recession, employment has fallen by 2.4 million jobs. Since the end of the recession two years ago, there have been about 726,000 jobs lost, marking this as a period of “jobless recovery.”

How is employment defined in the household and payroll surveys?
The household survey counts people as employed during the reference period if they worked as a paid employee, worked on a farm, were self-employed, worked without pay in a family business, or worked in a private household. The household survey also counts people as employed if they are on unpaid leave during the reference period. The payroll survey, however, only counts people as employed if they were nonfarm workers who worked for pay for any part of the reference period (including persons on paid leave), excluding the other categories of workers measured by the household survey. To reconcile these differences, the household survey must be reduced by agricultural workers, the self-employed, unpaid family workers, private household workers, and those on unpaid leave.

On the other hand, the payroll survey counts each job separately when employees work at more than one job. The household survey counts each employee only once regardless of the number of jobs they hold. The household survey employment numbers must be increased to include multiple job holders to make it comparable with the payroll survey.

How does the household survey reconciliation alter employment numbers?
To better understand why the surveys display different trends, it is important to make the two surveys as comparable as possible. In this section, the household employment numbers are adjusted to account for the differences in the surveys. Specifically, this reconciliation subtracts agriculture, self-employment, private households, unpaid family workers, and those on unpaid leave, and adds multiple job holders to the reported household employment numbers. The household employment numbers are seasonally adjusted to make the two surveys comparable.

As shown in Figure 1, the household and payroll survey employment levels ebb and flow, sometimes drawing close together and sometimes shifting further apart. In January 1998, the difference between the two surveys was 7.3 million. In July 2000, the surveys were as close as 4.7 million, while in October 2003, the difference was back up to 7.9 million.

The household survey reconciliation brings the employment estimates much closer together (see Figure 2). For much of the series, payroll employment is higher than the reconciled household survey. The July 2000 gap shrinks from 4.7 million before the reconciliation to 2.4 million and the October 2003 gap shrinks from 7.9 million to 158,000. The levels of employment are much closer and the difference in trends for the recovery is reduced by about one-third. The household survey reconciliation indicates a gain of one million jobs since November 2001 and 100,000 jobs since March 2001. With the reconciliation, the difference in employment trends since November 2001 is 1.7 million, whereas without the reconciliation, there is a difference of 2.7 million between the household and payroll employment trends (see Table 1).

The employment trends of the payroll survey and the household survey reconciliation still produce divergent results, though the differences are smaller than before the reconciliation. The fact remains that the household and payroll surveys report different trends since
2001. Because the trends are different, it is important to report the employment numbers from the more accurate survey. The payroll survey remains the best indicator of employment trends.

Conclusion
The payroll survey has a clear advantage in measuring employment trends in the U.S. economy. The payroll survey employment numbers are based on one-third of total nonfarm payroll employment and are benchmarked to the complete enumeration of nonfarm payroll employment yearly. Overall, the payroll survey provides a more precise and less volatile measure of employment and employment trends than the household survey.

December 2003

Endnotes

1. Chao’s comment was that the “experts may argue about the advantages and disadvantages of each survey” (Chao 2003).

2. See, for example, Melloan (2003).

3. See The Heritage Foundation (2003) and Norris (2003).

4. The BLS has revised its updating process to produce these corrected estimates even faster than in previous years. For instance, the data up through March 2003 will be updated in February 2003 rather than June 2004.

References

Beach, William W. and Rea S. Hederman Jr. 2003. “Tax cuts working: Over one million new jobs.” The Heritage Foundation, WebMemo No. 363. Washington, D.C.: The Heritage Foundation.
http://www.heritage.org/Research/Taxes/wm363.cfm

Board of Governors of the Federal Reserve System, Division of Research and Statistics. 2003. Reconciliation of Household and Payroll Employment. Washington, D.C.

Bowler, Mary, Randy E. Hg, Stephen Miller, Ed Robison, and Anne Polivka. 2003. Revisions to the Current Population Survey Effective in January 2003. Washington, D.C.: Bureau of Labor Statistics.

Bureau of Labor Statistics. 2003a. The Employment Situation: August 2003. Washington, D.C.: Bureau of Labor Statistics. USDL 03-467.

Bureau of Labor Statistics. 2003b. The Employment Situation: September 2003. Washington, D.C.: Bureau of Labor Statistics. USDL 03-523.

Bureau of Labor Statistics. 2003c. CES Net Birth/Death Model. Washington, D.C.: Bureau of Labor Statistics.

National Bureau of Economic Research, Business Cycle Dating Committee. 2003.
http://nber.org/cycles/july2003.html

Chao, Elaine. 2003. Where the workers are. The Wall Street Journal, December 9.

Congressional Budget Office. 2003. The Budget and Economic Outlook: An Update. Washington, D.C.: Congressional Budget Office.

Kathleen Utgoff. 2003. Commissioner, Bureau of Labor Statistics before the Joint Economic Committee. U.S. Congress, September 5.

Melloan, George. 2003. That `jobless recovery’ isn’t so jobless after all. The Wall Street Journal, December 9.

Meltzer, Allan H. 2003. A jobless recovery? The Wall Street Journal, September 26.

Norris, Floyd. 2003. Grasping at the statistics on the self-employed. The New York Times, December 6.

Getz, Patricia. 2003. Personal correspondence with the Division Chief, Current Employment Statistics, December 9.

Samuelson, Robert. 2003a. Economic turnaround? The Washington Post, July 30.

Samuelson, Robert. 2003b. The ‘big media’ myth. The Washington Post, August.