Assessing Job Quality: How Factcheck.org got it wrong
By Jared Bernstein Elise Gould Lawrence Mishel
July 27, 2004

July 27, 2004 | EPI Issue Brief # 200

Assessing Job Quality
How Factcheck.org got it wrong

by Elise Gould, Lawrence Mishel, Jared Bernstein, and Lee Price

Download a Microsoft Excel spreadsheet containing the full data set used in this Issue Brief. 

Using jobs data from the Bureau of Labor Statistics (BLS), the voter advocacy group Factcheck.org recently weighed in on the debate over the quality of the jobs being added to the U.S. workforce. Factcheck.org purported to find "good evidence that job quality has increased over the past year or more." 1 However, as we show below, Factcheck.org's analysis is flawed, and the BLS data do not support this conclusion. To the contrary, the evidence shows that the industry and occupation categories growing most quickly over the past year pay less than those growing at a slower pace.

Factcheck.org analyzed a BLS data set of 150 industries and occupations, such as sales occupations in the construction industries. We refer to these occupations within industry categories as sectors. Factcheck.org sorted these sectors by 2003 median wages, put 75 in the top category and 75 in the bottom (irrespective of sector size), and stated that the sectors above the median added more jobs in the last year than those below the median, leading to their conclusion of an overall improvement in job quality.

A better method of assessing job quality
For reasons elaborated on below, Factcheck.org's method reaches the wrong conclusion about the impact of net job growth on job quality because it fails to capture the impact of job growth on wages. Applying a better method to the same data as Factcheck.org over the same time period they analyzed—June 2003 to June 2004—we find that the weekly earnings of expanding or fast-growing sectors are 7.2% lower than the shrinking or slow-growth sectors (our key results are shown in Table 1). As shown in the first column of Table 1, over the past three years (June 2001 to June 2004) the gap between expanding and contracting sectors was -9.5%. Since our calculations have been performed on precisely the same data used by Factcheck.org, why do we get different results? We answer this question by discussing the shortfalls of the Factcheck.org methodology.

Unlike Factcheck.org's analysis, our approach is based on the fact that the average wage in the economy is a function of the average wages of the various sectors (occupation/industry groupings, in this case).2 The importance of each sector's contribution to the overall wage level is determined by its size, or more precisely, its share of total employment. Whether changes in employment in that sector contribute to overall higher wages depends on two factors. One factor is whether that sector has an expanding or contracting share of employment. The second factor is whether the expanding sector has higher or lower wages than the contracting sector. For example, when higher-wage sectors have expanding shares (or contracting sectors have lower wages), then overall wages will become higher and job quality improves. Conversely, when higher-wage sectors have contracting employment shares (or lower-wage sectors have expanding shares), then overall wages are lowered and job quality deteriorates.

Here is a step-by-step description of our approach (displayed in Table 2, which shows the 10 largest sectoral changes and an aggregate of the other changes). The first step is to calculate the employment shares—that is, employment in a given sector as a percent of total employment in each time period. We then calculate the difference in the share of jobs for each sector between June 2003 and June 2004. When the change in share is negative, we say that sector is contracting relative to changes in total employment. Likewise, when the change in share is positive, that sector is expanding. Since we are dealing in shares, the sum of the expanding shares must exactly counterbalance the sum of the contracting shares.3

Next, we calculate the change in share for each sector in contracting industries relative to the total change of all contracting industries. For example, the "office and administrative occupations/retail trade" sector had a change in share equal to -0.182%. We simply divide this by the total contracting share change (-2.328%) and find that the change in this sector represents 7.8% of the total contracting share change. The same calculation is conducted for the remaining contracting sectors and all the expanding sectors. These calculations let us know how important the change in each sector is compared to the total change in employment shares.

The fourth column of the table is the median weekly wage (again, taken directly from the same BLS data as used in Factcheck.org's analysis). We weight the wage by the percent share change represented by each sector, which represents the importance of that sector to the total change in shares. Again, using the example of the sector labeled "office and administrative occupations/retail trade," we multiply 7.827% by $369 and get $28.88. We sum up all of the weighted wages to get the wage for the average contracting-sector job ($612.70). Performing the same exercise on the expanding sectors, we find an average wage of $568.29, which is 7.2% lower than the average wage for the contracting sectors.4

Factcheck.org's shortfalls
There are several reasons why Factcheck.org's analysis is not an appropriate metric for job quality:

Factcheck.org analyzes changes in employment, not employment shares.
Factcheck.org examines changes in the absolute level of employment in sectors. Thus, if a higher-wage sector adds a few jobs but is still shrinking as a share of total employment (i.e., job growth was below average), then the Factcheck.org analysis would show that sector improving job quality, even though its declining share was actually putting downward pressure on overall wage growth. As we have shown with our analysis, examining the same data in terms of changing employment shares gives the opposite conclusion, and one that is far more reliable since this method weights each sector's pay by its contribution to the overall wage. The difference is that Factcheck.org analyzes sectors according to whether employment grew or did not grow, when the appropriate question for job quality is whether sectors grew faster than average or slower than average.

Industry sectors are too aggregate.
The BLS data used by Factcheck.org are at too aggregate a level to provide a thorough assessment of job quality. For one, there are only 14 industry categories. We have analyzed the industry dimension of job quality and have found that the erosion of job quality is underestimated with such aggregated industry sectors. This underestimation occurs because there are important shifts occurring within the large industry sectors, such as "temp work" within the broader category of "professional services." For example, when we perform our job quality calculation on the eight most aggregated industries from June 2003 through June 2004, we find that expanding industries pay slightly more than contracting ones, that is, the opposite result than the one we've stressed throughout. When we disaggregate to 20 industries, we find the expanders pay 10.4% less than contracting industries.

Factcheck.org's use of the data is too limited.
Examining job changes above and below one threshold is an inadequate way to summarize changes in the economy. After all, there may be (and are) large gains that occur among the sectors that have wages above (or below) the median wage. Factcheck.org's analysis will not capture such changes across the pay spectrum. Our method, however, relies on changes in every sector and summarizes job quality changes economy-wide. One way to illustrate this is to examine changes in job quality within the two large sectoral groupings that are the basis of Factcheck.org's analysis: all sectors with wages either below or above the median. As shown in Table 3, the expanding and contracting sectors with wages below the median had roughly the same wage ($408 versus $403). However, there is a very large wage gap of $164, or -19.3%, between the expanding and contracting sectors whose wages are above the median. Thus, the jobs being created in the sectors with wages above the median (about 54% of all jobs) are declining in job quality. Factcheck.org's analysis does not capture this important dynamic.

Real wages are declining within certain sectors.
All of the sector-based analysis, including that of Factcheck.org and our own, misses the movement of real wages within the sectors. Obviously, a job that pays less this year than last year is of lower quality. So far, we have been looking at changing employment levels or shares within industry/occupation sectors, but there has been a striking decline in average hourly wages over the last year. In the same period of time Factcheck.org frames their analysis, June 2003 to June 2004, real average hourly wages have fallen from $15.83 to $15.65. Similarly, real average weekly wages have fallen from $533.58 to $525.84. This decline is due to a combination of: (a) the lingering effects of the jobless recovery and the considerable existing labor slack that has lowered workers' bargaining power; (b) rising inflation that lowers workers' purchasing power; and (c) the fact that faster-growing industries pay less, on average, than shrinking or slower-growing industries.

Though inflation has picked up in recent months, this negative trend in real wages is driven largely by the marked slowdown in the growth of wages throughout the U.S. labor market. Nominal wage growth has slowed sharply, from an annual average of 2.9% in the second quarter of last year to 2.1% in the same quarter this year. Inflation over this same period has accelerated from 2.2% to 2.8%. Thus, even if inflation were back to its year-ago level, wages would still be stagnant at best.

If the quality of net new jobs were as good as Factcheck.org concludes, it is difficult to explain why wages for so many workers are falling behind inflation. In fact, our analysis, along with that of others in this debate,5 uses a better metric and comes to the opposite conclusion. The fact that jobs are growing again is unequivocally positive, but their quality remains problematic.

Endnotes

1. See their analysis at http://www.factcheck.org/article.aspx?docID=208. Note that the recent job quality debate has focused exclusively on wages. More broadly, however, the concept could include many other job quality determinants, such as working conditions, potential for upward mobility, and the full spate of benefits (health care, pensions, etc.).

2. The BLS data provide only median earnings (not averages) but this neither affects our approach nor influences our results.

3. In the table, see that the sum of the contracting shares equals -2.238% where the sum of the expanding shares equals 2.238%.

4. One way to see how this works is to look at a particularly high-wage sector that is losing a large share (e.g., management occupations/manufacturing, where the average wage is $1,125 and its weight is 6.4%) and compare it to a particularly low-wage sector that is gaining a large share (service occupations/professional and business services: average wage is $325 and its weight is 4.5%).

Since both have large shares, their influence on the average is significant, but since their wages are at opposite sides of the spectrum, they can drive the average wage down or up, respectively.

5. See Roach, Steven. "More Jobs, Worse Work." The New York Times, Op-Ed. July 22, 2004.

Download a Microsoft Excel spreadsheet containing the full data set used in this Issue Brief.