The type of number that often surfaces in discussions about gender equality:

77%–the overall ratio at the Bank of women’s wages to men’s

The surprising number from a new working paper by Jishnu Das, Clement Jobert, (both from DECRG) and Sander Florian Tordoir (from HR):

96%–the ratio of women’s wages to men’s among staff who entered the Bank at grade GF and have worked here for 15 years

You can find the paper, “Compensation, Diversity and Inclusion at the World Bank Group” at this link:

http://documents.worldbank.org/curated/en/181701494593406327

It is well worth a look. I will summarize a small subset of their results.

It Takes Research to Produce Good Numbers

Too often, the policy discourse on economic development reminds me of the scene in the movie Amadeus in which the Emperor complains that Mozart’s music has “too many notes.” We have too many words.

In 2008, the US Embassy demonstrated the power of an alternative strategy–publishing a few good numbers. Staff started posting on the embassy’s website the measurements they were already collecting on PM2.5, the concentration of the smallest and most dangerous airborne particles. This number, updated hourly, changed the discourse on air pollution in China and is now changing policy. It did not need to be dressed up in a long-winded report.

What does it take to produce a few good numbers? Hard work, open disagreement, clearly stated assumptions, a belief that evidence wins any fight with theory, and a commitment to publishing results no matter whose feathers they might ruffle. In a word, it takes research.

Salaries at the Bank for Women Relative to Men

The GF cohort stands out not only because of the relatively small salary gap after 15 years. Among the big entry cohorts, initial hires at this level are closest to a 50/50 gender balance.

The overall ratio of 77% for women’s wages relative to men’s reflects departures from balance for initial hires in the other grades. To make progress toward gender equality, managers in the Bank can examine the initial hires into each grade and respond in two ways:

The obvious contrast to the cohort that enters at level GF is the even larger cohort that enters at level GG. The salary gap for GG’s looks like the one for GF’s–provided that you run time backwards. Women start at an average salary that is about $5000 less than what men receive but then catches up so that after 15 years, the average salaries are almost identical.

It is hard not to be curious about this difference in the trend in the salary gaps for these two cohorts, but for practical purposes the more important difference is the one revealed by Figure 1 (which adjusts two graphs from the paper so they have the same vertical scale.) Female hires at level GG have been and remain much much farther from 50% than hires into level GF.

Figure 1: Hiring into Grades GF and GG

Because the GG cohort is so large, it has a big effect on the overall gender balance of Bank staff. Because GG’s are relatively well paid, this affects the ratio of average salaries for women relative to men.

Because GG hires are older and have more experience, a switch to more hiring at grade GF would imply a transition to a workforce that acquires more of the skill from experience and learning-by-doing via work inside the Bank. It is possible that relying more heavily on internal experience would come at an efficiency cost. But to make a convincing case that the clear advantages from greater gender balance that would follow from more hiring at grade GF level would be outweighed by such costs, one would need to present compelling evidence about the magnitude of these costs. A fact beats a theory every time.

A strategy that switches to more hiring at grade GF might also need to be accompanied by measures that retain Bank staff more effectively. As I’ll note below, the attrition rate is surprisingly high.

Relative to hires at level GF, the other important comparison is to staff who enter in the grades GA-GD where women dominate. In the late 1980s, 90% of initial hires were women. More recently, this percentage has remained in the still high range of 70-80%.

Figure 2: Fraction of Women among Initial Hires, By Grade

Although the gender imbalance in grades GA-GD remains large, its effect on the the overall ratio of women to men (and hence on the overall salary gap) at the Bank has been falling because of a switch toward hiring at other entry grades. From the late 1980s to the 2015, the percentage of all hires who enter in grades GA-GD has fallen in half, from the high 30s to the mid teens.

Figure 2 also shows that based on the recent trends, there are grounds for optimism that the gender balance for initial hires into grades GG and higher can continue to approach 50/50.

Why Has the Average Wage for Women Been Catching Up?

Figure 3 shows that for all Bank employees, women have been catching up with men. In 1987, the average salary for women was around half that for men from part 1 countries. The paper reports that in 2015, the ratio stood at 77%.

Figure 3: Average Salaries for All Bank Staff

Without information on individual differences beyond gender, citizenship, and years of service, we can understand neither the policies that reduced this overall gap nor the options for continuing to reduce it in years to come.

The analysis from the paper that includes information about the entry grade for each individual shows that the reduction in the overall gap is due to success in equalizing the percentages of women and men hired into grades GF, GG and GH and switching away from hiring into grades GA-GD where a large imbalance persists.

How Research Informs the Production of Data

The authors could produce these results because they started with access to a larger and more detailed sample of job histories from the Bank’s HR systems. It took painstaking effort to prepare this raw data for the analysis reported here.

But assembling a useful data set takes more than bookkeeping. Any analysis that distills a large body of data down to a few salient numbers relies on assumptions. Making the right assumptions takes judgment and experience. For the results to be credible, authors must highlight the assumptions they make and show how sensitive their conclusions are to deviations from those assumptions.

One of the most impressive parts of this paper is the clarity with which Jishnu, Clement, and Sander explain the assumption they make about attrition and the sensitivity of their results to deviations from their assumption. The attrition rate is a concern because it is so high. Roughly 8-10% of the staff considered here–those working on the headquarters salary plan–leave the Bank each year. After 15 years, roughly half the people in each entry cohort are gone.

If leaving the bank is independent of a person’s expected future job performance and pay, it makes sense to do what the authors do–track wages for those who remain at the Bank and ignore attrition.

But if attrition is related to expected future salary growth, the interpretation of the results could change. If the people who leave the Bank were to have lower wage growth thereafter, the average salary for those who stay would be an overestimate of the average salary for the cohort. If, in addition, women were more likely to leave, results for those who stay would overstate wage growth for women relative to men and understand the true wage gap.

Figure 4 shows the fraction of the staff that remains at the Bank after 15 years for cohorts that entered between 1998 and 2000. The attrition rate is not independent of the individual characteristics gender, nationality, and entry grade.

Figure 4: Fraction of Each Cohort that Remains After 15 years

This raises concern about whether attrition is independent of expected future wage growth. One reassuring finding based on the salary data alone is that salaries for staff from the same entry grade and with the same years of service are the same for leavers and stayers.

This leaves open the possibility that current pay does not capture some dimension of performance that is related to expected wage growth. As a follow-up test of the similarity of stayers and leavers, the authors collected data on a subsample of years in which they could observe individual performance ratings. Another surprising result that emerges from this analysis is that pay within a salary grade varies substantially with an individual’s history of performance ratings. (See the paper for the details.) The specific point illustrated in Figure 5 is that among people who enter in the same grade and have the same years of service, those who leave and those who stay have the same average pay and the same average performance rating:

Figure 5: Salary and Performance Rating for Attiters and Stayers

Research Is Central to the Bank’s Mission

Humanity’s greatest achievement was the consensus that emerged from the Enlightenment and the Scientific Revolution that statements of fact have an objective truth value that people, working together, can uncover when they follow the lead of Galileo, Newton and Voltaire by

We live in a time when cheap appeals to emotion, accompanied by cargo-cult imitations of science, are making headway in the contest for public attention. As a result, we are having trouble maintaining a consensus around such basics as that parents should vaccinate their children against measles.

The Bank can lead by committing to the practical power of science. It can demonstrate this power by producing numbers that spur action. These numbers must be grounded in fact and logic. They must reveal information about the consequences of the actual choices people have to make.

To live up to this commitment, the Bank must retain its internal capacity for undertaking the full spectrum of work illustrated by the paper by Jishnu, Clement, and Sander–from data collection and preparation, to an analysis that yields preliminary conjectures, then back to the search for disconfirming evidence, ending with numbers that capture the consequences of practical decisions. For these numbers to be credible, Bank publications must also describe all the steps in the production of these numbers with the clarity and precision that these authors use to actually deliver on the oft-made promise of transparency.