(Photo via iStock)

Correction, 7/8/14, 11:55 a.m. EDT: An earlier version of this story provided inaccurate information on the number of black equity partners at Gibson Dunn & Crutcher. The firm has four. The story has been updated accordingly. We regret the error.

Call me the party pooper. The proverbial fly in the ointment. Yes, this is a celebratory edition of The American Lawyer, the annual A-List issue in which we all raise a glass to toast the 20 best all-around law firms in the country. They are shiny examples of the profession, hailed not only for their financial success but also as exemplars of diversity, social conscience and good morale.

All fine things. So why do I view this laudable venture with fatigue and a bit of dejection? Because once you look behind the scores and self-congratulatory remarks of the winning firms, you get a sense that the winners aren’t quite as stellar as they first appear. The big myth is that the A-Listers have mastered—and exceeded—the fundamental measures of a great firm. (The A–List consists of the highest-scoring firms for revenue per lawyer, pro bono, diversity and associate satisfaction, with double weight given to revenue per lawyer and pro bono.) “Those four measures are good proxies for quality of life indicators,” says James Sandman, former managing partner of Arnold & Porter, an A-Lister seven times (though it didn’t make it this year).

They might be good measures. But firms on the A-List often fail to get good grades in some categories. Two firms—Kirkland & Ellis and Irell & Manella—fell below the average score for Am Law 200 firms (133) in associate satisfaction. (Kirkland scored 122, Irell 126.) Other firms squeaked by with just above-average performance for associate morale, including top ranked Milbank, Tweed, Hadley & McCloy (No. 4 on the A-List), which scored 139 and Morrison & Foerster, which got 135.

Moreover, when it comes to the tricky issue of diversity, few firms stand out. Though they all met the average score for diversity (101), several firms are minimally diverse. One A-List firm, Robins, Kaplan, Miller & Ciresi, barely hit that average, with a score of 105. And if you look at how A-Listers fare with blacks, Orrick, Herrington & Sutcliffe would get an “F.” (It has no black equity partners.) Many more firms are contenders for a gentleman’s “C” for having just one black equity partner. (Among them are Debevoise & Plimpton; Hughes Hubbard & Reed; and Milbank.)

The two categories that A-List firms have little trouble excelling in are revenue per lawyer and pro bono. Firms seem to have made those into top priorities. And speaking of priorities, how do firms fare on the percentage of female equity partners? Though that’s not part of the A-List consideration, I think it should be. After all, women have constituted about half of law school graduates since the early 1990s, and we’re not close to parity.

If female equity partner rates were factored into the scores, several A-Listers would stand out—for the wrong reasons. The average percentage of female equity partners is 16.5 percent, according to our recent NLJ 350 survey. A-Listers such as Orrick, Milbank and O’Melveny & Myers have spectacularly low rates in that category—9.6 percent, 10.5 percent and 11.6 percent, respectively.

How can firms celebrated for their diversity and associate morale have such subpar scores on those very issues? If the crème de la crème of the profession can’t excel on the fundamentals, what example are they setting? Instead of just looking at overall scores, perhaps we should demand that A-Listers actually get an “A” or at least an “A-” in each of the four categories (and let’s add the rate of female equity partners into consideration).

At the same time, firms need to demand more of themselves, taking a deeper look at what success means.

To accept mediocrity from the A-List is grade inflation. And we all know that firms believe in the sanctity of grades, right?