'After the J.D.' Study Offers Weak Evidence of J.D.'s Value

'After the J.D.' Study Offers Weak Evidence of J.D.'s Value Matt Leichter

At the ABA’s midyear meeting in early February, the Fellows of the American Bar Foundation presented the preliminary culled findings from the third wave of the “After the J.D.” study (AJD), according to the ABA Journal.

That study tracks a robust sample of more than 3,000 people who passed a bar exam in 2000 after graduating from a wide variety of law schools, including some that were unaccredited. This cohort was first surveyed in 2003 (AJD I) [ PDF], again in 2007 (AJD II) [ PDF] and, most recently, 2012 (AJD III).

The good news to be gleaned from the latest batch of preliminary findings: On a scale of 1 to 7, with 7 being highest, respondents gave the statement “I would go to law school if I had to do it all over again” an average rating of 4.91. Those queried gave relatively higher ratings to questions about whether they felt attending law school had been a good investment and whether they were satisfied with their decision to become lawyers.

The not-so-good news is that as of 2012, only a minority of respondents were still in private practice, and roughly one in four had left the practice of law entirely (by contrast, most respondents were in private practice in 2003). Moreover, the gap in median full-time earnings between graduates of elite law schools and those who earned their degrees at less reputable schools had widened considerably, as had the gap in earnings between graduates of nonelite schools based on GPA.

Reading the ABA Journal’s account of the presentation, especially its comparisons to the AJD I findings, might lead one to conclude that although those who passed the bar in 2000 are doing fairly well today, they didn’t escape the Great Recession’s effects lightly. In truth it’s hard to say how the financial panic affected the AJD cohort until more data become available, but the available evidence shows that the attrition process was in effect as of 2007—an indication that the legal market’s structural problems had already begun to surface by then.

Somewhat surprisingly, the lead AJD researchers viewed the second batch of data produced by the study more optimistically. In a paper titled “Buyer’s Remorse? An Empirical Assessment of the Desirability of a Lawyer Career” (“Buyer’s Remorse”), those researchers argue against the belief advanced by some legal academics that law students suffer from “optimism bias” that leads them to take on excessive debt to attend law school on the expectation that they will find high-paying jobs in the legal industry. After the reality of the legal job market sets in, supposedly, the students come to regret their decision.

To summarize: In “Buyer’s Remorse,” the researchers offer several reasons to be wary of the optimism bias hypothesis.

First, critics focus too much on short-term career metrics such as debt and starting salaries rather than considering the data on long-term career outcomes that the AJD provides (the authors dismiss empirical research on law graduate and lawyer oversupply as “complaints from law school graduates”). Specifically, respondents’ incomes across all practice sizes grew significantly over the inflation rate between AJD I and AJD II, and some lawyers were able to move to larger firms during that time period.

Second, although there was a clear relationship between student loan debt and the prestige of the law school attended, graduates of less reputable schools could decrease their debt-to-income ratios more quickly than graduates of more prestigious schools.

Third, AJD II survey data show that lawyers are generally satisfied with their career choices. Thus, the authors argue the AJD cohort’s success demonstrates that law school critics are confusing the effects of the Great Recession with structural changes in the legal sector (though law school certainly costs more than it did pre-2000).

While the optimism bias hypothesis doesn’t need my help, a careful look through the AJD reports and “Buyer’s Remorse” calls the researchers’ conclusions into question: For a lot of people who took the bar in 2000, law school was not obviously the best option for them, irrespective of their own opinions to the contrary.

Earnings and Job Mobility

To start, the AJD participants’ full-time earnings growth presented in the AJD II findings and “Buyer’s Remorse” is nothing short of fantastic. Compare the total money earnings of 25-to-34-year-old professional school graduates who worked full-time year-round from the Census Bureau’s Person Income tables to the AJD II’s median full-time incomes in Table 5.1 (AJD II used 2006, not 2007, incomes).

Bachelor’s Degree $41,516 $44,125 +6.3%
Master’s Degree $48,615 $51,244 +5.4%
Professional Degree $64,158 $61,352 -4.4%

How can professional school graduates’ earnings generally decline while the AJD cohort’s boomed? There are three reasons.

First, though the AJD researchers never define “full-time,” it appears to exclude people who worked two or more jobs to make a living. Meanwhile, the Census Bureau uses a looser definition: at least 35 hours of work per week for 50 weeks a year.

Second, the AJD II table doesn’t provide the number of workers in each employment category for either year. I’ll discuss the attrition issue below, but a lot of the participants left large firms for lower-paying occupations, meaning their incomes probably stagnated or declined.

Third, the Census Bureau’s tables have a wide confidence interval for professional school graduates due to a small sample size, making those numbers less reliable. For example, the median income for professional degree holders in the 25–34 bracket was $70,000 in 2005. The median full-time income for an AJD II participant, meanwhile, was nearly $30,000 greater than that—and yes, doctors generally make more than lawyers.

There are two other striking comparisons to make between the Census Bureau and AJD earnings data. AJD participants earned more when working in nonlawyer business positions than lawyers in many practice areas did. Also, depending on their employment category and percentile, many lawyers made less money than the median bachelor’s or master’s degree holder in the same age bracket, a topic I’ll discuss shortly.

As to attrition rates, even by 2007, 16.5 percent of AJD respondents reported not practicing law. In 2003, that figure was 9 percent (AJD II, Table 3.1). By combining tables 1 and 2 of “Buyer’s Remorse,” we see which employment categories saw the most attrition by law school reputation:

PRACTICE AREA TOP 10 11 – 20 21 – 50 51 – 100 TIER 3 TIER 4 TOTAL
03 07 03 07 03 07 03 07 03 07 03 07 03 07
SOLO 0% 2% 3% 5% 2% 6% 5% 8% 5% 10% 8% 18% 4% 8%
SMALL FIRM 3% 7% 14% 11% 22% 18% 25% 21% 31% 21% 37% 26% 23% 18%
21-100 FIRM 6% 6% 11% 10% 13% 12% 16% 11% 15% 11% 9% 6% 13% 10%
LARGE 101-250 FIRM 11% 5% 14% 8% 9% 7% 7% 6% 6% 4% 4% 2% 8% 5%
MEGA 251+ FIRM 55% 29% 37% 19% 22% 15% 13% 11% 8% 10% 3% 2% 20% 13%
GOV’T LEG. SERV. & PUB. INT. 11% 16% 10% 14% 19% 15% 19% 19% 17% 17% 22% 18% 17% 17%
NON-PROFIT 10% 15% 6% 11% 6% 9% 6% 6% 4% 6% 7% 10% 6% 9%
BUSINESS 2% 13% 2% 15% 4% 11% 4% 10% 8% 12% 5% 8% 4% 11%
OTHER 2% 6% 3% 7% 3% 7% 5% 9% 6% 9% 5% 10% 4% 8%
PRACTICE AREA TOP 10 11 – 20 21 – 50 51 – 100 TIER 3 TIER 4 TOTAL
SOLO 2% 2% 4% 3% 4% 10% 4%
SMALL FIRM 5% -3% -4% -4% -10% -11% -5%
21-100 FIRM 0% -1% -1% -5% -3% -3% -3%
LARGE 101-250 FIRM -6% -6% -2% -1% -2% -2% -3%
MEGA 251+ FIRM -27% -17% -6% -2% 2% -2% -7%
GOV’T LEG. SERV. & PUB. INT. 6% 4% -4% 0% 0% -4% 0%
NON-PROFIT 5% 5% 3% 0% 2% 3% 2%
BUSINESS 11% 14% 7% 6% 4% 4% 7%
OTHER 4% 4% 4% 4% 3% 5% 4%

(Note: AJD bases law school prestige on the U.S. News & World Report law school rankings for 2003.)

By the time of AJD II, the percentage of Tier 3 graduates in megafirm positions grew, but that’s probably a fluke and certainly not an outcome around which to plan a career. By contrast, what law graduates can bet on is that they’ll leave larger firms for smaller ones, start their own practices, and leave private law practice for business and other positions. Graduating from an elite law school amplifies these outcomes.

Additionally, there isn’t as much “upward mobility” for lawyers as the researchers claim in “Buyer’s Remorse.” For example, the authors write, “[W]e see that 38 percent of AJD lawyers who began their careers in medium firms have moved to large firms by wave 2,” yet looking at table 4 in “Buyer’s Remorse,” it appears that only 33 out of 192 lawyers (17 percent) made such a move. Table 4 shows just how rigid the employment structure is for AJD lawyers. Undoubtedly it will look starker when AJD III is published.

Student Loans

The discussion about the AJD cohort’s student loans is largely moot because the student loan system has changed dramatically in the intervening years. When they started law school, AJD participants could cover in-state tuition costs at public law schools with Stafford loans alone, and only half of the private law schools’ full tuition charges required reliance on private loans or some other funding source to cover a shortfall.

Today, the government lends as much money to law students as their schools are willing to charge plus living expenses. When AJD participants began law school, they could expect to discharge their student loans in bankruptcy after seven years without showing an undue hardship. By contrast, today’s law students’ debts are mostly nondischargeable, but they are almost all eligible for income-sensitive repayment plans. The academics behind the optimism bias hypothesis tend to see such repayment plans as soft defaults and are concerned with the tax penalties that law school debtors will face in 20 years as well as the inevitable write-downs the government will take on the loans.

That’s not to say the researchers’ assessment of the AJD cohort’s student debts is sound. For example, in “Buyer’s Remorse” they focus on changes in respondents’ debt-to-income ratios, finding that graduates from less prestigious law schools were more aggressively paying off their debts. But a glance at the debt-to-income ratios themselves reveals something else: Graduates from unranked law schools were more likely to owe more, earn less and work in lower-paying positions like small practices. These are indicators that even in the late 1990s, these law schools—notably the private ones—offered poor long-term opportunities.

Career Satisfaction

The researchers who prepared “Buyer’s Remorse” tout the AJD participants’ survey responses that depict satisfaction with their choice to become lawyers as evidence that they don’t suffer from “optimism bias.” There’s nothing wrong with the respondents’ career satisfaction, but when discussing the value of a law degree, it’s irrelevant at best. What matters are the alternatives that AJD participants had at the time they made the choice to go to law school.

As the full-time salary data above prove, it’s clear that many of the AJD participants would have earned more (and possibly worked fewer hours) if they had gone back to school for a bachelor’s or master’s degree in a more marketable subject. It’s not as if doing so would have been difficult: In AJD I three-quarters of the respondents indicated that they graduated in the top quarter of their college classes. Additionally, the opportunity cost of attending law school in the 1996–2000 period was quite high because jobs were plentiful for college graduates then. The overall earnings for college graduates under 35, including part-time workers, was higher then than it is today, adjusting for inflation. In calculating the value of their law degrees, many AJD participants sacrificed much and gained very little.

Consequently, instead of discrediting the notion that the legal education system suffers from structural defects, the AJD evidence portrays a profession that wastes substantial amounts of human capital. It may be true that lawyers, particularly public service attorneys, are happy to work for less than they would make in other careers, but the AJD data don’t suggest the participants were fully informed of the choice they were making. If it’s not optimism bias, then certainly some kind of “labor market illiteracy” encourages people into the legal profession when their talents could be better utilized in other careers. The law schools and the law firms, for their parts, bear little risk of poor human capital misallocation.

Disturbingly, the AJD surveys might simply show that participants suffer from “choice-supportive bias” and “outcome bias”—to name other cognitive biases—that help people rationalize poor choices long after they have been made. These biases make it harder to identify and resolve shortcomings with lawyer training.

Of course, the Great Recession has reduced recent law school graduates’ career outcomes for the worse. No one seriously denies cyclical effects on lawyer employment, but there is some debate over how long those effects will linger. What is true, however, is that even in what one of the AJD researchers referred to as the “golden age” for law school graduates, things weren’t as wonderful as they appear. It’s unfortunate that the AJD’s lead researchers aren’t aware of the serious questions their work raises. For example, how many law schools whose graduates are apt to find themselves in small practices prepare their students for such careers? Does the large proportion of lawyers entering small law or leaving the law altogether imply there are too many law schools? Do elite law schools inform their students that they will most likely leave large law firms after a few years? To what extent does a legal education specifically prepare lawyers to work as nonlawyers? In what ways does it do so better than business school or other programs? How can would-be lawyers be better informed of the risk of a poor decision to go to law school? Shouldn’t law schools bear more of those risks?

These questions and many others the AJD illustrates can’t be answered if the structural forces affecting the profession are ignored.

Matt Leichter is a writer based in Brooklyn, New York. He received his dual degree in law and international affairs from Marquette University. He operates The Law School Tuition Bubble, which archives, chronicles and analyzes the deteriorating American legal education system. It is also a platform for higher education, student debt and fiscal reform.