Attorney General Eric Holder Jr. proposed one of the largest overhauls of the country's criminal legal system on Monday, pushing for "sweeping, systemic changes" in sentencing that could become a centerpiece of his tenure in the Department of Justice.

During an address to the American Bar Association, Holder announced a major policy shift within the U.S. Department of Justice and promoted legislation aimed at ending incarceration rates that he called "both ineffective and unsustainable."

Existing policies, he said, cause a "significant economic burden" that as of 2010 amounted to $80 billion per year and carries "human and moral costs that are impossible to calculate."

"It's clear—as we come together today—that too many Americans go to too many prisons for far too long, and for no truly good law enforcement reason," Holder told the ABA convention in San Francisco. "And it is well past time to implement common-sense changes that will foster safer communities from coast to coast."

Those changes include a new approach to mandatory minimum sentences for drug-related crimes and new guidelines for when criminals should face state or federal charges.

The reaction from the legal and political communities appeared overwhelmingly supportive of Holder's effort. The Civil and Human Rights Coalition called it "the most significant proposal ever put forth by the Justice Department to reform our nation's disastrous criminal justice system," while the ACLU said it "thrilled by these long-awaited developments."

The president of Justice Fellowship, a conservative criminal justice reform group, said it was time for the country to be more prudent in sentencing. "The president, his predecessor and literally hundreds of thousands of Americans have violated the nation's drug and alcohol laws at some point in their lives," Craig DeRoche said. "The use of incarceration for petty crimes works against fiscal responsibility, increases dependence on government entitlements, breaks apart families and prevents future self-sufficiency."

Mark Harris, a former federal prosecutor who is co-chairman of the appellate practice at Proskauer Rose in New York, called it a milestone in the movement away from mandatory minimum sentences. "What's most remarkable is the reflection of the fact that on both right and left we've come to a consensus, it seems, that mandatory minimum sentencing is a bad idea," Harris said.

Holder announced that he has modified Justice Department charging policies so that low-level, nonviolent drug offenders no longer would face mandatory minimum sentences unless they have ties to large gangs or cartels.

"They now will be charged with offenses for which the accompanying sentences are better suited to their individual conduct, rather than excessive prison terms more appropriate for violent criminals or drug kingpins," Holder said. "By reserving the most severe penalties for serious, high-level, or violent drug traffickers, we can better promote public safety, deterrence and rehabilitation—while making our expenditures smarter and more productive."

U.S. attorneys will craft guidelines to determine when federal charges should be filed—and when they should not—in the interest of being smart and efficient when it comes to battling crime, he said.

"This means that federal prosecutors cannot—and should not—bring every case or charge every defendant who stands accused of violating federal law," Holder said. "Some issues are best handled at the state or local level."

The department has updated its framework for considering compassionate release for inmates facing extraordinary or compelling circumstances who pose no threat to the public, Holder said, including revised criteria for elderly inmates who did not commit violent crimes and who have served significant portions of their sentences.

Holder pledged to work with members of Congress and both major political parties to advance legislation aimed at giving judges more discretion in applying mandatory minimums to drug offenders. He called a bill by senators Dick Durbin, D-Ill., Patrick Leahy, D-Vt., Mike Lee, R-Utah, and Rand Paul, R-Ky., "promising legislation" that "will ultimately save our country billions of dollars while keeping us safe."

Leahy, chairman of the Senate Judiciary Committee, said he would hold hearings in September on the Justice Safety Valve Act of 2013, which would give judges flexibility in all crimes in which mandatory minimum is considered unnecessary.

"While I believe this broad approach is the better way to reform our sentencing laws, I commend the Attorney General today for his efforts to combat injustice in drug sentencing and prevent waste of taxpayer dollars," Leahy said in a written statement.

House Judiciary Committee Chairman Representative Bob Goodlatte, R-Va., agreed with many of the policies Holder outlined, but said he should work with Congress to reform the criminal code. Goodlatte highlighted his committee's Overcriminalization Task Force, which already is "in the process of taking a broad look at our criminal code, allowing for input from experts, and is already considering sentencing and prison reform issues."

Harris, of Proskauer, said legislation would help clarify one of the big uncertainties about Holder's move—whether lawyers will push this to the next step of retroactively going back to reduce existing prison sentences. Harris led a team of lawyers who won a June 2012 U.S. Supreme Court case that addressed changes in crack-cocaine sentencing laws, representing Corey Hill in the consolidated U.S. Supreme Court cases of Hill v. United States and Dorsey v. United States.

"One of the tough questions is, what is the mechanism for doing that? Literally, how do you get the case back in court," Harris said. "It would carry a lot more weight if there was a law that went with it."

Samuel Buell, a former federal prosecutor who now is a professor at Duke University Law School, said Holder's move was "welcome" and would bring flexibility and consistency to federal drug sentences.

"The truth is that prosecutors in several big city U.S. Attorneys' offices who well understand the difference between serious drug cases and small ones have for years been using this legal tactic for avoiding mandatory minimums in their plea agreements, more or less under the radar," Buell said in a statement.

Jamie Gorelick, a Wilmer Cutler Pickering Hale and Dorr partner who was one of the longest serving deputy attorneys general, called it a significant statement on federal enforcement policy.

"It coincides with a movement within the states, among federal judges, within the sentencing commission, to take a second look at policies that have been in place for a long time," she said. "This is a good moment for everyone to deal with that."

"It is a well thought-out statement to one of the most important audiences to which an attorney general can speak," Gorelick said, adding that former attorney general Janet Reno also reserved her important statements for the ABA audience. "So I think it is a very significant statement and commitment."

Holder's concerns were not out of the blue by any means, said Covington & Burling partner James Garland, who served as Holder's deputy chief of staff for two years.

This is part of an incremental steps the AG taken during his entire career—from his time on the bench, as U.S. attorney for the District of Columbia and even in a 2010 memo giving prosecutors more discretion in charging, Garland said. It's something he's wanted to do since he became attorney general, and one of the issues that has motivated him the most.

"This speech is a very logical extension of what he's built his whole career around," Garland said.