Attorney General Eric Holder proposed one of the largest overhauls of the country's criminal legal system on Monday, pushing for "sweeping, systemic changes" in sentencing that could become a centerpiece of his tenure at the U.S. Justice Department.

During an address to the American Bar Association, Holder announced a major policy shift within the Justice policies and promoted legislation aimed at ending incarceration rates that he called "ineffective and unsustainable."

Existing policies, he said, cause a "significant economic burden" that as of 2010 amounted to $80 billion per year and carries "human and moral costs that are impossible to calculate."

"It's clear…that too many Americans go to too many prisons for far too long, and for no truly good law enforcement reason," Holder told the ABA convention in San Francisco (Read Holder's Remarks). "And it is well past time to implement common-sense changes that will foster safer communities from coast to coast."

Those changes include a new approach to mandatory minimum sentences for drug-related crimes and new guidelines for when criminals should face state or federal charges.

The reaction from the legal and political communities appeared overwhelmingly supportive of Holder's effort. The Civil and Human Rights Coalition called it "the most significant proposal ever put forth by the Justice Department to reform our nation's disastrous criminal justice system," while the American Civil Liberties Union said it is "thrilled by these long-awaited developments."

The president of Justice Fellowship, a conservative criminal justice reform group, said it is time for the country to be more prudent in sentencing.

"The president, his predecessor and literally hundreds of thousands of Americans have violated the nation's drug and alcohol laws at some point in their lives," Craig DeRoche said. "The use of incarceration for petty crimes works against fiscal responsibility, increases dependence on government entitlements, breaks apart families and prevents future self-sufficiency."

Mark Harris, a former federal prosecutor who is cochair of the appellate practice at Proskauer Rose in New York, called it a milestone in the movement away from mandatory minimum sentences.

"What's most remarkable is the reflection of the fact that on both right and left we've come to a consensus, it seems, that mandatory minimum sentencing is a bad idea," Harris said.

Holder announced that he has modified Justice charging policies so that low-level, nonviolent drug offenders no longer would face mandatory minimums unless they have ties to large gangs or cartels.

"They now will be charged with offenses for which the accompanying sentences are better suited to their individual conduct, rather than excessive prison terms more appropriate for violent criminals or drug kingpins," Holder said. "By reserving the most severe penalties for serious, high-level or violent drug traffickers, we can better promote public safety, deterrence and rehabilitation—while making our expenditures smarter and more productive."

U.S. attorneys will craft guidelines to determine when federal charges should be filed—and when they should not—in the interest of being smart and efficient when it comes to battling crime, he said.

"This means that federal prosecutors cannot—and should not—bring every case or charge every defendant who stands accused of violating federal law," Holder said. "Some issues are best handled at the state or local level."

The Department of Justice has updated its framework for considering compassionate release for inmates facing extraordinary or compelling circumstances who pose no threat to the public, Holder said, including revised criteria for elderly inmates who did not commit violent crimes and who have served significant portions of their sentences.

Holder pledged to work with members of Congress and both parties to advance legislation aimed at giving judges more discretion in applying mandatory minimums to drug offenders. He called a bill by senators Dick Durbin, D-Ill., Patrick Leahy, D-Vt., Mike Lee, R-Utah, and Rand Paul R-Ky., "promising legislation" that "will ultimately save our country billions of dollars while keeping us safe."

Leahy, chairman of the Senate Judiciary Committee, said he would hold hearings in September on the Justice Safety Valve Act of 2013, S.619, which would give judges flexibility in all crimes in which mandatory minimum is considered unnecessary.

"While I believe this broad approach is the better way to reform our sentencing laws, I commend the attorney general today for his efforts to combat injustice in drug sentencing and prevent waste of taxpayer dollars," Leahy said in a written statement.

House Judiciary Committee Chairman Representative Bob Goodlatte, R-Va., agreed with many of the policies Holder outlined, but said he should work with Congress to reform the criminal code. Goodlatte highlighted his committee's Overcriminalization Taskforce, which already is "in the process of taking a broad look at our criminal code, allowing for input from experts, and is already considering sentencing and prison reform issues."

The support for Holder's ideas was not universal.

Senator Charles Grassley of Iowa, top Republican on the Senate Judiciary Committee, said whether the law needs to be changed should be decided by the Congress, along with the president.

"Instead we're seeing the president attempt to run roughshod over the direct representatives of the people elected to write the laws," Grassley said. "The overreach by the administration to unilaterally decide which laws to enforce and which laws to ignore is a disturbing trend."

Harris, of Proskauer, said legislation would help clarify one of the big uncertainties about Holder's move—whether lawyers will push this to the next step of retroactively going back to reduce existing prison sentences. Harris led a team of lawyers who won a June 2012 U.S. Supreme Court case that addressed changes in crack-cocaine sentencing laws, representing Corey Hill in the consolidated U.S. Supreme Court cases of Hill v. United States, 11-5721, and Dorsey v. United States, 11-5683.

"One of the tough questions is, what is the mechanism for doing that? Literally, how do you get the case back in court," Harris said. "It would carry a lot more weight if there was a law that went with it."