A federal appeals court panel on Monday erased the forced-labor conviction of a Michigan resident who made four relatives cook, clean dishes, wash clothes and perform other chores.

The defendant, Jean Claude Kodjo Toviave, originally from the West African country Togo, beat the relatives when they misbehaved or failed to follow the rules imposed on them, the U.S. Court of Appeals for the Sixth Circuit said.

Toviave, the appeals court said, struck the victims with “plunger sticks, ice scrapers and broomsticks.” The court called Toviave’s behavior “deplorable” and “reprehensible.” The federal government, however, overreached in bringing the case, the panel concluded.

“The mere fact that Toviave made the children complete chores does not convert Toviave’s conduct—what essentially amounts to child abuse—into a federal crime,” Sixth Circuit Judge John Rogers wrote. The facts in the case, the judge said, “amount to nothing more than household chores.”

From the Sixth Circuit’s ruling:

First, forcing children to do household chores cannot be forced labor without reading the statute as making most responsible American parents and guardians into federal criminals. Second, requiring a child to perform those same chores by means of child abuse does not change the nature of the work. And third, if it did, the forced labor statute would federalize the traditionally state-regulated area of child abuse.

Michigan state authorities investigated Toviave, a University of Michigan janitor who also worked as a tennis instructor, on child-abuse claims and removed the children from his home, the appeals court said. The U.S. Department of Homeland Security got involved to find out whether the children illegally entered the country. Federal agents who executed a search warrant at Toviave’s home reported finding false immigration documents.

Toviave pleaded guilty to mail and wire fraud charges; in exchange, prosecutors dropped a human-trafficking charge. He was found guilty at trial on four counts of forced labor and sentenced to a little more than 11 years in prison. The four victims testified at trial, the Detroit Free Press reported.

The trial judge had ordered $130,000 in restitution for the victims—an amount that represented the minimum wage for the number of hours worked. In 2013, at the time of Toviave’s sentencing, a Homeland Security official called the prison sentence “a victory not only for the young victims in this particular case, but also for human dignity.”

The appeals court in its decision examined hypothetical scenarios in which parents, under the federal government’s interpretation of the forced labor law, could be prosecuted for “innocuous, widely accepted” rights to require children to fulfill household chores—taking out the garbage, for instance, or mowing the lawn. “The forced-labor statute could not have been intended to overturn this long-standing parental right,” Rogers wrote.

The panel distinguished the case against Toviave from other prosecutions that included household-based forced-labor charges. The Fifth Circuit in 2011 upheld a forced-labor conviction in a case in which the defendants prohibited a “poor, illiterate, Nigerian widow” who performed household chores from “making contact with outsiders.” The Seventh Circuit in 2008 sustained a forced-labor conviction of two physicians who restricted travel by their Filipino housekeeper.

In Toviave’s case, the Sixth Circuit said, “the parties appear to agree that the children spent a substantial amount of time at school, doing homework and studying.”

The court said Toviave “had an almost single-minded fixation on making sure the children got an education,” hiring an English tutor for them.

“One may ask what harm is done if child abuse without forced labor is deemed to be a federal crime,” wrote Rogers, joined by Senior Judge Richard Suhrheinrich and Judge Jeffrey Sutton. “The harm is the federalization of state law.”

Toviave’s appellate lawyer, Christopher Keleher, wrote in court papers in the Sixth Circuit:

While Toviave’s conduct was many things—extreme, outrageous, and cruel—it was not forced labor. As a juror correctly recognized, child abuse is a state offense. But blinded by Toviave’s barbarism, the prosecution lost sight of the fact. Extra schoolwork is not a labor or service. Keeping one’s room and house clean is not a labor or service. A ban on visitors and strict discipline is not a labor or service. Toviave derived no benefit from imposing these rules.

Sentencing law scholar Douglas Berman, writing at his Sentencing Law and Policy blog, said the Sixth Circuit ruling “reinforces my fear that that modern federal prosecutors may often have too much discretionary criminal justice power as well as my optimism that a wise modern judiciary can and will often play a critical role in checking that power.”

Contact Mike Scarcella at mscarcella@alm.com. On Twitter: @MikeScarcella.