After the relentless pounding the Securities and Exchange Commission has received for failing to expose Bernard Madoff’s fraud, it was probably only a matter of time before someone took the agency to court.

That time came Wednesday, when attorneys at Herrick Feinstein filed a complaint in the Southern District of New York accusing the agency of “serial, gross negligence.” Herrick represents two of Madoff’s victims—disabled retiree Phyllis Molchatsky and Dr. Steven Schneider—who lost more than $2.4 million between them.