Hundreds of thousands of Americans may have had their tax refunds wrongfully taken by the federal government, according to a class action filed Wednesday in Washington federal district court.

The lawsuit accuses the Social Security Administration of taking steps to redirect tax refunds without providing the required notice or explanation. The refund money was used to satisfy debts allegedly owed to the Social Security Administration for past overpayments of benefits. In some cases the alleged debts involved survivor benefits that plaintiffs received decades ago as children.