The U.S. Supreme Court on Monday turned away a challenge to the constitutionality of the Consumer Financial Protection Bureau.

Even though the Trump administration agreed with the challengers that the bureau’s director has too much authority, the government had urged the justices not to take the case. The Justice Department pointed to the likely recusal of Justice Brett Kavanaugh, who would not have taken part in ruling on the merits.