Southern District Judge Harold Baer’s habit of prodding lead counsel in class actions to staff cases with women and minority lawyers came under fire at the U.S. Supreme Court on Monday. Justice Samuel Alito Jr. dressed down Baer in a six-page statement, not-so-subtly hinting that Baer should knock it off.

The warning came as the high court denied certiorari in the underlying case, an antitrust class action against Sirius XM Radio Inc. that settled in a deal valued at $180 million in May 2011. The unsuccessful cert petition was filed by Theodore Frank of the Center for Class Action Fairness, who argued that Baer’s diversity initiative was improper.