Correction: An earlier version of this article incorrectly identified the federal district court where Charles Breyer, whose brother is Justice Stephen Breyer, is a judge. Charles Breyer sits on the U.S. District Court for the Northern District of California.

The growing pace of recusals by U.S. Supreme Court justices this term is fueling new concern about the court’s ability to function at full strength.