Attorney General Eric Holder Jr. hasn’t formally announced whether he wants to remain at the helm of the U.S. Justice Department, but there’s already change under way in the top ranks below him — and talk around Washington has some division leaders mulling their next moves.

Christopher Schroeder, who ran the department’s Office of Legal Policy, the section that supervises the vetting of judicial nominees, is returning to Duke Law School. Holder’s chief of staff, Gary Grindler, who served as acting deputy attorney general in 2010 after David Ogden returned to Wilmer Cutler Pickering Hale and Dorr, announced his departure last week.