A South Florida man who wanted to build a mid-century modern style mansion in Palm Beach lost out with a majority opinion from the U.S. Court of Appeals for the Eleventh Circuit, which found Tuesday that the house did not conform to the Palm Beach’s Architectural Review Commission’s regulations because it was “not in harmony” with neighboring properties.

It’s a ruling that highlights the breadth of authority that architectural commissions and other entities have to determine zoning regulations.