Civil rights attorneys are threatening to sue U.S. Rep. Peter King, R-New York, who represents parts of Long Island, over his decision to block users from interacting with his political Facebook page, calling it an “affront to the First Amendment” of the U.S. Constitution.

The New York Civil Liberties Union said in a letter to King this week that if he didn’t return access to the Facebook page for those users, they would file a lawsuit in federal court on behalf of a handful of those individuals.