Updated: July 23, 2015 11:16 AM ET | Originally published: July 23, 2015 10:27 AM EDT

The union representing border patrol agents has backed out of a tour with Donald Trump when he visits the U.S.-Mexico border Thursday.

“After careful consideration of all the factors involved in this event and communicating with members of the National Border Patrol Council (NBPC) at the National level, it has been decided by Local 2455 to pull out of all events involving Donald Trump,” the Laredo, Texas branch of the union said in a statement.

Trump’s office put out a statement Thursday saying, “Despite the great danger, Mr. Trump is traveling to Laredo, TX to proceed with the visit to the border” and that Local 2455 “[was] totally silenced directly from superiors in Washington who do not want people to know how bad it is on the border — every bit as bad as Mr. Trump has been saying.”

Trump’s trip to Texas was sure to stir up controversy, as the billionaire real estate mogul and presidential candidate has taken a hard line on immigration since his announcement speech, when he first referred to illegal immigrants as “rapists.”

More Must-Read Stories From TIME

Write to Tessa Berenson at tessa.berenson@time.com.

Read More From TIME

Related Stories