In his final State of the Union address to Congress on Tuesday, President =Obama renewed his call to close the Guantanamo Bay prison in Cuba, one of the promises he made repeatedly during his first presidential campaign in 2008.
“I will keep working to shut down the prison at Guantanamo: it’s expensive, it’s unnecessary, and it only serves as a recruitment brochure for our enemies,” Obama said during the address. But shuttering America’s most well-known military prison has proved more difficult than Obama anticipated. Guantanamo, which opened 14 years ago, still holds 103 of the 780 detainees who have cycled through the prison, according to the New York Times, far from the President’s goal of zero.
Read more: How to Watch Tonight’s Republican Debate Online
White House Chief of Staff Denis McDonough said on Sunday that Obama will present Congress with a plan to close the prison soon, and he might use his executive authority to do so if necessary before his term ends. However, the Obama administration still faces a series of obstacles, both old and new, that stand in the way of shuttering the facility.
Congress recently passed a law banning Guantanamo’s closure
Congressional Republicans have repeatedly argued that releasing or transferring prisoners poses a threat to national security, and have fought against the installation’s closure. In the larger defense bill that the President signed into law in November, congressional Republicans included a measure, opposed by Obama, that prohibits the use of funds to close or abandon the prison, transfer detainees to the United States (or Libya, Somalia, Syria and Yemen) or build or modify facilities to house detainees in the United States.
“The restrictions contained in this bill concerning the detention facility at Guantanamo are, as I have said in the past, unwarranted and counterproductive,” Obama said in a statement when he signed bill into law.
The remaining detainees are some of the most difficult to relocate
Many of the 103 remaining detainees require a permanent alternative to Guantanamo because they have not been approved for transfer or release, and the topic of where they could be securely detained or prosecuted is still fiercely debated. As the Washington Post reports, lawmakers have generally opposed housing detainees near their constituents, while others have raised concerns that the prisoners would be additional legal rights if transferred to the U.S. Former Attorney General Eric Holder personally promised that the Justice Department would not relocate detainees to one high-security prison in Illinois, stymieing one administration proposal.
Read more: The Portaits of Gitmo Detainees
His former White House counsel resigned amid criticism over the plan to close the prison
Former White House counsel Greg Craig resigned from his post in 2009 amid criticism for the way he handled Obama’s plan to close Guantanamo, which was met with strong congressional resistance as the administration moved forward quickly amid political divisions.
The plan’s supporters saw Craig’s resignation as a sign that the administration was not as committed to the closure as promised, while others viewed the resignation as an indication that he had failed to anticipate the complex Congressional debate over Guantanamo, according to Politico.
Thawed relations with Cuba haven’t eased the debate
The United States has had the military base in Cuba since 1903, when it signed a deal with the Cuban government allowing it to construct a base in exchange for an annual payment.
Even as ties between the two nations resume, experts think the U.S. military won’t want to surrender the naval base because of its strategic location and proximity to the Caribbean, Latin America and South America — even though Cuba has wanted the land back since Fidel Castro took power more than half a century ago.
The Bowe Bergdahl affair hasn’t made things easier
The controversial exchange of Bowe Bergdahl for five Taliban detainees angered many Republicans and has put Guantanamo transfers in an even more unfavorable light among opponents. Bergdahl was released after five years in captivity in exchange for five detainees who were being held at Guantanamo Bay.
Following the swap, Republican U.S. Sen. Lindsey Graham said he was “very disturbed about the policy of releasing prisoners from Guantanamo Bay while the war is still raging,” saying the decision “undercut the war effort.” Sen. Saxby Chambliss said the prisoner trade “is one of the reasons why a number of us have been so strongly opposed to the release of individuals there.”
The argument has carried over to the 2016 presidential campaign trail, where Republican frontrunner Donald Trump has berated the swap.
“We get a traitor, a no-good, rotten traitor like Bergdahl,” he said. “And they get five killers that they most wanted in the whole world, who are right now back on the battlefield, trying to kill everybody, including us. Okay? What kind of a deal is this?”
More Must-Reads from TIME
- Donald Trump Is TIME's 2024 Person of the Year
- Why We Chose Trump as Person of the Year
- Is Intermittent Fasting Good or Bad for You?
- The 100 Must-Read Books of 2024
- The 20 Best Christmas TV Episodes
- Column: If Optimism Feels Ridiculous Now, Try Hope
- The Future of Climate Action Is Trade Policy
- Merle Bombardieri Is Helping People Make the Baby Decision
Write to Katie Reilly at Katie.Reilly@time.com