Gowdy, a former federal prosecutor and state attorney, was on the shortlist of contenders to replace Comey.
Gowdy said he spoke with U.S. Attorney General Jeff Sessions about the position and shared with him "my firm conviction that I would not be the right person."
Lawmakers in both parties have urged Trump to choose an FBI director who does not have a political background in order to ensure the job remains nonpartisan.
"Our country and the women and men of the FBI deserve a Director with not only impeccable credentials but also one who can unite the country as we strive for justice and truth. I am confident that person will emerge," Gowdy, a Republican, said in a statement Monday night.
"I want to thank the scores of law enforcement offices and prosecutors who have contacted me over the past couple of days. Perhaps we can work together again in the future."