Tina Fey Says Goodbye to Letterman by Taking Off Her Dress

May 8, 2015 9:13 AM EDT

Tina Fey has been on The Late Show With David Letterman 20 times, and now that he’s retiring, she says she’ll never wear a fancy dress on a late night show again.

“I dress up like this out of respect for you,” she said from the guest seat Thursday night. “This is it, because…what, am I gonna put a dress on for Jimmy? That’s creepy, he’s like my brother.”

Since this would be her last time wearing something nice on a talk show, she decided to give Letterman the dress as a gift. He helped her remove her frock onstage to reveal what she called “almost medical” contraptions of shapewear that read, “Bye Dave / #lastdressever.”

Photos: David Letterman’s Early Career

One of David Letterman's earliest roles before he made it as a television host was as a guest star on Mork & Mindy.CBS/Getty Images

Before he received his own show, Letterman was a regular guest host for *The Tonight Show Starring Johnny Carson*. Above, he is seen interviewing Betty White.Paul Drinkwater—NBC/Getty Images

Truma Capote and David Letterman on the David Letterman show. — In 1980, Letterman finally got his own morning comedy show, *The David Letterman Show*. Above, he is seen interviewing Truman Capote.Time & Life Pictures/Getty Images

Late Night with David Letterman — After the cancellation of his morning show, Letterman was given a new show and time slot, *Late Night With David Letterman* which immediately followed *The Tonight Show Starring Johnny Carson*. The first version of the *Late Night* franchise, debuted on February 1, 1982 and would eventually be succeeded by Conan O’Brien, seen above.Al Levine—NBC/Getty Images

Ed Sullivan Theater which is owned by CBS has the Late Show — Letterman left NBC for CBS to start the *Late Show with David Letterman* when NBC gave the reigns of *The Tonight Show* to Jay Leno instead of him after Johnny Carson's retirement. The first show debuted on August 30, 1993, with Letterman's retirement announced for 2015.New York Daily News Archive/Getty Images

Photos: David Letterman’s Early Career

More Must-Reads from TIME