How Brad Garrett Ticked Off Bill Cosby, Frank Sinatra and John Gotti

Ideas

May 1, 2015 2:27 PM EDT

America met Brad Garrett as Ray’s goofy brother Robert on Everybody Loves Raymond. Robert may have been bumbling, but he was rarely offensive. Garrett’s stand-up, however, can be a little saucier (and smarter). His new book of personal essays, When the Balls Drop: How I Learned to Get Real and Embrace Life’s Second Half, recounts some moments that left some high-profile comedy targets less than laughter-filled.

Bill Cosby:

When Garrett was still getting started, he got a last-minute call from Cosby, who was guest-hosting Johnny Carson’s The Tonight Show. He agreed to do a spot on the show, and the two took a jet together from Las Vegas. En route, Cosby asked the younger comedian about his act.

“I hear you do an impression of me.”

… “Yes, I do …”

“Yeah, don’t do that tonight on the show,” he said. “Just be yourself.”

But they were just hours away from taping, and Garrett had no other material that he hadn’t used on Tonight Show spots before. So he did his Cosby impression anyway, knowing it would kill (it did).

When I looked over to the Coz behind Johnny’s desk, I could see he had his head down, reading the blue cards that would intro the next commercial. He had stopped watching…

Backstage after the show, he said nothing to me. Silence on the car ride back to the airport. Zero conversation on the jet.

Consider that door closed.

Frank Sinatra:

Garrett frequently opened for Sammy Davis, Jr., which is how he met Frank Sinatra’s people. They booked Garrett to open for Ol’ Blue Eyes on New Year’s Eve 1989, and Garrett was ecstatic. That is, until Sinatra saw him backstage and asked his guys, “Where’s Dreesen?” Tom Dreesen was his regular opener, but he was booked for Glen Campbell that night. “No one told me,” Sinatra said. “Why the hell would you hire someone to open who I’ve never seen before? And on New Year’s Eve?”

Garrett was already sweating when he went onstage, but the act went relatively well. Sinatra booked him again one month later in Atlantic City, where Garrett faced a tough crowd (more about that later). His parting words to the audience were, “You’ve been very … lifelike. Thank you. And stick around for Frank!”

This set off alarm bells for Sinatra. His close friend Jilly Rizzo went backstage after the show and confronted Garrett in his dressing room.

“What did you mean when you said, ‘Stick around for Frank’?”

“Oh, that? That was a silly joke. It meant nothing.”

“Well it musta meant somethin’. It was the last thing you said, right?” …

“I was making fun of myself. ‘Stick around for Frank!’ You know? ‘He’s comin’ out!'” Still nothing.

“But they’re already here. Where are they going? They came to see Frank. Why do they need to stick around for him? … Were you making fun that Frank is sometimes late to a show, like ‘Stick around for Frank,’ as in, ‘He’ll be here soon, don’t leave’ kind of thing?”

“Lord, no. That is not my style, Jilly,” I said. Though, of course, it was.

Just then Jilly looked to Mike, the sleeping mountain, and asked him, “What did you hear, Mike?”

And in broken human, he replied, “I thought I heard, “Stick it to Frank.'”

Now I was envisioning being crammed into the trunk of a Lincoln.

All was forgiven in the end, and the group went out for Chinese food.

John Gotti:

Perhaps the most dangerous enemy Garrett made was an audience member at his Atlantic City gig for Sinatra. He noticed some people weren’t even paying attention to his jokes:

There was a couple so drunk in the front row that they got up and started slow-dancing during my act as the guy sang “The Lady Is a Tramp” to his date. That’s something that stays with you forever. The guy was in a top hat and tails, holding a cane and wearing dark shades. I called him “Mr. Peanut.” His lady had a scary perm that shot out in all directions. “Next time,” I told her, “tease your hair. Don’t piss it off.” The folks around them were not laughing, but the rest of the audience and the orchestra behind the curtain howled.”

Garrett got an explanation after the show: “Turns out ‘Mr. Peanut’ was John Gotti. Apparently, I have nine lives.”

When the Balls Drop hits shelves on Tuesday.

More Must-Reads from TIME