July 9, 2014 8:17 AM EDT

Just last week, Joan Rivers walked out of a live interview with CNN, frustrated with what she deemed “very judgmental, very nasty, very opinionated, very negative”questions.

But Tuesday night, it was David Letterman’s turn to walk out during an interview with the 81-year-old actress.

As the late night host teased Rivers for her on-camera behavior, he suggested that perhaps the whole affair was a PR stunt to promote Rivers’ new book, Diary of a Mad Diva

. But Rivers said: “just the opposite.”

Watch the clip above for more on the exchange.

Contact us at letters@time.com.

Read More From TIME

Related Stories