January 10, 2017 12:58 PM EST

Billy Joel may have more than 120 different songs in his repertoire, but when Stephen Colbert asked him to name his top five, he seemed to have a pretty definitive answer. During a Monday night appearance on The Late Show, the 67-year-old pop singer chatted about his long hiatus from creating new music before revealing his ranking.

“I actually think about that on when I’m stage — what would I want to see me do?” he said. “I tend to like the album tracks, not the ones that are the hit singles.”

Joel then rattled off his list, which included, “Vienna,” “And So It Goes,” “You May Be Right,” “She’s Right On Time” and last but not least, “Scenes from an Italian Restaurant.”

Watch the full clip below.

Write to Megan McCluskey at megan.mccluskey@time.com.

Read More From TIME

Related Stories