Warren Buffett, chairman and chief executive officer of Berkshire Hathaway Inc., during an interview in New York City on Sept. 8, 2015.
Chris Goodney—Bloomberg via Getty Images
December 8, 2015 5:23 PM EST

When people first find out that Peter Buffett’s father is Warren Buffett — investing legend and the third richest man in the world — they’re shocked.

They’ll often say, “But you’re so normal!” Peter tells Stephen Dubner on a 2011 Freakonomics podcast.

This normality that people tend to gawk over is unsurprising once you hear about what it was actually like being raised by the “Oracle from Omaha.”

“Growing up, we really didn’t know what my dad did. It was quite mysterious,” the youngest of the three Buffett children tells Dubner.

“In fact, when my sister filled out a form, in fourth or fifth grade, about what our parents did, she put ‘security analysis,’ and it was assumed that what he did was check alarm systems,” continues the composer, musician, and author. “To a kid, it was like, ‘What do all the numbers on the page mean? And what exactly is the New York Stock Exchange, and buying and selling and all that?’ So we really didn’t know.”

The Buffett household maintained modesty and normalcy — “I walked to public school … I had the same English teacher that my mother had,” Peter explains — despite the financial successes that were going on behind the scenes.

And how would Peter and his siblings know what their father did and how financially successful he was, considering they grew up in a modest five-bedroom in Omaha that Buffett bought for $31,500?

It took a quarter century until Peter fully grasped what his father did: “[Finding out] was very gradual … I was probably about 25. The truth is, it just wasn’t around … We didn’t grow up around the exposition of wealth.”

Buffett is famously down to earth — he still lives in the same $31,500 home, and chooses a flip phone over a smart phone.

He also prefers to spend the majority of his day reading, a habit older than his 57-year-old son: “He read a lot, which he still does,” Peter says. “And I will say, if you walk in the house today you will see the same thing that I saw in 1965. He’s just this consistent human being in spades. It was incredible.”

This article originally appeared on Business Insider

More from Business Insider:

More Must-Read Stories From TIME

Contact us at letters@time.com.

Read More From TIME
You May Also Like