May 14, 2015 6:14 AM EDT

Revolutions do not generally start at bedtime. Who has the energy? Bedtime is for unwinding, taking stock, being told a story. But push too far past bedtime–get punchy, get wired–and things change, reality distorts, rules get broken.

Admittedly, staying up to watch a late-night show on TV does not feel like an insurrectionist act. But in the 1980s era of Late Night With David Letterman, crowding around the common-room TV with my college dorm mates, we had the sense that something amazingly wrong was happening. Late Night was like something that had snuck onto TV, an anomaly you’d better watch quick before somebody noticed and fired the disgruntled control-room staffer who switched it on.

Letterman was a baby boomer (born 1947), but he felt like a Gen X-er: sarcastic, alternative, outnumbered. Our parents watched Johnny Carson to be lofted into bed on the gentle arc of his golf swing. We watched Dave to be kept awake. Late Night was not a sleeping pill but an Alka-Seltzer suit you put on to jump into a water tank, letting it foam and effervesce around you.

It was an inside job, a nightly feast on the hand that fed. In 1986, when General Electric bought NBC, Letterman showed up at GE headquarters, Michael Moore–style, with a fruit basket, and got rebuffed at the door: “Aw, this is going to be fun to work with these people, isn’t it?” He was chucking watermelons from the inside, doing liposuction on a bloated profession and bringing it back to its anarchic roots. David Letterman was punk rock.

What was he rebelling against? the thing to remember is that Letterman came on the air at a time when TV was bad. Not all bad–in the ’80s, NBC launched Hill Street Blues, Cheers, The Cosby Show and Seinfeld. But “TV,” as a business, as a concept, was still in the blow-dried, Ted Baxter era of least-objectionable programming and leisure-suited phony bonhomie.

Letterman was a native-born speaker of TV-ese, and he turned that language against itself. One of the most repeated early-Letterman anecdotes is from his stint as an Indiana weatherman, when he predicted “hailstones the size of canned hams.” When the young comedian got his first national shot–The David Letterman Show on NBC in 1980–he and his then girlfriend and head writer, Merrill Markoe, created a talk show for TV’s native latchkey children–absurd, winking and referential. Letterman wheeled out Steve Martin for an interview in a bed littered with beer cans. He hosted a 50th wedding anniversary for a Long Island couple, at which the plastic flowers caught fire and had to be doused by stagehands. Art-comic Andy Kaufman stared him down in discomfiting silence for a solid minute, wiping his nose.

Did I mention the show aired live? At 10 in the morning? In place of Hollywood Squares? The program won two Emmys, and it was canceled in four months. NBC wisely decided this kind of thing might work better 9½ hours earlier.

The first face you saw on Late Night With David Letterman, which debuted Feb. 1, 1982, was not Dave’s. It was that of Calvert DeForest, the doughy actor who played mascot Larry “Bud” Melman, intoning in his David Lynchian warble, “We are about to unfold a show featuring David Letterman, a man of science who sought to create a show after his own image without reckoning upon God!”

It was a mission statement: the freaks would come out at night, the restless, the jobless, the up-too-laters. Late Night was like a national Rocky Horror midnight movie. It aired from New York City, not sunny Burbank, and played up its home’s Koch-era grime. It was a talk show, sure, with a monologue and guests and music (Paul Shaffer combining both the Doc Severinsen and Ed McMahon roles). But it was mainly a comedy show, informed by the alterna-comics of the ’70s, like Bill Murray, Letterman’s first guest.

There was an element of self-deflation in it all. Letterman would fill silence with an exaggerated “My oh my, we’re having some fun now!” He would send index cards flying at the backdrop with a crashing-glass sound effect for the “windows” that weren’t there. Look at us, was the message, on a fake set, doing the things people do on TV. The monster that mad-scientist Letterman created was sentient, a talk show that knew it was a talk show.

But above all, it was fun. “Television is a kind of toy,” Letterman once told the New York Times, and Late Night had a local-access spirit of playing without adult supervision. It mounted cameras on wires, on Dave’s head, on a monkey. It turned oddball staff writer Chris Elliott into a cult star playing characters like the Guy Under the Seats. Letterman had Teri Garr take a shower in the greenroom. He bathed in 250 gallons of onion dip and threw stuff off a five-story tower. Stupid Pet Tricks, Stupid Human Tricks, Viewer Mail, Top 10 lists–those early years flung a lot of things against the walls, as if they were Letterman’s Velcro suit, and many of them stuck.

The show invited outsiders in; Letterman’s talks with graphic novelist Harvey Pekar (recounted in the movie American Splendor) were transfixing, Pekar railing against the spectacle even as he participated in it. Letterman maintained an outsider persona, airing gripes with NBC and crashing Today with a bullhorn. (“I am Larry Grossman, president of NBC News, and I’m not wearing pants!”)

But here’s the secret: like most passionate rebels, Letterman loved what he was revolting against. He was a classicist, a TV scholar. (His bizarre suit stunts, for instance, were borrowed from Steve Allen’s “suit of tea bags.”) Even as he lampooned the guy-at-a-desk format, he loved being the guy at the desk.

True, love was not a word most would associate with his persona. Letterman became known as late night’s Ice Man. (Literally, in his CBS years, he kept the Ed Sullivan Theater cooled to well below 60°F.) He was private, curmudgeonly, by his own admission self-flagellating. There was an edge of anger to his comedy; when Cher called him an “asshole” to his face, she had a point. But when he was in the zone, was there a more terrific smile in the business? His face would open up, gap-toothed, relaxed, at home.

So of course the antihost wanted to become the host, to sit in his idol Johnny’s chair on Tonight. We know how that turned out: the dealmaking, the skulduggery, the falling out with Jay Leno (once one of Dave’s most reliably hilarious guests). The whole situation went terribly, then well, then terribly again, and finally for the best.

Debuting on CBS in 1993, Late Show with David Letterman was Late Night but bigger, with a stronger engine under the hood. For a couple of years, it was wildly successful; then something shifted. Maybe the show was too agitating for an earlier audience, too smoothed out for his old fans or both. Maybe Letterman, late night’s great alternative, was simply never going to be for everyone. His poorly reviewed “Uma/Oprah” stint hosting the 1995 Oscars symbolized the impossible task of being both insider and outsider at once.

Later in the ’90s, the Late Show seemed to drift–the audience could feel it, Letterman could feel it. America had voted for “Jaywalking” and “The Dancing Itos.” And except for a brief stint against Conan O’Brien in 2009, Letterman would never dominate the ratings battles again.

Yet he won the war. Two decades after Letterman left NBC, no one is out there doing a version of Leno. But an entire world of comedy, talk and media learned to do Dave. His influence isn’t only in his overt acolytes, like Jimmy Kimmel and O’Brien (who carried on the traditions of experimental comedy and losing The Tonight Show to Jay Leno). You see it in the referential humor of ESPN anchors and the deadpan delivery of public figures like President Obama. It’s in the DNA of The Daily Show (which nurtured Letterman’s eventual replacement, Stephen Colbert). It’s in viral comedy, from the cringe interviews of Zach Galifianakis’ Between Two Ferns to Adult Swim’s bizarro sitcom-credits parody “Too Many Cooks.”

Above all, Letterman shaped the affect popularly mislabeled as irony: snideness, sardonic distance and hyperawareness. To some detractors, Letterman was the culture’s Typhoid Mary of nihilism. In David Foster Wallace’s short story “My Appearance,” an actress is coached on how to succeed on Late Night: “Laugh in a way that’s somehow deadpan. Act as if you knew from birth that everything is clichéd and hyped and empty and absurd, and that’s just where the fun is.”

But Letterman did care. He used irony in defense of authenticity, sarcasm as a weapon against fakery. It was several years after he moved to CBS that Letterman managed to fuse his two sides–the smart-ass and the softie–and his second great period began.

Letterman’s revival started when his heart almost stopped. In 2000, returning to the air after emergency quintuple-bypass surgery, he brought out the doctors and nurses who saved his life, choking back tears. If the surgeons enlarged his heart a size or two while they had him open, the procedure took. After the 9/11 terrorist attacks, he was late night’s cathartic voice: “We’re told they were zealots fueled by religious fervor. And if you live to be a thousand years old, will that make any sense to you? Will that make any goddamn sense?” He said goodbye to singer Warren Zevon, dying of lung cancer, in a public deep dive into mortality and meaning. As Zevon summed it up: “Enjoy every sandwich.”

This wasn’t a wise guy getting sappy in his old age. This was an artist mastering his instrument. As he said, composing himself for his 9/11 monologue: “I just need to hear myself talk for a couple minutes.” Late-era Letterman was still a snarkmeister, but he also became a kind of spoken-word essayist. It could be disturbing, as when he admitted affairs with female staffers after a 2009 blackmail attempt. It could be sweet, as when he shared late-life fatherhood stories about his son Harry. It could be amusing, as when he spun a story about waking up one morning to find a bear in his Montana ranch house. “I don’t know how big,” he told Shaffer, “but he was the biggest bear I’d ever seen in my kitchen.” Increasingly, it could be political, most recently when he called out a law in his home state allowing businesses “religious freedom” to deny service to gay customers: “This is not the Indiana I remember as a kid.”

After all those years of invention, Letterman’s last experiment was to be himself, just a voice in front of a crowd telling a story. But in a way, Late Night renegade Dave and Late Show raconteur Dave were the same–one guy, without a map or a rule book, figuring it out in public and inviting you to watch. On May 20, he exits late night through a busy revolving door: Leno’s gone, Jon Stewart is going. He started as the single alternative to The Tonight Show; now there are talk shows on Comedy Central and Bravo. Soon Chelsea Handler will have one on Netflix.

Like all successful revolutions, Letterman’s ran its course. The idea of making a late-night show as a rebellion seems as ancient as VHS tape. (O’Brien, the grand old man of alterna-comedy, is now a niche product on TBS.) The guiding spirit now is Jimmy Fallon, a versatile performer and tireless glad-hander. Successful comics have a way of connecting forward a generation, and just as early Letterman channeled Gen X alienation, Fallon’s good-time variety showcase on Tonight exudes a millennial no-haters positivity. CBS’s new Late Late Show host, the chipper, ingratiating James Corden, owes far more to Fallon than to Letterman.

Colbert–also an ironist powered by a nuclear core of sincerity–has the best shot at carrying on Letterman’s legacy. But just as Letterman was never going to be another Johnny, no one is going to be another Dave. That’s partly just the business: late night is too atomized, diced into Facebook-shareable clips. And it’s partly because over three decades ago, while it seemed no one was watching, one gap-toothed punk genius decided to teach TV some stupid new tricks.

This appears in the May 25, 2015 issue of TIME.

More Must-Read Stories From TIME

Contact us at letters@time.com.

Read More From TIME
You May Also Like
EDIT POST