The long view of history tends to be the judge of a presidency. As President Obama embarks on a second term in the Oval Office, it may still be too early to draw conclusions about his legacy as Commander in Chief. What we do know is that Obama’s first term has been a historic one: the first African American to hold the country’s highest office, Obama and his Administration have battled a recession, passed health care reform and legislation to end the military’s “Don’t ask, don’t tell” policy, formally ended the war in Iraq and brought Osama bin Laden to justice.

( Barack Obama, 2012’s Person of the Year)

Through adversity and triumph, public victories and private setbacks, chief official White House photographer Pete Souza and his team of photographers have relentlessly documented the actions of the President, the First Lady and the Vice President since Obama took office in early 2009.

As the President runs for a second term, LightBox asked Souza to reflect on his time photographing Obama and share an edit of his favorite images that he and his staff made during the President’s first term; the photographs offer a fascinatingly candid insight into the life of the President while painting a portrait of Barack Obama the man, husband and father.

“I tried to, in putting together this edit, not only to show some of the high points or low points of his presidency thus far, but pictures that help people understand what he’s like, not only as a President but as a human being,” Souza tells TIME. “And how he relates to other people, how he relates to his family.”

(Person of the Year runner-up: Malala Yousafzai )

Souza’s process is aided by his long-standing working relationship with Obama — one that precedes the presidency. They met on Jan. 3, 2005, Obama’s first day in the Senate. For Souza, then a staff photographer at the Chicago Tribune‘s Washington bureau, it was the first day of a yearlong assignment to document the new Senator’s time in office.

As the assignment evolved, Souza — who had worked as a White House staff photographer during President Reagan’s second term — began recognizing something special about the Senator. An inkling of things to come, or potential for the future. He began looking for moments that would prove valuable in the course of history, photographs that would define Obama’s early years to those who only knew his legacy.

“I was looking for things that I knew that if he ever became President you would never see again,” he says. “[Obama was] walking down a sidewalk in Moscow in 2005 and no one recognized him. I realized that if he ever became President, you would never, ever see a photograph like that. The odds of becoming President are obviously pretty slim, but I knew he had the potential. And you can’t say that about too many people.”

Souza continued to photograph Senator Obama, who quickly became presidential-candidate Obama and then Democratic-nominee Obama. With Obama’s 2008 election victory, Souza returned to the White House as chief official White House photographer and director of the White House Photography Office.

The photographs that Souza has taken extend the lineage of White House photography that began in the 1960s, first in a somewhat scattered way during John F. Kennedy’s Administration and then more officially with Yoichi Okamoto, Lyndon B. Johnson’s photographer. Okamoto is considered the first photographer to capture the presidency with an eye for history. Souza is quick to acknowledge and praise his work and that of others who have followed, including David Kennerly (Ford), Bob McNeely (Clinton) and Eric Draper (George W. Bush).

An all-digital workflow is one thing that differentiates Souza’s work from the majority of his predecessors. Although he wasn’t the one to move the process to digital — Draper, Bush’s photographer, made the switch from film to digital — Souza made the first official portrait of an incoming President with a digital camera. The Obama Administration has understood the insatiable appetite for imagery that the digital age has wrought and embraces Flickr as a means of disseminating presidential photography.

The Administration encourages sharing behind-the-scenes photos now, he says. “[It wanted] to establish a way to become more transparent than any other Administration, so every month, we upload a new batch of behind-the-scenes photos. The response has been overwhelming.”

But alongside the ease brought by the digital era came one difficulty: the Presidential Records Act prohibits Souza and his team from deleting any photographs. “One of our bigger challenges is just the storage of all these images,” he says, noting the immense difficulty the team will experience moving millions of digital files to the National Archives at the end of Obama’s tenure.

Souza’s work with the President follows in the golden age of photojournalism’s best traditions, when photographers working for magazines like LIFE established relationships and spent inordinate amounts of time shooting beautifully crafted images of public figures.

“I spend a lot of time with [the President], around him, on vacations, sometimes on weekends, depending on what’s going on. He’s used to me being around,” Souza says. As his friend P.F. Bentley described it, “When the President is on, I’m on. And when the President’s off, I’m still on.”

Souza recalls one meeting that he missed because it had been rescheduled unbeknownst to him. “I was a little upset with the President’s secretary for not telling me that they had moved the meeting up, and [the President] heard us talking and he said, ‘What are you talking about? You were in that meeting.’ He’s so used to me being there that he thought that I had been in the meeting that I wasn’t even in. So I took that as a compliment.”

His access to Obama’s inner circle and day-to-day routine stems from the trust he built during their relationship prior to the presidency. “I’m there to seriously document his presidency. I’m not looking for cheap shots, and I think that’s the kind of relationship any White House photographer should have with the President they’re covering,” he says. “That they have a level of access and trust that will lead to important photographs for history.”

Souza is aware of the significance of the photographs he and his team are taking, but he’s also focused on capturing the small and incidental moments that make the Obama Administration unique. “There are days that you certainly think about the importance of what’s taking place — you’re serving an important role in visually documenting this period of time for history,” he says. “But at the same time, a lot of the pictures that tell you a lot about a President are not [made] during those times. They’re when he’s having a private moment with one of his daughters, or when something unexpected happens that may not be, you know, important in terms of history’s sake.”

“I think that’s what keeps you on your toes. You never know when those moments are gonna occur, because they don’t always occur when big things are happening,” he says. The image of Obama playing in the snow with Sasha and Malia is a testament to Souza’s approach. The photograph is not simply of the President but of a moment shared between a father and his daughters.

These personal images round out Souza’s portrait of the President and give it greater depth. While preparing this edit for LightBox, he acknowledged that it was hard to present what a presidency is about in just a handful of pictures. “I don’t gravitate toward any singular image right now,” he says. “I try to look at a body of work, and so I’m proud of this edit that I submitted. To me, it’s all these photographs together which tell you something about this man, this President, and I guess to a certain extent, about me and what I think is important.”

Although Souza’s edit comprises more than 100 images, it is by no means a comprehensive record of Obama’s time in office. “I’m sure that I left out some important moments,” he says. “I don’t think I included anything from the Nobel Peace Prize ceremony, and that’s historic in itself — he won the Nobel Peace Prize. But it just didn’t fit in with the series of pictures that I wanted to present.”

Says Souza of the President: “He has certainly created history just by being the first African-American President. Hopefully in future generations, we’ll soon have a woman President or a Hispanic President, and it won’t matter that much. But I think that if you’d ask him, he wants to be remembered for the things that he’s done.”

For Souza, it’s difficult at this point to reflect on the last four years and the photographs he and his team have made. “One of the difficult things, doing this every day, is having a chance to really sit back and take it all in. Putting these photos together helped that a little bit,” he says. “You’re a little bit overwhelmed about everything that happened in four years, because a lot of stuff has happened. I hope there will come a time where, when I’m not doing this job any longer, I’ll be able to sit back and reflect on everything that he’s been through and everything that I’ve been through.”