J.R. Rotem on Producing Hits for Gwen Stefani, Empire and the Key to Career Longevity

Music producer J.R. Rotem, who’s crafted hits for everybody from Rihanna to Fall Out Boy, was having quite a week when TIME caught up with him during a break from the studio: he helped Gwen Stefani score her first No. 1 album as the producer of several tracks on This Is What the Truth Feels Like, including the lead single “Used to Love You.” He’s also a music producer for Empire, which returned last Wednesday from its winter hiatus. Below, Rotem chats about navigating more than a decade in the industry, the challenges of writing music for TV and the artist he’s most excited about in 2016.

TIME: Congratulations on having a No. 1 album with Gwen Stefani. You’ve been in the studio with a lot of great people, but what stands out about working with her? I know she journals a lot and will crack open her diaries to look for lyrics.

J.R. Rotem: It’s hard to describe. There’s the tangible stuff—yeah, she writes in her journal, she’ll come into the studio and go, “Hey guys, I had this realization last night.” It’s very much bringing what’s going on in her mind into the studio. It was coming in without a plan, just following whatever emotions we were all feeling. She wasn’t like, “Hey, let’s just try and make hits,” she was really intent on telling her story.

You don’t expect that level of depth or honesty with a superstar. You think she’s going to be on such a high level, but I didn’t really expect that it would be so connected on an emotional and spiritual level too. It was deeper than the music. It’s a combination of how amazing of an artist she is and also what she’s going through right now. It’s not like she’s in this very stable, boring place in her life—it was the most turbulent time!

I make a lot of songs for a lot of people. Sometimes it’s just a couple songs. This was one of those rare experiences where I was producing half the album for an iconic superstar who was going through one of the most transitional things in her life in a very public way. It was just a very unique experience.

(Read next: Gwen Stefani: How Making My New Album Saved My Life)

When you’re working on songs for Empire, what’s the difference between making music for a fictional character and writing with a real artist?

With the show, I’m speaking with the directors and producers of the show. They’re telling me, “We have a scene where Jamal does this, Hakeem does this, and we need a song that fits that scene.” So I’m trying to create songs that fit a scene, that fit what the character is doing. When I’m in the studio with Gwen Stefani or Nicki Minaj, it’s more about telling their story on their album. I’m still producing and guiding [with TV], but there’s more of a specific slot that we need to represent. You’re not going to write the song so on-the-nose that the song is going to explain what the scene is about. You have to find the balance of finding a song that’s artistic and abstract—it has to stand on its own—but at the same time is going to at least loosely tie into that particular story.

Interesting—I would have guessed that you would have more freedom. You don’t have an artist coming in and saying, “I want to go in this direction, help me get there.”

When the artist is making an album, I’m not going to say they have all the time in the world, but there’s a freedom—they have several months if not a year. You don’t know what songs are going to come out. They may record 50 songs and choose 15. When you’re dealing with TV, they’re constantly moving. They’ll tell me, “This is what the song needs to be about, we need to sync it to the video in a day or two.” You don’t have all the time in the world to vibe on one song. Some people might find those deadlines difficult, but I like the structure of, “We need to get this right by tomorrow.” There’s a momentum to that. There’s so many people involved and so much money being spent that they don’t have time to wait around. You really need to deliver. I like the pressure.

You’ve been writing and producing in the pop world since the early 2000s. Not that many producers have that kind of longevity—to what do you attribute yours?

For me, I look back at my roots. Before I was a producer, I was starting off as a classical pianist at a very early age. Later on, I went to the Berklee College of Music and became a jazz pianist. I just grew up with a love and understanding of a lot of different kinds of music. I was very inspired by hip-hop and rap but I loved pop and always wanted to be involved in those genres. When I first entered the music industry, it was a little more on the urban side—50 Cent, Snoop Dogg, Eminem and Rick Ross. I made the conscious effort to transition into pop, and that’s when I started working with Rihanna, Britney Spears.

There’s always ups and downs in the music industry. Every career has ups and downs. I think there was a point where maybe I got stale or needed a new breath of life, I needed to reinvent myself. So then I went on to do “Centuries” for Fall Out Boy, which was a little more rock than I’ve done, to doing TV stuff like Empire and Gwen Stefani while still maintaining the hip-hop. I always had a desire to be versatile. I guess that hunger of wanting to produce different kinds of music just never went away. I just want to make people feel something. If it’s a rap song that does that, great. If it’s me at the piano like Leona Lewis’ “Better in Time,” great. I don’t really care what the genre is. One of my newest songs is a remix of a country song with Nelly, “Die a Happy Man.” I just like taking on different kinds of projects.

I feel like the music industry can be a little unforgiving when it comes to producers getting typecast or associated with a particular sound. Artists seem to have an easier time shaking off certain styles when they stop being trendy.

I would say even for myself, there was a minute there where I was associated with, “Okay, Jason Derulo, Sean Kingston, Iyaz are your artists—you’re the guy who breaks urban-pop males, that’s the box you’re in.” I’ve suffered from typecasting as well. But you have to switch up your sound, you have to change with the times. That’s a very difficult thing to do. It’s one thing to listen to the radio and say, “Everything was urban for one moment and now everything is Euro-dance.” It’s not as simple as, “Okay, let me make all my beats and all my flavors four-in-the-floor.” You can’t copy what’s out there, because then you’re just a watered down version of someone who does it better than you because that’s how they naturally hear music.

There were times when I felt musically lost: “Everything is dance now, and I don’t naturally feel that kind of music. I respect it, I want to learn from it, but at the same time that’s not really who I am. How do I adapt my sound?” You never hit a point where you’re like, “Now I’ve found it, I’m good, I can keep doing that.” Music is constantly changing, it’s a moving target. You constantly have to change.

With that in mind, is there a part of you that still wants to have some through line in your work? Some idea of what makes a J.R. Rotem song?

I don’t think I need to go out of my way to establish the J.R. sound. If I did that, I would only be making music that sounds dated. For me, when I go into a room and talk to somebody, I don’t have to try to be “more J.R.” I just naturally do that. Music is the same way. I don’t have to try to be more “me” than I already am. I think it’s best to do the opposite—let me go against what I would naturally do. I’m versatile, I can easily adapt, but I think a weakness of mine is that I tend to be a creature of habit. I can rely on things that have been working or things I feel comfortable doing. You might talk to somebody else on the outside looking in and think, “Oh yeah, J.R., he’s known for flipping samples.” There’s truth to that, I’ve had a good amount of success remaking samples, but I don’t go into it saying, “I’m going to be the sample guy.” I don’t go out of my way trying to interject a J.R sound. That’s going to happen naturally without me trying.

You produced one song that I thought was going to be everywhere—Who Is Fancy’s “Boys Like You,” featuring Meghan Trainor and Ariana Grande. What happened to that song?

I was so happy to be a part of the song, and getting both Ariana and Meghan as features made it even bigger. Once I do a song as a producer, there’s no telling where it will go. I lose control at that point. That’s how it works in the music industry. It may end up blowing up overnight. It may take several months or a year to get big. I myself have been wondering what’s up with that song, and I don’t really have an answer other than I had high hopes for it. There are so many factors. It has to do with the relationship between the artist and the label. Pop radio only plays a certain number of songs, so if pop radio is concentrating on Meghan Trainor’s first single, they might feel it’s a conflict to have another song where Meghan is featured because that’s too many slots of Meghan. You know how it is: sometimes when you listen to pop radio it feels like you’re hearing the same six, seven songs over and over again. I’m not sure that’s the reason—I’m just speculating. But I was really proud to be a part of that song. I love the honest message that Fancy was delivering. It’s kind of pushing boundaries, definitely edgy. I love being a part of things like that.

Is there anybody you’re working with right now that you’re really excited about?

When I sign artists, I don’t like to sign five or 10 and dilute my attention and see which ones pan out. I have to be selective. I’ve had some successes with artists like Jason Derulo, and then there have been other times when I spent a long time on an artist that maybe wasn’t worth the time. I was holding out and waiting until I found someone special, and I’m very happy to say I found it. His name is Marteen Estevez. There’s a Grammy-nominated singer named Kehlani—Marteen is Kehlani’s cousin. He’s about to turn 15, and he’s got one of the most incredible R&B voices I’ve ever heard. To me, he’s as much of a superstar as anyone I’ve ever worked with. The tone of his voice and the stories that he’s telling are some of the most potent that I’ve ever worked with. I’m as inspired as ever.

One last thing: are you doing anything to celebrate the 10th anniversary of your work on Kevin Federline’s Playing With Fire?

[Laughs] Uh, no, I haven’t planned anything yet.

It’s been a decade. A decade!

That’s amazing. Man, time really flies.