How Hedge Funds' Lack of Diversity Affects All of Us

January 5, 2022 12:37 PM EST

A recent Knight Foundation study found that only 1.4% of America’s wealth is handled by asset management firms owned by women and/or people of color, even though there is zero difference in the performance of the more diverse firms. Former hedge fund employee and current sociologist Megan Tobias Neely’s new book Hedged Out: Inequality and Insecurity on Wall Street takes on a small but crucial slice of that issue as it explores how the structure and practices of hedge funds contribute to the continuing lack of diversity within the industry’s ranks, and how that affects the rest of us. She spoke with TIME about the issue.

In 2003, there were 3,000 hedge funds and they were managing about $500 billion dollars. Today there are something like 12,000 hedge funds, and they have about $4.3 trillion under management. Why has it grown so much?

They’re really a phenomenon of the last 20 years. They’ve just skyrocketed. What I argue in the book is that it has to do with a broader trend of investors moving their money through trust-based networks rather than bureaucratic institutions. Now all the large investment banks have hedge fund units. It’s largely been made possible because of deregulation.

In your book, you write that hedge funds are a big driver of inequality. How?

The average household in the US takes home around $51,000. That’s a whole household’s earnings, whereas in hedge funds, the average portfolio manager takes home $1.4 million. Even entry level analysts take home around $680,000, which is almost at the threshold of the top 1% of household earnings. So those incomes alone contribute to inequality. But I looked specifically at how the everyday practices of the funds effectively hedge some people into their industry and allow them into these insider circles where these top incomes are created, while also hedging other people out and making it harder to get access to those incomes.

Why would we care about inequality within a hedge fund? Don’t we have much bigger problems?

Yeah, the women and the men of color I interviewed said, ‘I have no right to complain. I’m doing just fine.’ There are worse issues in society. But this is an industry almost exclusively run by white men. White men manage 97% of industry assets. It’s kind of mind boggling, how much control they have over it. The real focus of this book is that when you restrict who can access such enormous rewards and power, what you end up with is a sphere of society where people really can dictate their own terms, and this is happening in this context at a level that’s unprecedented. The concentration of wealth and authority among these elite white men in the hedge fund industry is on a whole different scale. And they can insulate themselves from oversight. They can demand these high fees, pay fewer taxes. Part and parcel of the social world they’ve created for themselves is that it’s walled off from others and people who aren’t like them. They don’t realize that they’re creating this environment for just a very select few.

How do they justify still having organizations like that? Are they not attempting to be more diverse?

They see it in a number of ways. They say they want to get out of the big investment banks. They describe themselves as very jaded with bureaucracy and with big government. So they create these small firms that they think are more equal in many ways, that anybody can walk in. But what they don’t realize is that when you’re the leader, you may be comfortable in a setting that other people are not. When you’re hand picking people who are exactly like you, who will behave and act and think like you, you just think ‘oh, it’s a good fit.’ Cloaked behind designations of who might be ‘a good fit’ are other sort of sexist and racist beliefs that you may not even realize you have. That’s part of what keeps people out and then allows them to have like-minded workers, who then say, ‘Yeah, these incomes make sense. They’re justified. We’ve worked really hard. We work, you know, 80 hours a week. We deserve this.’

What is the effect of risk on the hedge funds’ insularity?

At hedge funds, they would all say, ‘Oh we’re calculated risk takers; we’re not taking too bold of a risk.’ They all feel it is very justified. But what is often implicit in this picture is that because white men have worked in hedge funds for so long, they’re understood as being the most capable of mastering risks. One thing that came up again and again in my interviews were stories of women hedge fund managers who were perceived as too reckless. There was this assumption that women are risk averse. But when a woman takes the same risk as a man, her behavior was perceived as more risky. Similarly, Black traders I interviewed would take the same risks as their counterparts, but be perceived as aggressive. And so in these perceptions of risk taking are implicit notions about gender and race and who can be trustworthy or qualified as risk takers.

Is this a problem limited to hedge funds? Is it not true also of investment banking?

What’s really distinct about hedge funds is they are unfettered from as much regulatory scrutiny and oversight. Many people don’t understand what hedge funds do; it’s deliberately opaque. They’re investing in assets the worth of which is hard to evaluate in any given moment. So they’re hard to regulate. They can use the opacity to their advantage in terms of the taxes they pay. Also, the average hedge fund only last five years. They’re not very sustainable. So the workers prepare to change firms all the time to protect themselves from that uncertainty. And they do that by creating these close-knit networks. A lot of hedge fund managers hire people who they perceive as being like them, and then they might mentor them and invest in them later to start their own firm. And it’s these tight knit networks that then allow for the funneling of resources and ultimately these higher incomes.

Isn’t it true that because they manage a lot of pensions and state funds and that money is then redistributed to working people, that they could actually have a positive impact on income inequality?

Yeah, the average person assumes that they’re not affected by hedge funds, and they probably can’t afford to invest directly in a hedge fund. But most hedge fund investors are large institutions—pension funds, municipal or state government funds or federal government funds, nonprofit endowments. In some way most of us are impacted by either the investment returns they make or the investments that they make on society. And they are very aware that their average investor is the pension fund holder. But they charge such high fees, it doesn’t necessarily help those pension fund workers that they express the need to do right by. I was in one executive’s office asking about the impact on society of her work. And she lowered her voice said, ‘I just think the fees are too high. But don’t say that around here.’ Others said essentially, ‘We’re just moving money around, but we’re charging so many fees, so many layers are skimmed off, that ultimately it doesn’t benefit the people we’re supposed to serve.’

Wouldn’t most successful hedge fund employees say that they take big risks, and therefore, they reap big rewards?

I loved hearing how portfolio managers describe the risks they take. They’re really socialized into taking risks. One woman described how the first time she lost a million dollars in a trade, she was just terrified. She thought she was going to get fired. And when she told her manager he said, ‘Welcome to the game,’ and they celebrated. This was something people often talked about, this kind of social conditioning to risk taking and how you get used to losses. They had so many million dollar trades where their models played out, and that was so satisfying—and they described it in this kind of gamer or scientist way—that by the time they reached a billion dollar trade, they were confident that it would play out. They described that kind of risk taking as very calculated and well planned. And then they got to relish watching as the model played out as they as they planned it would. So they don’t see it as reckless risk taking by any means.

You claim in your book that hedge funds drive down payrolls of regular folk. How does that work?

This is something that we don’t often hear as much about. It’s very similar to what private equity firms do. Some activist hedge funds will become involved in a company’s management and put pressure on the company to do things to maximize profits for them as the investor. This often involves laying people off, outsourcing labor or streamlining processes in ways that really weaken the average worker’s bargaining power. Another way is by putting pressure on the company stock, which executives take as a message that they need to do something to turn that company around, which often means weakening labor conditions for the average worker.

Would America be better off if it just didn’t have hedge funds?

I think we’d be better off if we put more put more protections in, to curb what they can do, how much they pay in taxes. There’s things that we can do to empower average workers and make sure that they have the rights and benefits that they need in society and that would curb the power of big finance. People in the hedge fund world and in finance have a lot of resources. They’re very wealthy, they have elite networks, and so they have the ability to change policy in ways that pay for their work. This ultimately disadvantages everyone else. Putting in more protections to support the interests of others is the most important action to take in terms of curbing this kind of inequality.

What do you think of the retail investors who are propping up stocks like GameStop and AMC in order to hurt the short sellers, and citing 2008 as the source of their anger at hedge funds?

My main thought on this kind of Robin Hood trend to democratize finance is that capitalism and the stock market are not ever going to be democratic. There’s just too much money held by the hedge funds, by the big banks, by the institutional players. And so there’s no way that the other people getting into the stock market is actually going to end up in their favor. I mean, hedge funds invest in ways that a single fund can completely move the price of a stock. And ultimately that everyday investor who invests in a few stocks—I think that’s great in theory, I just don’t see it as an avenue for addressing inequality.

In the book, you talk about how very different this is from the Gilded Age, where the people who inherited money had incredibly opulent lifestyles. Is it that today’s hedge fund managers are still into wealth but also draw more of their satisfaction from seeing their model work?

Absolutely. I think that they think about it in such hypothetical terms. It’s hypothetical money. It’s all electronic. One of the most surprising things in my research was how many academics are in finance and in hedge funds. I’d say about a third of the people I interviewed came from academia. Others were lawyers. When I asked them how they ended up in finance, it was because funding ran out for them. One man I met had worked at NASA, and then the funding ran out. Others talked about how they didn’t make enough money in government research jobs or in universities; they were talking about entry level salaries in the $20,000 to $40,000 range when they started, and that they had student loan debts. We don’t need to pay researchers like hedge fund managers, but if we had a system that provided more funding to support research in another in a number of capacities, it may help to make sure those those paths are livable for more people, especially those who are most likely to be indebted and have other commitments.

You worked at a hedge fund for a while. People might say that this book is sour grapes because you couldn’t cut it. Would you like to respond?

I kind of stumbled into hedge funds to be honest; it wasn’t something I planned to do. I always wanted to do a PhD and become a social scientist. But I wanted a job that involves data analysis, to make sure I could cut it in terms of doing research like this. And so I sort of stumbled into the hedge fund unit and I worked there for almost three years during the financial crisis. And it really just piqued my interest in terms of what the relationship is between finance and society and inequality at large. The people I worked with were all great colleagues. I’ve stayed in touch with them.

How Hedge Funds’ Lack of Diversity Affects All of Us