More than 40 million Americans carry a combined record $1.2 trillion in student loan debt, with the average borrower in hock for $29,000. Now, a new study from the Washington Center for Equitable Growth shows that education-related debt is strangling some people of color, squeezing tighter on their necks than the general population.

For the second release in the organization’s “Mapping Student Debt” project, researchers Kavya Vaghul and Marshall Steinbaum analyzed zip code maps, student debt data from Experian and income info from the American Community Survey and found that areas with large populations of Latino and Black residents have higher rates of past due balances on school loans.

But they didn’t stop there. When they controlled for income, they found that it wasn’t the expected low earners who had the most trouble paying back the debt. Among people of color, it’s actually the middle class that is falling behind, causing the researchers to wonder if going in to debt to finance education is the right way to build generational wealth. As they write in the report:

These data tell us that at least with respect to longstanding group and individual income and wealth gaps between minorities and the overall population, debt-financed higher education is not the solution, and may even be contributing to the problem. The fact that, among minorities, the middle class is most strongly affected implies the problem is structural racism, not poverty. Any solution to the student debt crisis has to recognize that.

The researchers say this structural racism is baked into four main areas:

Higher education system: Not only is there a college completion gap between people of color and Whites, but much of the education debt for the former is concentrated among those who either attended nontraditional or for-profit schools or were unable to finish. This impacts loan-holders’ ability to earn a living and pay back accumulated debt.

Credit markets: “Even after controlling for key risk factors, African Americans and Latinos are disproportionately served by high-cost credit providers who provide less generous terms and more onerous repayment requirements, implying that discrimination occurs through market segmentation and sorting.”

Labor markets: Higher unemployment rates and lower earnings plague people of color. “These disadvantages extend across college majors, occupations and the type of higher education institution that these recent graduates attended. In combination, these factors leave minority students and their families substantially more vulnerable to delinquency than comparably situated White students and their families.”

Racial wealth gap: “Holding income constant, African-American and Latino households have substantially lower levels of wealth than do White households, including financial assets that can act as a buffer against student loan delinquency in the event of job loss or some other misfortune.”

Read the full report here.