DURHAM, N.C.—By some measures, the price of housing in this growing city is still a steal. Median home values in the area remain below the median price for homes across the country. And rent here is cheaper that in most cities, too.

But look around at the trendy bars and coffee shops cropping up across downtown, the cranes and construction work on nearly every corner, and it’s a good guess that Durham’s affordability may soon be a thing of the past, just like the city’s status as a working-class, tobacco-manufacturing town.

That’s a troubling proposition for families that have lived and worked here for generations, getting by on blue-collar salaries. The growth in those kinds of jobs has largely dried up, replaced by openings at start-ups and biopharmaceuticals. And the cost of living is getting higher, to boot. The revitalization of downtown Durham, which had long been largely abandoned a little more than a decade ago, was widely applauded, bringing money and jobs but not really coming at anyone’s expense. But now that the surrounding neighborhoods—traditionally black and working-class—are becoming whiter, richer, and pricier, the changes are resulting in real harm to long-time residents.

Still it’s hard to frame the changes to neighborhoods that have suffered from poverty and disinvestment as all good or all bad. “I know it’s easy to harpoon gentrification as a bad thing, but taking houses away from slumlords and renting them out does a lot of good,” says Brittany Kielhurn, a 30-year-old landlord in Durham. “Yes, it’s displacing people of very low incomes, that’s a problem. But it also means less abandoned buildings and overall blight.”

In Walltown, a historically black neighborhood, the changing demographics are evident. “It was almost exclusively African American up until the 1980s,” says Jonathan Wilson-Hartgrove, a minister and organizer who has lived in Walltown for over a decade. The community was largely made up of those who worked service jobs at Duke University, doing things like cleaning and laundry for the school. “They referred to Duke as ‘the plantation,’” he says, chuckling. Now, the mix has changed, and most of his new neighbors, Wilson-Hartgrove tells me, are white. Walltown is prime real estate, bordering Duke’s east campus and sitting just a mile away from downtown. “To pay $800 or $900 down here is a significant savings for a grad student,” Wilson-Hartgrove says. “But the people who had been paying $300 a month are getting pushed out.”

Walking through the neighborhood, there are houses that are older and more rundown adjacent to others in the midst of massive renovations. Young black and Hispanic men walk and drive around, older black residents sit on their porches, and young white residents bike by. Tensions have risen, Wilson-Hartgrove says, as white neighbors have moved in and policing in the area has increased; the young black and Latino men say they’re being harassed and white residents say they don’t always feel safe. Then there’s the issue of whether or not the populations that have historically inhabited this neighborhood will be able to stay put.

Wilson-Hartgrove tells me about a man named Pervis who lived in the neighborhood for 30 years, renting an apartment and working as a janitor at the university. He paid about $275 a month in rent, Wilson-Hartgrove remembers, until his landlord asked him to vacate so that he could make renovations a few years ago. When the tenant asked if he’d be able to move back in afterward, he was told that he could, for the new price of $800 a month. After the upheaval of finding a new place, he died of a heart attack shortly after moving.

* * *

For years, Durham’s low-cost housing and abandoned properties put some neighborhoods in the sights of those with enough cash to buy property, rehab it, and then sell it or rent it out. Kielhurn, a graduate of Duke who has lived in the Research Triangle since she was a kid, has amassed dozens properties in the past five years.

Kielhurn says that she got her start buying lots that were abandoned or dilapidated, and cheap. Sometimes they were foreclosures, but sometimes they were active rentals in horrible condition, she says. Landlords would keep the rent down, asking for, say, $400, but in return they refused responsibility for any upkeep or renovations. Often, desperate tenants felt like they had no means of forcing their landlord’s hand. In one home that she toured, Kielhurn says, the bathroom wasn’t structurally sound, so when the tenant stepped into the tub, it would sag a few inches. She gave up and went to a friend’s to shower. When the tenant finally started withholding rent because the bathroom wasn’t being repaired, her landlord started eviction proceedings.

Kielhurn says these stories of dilapidated, unsafe, unsanitary rentals are fairly common. And the poor condition of some of the housing stock in poorer neighborhoods is what allows her, and other buyers, to grab up properties for such low prices. She’s bought many of her properties for under $50,000 and spends the bulk of her funds on renovations. When she rents them out again, she charges what she feels is a fair price for all the work she’s put in, and for the fact that she’ll be more attentive than previous landlords. So prices escalate to $800, $900, or $1,200 a month.

She’s well aware of the problems this could cause as supply of cheap housing dwindles, but notes that putting up new roofs, fixing structural deficiencies, and fixing things as they break are costly. “After paying property taxes, insurance, and financing costs, there’s often no money left for landlords to adequately maintain their properties while also charging very low rents,” Kielhurn says. It’s frustrating, but Kielhurn has a point. In order to be profitable, landlords must either keep costs low (which often means forgoing upkeep and letting buildings decay), or charge enough to foot the bill of maintaining units, paying for their own expenses, and leaving a little room left over.

She takes me to a house she’s planning to buy soon in East Durham. It’s massive and just a few weeks ago served as a rooming house, allowing tenants to rent bedrooms by the week. As Kielhurn fiddles with the lockbox, we say hello to an older black couple sitting on the porch next door, and I start to count the bullet holes in the front door and siding.

The inside is dark and musty, and the space has been carved up with temporary walls to create additional rooms. Most have flimsy locks on their doors and the entire house is covered in old, stained, smelly carpet. Kielhurn is excited about it. Once she’s through with renovations, she’ll be able to rent it in a completely different way, she tells me as we walk back to the porch.

Some of the same houses that Kielhurn and other landlords have been able to snap up on the cheap aren’t as accessible to Durham residents who might want to buy and live in them. “A lot of the challenge is that you can't get mortgages on houses that aren't in pristine condition,” Kielhurn says. “Many of the properties that I'm getting you would have a lot of trouble purchasing with conventional financing for silly reasons: peeling paint on the exterior, not having the right electrical outlets in bathrooms and the kitchen.” For instance, on one foreclosed property, all it took to make the home bank-approval-worthy was patching a hole in the wall. It cost Kielhurn $75.

But many would-be buyers don’t know that, and many fixes, like damaged floors or roofs are a bit pricier, so a buyer would need to have about a few thousand dollars, in addition to their down payment, to fix it up. But that hurdle can be a big one for a family, leaving them renters instead of owners with a mortgage that would be much cheaper than renting.

Organizations like Habitat for Humanity, the Durham Community Land Trust, and the Self-Help Credit Union are active all over Durham, fixing up properties and helping low-income residents become homeowners, but they can’t completely slow the tide in the city’s housing market, or scale up to meet every need. “When we were doing this work in the ‘90s and the 2000s there were no other investors coming in and trying to buy properties and flip them,” says Tucker Bartlett, a vice president at Self-Help.  “All of the gentrification pressures did not exist in those neighborhoods at that time.”

The need for affordable housing is evident, but while many cities have programs that offer tax incentives or density bonuses for including affordable housing, Durham doesn’t have a concrete affordable-housing plan. And Kielhurn says that while many of her tenants are low-income, the Section 8 voucher program can be a hassle. The waiting list is long, and for landlords, the paperwork is daunting, and the entire process can be length. While they wait for everything to check out, owners aren’t collecting rent on the unit, she says.

* * *

A big part of Durham’s problem is that though this wave of revitalization, which started downtown and is trickling outward, was sought after and carefully planned, it seems that no one thought about or planned for the very real dangers that inviting an influx of investment and revitalization to the area might bring.

Even among longtime property owners, holding onto housing in Durham is becoming a challenge. The city only assesses taxes every eight years, which means that property taxes haven’t been adjusted in many now-popular areas with much higher home values. The city council has now realized that waiting so long to re-evaluate properties can be a problem during growth periods, because instead of acclimating to gradual tax hikes as property values grow, owners are blindsided with a substantial increase all at once. There’s talk of shifting the schedule to every four years, but that likely won’t help those who are hit with a tax bill they’re ill equipped to pay in 2016.

“We’re seeing displacement and there doesn’t seem to be a point where this is going to stop unless we do something about it,” says Jillian Johnson, a city councilwoman. “It seems like the natural market curve prices a huge number of people out of Durham very quickly, and I think the process is accelerating.”

Right now, the city is ill-equipped to handle the housing problem. Durham doesn’t have a robust affordable-housing program or the ability to incentivize—or force—builders to make housing affordable in the way that other cities going through similar changes do. The city, for instance, doesn’t have inclusionary zoning laws, which would require that developers building new housing or doing big rehabilitations on old buildings make a portion of the new units affordable. And Johnson says that Durham also lacks strong rent-control regulation, which would prevent the cost of a unit from climbing astronomically over a short period of time. The result is a massive amount of demand for a dwindling supply of lower-priced housing. “We’re finding that for folks at 50 percent or below of the area median income that there’s a severe shortage of affordable housing,” Johnson says. “This is something that we allow to happen by having very few checks on the market for real estate.”

And it’s not just homeowners who are struggling. Businesses are seeing the rent on their commercial spaces climb as downtown grows, with fancy renovated and new buildings that charge significantly more for retail spaces. It creates a conundrum for businesses who want to participate in the growing consumer base of the revitalized downtown but simply don’t have the money that a large chain or investor-backed venture might.

Ann Woodward, the executive director of the Scrap Exchange, a nonprofit that serves Durham’s art community, says that the organization has struggled with property issues for years, but recently bought 23,000 square feet of space in a mostly abandoned shopping mall, which has let them stabilize their rent and help other creative organizations by renting out space at below-market rates. Demand has been high, proving that small-business owners are desperate for more affordably priced space. “No one wants to go back to the Durham of 1996 when it was empty, but there’s a fear that local businesses won’t be able to stay downtown,” says Lisa Sorg, a journalist and longtime Durham resident.

There are, of course, plans and discussions about how to fix Durham’s affordability problem. There’s talk of pushing for inclusionary zoning, and building more voucher housing on land owned by the Durham Housing Authority. The city has hired an affordable-housing consultant and is investigating ways to create mixed-income communities so it doesn’t foster concentrated poverty.

Those would all be positive steps, but they are all reactionary. By and large, people aren’t mad about the arrival of business, jobs, and population growth to the city. They are mad that the city has actively courted these changes over the past decade or so, but done little to protect residents who’ve called the city home even through its toughest times and now find themselves being pushed out. “I wish that we had done more 10 years ago,” Johnson, the councilwoman, says. “I feel like there are important questions that are left to the end, where it’s much more difficult to make headway on them.”

It’s also painful to watch a southern city that once housed a thriving minority middle class and blue-collar workers, push out members of its population who aren’t white or affluent. Sorg says that Durham has long prided itself on the idea of inclusion, but it’s unclear that that’s been the guiding principle of the revitalization. “If you think about great cities, they are economically diverse and racially diverse. The concern is that’s not going to happen here.”