Clinton Court is a secluded courtyard and former carriage house that seems to spring right out of another time. Built back when New York City was all farmland, hidden amidst imposing apartment complexes and skyscrapers of Hell’s Kitchen, the courtyard is a relic of the days when horse-drawn carriages were the primary means of transportation.

The courtyard, located at West 46th Street between 9th and 10th Avenues, is not open to the public, and is barely visible from the street through a wrought-iron gate. When you pass through, you reach a small patio that feels like an anachronism, especially because it’s a stone’s throw from the charged brilliance of Times Square. Inside, there are benches, a winding staircase, glass-paned doors, and spidery green trees that cast a layer of shade over it all.

That’s the magic of New York City, though: you never know when you’ll unexpectedly find a magical, haunted nook, and almost every square foot of the city has a story to tell.

Clinton Court certainly has its stories. It was built sometime before 1800 by the builder Robert Auld. In the 19th century, small buildings like this one were often constructed behind larger ones. It originally functioned as a stable, and was once home to New York Governor and Vice President George Clinton. Today, it is a multi-residence, private community, with 18 apartments inside.

But Clinton Court isn’t just a historic time capsule: it’s also allegedly one of the most haunted places in New York. (And that’s saying something). According to legend, at least three women have jumped to their deaths from the roof, and long ago, an old mutinous sailor named Old Moor was hanged in the courtyard. His ghost allegedly appeared to the wife of the resident family’s coachmen, scaring her so much that she fell down the stairs and joined the already teeming masses of ghosts that wander and convene here (supposedly).

If all this makes you want to go seek out the ghosts, use caution. That same family’s grandchildren impersonated ghosts for many years until one night an apparition rose out of the ground, scaring one of them so badly that she also fell down the stairs.

With its dark, brooding red brick architecture, winding clusters of ivy, French Quarter-esque vibe, and ghoulish eeriness, the place is perfect for urban explorers and those seeking to escape the heat of New York City. In addition, the location is increasingly generating more publicity, so you might just see it as the set of a new blockbuster or indie film in the years ahead.

For more, check out this list of the top 10 hidden gardens in NYC and The Top 10 Secrets of Hell’s Kitchen in NYC