On the Upper West Side, a series of mansions stretch along Riverside Drive, offering a dazzling view of the Hudson River. Built in the late 19th century by a group of eight millionaires, the stately homes serve as a alternative to the exclusive mansions along upper Fifth Avenue. Out of the eleven houses built, No. 323 has a rather intriguing and scandalous history that makes it stand out from the rest of the mansions, as underscored by its distinctive red brick facade.

Part of a lavish row of residences designed by C. P. H. Gilbert, No. 323 on West 74th Street was once rented by steel magnate Charles M. Schwab while his gargantuan French Renaissance style mansion, located directly across the street, was being built. The residence give him the unparalleled ability to oversee what would eventually become the largest mansion ever constructed in New York City.

When Schwab finally moved into his newly constructed home, No. 323 was purchased by Robert E. Tod, who, along with his brother, ran a banking firm by the name of J. S. Kennedy & Company. In 1913, Tod sold the residence to a real estate firm, which, in turn, sold the home to an anonymous buyer giving rise to much speculation surrounding the purchase. The Sun, in a 1914 article, printed the following: “There is one other real estate mystery in Manhattan. It involves the ownership of the fine dwelling at 323 West Seventy-fourth street…Various stories have been heard as to the real buyer, but none of them could be verified. In fact, as has been said, it is one of the mystery properties of the city.”

The story of the mystery owner would not come to light until years later, but when it did, the scandal stoked juicy parlor gossip for a very long time. It all began in the late 19th century:

When industrialist Jay Gould died in 1892, his lofty $72 million fortune was divided among his six children, including his son, George Gould, who married socialite Edith Kingdon. However, as Edith’s once striking figure started to change, so did Gould’s fidelity. He soon grew interest in a showgirl and dancer named Guinevere Sinclair, who, despite her career choice, came from a prominent family.

The two eventually kindled a romance and Gould surreptitiously purchased the sumptuous mansion on No. 323 West 74th as a present for his mistress. The affair continued over the years: the couple welcomed a son named George Sinclair in 1915, as well as a daughter named Jane in 1917. While the matter was kept secret from New York City society, the illegitimate relationship was known to his wife, Edith, and even to his family who referred to the children as “George’s bastards.”

In 1921, Edith Kingdon collapsed and died as a result of a heart-attack. It was later discovered that she had been wearing an extremely tight rubber suit underneath her dress at the time. Although Edith had been striving to regain her husband’s favor for years – resorting (in vain) to wearing the suit in an attempt to lose weight – Gould’s adoration for Sinclair was unperturbed. He went on to marry her merely six months after his wife’s death. Following their quick ceremony, Gould and Sinclair immediately left for Europe, leaving New York ballrooms buzzing with the scandalous turn of events. The marriage, however, did not last long and George Gould died a year later alone in France; thus began the bitter fight for his inheritance.

Because the Gould family felt contemptuously towards Sinclair and her children, they fought vehemently in court for many of George Gould’s properties. The inheritance was ultimately settled with Sinclair receiving $ 1 million in liberty bonds and the children receiving a $4 million trust fund. Not one to stay widowed for long, she went on to marry the English viscount George St. John Broderick Dunsford later that year.

After the settlement, the mansion was divided into “non-housekeeping apartments” and five years later, it was altered to “apartments.” Later, a penthouse was added to the roof, making a triplex apartment with the fourth and fifth floors. Today, the asking price for the six-story luxury mansion is a staggering $19.995 million. While there are yet to be any prospective buyers, the mansion has been rented to celebrities, including Owen Wilson and Woody Harrelson.

Next, check out 17 Gilded Age Mansions of Millionaire Row on NYC’s 5th Avenue and 6 Lost Mansions of the Upper West Side and Upper Manhattan.