Built in 1844 for General Robert Taylor, the Greek Revival Taylor-Grady House later became the residence of Henry W. Grady, famous as the voice of the New South. After the house was sold by the Grady family, it changed hands many times and was eventually left vacant. During the vacancy vandals stole mantel pieces and other items.

In 1966, when Open Spaces grants became available, the City of Athens bought the property and immediately leased it to the Athens Junior Assembly (now Junior League of Athens) which undertook its first renovation and oversaw the property until 2022, at which time it again became vacant. A major renovation funded by a special local sales tax program took place in 2004. Today, the house and grounds, while still owned by Athens-Clarke County, are operated by the Taylor-Grady House National Historic Landmark Foundation, a non-profit 501(c)(3) organization which preserves this architectural gem and its valuable collections as a thriving historic space for the Athens community.