Although in the early years, the layout of the gardens and grounds were the purview of the estate's masters, the ladies of Hampton became principally responsible for plantings by the mid-19th century.

Eliza Ridgely, third Mistress of Hampton, was an avid horticulturalist. For her beloved gardens, she acquired newly introduced varieties and rare species from around the world, many housed in newly constructed greenhouses. Her gardening legacy remains apparent, with creation of the Victorian carpet bedding displayed in the falling gardens, and in the plantings of many magnificent trees throughout the site.

Margaretta Howard Ridgely continued to maintain the parterre [formally patterned flower garden] gardens in brilliant fashion with a large staff of professional gardeners who planted thousands of colorful annuals each year. Helen West Stewart Ridgely, Fifth Mistress of Hampton, eventually modified the gardens to reduce the intense maintenance they required. With the last generation of Ridgelys at Hampton, Lillian Ketchum Ridgely sought to restore gardens that by then had become dilapidated and continued the use of the historic greenhouses.