The stone building commonly known as Forbes Mill is actually the two-story stone storage annex which was added to the original four-story stone mill building in the fall of 1880. The original "Santa Rosa" Flour Mill building, torn down between 1915 and 1929, was constructed in 1852-54. James Alexander Forbes, former Vice-Counsel in San Francisco for the British Government, came to the Santa Clara Valley in the early 1840s. An educated Scotsman who landed in California in 1831 when he left the whaler ship Fanny, Forbes was involved in real estate deals across the present day Santa Clara County. Recognizing the need for a facility to process the grain of the growing number of farms south of San Jose, Forbes found a convenient spot for a mill near Los Gatos Creek on the Rancho Rinconada de Los Gatos. This site lay astride the old Mission trail between Santa Clara and Santa Cruz. In 1850 he purchased 3,000 acres in the area of the Los Gatos Creek and proceeded to build a mill.
The opening of Forbes Mill in 1854 was an auspicious moment in the history of the Santa Clara Valley. Built on the site of a Mexican rancho amidst the sylvan landscape of the valley, the mill represented the shift of land ownership from the Californio Dons to the new settlers arriving after the Mexican-American War. The first three stories of the mill were built of stone quarried from the Los Gatos Canyon. The top story was wood, cut from trees in the mountains above Los Gatos. In 1880 an annex was attached to the north wall of the mill building. The town that grew around this building was first called Forbes Mill, then Forbestown, and finally Los Gatos. Unfortunately, Forbes knew nothing about flour milling, over extended himself and was forced into bankruptcy. The mill passed from hand to hand until 1870 when a stock company headed by W. S. McMurtry and J. W. McMillen, took it over and made a success of it. It continued as a flour mill until 1887 when it became successively a power plant for the Los Gatos Ice and Power Company, a brewing and bottling company, the Los Gatos Gas Company and finally the P.G. & E. substation for Los Gatos. The mill remained a storehouse for P.G. & E. until after World War II. It was then abandoned and finally in 1971 it was revived as a youth center for Los Gatos. Today the old mill annex operates as a museum, appropriately named the Forbes Mill Museum. On June 10, 1950, the California Centennial Commission commemorated the mill's contribution to the State's rich past by designating it as State Historical Landmark number 458.