By 1824 the lead miners had moved north from Galena digging into what would become Wisconsin. Their first settlement was Hardscrabble, soon renamed Hazel Green.

In 1846 lead mining was booming. Hazel Green was a bustling town on the stage coach trail from Milwaukee. That year the town saw the opening of its second inn, the Wisconsin House. Soon the boom moved westward and the railroad replaced the stage coach. In 1853 Jefferson Crawford bought the inn as his family residence.

Foremost amongst the miners of the Hazel Green District, Jefferson Crawford was a community leader and friend of U.S. Grant. They often visited together. General Grant is last known to have visited at the house in August 1868, at Jefferson Crawford's death.

In 1876 a tornado swept through Hazel Green, and leveled most of the town. The native oak timbers of the house proved to be as strong as rumored, and the house was among the few left standing.

For much of its history, the inn was the home of Miss Helen Crawford, Jefferson's granddaughter. Miss Crawford taught at the local high school into the 1950's. Much beloved by her students, many of the townspeople share wonderful stories about her. Miss Crawford died in 1958.

For a while the inn continued as a family residence, then became an apartment house, and eventually fell into disrepair. After rehabilitation is 1985, the inn reopened. It soon became well known for its hospitality and good food. In 1992 Ken and Pat Disch bought the inn. The tradition of homey and comfort and relaxation for the traveler continues. Both Milwaukee Magazine and Travel & Leisure have featured the inn.