The area was settled by 120 Norwegian immigrants from Lisbon, Illinois on June 7, 1855; including Osmond and Anna Sheldahl and five of their children. It incorporated as a city on January 18, 1882.
The Chicago, Milwaukee, St. Paul and Pacific Railroad established a rail line through Sheldahl during the 1880s. In 2005, Union Pacific Railroad sold a 25-mile discontinued rail corridor—which included the line through Sheldahl—to the Iowa Natural Heritage Foundation, which enabled the construction of the recreational High Trestle Trail. Access to the trail, including a parking area and washroom, is located in Sheldahl