Cheboygan began as a Native American settlement. White men came in 1844 and it was made the county seat in 1853. There was a theater built in town in 1877.

In approximately 1890, Cheboygan became the home port for ferryboats to nearby Bois Blanc, an island in the Straits of Mackinac. Ferry boat services continued as of 2007.

Early in the 20th Century, it was home to the pioneering brass era cyclecar maker Flagler.

In 1944, Cheboygan became the home port of the former U.S. Coast Guard cutter and icebreaker Mackinaw, serving from 1944-2006. Beginning in 2006, the port continued this role as the home dock of the new Mackinaw, a successor cutter.

The name of the city shares the name of the county and probably has its origin from the Cheboygan River, although the precise meaning is no longer known. It may have come from an Ojibwe word zhaabonigan meaning "sewing needle". Alternatively, the origin may have been "Chabwegan," meaning "a place of ore."