Charles H. Bullis, his wife Ellen, and their three children left Manchester, Vermont in November 1838 to go west to Ohio. Traveling across New York State via the Erie Canal, the family stopped in Macedon to visit Charles's sister. The Bullis family decided to settle in Macedon. On April 1, 1839, Charles purchased a 59-acre farm next to the Erie Canal for $2,631.64.

Construction of the Bullis cobblestone house began in 1839 and was completed in 1842. In 1863, Bullis deeded the home to his son, Dr. Abraham Bullis who had a medical practice in nearby Farmington. The property continued to be passed from one Bullis generation to the next until 1984, when it was sold.

The cobblestone structure is unique in that the quoins and window and door lintels are constructed of brick instead of limestone or sandstone, giving the building a rather unusual appearance.

Adding to the history of the house is the fact that Civil War veteran, Brigadier General John L. Bullis lived here with his grandparents from 1857 until 1860 while he attended Macedon Academy. Bullis was a prisoner of war after the Battle of Gettysburg. According to his May 27, 1911, obituary in the New York Times, Bullis was with General Nelson A. Miles when he took the famous Apache chief – Geronimo - into custody in 1886.