The earliest information we have about the Center’s building comes from the Brown University archive. It is a letter written by Susan Ely, granddaughter of the building’s first owner, Joseph Cady.

Mr. Cady served as what was then known as the Steward of the University from 1812–1826. His job was to look after the students’ well being including their rooms and meals, collect their tuition payments, as well as take care of the University grounds. Around the time of his appointment he purchased a lot on George Street near where Rhode Island Hall stands today. Mr. Cady purchased the lot from Mrs. Sarah Hopkins, the daughter-in-law of Esek Hopkins, the captain of the slave ship Sally commissioned by Nicholas, John, Joseph, and Moses Brown decades earlier in 1764.
In 1866, after Mr. Cady’s death the present building moved to the corner of Waterman and Prospect street. His granddaughter's letter tells us that “the old house commenced a journey across the back campus to a new location.” It remained there for just one year before again moving to its present location at 94 Waterman Street in 1867. For over a century Joseph Cady’s descendants lived in the home. These included the prominent physician and founder of the Providence Medical Association, Dr. James W.C. Ely ’42 and Joseph Cady Ely ’57, a prominent lawyer who helped reform the state’s laws and judicial system as part of an 1890 commission. The last Ely family member to live in the home, Ruth, bequeathed the building to the Providence Athenaeum upon her passing in the early 1970s, an organization she and her family had supported for many decades.
For many in the community, the building is most recently remembered as a real estate company before it was transformed into Simmons Center's present home.