This is St John's Roman Catholic Church that stood on Federal Hill in Providence, Rhode Island, from 1875 until its demolition in 1993. It was Lovecraft's inspiration for the brooding church that so intrigued, writer and painter, Robert Harrison Blake in 'The Haunter of the Dark', written in 1935.
You can still visit the site of the church which is now a small park.