Colliton House, Dorchester. The plaque on the wall (just to the left of the road sign) gives this information:
Formerly the town house of the Churchill family. Mainly 17th century with major 18th century alterations. The site is believed to be the location of the medieval hospital of St. John the Baptist, fragments of which may still remain in the building and its foundations. The building has been owned by the County Council since 1933.I was intrigued by the blocked up windows, and wondered if they may have been filled in to avoid window tax. I couldn't find anything about them on Google, but I did discover that the house appears in Hardy's The Mayor of Casterbridge as High Place Hall.