Elizabeth Bay House is a historic colonial home that is part of Sydney Living Museums. It has been incredibly preserved and all the rooms are furnished as they would have been in the 1830s. There are various artefacts on display and incredible history to learn. It's open Fridays, Saturdays and Sundays 10am to 4pm. You can pay a small fee for admission or use a Sydney Museums Pass if you have one. You can do a guided tour if you ask the staff, or explore on your own. Right across the road is Arthur McElhone Gardens which have a fantastic view of the harbour and are beautifully manicured with a romantic stone bridge over a koi fish pond. It is perfect for a picnic before and after. You will feel like you've stepped backwards in time.