Take a picnic basket down to Balmoral on a hot summer day and take shelter under the trees that line the foreshore while peering out across the harbour.
There are a couple of busses that go there from wynyard, mosman, crows nest and other so get yourself to one and shoot down to the beach.
If it's a warm day take some swimming gear and a towel, you can swim off the baths or in from the beach. The light on middle head and the other side of the shore is pretty special at sunset if you're around for it.
$15 each should be enough to cover transport there and back, a cheap bottle of wine and some nibbles.