This prominent headland is formed of granite. The granite originated when magma was squeezed up from deep in the continental crust during the mountain building episode which occurred about 500 million years ago. This is known because sediments left by the ice age, which occurred at that time, rest immediately on top of it, and boulders of granite were carried, by the ice, northwards. The smooth rounded profile of the Bluff is attributed to the erosive action of this ice sheet.

On top of the Bluff a plaque, commemorating the encounter between Matthew Flinders and the French explorer Nicolas Baudin in 1802, has been affixed to one of the many large rounded boulders, typical of weathered Victor Harbor granite.

Several granite islands can be seen from the Bluff. They also bear signs of having been smoothed by ice passing over them. Granite from West Island, to the south, was used for the base and the steps of Parliament House in Adelaide. It is now reserved for conservation of wildlife habitat.