Oystermouth Castle was built of stone in the early twelfth century on top of and older and previously unrecognized castle ringwork. The entire castle still stands to its original height, in an imposing position on a headland with wide views across Swansea Bay on the entrance to the Gower Peninsula.

The castle was the residence of the first Norman Lord of Gower. Successive Welsh uprisings destroyed the castle in 1116 and later in 1215 mainly by fire. These fires resulted in the building and rebuilding of the eminently picturesque castle during the Victorian era. Extensive repairs and extensions were made to the castle in preparation for the visit of King Edward I who stayed there for two days after the subjugation of Wales.

Currently the castle is the responsibility of the Swansea City Council and is supervised by the Friends of Oystermouth Castle. They have maintained the grounds and the castle so that it is a pleasant and interesting place to visit and explore.