It was also once known as a shipbuilding centre. Two of Francis Drake's ships, including the "Golden Hind", were built here.
The Moot Hall, dating from the 16th century, now stands beside the beach, but was originally in a Market Place in the centre of the town, such has been the constant erosion by the sea.
The Aldeburgh Festival is internationally famous. This was started in 1948 by the composer Benjamin Britten and singer Peter Pears and is held every June.
St. Peter and St. Paul Church stands on a hill overlooking the town. Most of the church is 17th century, but it has a 14th century tower.
George Crabbe was born at Aldeburgh, and returned here after being chaplain at Belvoir Castle. His poem "The Borough", written in 1810, was the inspiration for Britten's opera "Peter Grimes", set in the town.
Also from Aldeburgh was Elizabeth Garrett Anderson, England's first woman to become a Mayor, and also the first to practise as a doctor.
Just South of the town on the beach at Slaughden is a Martello Tower, one of many set up around what was once known as the Saxon Shore, as defences against likely Napoleonic invasion.
M. R. James, writer of ghost stories, often stayed in Aldeburgh, which he used as the setting for "A Warning to the Curious".