When William the Conqueror came to the English throne in 1066, he began his defence of the kingdom by building fortified castles. He started the building of the Tower of London soon after his coronation. The Central Keep of the Tower, started in 1078, was built from limestone imported from Caen in Normandy.

For a long time after William's death English kings continued to add to the Tower buildings. During the following three centuries the Tower was gradually extended beyond the old Roman city walls.

Thirteen towers have been built over the years and they stand inside the outer fortifications. The most famous of these are the Bloody Tower, the Beauchamp Tower and the Wakefield Tower. A moat surrounded the outer wall and a watergate, built in the thirteenth century, connected the River Thames to the Tower itself. This watergate was known as Traitor's Gate because many prisoners were taken to the Tower by this route, in some cases never to set foot outside the Tower again.