It was built to serve pilgrims making their way to the famous shrine at Walsingham. It was sited just outside the town walls so that those arriving late, after the town gates were locked, might find shelter for the night. At the top of the little tower is a chapel so that prayers could be offered before the next leg of the journey.
When I last visited in early March of this year it was beneath dull and overcast skies, not ideal for photography. The door was firmly locked and I'd been told that it was used as a store for all kinds of rubbish.
On my way back from Wisbech recently - not a very direct route I'll admit but it's what you have to do if you rely, as I do, on public transport - I thought I'd see if I could get a better picture as I had nearly an hour to wait for the train.
Not only did it look a lot better in the sunshine but, wonder of wonders, it was open to the public. It's actually now open from 1:00 till 4:00 on Wednesday, Saturday and Sunday from mid-May to mid-September. There was no admission charge though there was a jar for donations.
The Chapel was built by one Robert Curraunt for the then Benedictine prior of Lynn, William Spynke. When the monasteries were dissolved during the 1530s the ownership of the building passed to the town council and it was subject to the indecision and neglect that one might expect from such a body at that time.
It was first of all partly dismantled, then used as a water-tower. Later it became a stable, an observatory and lastly a store-room.
On the ground floor is what was once the chapel, then you ascend by some rather uneven brick stairs, curving around inside the thickness of the walls, to what is known as the Priest's Room.
This is believed to be where the vestments and other valuables were stored. A further set of stairs leads up to the crowning glory of the structure, the stone chapel built in 1506 in the shape of a cross.
The stained glass is a modern addition, designed by Colin Shewring, in the 1980s. It depicts a lily which is the symbol of the Virgin Mary.
"Aha!" thinks I looking upwards, "it's got a little fan vault, a miniature of the roof of King's College Chapel". But what I didn't know was that it was probably built by John Wastell, the man responsible for the King's ceiling as well as the Retrochoir at Peterborough.
Then it was time to descend another staircase - yes, it really does have one leading up and one leading down! - to the outside world, the twenty-first century and the 16.30 train.