The Chateau de la Batiaz is a small but very well restored 13th century castle and well worth the visit. Our boys loved it... So did we.
The castle is at the nexus of the northern and eastern parts of the Rhone (a major European river) valley. Its strategic importance dates back to ancient times when Martigny was a Roman town, when it was the gateway to one of the only passes into Italy. In fact the amphitheater is still used as an open air cinema.
The Chateau de la Batiaz withstood a siege by the Count of Savoie in the fourteenth century but a few years later the Duke got his way by moving enough of his men into town and having them vote him Martigny's new protector. The castle soon returned to local control though where it pretty much stayed until it was burned in 1518.
In addition to awesome views and cool battlements the castle has the requisite collection of torture instruments and, best of all, a set of medieval weapons that you can play with (unsupervised!)