The first fortified location in Cadore was situated on top of a hill in the middle of the Ampezzo region, at the junction of the Boite and Piave valleys; later, here will be erected the "Pieve" (people's church, from the Latin “plebis”).

Pieve becomes the capital of the region, set in a strategic point. The castle that was built on the highest point of a hill, on sacred place in Roman times, served as a control and defensive point.

In case of danger, the population could find refuge within the fortifications. The last such event took place during the war against Maximilian's. As it appears in books and old documents, the castle was quite large, with two floors built around an inner courtyard. The entrance is from the north-west.

A beautiful statue which seems to have belonged to the chapel of the castle, might be the work of Mino da Pisa. This statue, along with other items that were part of the chapel altar, thought to belong to the Vecellio school, are inside the existing church.

The castle was the center of military power in the region and one of the most famous historical personalities of the time was Bertrando patriarch, who in 1347 visited the Cadorini, coming from Aquileia. The castle was often under Venetian rule, resisting the siege of Maximilian (year 1511).

The Italian painter Titian was born in 1485 at Pieve di Cadore.