It is located at the highest point of a promontory that provides a natural defense, with strong cuts to the north and east, and which is extended to the west by a relatively wide and flat spur, from which the entrance tower to the castle is accessed.

A Almohad occupation developed between the XII-XIII centuries, the oldest recorded, has been documented. In this period the fortress would be built, which is organized in three enclosures with towers of troncopyramidal section, steps from north to south and made in high-quality tapial with a lime mortar lining simulating a false silery split. To the south of this set would extend the Islamic village, around the area where a vaguada was running.

In the 14th century several works will be carried out in the village, such as the construction of the first church of the town or the wall surrounding the area of access to the castle and the village. It will also be when various repairs are made in the castle and the so-called Torre del Picacho is raised on the top of the rock.

In the 15th century, at the point where the access door to the fortress was located, which was inserted in the canvas that separated the population from the square of arms, it was replaced by a torre-gate, which currently continues to function as access to the castle. This structure has rectangular plant, the first body, the only original part that is preserved, since the upper bodies are very restored, was built slightly wider than the rest with the aim of giving it greater strength.

In the following century the church of Santo Domingo was built on the south flank of the castle. With a typical Renaissance living room, it is organized in three ships, the centre of double width than the sides, and flat tester. The header has two cylindrical turreons on the corners. Its state of deterioration in the 19th century would lead to the suppression of the cult in the church and the abandonment of it, using it as a municipal cemetery until the mid-20th century.

The origin of the population appears to be related to the construction of the Almohade castle, which rises on a rocky spur dominating the town. It is most likely to be built as a center of a castral district that would cover much of what will then be the Cazorla advance. The high level of construction seems to indicate the State's participation in its construction in order to strengthen the Al- Andalus border.

In 1231, Fernando III, before conquering the area, granted the lordship of the village of Quesada to the church of Santa María de Toledo. Months later the prized Archbishop of Toledo Don Rodrigo Ximénez de Rada began the conquest of this territory, including La Iruela, thus taking over the lands that will later constitute the Lordship of the advance of Cazorla. The conversion of Cazorla to the capital of the advance must have had an impact on the development of the populations of its surroundings, so that the castle of La Iruela would be almost abandoned when the strengths of Cazorla were built.

In the second half of the 14th century, during the civil war that faced King Peter I with his stepbrother Henry II Tratámara, while Cazorla remained faithful to the king, Iruela supported the Archbishop of Toledo, who had placed himself on the sublevate side. After the end of the conflict after the victory of Henry II, in thanks for the support given, Archbishop Don Pedro Tenorio granted him the status of a villa in 1378.

From now on, the town will experience a progressive development, which will be reflected in the emergence of new buildings and in the reform of its defensive structures. After the conquest of Granada the advance would remain until 1812, but already without the tutelage of the figure of the military advance, a direct result of the loss of its strategic and border position.

The population of La Iruela tripled in the 16th century, growth that resulted in the construction of the first urban town, breaking with the walled structure that had characterized the population during the Middle Ages.

In the course of the War of Independence the population would be attacked on several occasions by the Napoleonic troops, suffering significant damage.