During the 17th and until the middle of the 18th century, the Palace was used as a Bank, proof of this are the two inscriptions existing on the walls of the central courtyard.

At the end of the 18th century, the building was acquired by the Royal Hospice Board by passing it to the Provincial Charity and settling in it, the Women's Hospice.

In the early 19th century, modifications were made to the building, including the remodeling of the entrance door of the Palace, replacing the previously existing one and placing on its lintel an inscription in white marble in grey stone.

Likewise, on both sides of this inscription, there were two shields: on the right, that of Bishop Fray Benito Marin (1751-1769); and on the left, the shield of Castile.

In 1868, when the Provincial Benefficiency Boards were abolished, the management and administrative functions of the charitable establishments became the responsibility of the Provincial Diputations, thus becoming part of the heritage of the Provincial Institution. Between 1901 and 1903 the Hospice was extended, demolishing to this end houses adjacent to the Palace. This extension includes the construction of a Chapel adjacent to the building, opened in 1903 under the title "La Visitación."

In 1939 the center was called "Internado de Santa Teresa" and served as a residence for both sexes and chronic patients until the second half of the 20th century, thus, with the transfer of the chronicles in 1971 to San Juan de Dios and that of the elderly in 1972 to the new Residence of Santa Teresa, the building remains empty. In those years the works of restoration of the Arab Baths located in the basements of the Palace and the rehabilitation of the entire building, which would be completed in 1984 and which were awarded the same year with the Medal of Honor of the Europa Nostra Association.

The central courtyard of the Palace is made up of a square of thick walls in which the windows and doors of different rooms are opened. Within this square is inscribed another one formed by a two-level column gallery, with three columns on each side and a total of eight columns on each of the two levels.

For the construction of the lower level or floor gallery, the chosen Order was Toscano applying half-point arches very slightly reduced. On the four columns of the corners rest four arches: the two eldest, form the angle of the gallery; the two minors, are attached to the perimetral wall becoming columns "in palm," a solution that had been used by the Muslim builder who had made the great Tempered Room of the underlying Arab Baths. In the same way, the architect of the Palace used a constructive structure identical to that used in the Baths: for the columns and the capitals he uses the stone and for the arches and the slices the chosen material was the brick.

As for the gallery of the upper level or floor, although it has a scheme similar to that of the lower floor, it has modifications, as a variant of the Toscan Order was chosen. Both columns and arches are lower than those located at the lower level. At present, the intercolumns of this level are closed with an abalaustrated wooden railing.

It has an interesting gastronomic library, and a dream environment for celebrations, congresses, conferences or business events. A place of forced passage for gala dinners in congresses, it has hosted in recent years important events held in the city of Ubeda.

The museum, considered a Good of Cultural Interest, is divided into several spaces; three buildings that house different models of almazaras, a garden of olive varieties from several countries and the Bodega.

Along the route of the visit, steps are shown to obtain the oil, from the most traditional methods of olive cultivation to the modern marketing and packaging processes.

In the museum you can know the characteristics of olive oil, value its properties and deepen the different aspects that define its quality, gradation, color, etc. You can also physically know the different varieties of olive trees, as it has a garden with trees of each variety, but it deserves special attention, a 19th century winery known as the "Cathedral of Oil." In addition, the complex has a conference room for 230 people. Oil caters are made for groups, tasting and other activities related to the olive tree.