The existence of an original fortress on the site of today's castle is assumed from sometime around the middle of the 14th century.  The first written source is an entry into the land records from 1465, mentioning the division of the property of deceased Ctibor of Zásmuk, also of Vlčetín, between his two sons Petr and Václav.  The fortress then might have been sold into the ownership of Diviš Boubínský of Újezd, who sometime around 1530 sold it to the knightly family of Káb of Rybňan.

            Of the Káb family, the most interesting character was undoubtedly the Knight Jan, a capable manager, active builder, and loyal Habsburg servant, who represented the prestigious office of tax collector in the Bechyňe region.  His short life was tragically marked by the plague, which took five of his children in 1557.  This might have been the reason that the simple castle chapel on the hill above the lake was built (today the chapel of the Holy Trinity).  After Jan's death, Lhota was inherited by his three sons Bohuchval, Zikmund, and Jiří, who first had to compensate their older brother Jaroslav.  Zikmund died four years later, and the families of the two remaining brothers Bohuchval and Jiří lived in Lhota at odds.  The castle ceased to be their quiet home and became a theatre of squabbles, arguments, and personal assaults.  It could be some of the stories captured in the memories of the occasional observers which gave life to the stories of the godless castle lady possessed by the devil, her tragic end marked by the bloody stain under the window on the then snow-white facade.  This stain is said to have been the later reason to plaster the entire castle red.  Such folklore later became the main motif of the enthralling prose of the Deštná priest Bedřich Kamarýt.

            In 1597 Bohuchval's son Jan bought his uncle Jiří's share of the castle, and instantly sold the reunited dominion to Vilém Rut of Dírná.  The Rut family had owned Dírná since the 14th century, and had bought Deštná from the Rožmberks in 1595.  Červená Lhota lay directly in the center and thereby joined the three dominions together.  Lhota never was separated from Deštná again.  The last of the Ruts, Bohuslav, had to leave the Bohemian lands as an Ultraquist after the Battle of White Mountain.

            In 1621 Červená Lhota was inhabited by Antonio Bruccio, a knightly commander of the Empirical army, an Italian noble who, in the service of the Slavats, oversaw the confiscation of Tábor and from 1621 served as commander of the municipal forces in Jindřichův Hradec.  A fiery Catholic and great Marianic venerator, Bruccio proved to be a good diplomat and manager.  He successfully protected the region from post-war pillaging, and strengthened the economic prosperity of Deštná by building a luxurious spa.  The only remainder of the spa today is the chapel of St. John the Baptist, built directly above the source of the healing mineral waters.  The dominion, evacuated by war, was shortly repopulated and Bruccio's own promise of loyalty to his Catholic subjects contributed to the settling of the area.  In 1639 Bruccio died without an heir, leaving great contributions to the Deštná church of St. Otto and the Jindřichův Hradec church of St. John the Baptist, where he was buried.  With his death, Lhota lost is function as a residence and served his successors simply as a simple occasional cotttage.

                After Bruccio's death, Červená Lhota passed into the management of the royal chamber, from which in 1641 the renowned aristocrat Vilém Slavat of Chlum and Košumberk bought it.  His drive, diplomatic skill, education, and high intelligence led him to a high official career.  From the position of royal marshall to Karlštejn burgrave to the president of the Bohemian chamber, he became the highest court-master of the Empirical court and soon afterwards the highest chancellor of the Bohemian kingdom, where he remained despite several resignations until his death.  His marriage to Lucia Otýlia of Hradec meant the integration into one of the largest dominions in Bohemia at the time, that of Jindřichův Hradec.  Červená Lhota then became a sort of summer residence, a place of parties, celebrations, and relaxation namely for the ladies of the Slavat family.  Among Vilém's great-grandchildren there were no sons, so with the wedding of the second-born daughter Marie Markéta of Ferdinand Vilém, the oldest of Vilém's grandsons, the castle passed into the hands of the Windischgrätz family.  Bedřich Arnošt Windischgrätz and his son Leopold dragged the dominion into great debts due to their out-dated style of economics, so the custodian of his under-aged successor Josef recommended the sale of the dominion.  In 1755 the castle then was obtained by the free lords of Gudenus.  Franz de Paul, free lord of Gudenus, shortly afterwards initiated several constructions.  Not only was the spa chapel in Deštná repaired, but the church of St. Otto was decoratively furnished, most of which is still preserved today.  Other building activities were brought to an abrupt halt in 1774 by a great fire, which destroyed essentially all agricultural buildings.  In 1776 Červená Lhota welcomed a new owner, Baron Ignác Stillfried, a progressive aristocrat of Prussian Silesia, who immediately sold the Deštná spas into private hands.  This definitively marked the end of Červená Lhota's aristocratic flavor.  The Baron wrote his chapter of history into the castle mainly as a host and sponsor of the aging composer Karel Ditters of Dittersdorf, whose lifelong pilgrimage ended here at the castle after a four-year residency.

    In 1820 Ignác's son sold the dominion to Jakub Veith.  If this enterprising industrialist and sponsor of Czech artists had any plans with Lhota, he evidently didn't manage to realize any of them.  His daughter Terezie sold the castle again in 1835, this time into the princely hands of Heinrich Eduard Schönburg-Hartenstein.

    Heinrich Eduard Schönburg-Hartenstein, a major in the Austrian army, diplomat and chamberlain, settled in his newly-purchased dominion Černovice in 1823, and bought Červená Lhota in 1835.  The center of his dominion, however, was Černovice, which became Heinrich's second home after Vienna.  It is curious, then, that he sold Černovice in 1872 and the only item he left for his son Josef Alexandr of his South Bohemian property was Červená Lhota.  Josef Alexandr Schönburg-Hartenstein, member of the Crown Council, was also active in diplomatic services.  His youngest son and heir was Prince Johann, chamberlain, bearer of the Order of the Golden Fleece, Great Cross of the Order of Leopold, Order of the Iron Crown, Great Cross of the Maltese Order and Order of Christ.  His greatest diplomatic accomplishment was fullfilled in his role as ambassador at the Pope's table in the Vatican.  The First World War and consequent dissolution of the beloved monarchy caused Johann to withdraw into the solace of his dearest seat, Červená Lhota, where he evidently devoted himself to the reconstruction of the castle.  In 1937 he was buried into the newly-built tomb, and thus was spared the destructive events of the new war, which drew the curtains closed for the entire aristocratic history of Červená Lhota castle.

    After the confiscation of the castle by the Czechoslovakian state in 1946 a children's clinic was established here, a year later it was assumed by the National Cultural Commission, and in 1949 it was opened to the public.