The original castle was built in the 13th century as part of the defense wall around Brussels. In the following centuries, the castle was destroyed and rebuilt several times. Its position evolved from strategic castle to summer residence and outdoor residence. It was consecutively inhabited by several noble families. The Count of Egmond was one of the best-known owners.
At the end of the 18th century, the castle was owned by the Italian archer family Arconati Visconti. It became a center where scholars and artists met. At the end of the 19th century, marchioness Arconati Visconti, the last owner, took the initiative for a thorough renovation. She set up the castle as a museum for her huge private art collection. The marchioness donated the castle to the Belgian state in 1921. Three years later it was opened as a public museum.
