Once Max II bought it back, it was used as his summer home and hunting retreat, and by extension that meant it was also the summer home of his two sons (Mad King Ludwig and his mentally-ill brother Otto, who we discussed back in Munich)