Sarah Ferguson was told by the late Queen to "pack her bags and leave" Balmoral Castle after an embarrassing incident in 1992. The Duchess of York was spending the summer holidays at Balmoral with her daughters when a newspaper published paparazzi photos of her in a compromising position.