We do know the fort was used by Humayun, renovated by his successor Akbar I, and renovated again by Shah Jahan. The emperor in between Akbar and Shah Jahan was Jahangir. These four, plus Shah Jahan's son and successor Aurangzeb all lived here during their reigns. In fact, this is the only fort in all of India where all five of these emperors lived.