The permits are free and I don't think they can run out, but the reason you have to get a permit is to force you to have face time with a ranger who explains the rules around going into the caves. These rules are important to protect the bat population of the caves.