Unfortunately, the doors to Bab Mansour are usually closed and you can't pass directly through the gate like you can Bab Boujloud. So you have to follow the walls for a bit until you find an alternate way through.