There are a few things that can cause this kind of behaviour when you yaw.
The first, and most likely, is the reason that Scotty suggests.
There can be other causes though. To make the quad yaw, the flight controller slows down one pair of motors and speeds up the other pair. So if you want to yaw anti-clockwise for example, the flight controller speeds up the clockwise motors and slows down the anti-clockwise ones, like this:
The amount that one pair of motors are asked to increase is mirrored in the other pair as an equal decrease, so in theory, the downward thrust from all four motors is still the same. That's the theory.
As we all know, theory and practice can be quite different!
First of all, you are using ESCs that don't have active braking. This means that they will slow down more slowly than they will speed up. This should result in always gaining altitude when you yaw, so that's probably not the explanation.
Secondly, the relationship between throttle setting and generated thrust is not linear, but the flight controller assumes that it is. So if you are in a steady hover and you yaw slowly, it will have a different effect on altitude than yawing quickly because the amount that the motors are asked to speed up and slow down is not the same and the overall thrust produced will be different.
Thirdly, the flight controller is also assuming that each motor creates the same amount of thrust at the same throttle setting. Within a certain margin, this should also be true. If it isn't true, and one motor is producing more power than the others at a given throttle setting, then a yaw movement will make that motor produce much more or less thrust than expected. This will immediately start to create uncommanded movement on the pitch or roll axis which the flight controller will then start to compensate for. This will then lead to an inbalance in overall thrust and a corresponding gain or loss of altitude.
There are quite a few reasons why this could happen:
1. One of the motors is not mounted perfectly flat. If the propeller disc is not at exactly the same angle on all motors, the thrust generated by each motor won't be the same.
2. The propellers are not the same. "Same" means the same model from the same manufacturer and not just the same diameter and pitch. A GemFan 5030 and an HQ 5030 do not produce the same amount of thrust.
3. The ESCs have not been calibrated correctly.
4. One of the ESCs, even though calibrated, is damaged and cannot drive the motor as well as the others.
5. One of the motors is damaged. If you overheat a motor, then above a certain temperature, the magnets in the motor's bell will start to demagnetise. When this happens, the kv rating of the motor will increase. So if only one motor has been badly overheated, you could end up with three motors that have 2300kv and one which has 3000kv.
The easiest way to see if your yaw behaviour is caused by your fingers on the sticks or by a motor problem is to record a black box log.
The log includes your stick positions, so it's very easy to see if you are inadvertently changing the throttle as you move the left stick to yaw. The log also shows the throttle level being sent to each motor. If your quad is properly balanced so that its weight is evenly distributed between the four motors, then in a stable hover, the throttle level sent to each motor should be similar.
If one motor has a consistently higher throttle level than the other three while hovering and the quad is properly balanced, then for some reason, that motor is not producing the same amount of thrust as the other three and you should determine why and correct the problem.
This is the beauty of black box logging. It lets you look at both what you and your quad are doing and diagnose problems with hard data rather than guesswork.