I installed four new spedix 20A dshot capable ESCs, enabled dshot 600 and then updated to latest Blheli_S. 

I wanted to check gyro noise so each motor in Blhelisuite I raised to max throttle, gradually and then suddenly. Stayed at max for less than a second. One of the escs smoked. 

I'm surprised as I would think the motor, a 2300kv would not be exceeding 20a with no prop. Even if it did the ESC should be able to handle high burst current. Was that a bad thing to do? How else do you observe gyro noise on the test bench?

It's not the current. Without the prop current draw is very low (maybe 1 or 2 amps).
It's the RPM that kills the ESC.
You don't want to spin unloaded motors/ESCs at max throttle for too long.

Sometimes a brand new ESC will simply blow up the first time you use it. I'd much rather it happens on the bench than in the air.

As Tom said, without a propeller loading the motor, it won't draw much current, even at full throttle. As your ESCs are BB2 based BLHeli_S ESCs, the maximum unloaded RPMs present no problem. The motor won't spin any faster than it's kv rating multiplied by the number of volts and that is well below the maximum RPM limit for these ESCs.

The motors will get hot though if you run them at full throttle and no load for too long, so you should limit the amount of time that you run them on the bench at high throttle levels. You should also check to see how hot they get when testing like this and stop if they get so hot that you can't hold your finger on the bell for several seconds.

Re-reading your post, you said that you were increasing throttle to max both slowly and suddenly. There is a risk that raising the throttle very suddenly to maximum could cause a desync, especially when the motor is unloaded and RPMs can increase very quickly indeed. A desync is never a good thing and there is a chance that it can cause the ESC to fail. In flight, the PID controller won't go from minimum throttle to maximum throttle instantly. While you are testing in the motors tab though, there's nothing to stop you doing this and going straight from min to max might not be wise.