From my last flight, I ended up with a slightly puffed lipo battery, I am a bit concerned. 
Is it possible to catch fire in flight? Or is it only dangerous when when charging?

is it dangerous when it's not charging or discharging, e.g. is it likely that a LiPo battery can catch fire during storage?

I have not had a fire yet, but my suggestion is not to take any chances.

I think you are supposed to let the battery settle for half hour after crash.
Pay attention to notice any physical damage and watch for any signs of smoke or fire. Afterwards discharge completely and dispose.



Some people do continue to run puffy lipos but I wouldn't, your just asking for trouble.
If you don't want to throw out the whole pack, you can disassemble and remove the damaged cell and keep the good ones to match up with other good cells of the same brand/type/ratings.

I've built a few packs with my nano techs since they are all identical and similar IR. Don't mix and match different cells if they are different brands even if they are same ratings and mah.

Some people put it in salt water for a week to dispose, some use a discharger you can build yourself:
http://blog.oscarliang.net/lipo-discharger-light-bulb/