I bought a couple JJRC h36 (the same thing as an Eachine e010) as Christmas presents and also bought this battery package at the same time.  I get them in and see that the batteries are the exact same ones that came with the H36, but the connector is smaller.  
So now I've got 3 difference micro batteries and I need to figure out a way to use them all, but I don't even know what to look for (i.e. what the different types are called).  

There is the flat kind I have on my 410mAh batteries.
The boxy one with the single guide bar that comes with the H36 (like this).
The tiny one on the new batteries I bought that are the same battery but smaller connector (like this).

So does anyone know what the actual names for the different connectors are so I can make my own wires?

The amazon link: Micro JST 1.25mm 2-pin (this is the standard "TinyWhoop" connector - supposed to be not so great in the long term and limited current... but... seems to be working great for a very large population of users, honestly)
The JJRC H3g: mCPX (JST-PH 2-pin) mCPX is supposed to be much better (although bigger) than these others..... I don't know why... higher current, longer lasting?
I didn't see a link for the 410mAh ones ... but when you say "flat" i assume, Losi Style (https://www.amazon.com/Walkera-Battery-C...+connector)

If you can charge it - mCPX is probably the best (it's JST-PH) you can get those on digikey ... or you can build an adapter ... which can be a pain.

You can see Sir Oscar did that: http://intofpv.com/t-using-tiny-whoop-ba...chine-e010

Spot on with all of them! So now I just need to grab some Micro JST 1.25mm 2-pin female ends and some mCPX male ends and I can make an adapter and I'll be able to use all of my batteries!

Awesome, thank you!

Cool. Cool. Cool! Have fun!