I'm just getting into long range FPV and I'm in the process setting up my GPS rescue.
Since there is an option to enable Magnometer in BF, I'd like to use it. I'm currently using the Foxeer M10Q 180 module with built-in compass. I have already tested GPS recue on a switch, with barometer and magnometer enabled, and am getting ready to test failsafe on a switch next.

The thing is, yesterday after the fieldtest I found out in the Betaflight WIKI that it is NOT advised to use the magnometer at all.
However, a calibrated noise free magnometer data will benefit the GPS rescue sequence. And it seems to working fine..

Does anyone have any experience with magnometers? What are the risks? Does it help? Or should I just leave it off and rely completely on GPS (and baro) only as recommended by the Betaflight wiki.

I did not manage to set up the compass well on the Foxeer M10Q-120 GPS receiver. Compass readings fluctuated all the time.
On another quadcopter I used an external i2c compass and I also couldn't get good results to get the compass to always point north correctly. The compass readings were constantly changing, I could not achieve accuracy.

I ended up turning off the compass in Betaflight.

My experience is that the return home can be in a wide arc rather than a straight line. If the return height is set to 30 meters, and the drone is currently flying at a height of 50 meters, while the drone is descending from a height of 50 meters to a height of 30 meters, it can fly in any direction at that time. When my drone descended to a height of 30 meters - only then did it fly towards the house.

P.S.
compass = magnetometer.

I guess I was either really lucky doing those 3 GPS rescue tests in a row, or somehow I've managed to calibrate it correctly.
I also did the U-Center almanac download thing to get quicker GPS lock. Maybe thats why it worked as expected.
Until now I haven't seen any compass fluctuation like u mentioned, but now I am going to check it everytime before take-off.

It seemed like going flying back was in a straight line.. Descending however was a bit bumpy..
https://youtube.com/shorts/yU2kWkx5jYI?feature=share
Is this also normal behaviour?

The risk you run is that if you end up with some kind of magnetometer malfunction or issues with the readings during flight, you risk your quad having a flyaway if GPS Rescue is initiated. This is why the Betaflight devs advise against using a magnetometer. So it's up to you whether you want to take that risk or not. GPS Rescue works perfectly fine without the use of a magnetometer so unless you have another need for the compass direction (such as in your OSD) then it might be safer / wiser to switch off or disconnect the magnetometer. That is just my own opinion of course, but it's what I would do.

The "pulsing motors" behaviour as seen in your video is normal when GPS Rescue is automatically bringing the quad down to land under Betaflight 4.4.x.

I agree. Disabling it seems the wisest thing to do for now. 
Thanks for the input..

I agree, you should not use the magnetometer (compass) - it will cause problems. If you insist, you might try INAV