accumulated calibration fixes and improvements (mostly magnetometer)

[dagar]

This pull request combines a few mag sensor pipeline and calibration related improvements and fixes that started to overlap.

New functionality

1. sensor mag drop all remaining IOCTLS, move calibration downstream to sensors module entirely with parameters, and perform aggregation in standalone WorkItem rather than voted_sensors_update
   • mag calibration is never cleared, only updated after a new successful calibration
   • this largely mirrors the changes down for accel & gyro recently.
2. magnetometer calibration automatically set the rotation of external magnetometers relative to the first internal (see [WIP] commander: auto set external mag rotation during calibration if possible #15120 for more detail).
3. per sensor instance configurable priority (CAL_ACCx_PRIO/CAL_GYROx_PRIO/CAL_MAGx_PRIO)
   • this replaces the existing enable/disable parameters (CAL_ACCx_EN/CAL_GYROx_EN/CAL_MAGx_EN) and the uORB priority concept
4. new helper systemcmds/mag_test that simply runs and prints the heading of each sensor_mag instance until you quit I'll bring this in later.
5. remove conservative subscription interval during data acquisition
   • this allows you to go through the full mag cal quite a bit faster (I've compared data before and after on a couple setups)
6. store and use full soft iron calibration matrix (new parameters CAL_MAGx_{X, Y, Z}ODIAG)
   • currently in master we only keep the diagonals
   • mag cal soft iron distortions #8265, [WIP] Pr compasscal offdiag #9136, etc

Along the way I discovered a number of potential bugs and the need to properly carry forward additional data in calibration "slots" that led to the consolidation of these changes. One problem was again the potential inconsistency between sensor instances (uORB, /dev/magX, calibration slots) that can shift if sensor ordering changes (external mags changed or startup script).

EDIT (@MaEtUgR): fixes #14829

[bresch]

All good on my side, I tried a few orientations again and the detection works really well.
Maybe only point for later to improve: you currently don't need to perform a full rotation to complete one axis; doing some small left-right rotations is enough to pass to the next orientation, which isn't ideal.

[MaEtUgR]

doing some small left-right rotations is enough to pass to the next orientation, which isn't ideal.

Not ideal but not a regression, tricking it was already easy before.

[dagar]

doing some small left-right rotations is enough to pass to the next orientation, which isn't ideal.

Not ideal but not a regression, tricking it was already easy before.

It's a bit more obvious now because you can wiggle in place and force it through faster. It's also slightly harder though as it must accept samples from all mags in the same update.

https://github.com/PX4/Firmware/blob/bae2589feda03bb379e62f425ce2945caec747e8/src/modules/commander/mag_calibration.cpp#L268-L285

I found that as long as it was the full 6 orientation calibration it's actually pretty hard to mess up even if you squander a few sides completely without rotating. We definitely should encourage people to use the full 6 orientation unless it's not possible, and that's one of the reasons I tried to make it much faster.

I also considered running little attitude estimators in place and really requiring proper clean rotations, but that had implications for auto rotation later that I didn't have time to work through.

[dagar]

Thanks everyone!

[MaEtUgR]

Good timing, I just started testing it and I noticed:

• Nice UX during mag calibration! Calibration seems like it should be 👍
• After any calibration (gyro, accel, mag) I had a manual control lost just when it finished successful. Not sure if that's the timing on the autopilot or QGC side (I had virtual joysticks)
• It added "empty" parameters for an ACC2, MAG2 and MAG3 which are not present

[dagar]

After any calibration (gyro, accel, mag) I had a manual control lost just when it finished successful. Not sure if that's the timing on the autopilot or QGC side (I had virtual joysticks)

I believe this is QGC doing a full param sync after calibration. I'm going to start keeping a closer eye on mavlink performance and latency to see if we can do anything about this and other issues. In particular for manual control I don't yet know if it's the QGC stick broadcast getting blocked or the PX4 mavlink module not receiving/processing them properly.

It added "empty" parameters for an ACC2, MAG2 and MAG3 which are not present

This is an intentional reset of those slots. After you reboot they won't be present. I also plan to update the parameter system at some point so that we're not storing a reset parameter at all.

Thanks for testing.