Add indexes for user_management_user

[Red-M]

as larger instances struggle with slow queries

What this PR does

While running Grafana Oncall, I found that the select statements behind showing the current schedule of all users would generate sequential queries using regex to parse a JSON column and also using lower() on the email column, causing a high CPU load on the DB.

Which issue(s) this PR fixes

This doesn't have a github issue.
We found that a larger instance of Oncall (207 rows in user_management_user) could have this table take up to 30 seconds to complete the table scan and that these indexes reduced that down to 600ms-900ms (averaging at about 700ms)

Checklist

• [-] Unit, integration, and e2e (if applicable) tests updated
• [-] Documentation added (or pr:no public docs PR label added if not required)
• CHANGELOG.md updated (or pr:no changelog PR label added if not required)

[CLAassistant]

All committers have signed the CLA.

[matiasb]

Thanks for the contribution! A few minor comments added.

[matiasb]

Any reason for ordering fields differently? (ie. "organization", "is_active", "username" vs "username", "organization", "is_active")

+1 to add some indexing to the User model, wondering if keeping "role" and "permissions" out wouldn't be enough (thinking on space)? so we only add:
"organization", "is_active", "username"
"organization", "is_active", "email"

[Red-M]

The order of the indexing matters for performance reasons depending on the SQL engine.
Cardinality matters as it reduces the search time for rows, which this is attempting to eliminate as many rows as possible to remove the very costly table scan.

All 4 indexes are required as they're effectively used as a set of joins to reduce the number of rows down to (preferably) 1 row (in the best case).

[matiasb]

By any chance did you check if the recent updates around schedules improved your loading times (before applying these changes)? (e.g. besides some indexing added, there was some refactoring on how users details are fetched when RBAC is enabled)

OTOH, creating an index including a JSON column (permissions) is not possible in MySQL, migration fails with:

django.db.utils.OperationalError: (3152, "JSON column 'permissions' supports indexing only via generated columns on a specified JSON path.")

So it will require some extra work to enable it (eventually we may need to re-think how permissions are stored/checked).

[Red-M]

By any chance did you check if the recent updates around schedules improved your loading times (before applying these changes)? (e.g. besides some indexing added, there was some refactoring on how users details are fetched when RBAC is enabled)

No, technically speaking these changes aren't exactly what I had to apply since django won't accept:

CREATE INDEX user_management_user_urn_org_id_is_active_role USING BTREE
    ON `user_management_user` (`organization_id`, `is_active`, `username`, `role`);
    
CREATE INDEX user_management_user_lower_email_org_id_is_active_role USING BTREE
    ON `user_management_user` (`organization_id`, `is_active`, (lower(`email`)), `role`);

CREATE INDEX user_management_user_lower_urn_org_id_is_active_perms USING BTREE
    ON `user_management_user` (`username`, `organization_id`, `is_active`, (cast(json_unquote(permissions) as CHAR(500))) );

CREATE INDEX user_management_user_lower_email_org_id_is_active_perms USING BTREE
    ON `user_management_user` ((lower(`email`)), `organization_id`, `is_active`, (cast(json_unquote(permissions) as CHAR(500))));

OTOH, creating an index including a JSON column (permissions) is not possible in MySQL, migration fails with:

django.db.utils.OperationalError: (3152, "JSON column 'permissions' supports indexing only via generated columns on a specified JSON path.")

So it will require some extra work to enable it (eventually we may need to re-think how permissions are stored/checked).

Yeah I can see that in the tests, I'm not sure Django's ORM is giving a good experience here.

[Red-M]

Also #3067 is pretty poor form to pull the rug out from this PR.