Announcement about implicit addition of "_total" to counter metric names?

[erikjohnston]

We've just been bitten (matrix-org/synapse#4001) by the fact that Counter now implicitly adds _total to the end of the metrics name. This is unfortunate as people who run our services are now discovering a bunch of their metrics have disappeared from their monitoring dashboards, causing confusion. (Yes we should probably have pinned our dep explicitly to 0.3.x).

My question is really just whether this was publicised anywhere before the release? The Prometheus docs seem to only recommend that counters end in _total, as opposed to saying that it's effectively mandated by the official client libraries. Given the seeming propensity inside the python community to not pin dependencies to major versions I think a bit of warning would have been appreciated :)

Sneaky tangential question: is the plan to update the other clients to match this new behaviour?

[brian-brazil]

It's the first item noted in the release notes, and it is mentioned in the docs.

[richvdh]

I understand that the ship has sailed on this, but please please please think twice before introducing changes like this in the future: it comes as a huge surprise to everybody when their dashboards and alerting suddenly stop working because a bunch of metrics have suddenly changed name.

Perhaps you could consider a deprecation period where a warning is omitted when an application uses the old name, so that application developers have time to prepare for a forthcoming change.