Healthbeat 8.12.2 does not capture metrics for processes owned by root when running as a non-root user

[manoj-vunet]

Until version 8.1.2, Healthbeat could still collect metrics from root-owned processes when running as a non-root user. However, starting from v8.12.2, this is no longer the case—processes owned by root are not reported at all when Healthbeat is executed with non-root privileges.

This change is due to the transition from the legacy Beats package to the newer Elastic Agent system metrics package, which affects how process metrics are collected.
Process Metrics Collection PackagesElastic Agent System Metrics Package (Helathbeat 8.12.2)
      Repository: github.com/elastic/elastic-agent-system-metrics/metric/system/process
Beats libbeat Metric Package (Healthbeat 8.1.4)
     Repository: github.com/elastic/beats/v7/libbeat/metric/system/process

When running as a non-root user, the new Elastic Agent system metrics package (github.com/elastic/elastic-agent-system-metrics/metric/system/process) may be limited in its ability to retrieve process details for processes owned by other users. This behavior stems from Linux permission restrictions on accessing certain information in the /proc filesystem.
In contrast, the older Beats package (github.com/elastic/beats/v7/libbeat/metric/system/process) was often used in environments where it either ran as root or with elevated privileges. This allowed it to collect data from all processes regardless of ownership.

Can someone review this, provide insights into how both libraries function, and suggest a solution to resolve this issue?

[botelastic]

This issue doesn't have a Team:<team> label.