Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(metrics): Add metadata for buckets #3254

Merged
merged 11 commits into from
Mar 14, 2024
Merged

Conversation

Dav1dde
Copy link
Member

@Dav1dde Dav1dde commented Mar 12, 2024

Adds BucketMetadata to the metrics Bucket. The metadata can be used to track meta information about the bucket, like amount of aggregations (total merges).

The metadata currently only tracks the amount of "merges".

Bucket metadata is a purely Relay internal detail, clients are not expected to send it and Relay will ignore and reset the metadata if it is not coming from a trusted/internal Relay. This makes it possible to to implement the received timestamp for metrics while at the same time forwarding the metadata correctly from PoPs to Processing Relays without any additional configurations.

This PR contains:

  • Implementation of "merges" metadata
  • Extraction of Relay Id from Envelope requests
  • Resetting the metadata from unknown/non-internal sources

The metadata is currently unused, I will add all missing integration tests once it is possible to properly assert the metadata in an integration test. I have manually verified that the aggregation of metadata is working.

Change is smaller than it looks, most of it is just test snapshots.

@Dav1dde Dav1dde requested a review from a team as a code owner March 12, 2024 12:18
@Dav1dde Dav1dde self-assigned this Mar 12, 2024
@Dav1dde Dav1dde mentioned this pull request Mar 12, 2024
Copy link
Member

@jjbayer jjbayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! I'm unsure about the .await in from_request_parts.

What happens when a bucket goes into the pre-aggregator because a project state has not been loaded? Will the accounting be correct?

@Dav1dde Dav1dde requested a review from jjbayer March 13, 2024 09:11
@Dav1dde Dav1dde enabled auto-merge (squash) March 14, 2024 07:55
@Dav1dde Dav1dde merged commit 437fe72 into master Mar 14, 2024
20 checks passed
@Dav1dde Dav1dde deleted the dav1d/feat/bucket-metadata branch March 14, 2024 08:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants