Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(sampling): Add profile context early and expose getter for it #2715

Merged
merged 6 commits into from
Nov 13, 2023

Conversation

jjbayer
Copy link
Member

@jjbayer jjbayer commented Nov 10, 2023

In order to create dynamic sampling rules depending on the profile ID, add the profile ID to the transaction event's profile context and expose it via the event's Getter implementation.

Note that most transaction processing still occurs only in processing relays, and it might happen that events that have a profile ID at the time of sampling will not have a profile ID anymore after processing.

Fixes #2710

@jjbayer jjbayer changed the title Feat/ds profile feat(sampling): Add profile context early and expose getter for it Nov 10, 2023
/// Unique identifier for a profile.
///
/// Same format as event IDs.
pub type ProfileId = EventId;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this type alias to make the return type more obvious in function signatures.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we just new type it here instead?

Then it's actually a separate type and cant get mixed

contexts.remove::<ProfileContext>();
}
}
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This block was copied and adapted from process_profiles.

Comment on lines +1200 to +1206
if found_profile_id.is_none() {
// Remove profile context from event.
if let Some(event) = state.event.value_mut() {
if event.ty.value() == Some(&EventType::Transaction) {
if let Some(contexts) = event.contexts.value_mut() {
contexts.remove::<ProfileContext>();
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When profile processing fails, the profile ID is removed here. This means that some (invalid) profiles that were opted in by a dynamic sampling rule based on contexts.profile.profile_id might end up without a profile ID in the final payload.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this implies that we are performing sampling on profile ids that we don't know whether they are valid of not, am I right?

From the code flow, I see that we do filter the profile with context injection, we then move onto sampling (assuming the profile id refers to a valid profile), and then we process the profile and remove the context in case it's invalid?

Is this an expected behavior? Just curious

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this implies that we are performing sampling on profile ids that we don't know whether they are valid of not, am I right?

That is correct and a bit unfortunate. To fix this we would have to move the entirety of process_profiles to non-processing relays (and before dynamic sampling), and I'm not sure if that's currently possible. cc @jan-auer

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good, thanks for the explanation!

@jjbayer jjbayer marked this pull request as ready for review November 10, 2023 22:16
@jjbayer jjbayer requested review from a team and phacops November 10, 2023 22:16
Copy link
Contributor

@olksdr olksdr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

It might be a good idea to add a test for this, to make sure that profile id will be added where it's needed.

@jan-auer jan-auer self-assigned this Nov 13, 2023
Copy link
Member

@jan-auer jan-auer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The caveat of over-sampling is fine, at least it will still catch events with a profile.

In the longer run, a sampling rule like this might not have the ideal semantics. We would ideally want to sample events where there's a profile in the same trace. However, we're currently not able to guarantee such a rule.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make it possible to create Investigation Bias with has:profile.id
6 participants