Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(metrics): Add a metrics aggregator #958

Merged
merged 32 commits into from
Mar 26, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
671b613
feat(metrics): Metrics aggregator as actor
jjbayer Mar 18, 2021
5f8d13f
feat(metrics): Set bucket_interval
jjbayer Mar 18, 2021
594183d
wip: Flushing aggregator impl
jan-auer Mar 19, 2021
e0e5fad
ref(metrics): Unify metric values and types
jan-auer Mar 19, 2021
e6755e6
ref: Cleanup
jan-auer Mar 22, 2021
85b5a16
test(metrics): Get tests into a compiling state
jan-auer Mar 22, 2021
b125102
Merge branch 'master' into feat/metrics-aggregate-envelopes
jjbayer Mar 22, 2021
c5cd75c
fix: Flatten serialized metric
jjbayer Mar 22, 2021
855e877
fix: Handle non-representable instants
jjbayer Mar 22, 2021
a46a2e8
test: Fix tests
jjbayer Mar 22, 2021
6579f79
doc: Add missing documentation to bucketing.rs
jjbayer Mar 23, 2021
9c1ff2c
test: Try to insert wrong metric type into aggregator
jjbayer Mar 23, 2021
4934656
ref: Rename module 'bucketing' to 'aggregation'
jjbayer Mar 23, 2021
08a374f
ref: generic merge function
jjbayer Mar 23, 2021
f0914e7
test: Use relay-test setup
jjbayer Mar 23, 2021
bd0f7f6
ref: Rename and simplify unbounded instant
jjbayer Mar 23, 2021
b826bea
ref: Simplify MergeValue trait
jjbayer Mar 23, 2021
ed09081
feat: delay() utility in relay-test
jjbayer Mar 24, 2021
05482f3
test: Test bucket flushing
jjbayer Mar 24, 2021
5b7e034
doc: A bit more documentation
jjbayer Mar 24, 2021
8c87fd0
test: Check if bucket gets merged back if receiver denies it
jjbayer Mar 24, 2021
a03fb4f
fix: flatten MetricKafkaMessage
jjbayer Mar 24, 2021
ac2bab2
feat(metrics): Support serialization of buckets
jan-auer Mar 25, 2021
997b0f3
test: Fix doctest
jjbayer Mar 25, 2021
ee0f3b4
fix: Add metric type to bucket key
jjbayer Mar 25, 2021
d6978f6
ref: Declaration order
jjbayer Mar 25, 2021
66f6370
doc: Some more docs
jjbayer Mar 25, 2021
47d6fdc
ref: Remove redundant InsertMetric
jjbayer Mar 25, 2021
aed4194
Merge branch 'master' into feat/metrics-aggregate-envelopes
jan-auer Mar 26, 2021
2b3ed57
doc(metrics): More doc comments
jan-auer Mar 26, 2021
687d7bf
meta: Changelog
jan-auer Mar 26, 2021
e381b27
fix(metrics): Add imports for doctest
jan-auer Mar 26, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,8 @@
**Internal**:

- Emit the `quantity` field for outcomes of events. This field describes the total size in bytes for attachments or the event count for all other categories. A separate outcome is emitted for attachments in a rejected envelope, if any, in addition to the event outcome. ([#942](https://github.com/getsentry/relay/pull/942))
- Add experimental metrics ingestion without bucketing or pre-aggregation. ([#948](https://github.com/getsentry/relay/pull/948))
- Add experimental metrics ingestion with bucketing or pre-aggregation. ([#948](https://github.com/getsentry/relay/pull/948), [#952](https://github.com/getsentry/relay/pull/952), [#958](https://github.com/getsentry/relay/pull/958))
- Change HTTP response for upstream timeouts from 502 to 504. ([#859](https://github.com/getsentry/relay/pull/859))
- Disallow empty metric names, require alphabetic start. ([#952](https://github.com/getsentry/relay/pull/952))
- Add rule id to outcomes coming from transaction sampling. ([#953](https://github.com/getsentry/relay/pull/953))
- Add support for breakdowns ingestion. ([#934](https://github.com/getsentry/relay/pull/934))

Expand Down
15 changes: 15 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

53 changes: 51 additions & 2 deletions relay-common/src/time.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,22 @@ pub fn instant_to_date_time(instant: Instant) -> chrono::DateTime<chrono::Utc> {
instant_to_system_time(instant).into()
}

/// A unix timestap (full seconds elapsed since 1970).
#[derive(Clone, Copy, Eq, Ord, PartialEq, PartialOrd)]
/// The conversion result of [`UnixTimestamp::to_instant`].
///
/// If the time is outside of what can be represented in an [`Instant`], this is `Past` or
/// `Future`.
#[derive(Clone, Copy, Debug)]
pub enum MonotonicResult {
/// A time before the earliest representable `Instant`.
Past,
/// A representable `Instant`.
Instant(Instant),
/// A time after the latest representable `Instant`.
Future,
}

/// A unix timestamp (full seconds elapsed since 1970-01-01 00:00 UTC).
#[derive(Clone, Copy, Eq, Hash, Ord, PartialEq, PartialOrd)]
pub struct UnixTimestamp(u64);

impl UnixTimestamp {
Expand Down Expand Up @@ -54,6 +68,41 @@ impl UnixTimestamp {
pub fn as_secs(self) -> u64 {
self.0
}

/// Converts the UNIX timestamp into an `Instant` based on the current system timestamp.
///
/// Returns [`MonotonicResult::Instant`] if the timestamp can be represented. Otherwise, returns
/// [`MonotonicResult::Past`] or [`MonotonicResult::Future`].
///
/// Note that the system time is subject to skew, so subsequent calls to `to_instant` may return
/// different values.
///
/// # Example
///
/// ```
/// use std::time::{Duration, Instant};
/// use relay_common::{MonotonicResult, UnixTimestamp};
///
/// let timestamp = UnixTimestamp::now();
/// if let MonotonicResult::Instant(instant) = timestamp.to_instant() {
/// assert!((Instant::now() - instant) < Duration::from_millis(1));
/// }
/// ```
pub fn to_instant(self) -> MonotonicResult {
let now = Self::now();

if self > now {
match Instant::now().checked_add(self - now) {
Some(instant) => MonotonicResult::Instant(instant),
None => MonotonicResult::Future,
}
} else {
match Instant::now().checked_sub(now - self) {
Some(instant) => MonotonicResult::Instant(instant),
None => MonotonicResult::Past,
}
}
}
}

impl fmt::Debug for UnixTimestamp {
Expand Down
7 changes: 6 additions & 1 deletion relay-metrics/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,14 @@ license-file = "../LICENSE"
publish = false

[dependencies]
actix = "0.7.9"
hash32 = "0.1.1"
relay-common = { path = "../relay-common" }
relay-log = { path = "../relay-log" }
serde = { version = "1.0.114", features = ["derive"] }
serde_json = "1.0.55"

[dev-dependencies]
futures = "0.1.28"
insta = "1.1.0"
serde_json = "1.0.55"
relay-test = { path = "../relay-test" }
Loading