Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(server): Refresh projects based on internal timestamps #596

Merged
merged 2 commits into from
Jun 2, 2020

Conversation

jan-auer
Copy link
Member

@jan-auer jan-auer commented May 29, 2020

The last_fetch timestamp in project configs is set when it is written into the cache in Sentry. Since this timestamp may be arbitrarily old, this will force Relays to constantly refresh all project caches. While this is not such a big deal internally, since those requests go to a cache, it creates too much load for external Relays. Also, these time stamps are subject to clock drift and should not be trusted.

As such, the last_fetch timestamp has no useful semantics. Instead, the instant of loading the ProjectState should be used to determine the last fetch.

Since fetching projects is now a fast operation (both internally and externally), the overcomplicated logic for creating jitter is no longer appropriate. Instead, a simple time-based backoff for refreshes is absolutely sufficient.

Reverts #277

@jan-auer jan-auer requested a review from a team May 29, 2020 15:46
@jan-auer jan-auer self-assigned this May 29, 2020
@jan-auer
Copy link
Member Author

Example of what happened:

  • A project state is written into the cache with last_fetch = 29 Mai 2020 17:21:38
  • Relay fetches this state at 29 Mai 2020 17:42:10, but with last_fetch = 29 Mai 2020 17:21:38
  • The project state is immediately soft-outdated and refreshed.

Copy link
Member

@untitaker untitaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would not necessarily attempt removing the jitter in the same deploy but this makes perfect sense

@jan-auer
Copy link
Member Author

Verified behavior manually for external Relays, today. We can do a staged rollout on Tuesday with @tonyo.

@jan-auer jan-auer merged commit 896d324 into master Jun 2, 2020
@jan-auer jan-auer deleted the fix/dont-jitter-project-requests branch June 2, 2020 08:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants