Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change the behavior when computing if a file is outdated #26

Merged
merged 6 commits into from
Sep 10, 2022
Merged

Change the behavior when computing if a file is outdated #26

merged 6 commits into from
Sep 10, 2022

Conversation

ronisbr
Copy link
Contributor

@ronisbr ronisbr commented Jul 20, 2022

Currently, we have the following behaviors given an update interval:

  • :daily: The file is updated if the current day number is grater than the file day number.
  • :weekly: The file is updated if a Monday has passed after the file date.
  • :monthly: The file is updated if the first day of a new month has passed after the file date.
  • :yearly: The file is updated if the first day of the year has passed after the file date.

The problem are with files that are updated in those intervals, but at the end of the window. For example, if a file is updated every December 30, then RemoteFiles will download it repeatedly until December 30 of the next year.

This commit treats the intervals relative to the current date of the file. Hence, if the file mtime is December 30, 2021, and the update is set to :yearly, then RemoteFiles will only download a new copy after December 30, 2022. Thus, the new behaviors are:

  • :daily: The file is updated if 1 day has passed from the file date.
  • :weekly: The file is updated if 7 days has passed from the file date.
  • :monthly: The file is updated if 30 days has passed from the file date.
  • :yearly: The file is updated if 365 days has passed from the file date.

A problem with this approach exists if the download tool stamps the mtime with the time in which the file was downloaded instead of the time in which the file was modified at the source. However, both curl and wget seems to keep the original time stamp.

I also modified in this commit how the current time is obtained. It uses now(UTC) instead of now() because the file timestamp (obtained with unix2datetime) will always be in UTC.

Closes #25

@ronisbr
Copy link
Contributor Author

ronisbr commented Aug 26, 2022

Bump! :)

@helgee helgee merged commit 700dc73 into helgee:master Sep 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Change behavior of weekly update
2 participants