Change the behavior when computing if a file is outdated #26
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, we have the following behaviors given an update interval:
:daily
: The file is updated if the current day number is grater than the file day number.:weekly
: The file is updated if a Monday has passed after the file date.:monthly
: The file is updated if the first day of a new month has passed after the file date.:yearly
: The file is updated if the first day of the year has passed after the file date.The problem are with files that are updated in those intervals, but at the end of the window. For example, if a file is updated every December 30, then RemoteFiles will download it repeatedly until December 30 of the next year.
This commit treats the intervals relative to the current date of the file. Hence, if the file
mtime
is December 30, 2021, and the update is set to:yearly
, then RemoteFiles will only download a new copy after December 30, 2022. Thus, the new behaviors are::daily
: The file is updated if 1 day has passed from the file date.:weekly
: The file is updated if 7 days has passed from the file date.:monthly
: The file is updated if 30 days has passed from the file date.:yearly
: The file is updated if 365 days has passed from the file date.A problem with this approach exists if the download tool stamps the
mtime
with the time in which the file was downloaded instead of the time in which the file was modified at the source. However, bothcurl
andwget
seems to keep the original time stamp.I also modified in this commit how the current time is obtained. It uses
now(UTC)
instead ofnow()
because the file timestamp (obtained withunix2datetime
) will always be in UTC.Closes #25