Merge pull request #1216 from resource-watch/cit_002

update outdated check

Author: weiqi-tori

check_if_nrt_datasets_current/contents/src/__init__.py

@@ -146,7 +146,7 @@ def main():
             allowed_time = datetime.timedelta(days=10)
         # Fire Risk Index often goes offline for a few days
         elif 'for.012' in r['data']['attributes']['name']:
-            allowed_time = datetime.timedelta(days=10)
+            allowed_time = datetime.timedelta(days=30)
         # Conflict and Protest Events Dataset (working on it, hold for 20 days)
         elif 'soc.016' in r['data']['attributes']['name']:
             allowed_time = datetime.timedelta(days=20)

check_if_nrt_datasets_current/outdated_nrt_scripts.csv

@@ -1,18 +1,18 @@
 WRI ID,Public Title,Date Added to Sheet,Last Check,Known Reason?,Script in pause
-cit.003a.rw0.nrt,Air Quality: Fine Particulate Matter (PM2.5) Station Measurements,11/9/2023,11/25/2024,OpenAQ updated its API and data fetching options. The script cannot directly get data from their S3 bucket anymore. Need to register an account and use their API instead.,TRUE
-cit.003b.rw0.nrt,Air Quality: Particulate Matter (PM10) Station Measurements,11/9/2023,11/25/2024,OpenAQ updated its API and data fetching options. The script cannot directly get data from their S3 bucket anymore. Need to register an account and use their API instead.,TRUE
-cit.003c.rw0.nrt,Air Quality: Sulfur Dioxide (SO₂) Station Measurements,11/9/2023,11/25/2024,OpenAQ updated its API and data fetching options. The script cannot directly get data from their S3 bucket anymore. Need to register an account and use their API instead.,TRUE
-cit.003d.rw0.nrt,Air Quality: Nitrogen Dioxide (NO₂) Station Measurements,11/9/2023,11/25/2024,OpenAQ updated its API and data fetching options. The script cannot directly get data from their S3 bucket anymore. Need to register an account and use their API instead.,TRUE
-cit.003e.rw0.nrt,Air Quality: Ozone (O₃) Station Measurements,11/9/2023,11/25/2024,OpenAQ updated its API and data fetching options. The script cannot directly get data from their S3 bucket anymore. Need to register an account and use their API instead.,TRUE
-cit.003f.rw0.nrt,Air Quality: Carbon Monoxide (CO) Station Measurements,11/9/2023,11/25/2024,OpenAQ updated its API and data fetching options. The script cannot directly get data from their S3 bucket anymore. Need to register an account and use their API instead.,TRUE
-cit.003g.rw0.nrt,Air Quality: Black Carbon Station Measurements,11/9/2023,11/25/2024,OpenAQ updated its API and data fetching options. The script cannot directly get data from their S3 bucket anymore. Need to register an account and use their API instead.,TRUE
-cli.039.rw1.nrt,Standardised Precipitation-Evapotranspiration Index,11/21/2023,11/25/2024,Source upgraded and it needs an account for the latest data,TRUE
-cli.040.rw0.nrt.dash,Sea Level,5/25/2023,11/25/2024,"source site has major update on their data storage, we paused on updating it since this dataset may not be kept in the data catelog ",TRUE
-cli.041.rw0.nrt.dash,Land Ice: Antarctica,5/19/2023,11/25/2024,"source site has major update on their data storage, we paused on updating it since this dataset may not be kept in the data catelog ",TRUE
-cli.042.rw0.nrt.dash,Land Ice: Greenland,5/19/2023,11/25/2024,"source site has major update on their data storage, we paused on updating it since this dataset may not be kept in the data catelog ",TRUE
-cli.043.rw0.nrt.dash,Minimum Annual Arctic Sea Ice Extent,7/4/2023,11/25/2024,"source site has major update on their data storage, we paused on updating it since this dataset may not be kept in the data catelog ",TRUE
-com.024.rw0.nrt,Oil Palm Concessions,1/12/2021,11/25/2024,the dataset is now only available as vector layer tiles,
-com.025.rw0.nrt,Wood Fiber Concessions,1/12/2021,11/25/2024,the dataset is now only available as vector layer tiles,
-foo.003.rw1.nrt,Current and Projected Food Insecurity,7/4/2023,11/25/2024,"source site has major update on their data storage, we paused on updating it since this dataset may not be kept in the data catelog ",TRUE
-ocn.002.rw0.nrt,Wave Height,8/19/2021,11/25/2024,the version we use is no longer updated by the data source,TRUE
-soc.062c.rw1.nrt,"Recent Internal Displacement from Conflict, Natural Disasters, and Development",8/10/2022,11/25/2024,the data source has not been updated since 6/30/2022,TRUE
+cit.003a.rw0.nrt,Air Quality: Fine Particulate Matter (PM2.5) Station Measurements,11/9/2023,12/23/2024,OpenAQ updated its API and data fetching options. The script cannot directly get data from their S3 bucket anymore. Need to register an account and use their API instead.,TRUE
+cit.003b.rw0.nrt,Air Quality: Particulate Matter (PM10) Station Measurements,11/9/2023,12/23/2024,OpenAQ updated its API and data fetching options. The script cannot directly get data from their S3 bucket anymore. Need to register an account and use their API instead.,TRUE
+cit.003c.rw0.nrt,Air Quality: Sulfur Dioxide (SO₂) Station Measurements,11/9/2023,12/23/2024,OpenAQ updated its API and data fetching options. The script cannot directly get data from their S3 bucket anymore. Need to register an account and use their API instead.,TRUE
+cit.003d.rw0.nrt,Air Quality: Nitrogen Dioxide (NO₂) Station Measurements,11/9/2023,12/23/2024,OpenAQ updated its API and data fetching options. The script cannot directly get data from their S3 bucket anymore. Need to register an account and use their API instead.,TRUE
+cit.003e.rw0.nrt,Air Quality: Ozone (O₃) Station Measurements,11/9/2023,12/23/2024,OpenAQ updated its API and data fetching options. The script cannot directly get data from their S3 bucket anymore. Need to register an account and use their API instead.,TRUE
+cit.003f.rw0.nrt,Air Quality: Carbon Monoxide (CO) Station Measurements,11/9/2023,12/23/2024,OpenAQ updated its API and data fetching options. The script cannot directly get data from their S3 bucket anymore. Need to register an account and use their API instead.,TRUE
+cit.003g.rw0.nrt,Air Quality: Black Carbon Station Measurements,11/9/2023,12/23/2024,OpenAQ updated its API and data fetching options. The script cannot directly get data from their S3 bucket anymore. Need to register an account and use their API instead.,TRUE
+cli.039.rw1.nrt,Standardised Precipitation-Evapotranspiration Index,11/21/2023,12/23/2024,Source upgraded and it needs an account for the latest data,TRUE
+cli.040.rw0.nrt.dash,Sea Level,5/25/2023,12/23/2024,"source site has major update on their data storage, we paused on updating it since this dataset may not be kept in the data catelog ",TRUE
+cli.041.rw0.nrt.dash,Land Ice: Antarctica,5/19/2023,12/23/2024,"source site has major update on their data storage, we paused on updating it since this dataset may not be kept in the data catelog ",TRUE
+cli.042.rw0.nrt.dash,Land Ice: Greenland,5/19/2023,12/23/2024,"source site has major update on their data storage, we paused on updating it since this dataset may not be kept in the data catelog ",TRUE
+cli.043.rw0.nrt.dash,Minimum Annual Arctic Sea Ice Extent,7/4/2023,12/23/2024,"source site has major update on their data storage, we paused on updating it since this dataset may not be kept in the data catelog ",TRUE
+com.024.rw0.nrt,Oil Palm Concessions,1/12/2021,12/23/2024,the dataset is now only available as vector layer tiles,
+com.025.rw0.nrt,Wood Fiber Concessions,1/12/2021,12/23/2024,the dataset is now only available as vector layer tiles,
+foo.003.rw1.nrt,Current and Projected Food Insecurity,7/4/2023,12/23/2024,"source site has major update on their data storage, we paused on updating it since this dataset may not be kept in the data catelog ",TRUE
+ocn.002.rw0.nrt,Wave Height,8/19/2021,12/23/2024,the version we use is no longer updated by the data source,TRUE
+soc.062c.rw1.nrt,"Recent Internal Displacement from Conflict, Natural Disasters, and Development",8/10/2022,12/23/2024,the data source has not been updated since 6/30/2022,TRUE