| title | description | ms.service | ms.topic | author | ms.author | ms.date |
|---|---|---|---|---|---|---|
Example Jupyter notebooks using NOAA data |
Use example Jupyter notebooks for Azure Open Datasets to learn how to load open datasets and use them to enrich demo data. Techniques include use of Spark and Pandas to process data. |
open-datasets |
sample |
sdgilley |
sgilley |
05/06/2020 |
The example Jupyter notebooks for Azure Open Datasets show you how to load open datasets and use them to enrich demo data. Techniques include use of Apache Spark and Pandas to process data.
Important
When working in a non-Spark environment, Open Datasets allows downloading only one month of data at a time with certain classes in order to avoid MemoryError with large datasets.
| Notebook | Description |
|---|---|
| Load one recent month of weather data into a Pandas dataframe | Learn how to load historical weather data into your favorite Pandas dataframe. |
| Load one recent month of weather data into a Spark dataframe | Learn how to load historical weather data into your favorite Spark dataframe. |
| Notebook | Description |
|---|---|
| Join demo data with weather data - Pandas | Join a 1-month demo dataset of sensor locations with weather readings in a Pandas dataframe. |
| Join demo data with weather data – Spark | Join a demo dataset of sensor locations with weather readings in a Spark dataframe. |
| Notebook | Description |
|---|---|
| Taxi trip data enriched with weather data - Pandas | Load NYC green taxi data (over 1 month) and enrich it with weather data in a Pandas dataframe. This example overrides the method get_pandas_limit and balances data load performance with the amount of data. |
| Taxi trip data enriched with weather data – Spark | Load NYC green taxi data and enrich it with weather data, in Spark dataframe. |