Tutorial — Use the Python Module to easily query and download the rainfall satellite data for quick analysis

Rainfall is an important indicator of several use cases including agricultural productivity growth analysis, climate change analytics, etc. Understanding the rainfall patterns and fluctuations in a region helps to make strategic plans for the agriculture seasons.

There are several data sources that provide rainfall satellite data. CHRS portal is one among them that provides global rainfall indicators based on remote sensing satellite data. In this tutorial, let’s look at how to download rainfall satellite data from CHRS Data portal using python.

feature image - Download Rainfall Satellite Data from CHRS Data Portal using Python. Screenshot of the iRain tool by CHRS.

What is CHRS?

CHRS stands for Center for Hydrometeorology and Remote Sensing. It’s a centre set up by the University of California, Irvine (UCI). CHRS with the help of its faculty and researchers has built a range of products/systems aimed at solving the problems related to hydrology processes. These products utilize remote sensing data and machine learning (neural networks) models to understand and predict the impacts of various activities on water resources.

The CHRS data portal provides precipitation and related products through multiple projects. The PERSIANN(Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks) system for example computes an estimate of rainfall rate based on remote sensing data. Other systems include PERSIANN-CCS, PERSIANN-CDR, PDIR-Now, PERSIANN-CCS-CDR. This data is published in the public domain and made available for everyone to use.

💡 Note: CHRS also provides two exploratory tools to visualise the data directly on the browser — iRain, RainSphere.

Metadata

PERSIANN

Data Period: March 2000 — Present

Coverage: 60°S to 60°N

Resolution: 0.25° x 0.25°

HTTP Download (full globe): hourly, 3-hourly, 6-hourly, daily, monthly, yearly

FTP also available: ftp://persiann.eng.uci.edu/CHRSdata/PERSIANN

Latest Update: Near real-time with a 2-day delay

PERSIANN-CCS

Data Period: January 2003 — Present

Coverage: 60°S to 60°N

Resolution: 0.04° x 0.04°

HTTP Download (full globe): hourly, 3-hourly, 6-hourly, daily, monthly, yearly

FTP also available: ftp://persiann.eng.uci.edu/CHRSdata/PERSIANN-CCS

Latest Update: Real-time

PERSIANN-CDR

Data Period: January 1983 — Present

Coverage: 60°S to 60°N

Resolution: 0.25° x 0.25°

HTTP Download (full globe): daily, monthly, yearly

FTP also available: ftp://persiann.eng.uci.edu/CHRSdata/PERSIANN-CDR

3hr PERSIANN-CDR: 3hr PERSIANN-CDR, README

Latest Update: September 2020

PDIR-Now

Data Period: March 1st 2000 — Present

Coverage: 60°S to 60°N

Resolution: 0.04° x 0.04°

HTTP Download (full globe): hourly, 3-hourly, 6-hourly, daily, monthly, yearly

Data file format: PDIRNow README file

FTP also available: ftp://persiann.eng.uci.edu/CHRSdata/PDIRNow

Download the Data

Query Data on Portal

Any user can download the rainfall satellite data for any part of the globe by creating a query on the CHRS Data Portal. For each dataset, you choose you can select the available temporal granularity from the dropdown as well as choose a filtering domain. The filtering domain can be the whole globe, any specific country, region or location. You can even provide a shapefile to query the data.

Once you select the time step, domain and dataset, provide the start and end dates along with the download data format from the available options (ArcGrid, NetCDF, Tif).

After completing the details for the query and clicking on the Download button, you will be then be taken to a dialogue box to provide an email address. You will receive a link to download the data through an email at the provided address, upon successful submission.

You would need to manually navigate through the UI to query the data and download it via the email prompt process. Also, the download link is sent via email which only slows down the process. There’s an alternative method through which you can download data to avoid the email prompt.

HTTP Download

The CHRS also makes the data available through an HTTP server, which you can use to download the data. The data available is for the whole globe as per available time steps for each dataset. You can refer to the Metadata section above to get an overview of the time steps available and their respective HTTP Server links.

Though this method helps you overcome the email verification process, you will have to manually download files for each of the observations. And the data is available in ‘.bin.gz’ format compared to the flexibility of the query you can make on UI.

Using Python

Both the methods mentioned above have some limitations. So, I built a utility library to handle the query and downloading of rainfall satellite data from the CHRS data portal using python. The utility library chrs-persiann-util lets you make query the available datasets in different time steps and file formats.

To install this library clone the GitHub project to your local and from the root directory run the following command from your terminal;

python setup.py install

Or alternatively, you can directly install from the Github link as well;

pip install git+https://github.com/samashti/chrs-persiann-util.git

Once installed, you can use the library to download the data as follows;

from chrs_persiann import CHRS

params = {
    'start': '2021010100',
    'end': '2021010300',
    'mailid': 'test@gmail.com',
    'download_path': '~/Downloads',
    'file_format': 'Tif',
    'timestep': 'daily',
    'compression': 'zip'
}

dl = CHRS()

# PERSIANN
dl.get_persiann(**params)

Read more on the usage of the library and input query parameter standards on Github.

Download & Visualise Rainfall Data

Let’s visualise the rainfall satellite data for the month of January. You can find this example on Google Colab as well.

Let’s first initialize the required python modules. We are using the rasterio to read the raster data once it’s downloaded.

import rasterio
import zipfile
from chrs_persiann import CHRS

Here, We’d like to download the PERSIANN data for the month of January 2022. So, I pass in the query parameters accordingly to the utility library and download the rainfall satellite data in zip format.

params = {
    'start': '2021010100',
    'end': '2021013100',
    'mailid': 'test@gmail.com',
    'download_path': '/home/',
    'file_format': 'Tif',
    'timestep': 'monthly',
    'compression': 'zip'
}

dl.get_persiann(**params)

And you should see the query being made and the process of order through the logs as below;

Querying data & Placing the order...
Query Params:

start date - 2021010100
end date - 2021013100
time step - monthly
data type - PERSIANN
file format - Tif
compression format - zip
download path - /home/

Query complete.
Order Details - User IP: 108598346, File: 2022-03-10074331am
Generating Data url...
File url Generated - <https://chrsdata.eng.uci.edu/userFile/108598346/temp/PERSIANN/PERSIANN_2022-03-10074331am.zip>
Downloading compressed data file - /home/PERSIANN_2022-03-10074331am.zip
Download Complete ------------------------------------------

Now, we check the downloaded zip file and see the available raster files.

file = '/home/PERSIANN_2022-03-10074331am.zip'
z = zipfile.ZipFile(file)
z.filelist

We see that PERSIANN_1m202101.tif is part of the zip file contents and we are interested in reading this file using rasterio.

rasterfile = z.filelist[0].filename
raster = rasterio.open(f'/vsizip/{file}/{rasterfile}')

Once the raster is read successfully, we can utilize the rasterio and matplotlib libraries to quickly visualize the raster.

from rasterio.plot import show
from matplotlib import pyplot as plt

fig, ax = plt.subplots(figsize=(30,9))

# use imshow so that we have something to map the colorbar to
image_hidden = ax.imshow(raster.read(1), cmap='viridis')

# plot on the same axis with rio.plot.show
image = show(raster.read(1), transform=raster.transform, ax=ax, cmap='viridis')

# add colorbar using the now hidden image
fig.colorbar(image_hidden, ax=ax)

Summary

In this quick tutorial, we learnt how to download the rainfall satellite data from the CHRS portal through multiple methods including the use of a python module. The chrs-persiann-util will help you make queries on the dataset and download for a quick analysis right from within your code. If you like to experiment with the code, you can check out the code on this Google Colab Notebook.

‍

If you liked this blog, like & subscribe to the blog to get notified about future blog posts. You can find me on LinkedIn, Twitter for any queries or discussions. Check out my previous blog on Why you need to use geopackage files instead of shapefile & geojson here.

Read other related posts: How to use QGIS spatial algorithms with python scripts?, How to geocode addresses for free?

‍

‍