From the course: Data Pipeline Automation with GitHub Actions Using R and Python

Querying data with R and Python - GitHub Tutorial

From the course: Data Pipeline Automation with GitHub Actions Using R and Python

Start my 1-month free trial

Querying data with R and Python

“

- [Instructor] So far in the previous videos, we viewed the API functionality and saw how to set and send a get request via the browser. We'll dedicate the rest of this chapter to learn and practice how to query data from the API using R and Python. To pull data with R we will use the EIA API library that provides a set of functions to query data from the current version of the API V2. The package is available on CRAN and we will use the current version 0.1.2. For Python, currently, there is no stable library that supports the V2 of the API. Therefore, to be consistent, I created a set of Python functions aligned with the ones available in the EIA API library, which we'll use to pull the data from the API with Python. Throughout this course, we will use the following three functions, which are available both in R and Python to pull data from the API. First, the EIA underscore metadata, which as its name implies returns metadata. In the coming example, we will see how to use the function output to set the get request parameters. Next is the EIA underscore get function, which enable us to send get request to the API. The function retains time series data. As we previously saw the API is a cup of 5,000 observation limitation per get request. This might be challenging when pulling hourly data, as we're going to do heavily in this course. This is where the failed and last function. The EIA underscore backfill function comes in handy, enabling us to pull observation per request beyond the API limitation. On the backend the function splits the request into a sequence of smaller requests and send them to the API using the AI get function, it then appends the return outputs into one table. Throughout the course, we will mainly use the EIA backfill and the EIA metadata functions. Moving forward, we'll have coding demos in both Python and R, you can choose to continue with either one of the languages and skip the other or watch both languages.

Contents