matplotlib on Software Development Notes

Create interactive chart with matplotlib and ipywidgets

Eric Callanan — Sunday, 18 Apr 2021

ipywidgets are interactive HTML elements that can be used in Jupyter Notebooks to interact with outputs such as tables and charts. This article briefly introduces ipywidgets and uses them to change the rolling day period in a chart for the rolling average of deaths from covid-19.

ipywidgets or (jupyter-widgets) are interactive HTML elements used to provide the user with tools to interact with data and visualize the effects of parameter changes.

Slider with Interact

The Interact function is a convenient way to quickly create an interactive widget. It takes a function as it's first parameter and then the function parameters with the possible values. Interact creates user interface controls for exploring code and data interactively.

The function below takes one parameter and returns a string stating the value of that parameter. ipywidgets uses widget abbreviations to generate the appropriate UI control, for example when an integer-valued keyword argument of 10 is passed to interact, it generates an integer-valued slider control with a range of -10 to 30.

1def f(x):
2    return f"The selected value is '{x}'"
3
4w.interact(f, x=10)

Note that if you change x to pass in a boolean value, then a checkbox control is created and a textbox is created if a string is passed in.

1def f(x):
2    return f"The selected value is '{x}'"
3
4_ = w.interact(f, x=100)
5_ = w.interact(f, x=True)
6_ = w.interact(f, x="Apple")
7_ = w.interact(f, x=["Apple", "Banana", "Pear"])

It is necessary to specify the start, stop and increment values in a tuple to explicitly create a slider control.

1_ = w.interact(f, x=(1,7,1))

Creating ipywidgets with interact

Creating ipywidgets with interact

There are many widgets documented in the Widget List that are distributed with ipywidgets. These can be combined to create a more complex user interface composed of numerous widgets. As these are HTML elements, the Layout and Styling of ipywidgets utilizes many of the styling mechanisms of web pages. The layout attribute uses a number of CSS properties such as height, width, visibility, border and margin as well as grid-layout and flexbox features. This code created four ipywidgets controls and lays them out using HBox and VBox.

 1slider = w.IntSlider(value=5, min=0, max=10, step=1, description="Number")
 2button = w.Button(
 3    description="The Button",
 4    style=w.ButtonStyle(button_color="Plum"),
 5)
 6dropdown = w.Dropdown(
 7    value="green",
 8    options=["red", "orange", "yellow", "green", "blue", "indigo", "violet"],
 9    description="Rainbow",
10)
11radiobuttons = w.RadioButtons(
12    value="thu",
13    options=["mon", "tue", "wed", "thu", "fri", "sat", "sun"],
14    description="Which Day",
15)
16
17w.HBox([slider, button, dropdown, radiobuttons])
18
19w.VBox([slider, button, dropdown, radiobuttons])

Horizontal and vertical layout of ipywidgets

Horizontal and vertical layout of ipywidgets

The vertical and horizontal boxes can be nested to build up more complex layouts. This shows a horizontal containing two vertical boxes. Border ans padding styles can be applied to the boxes to make it easier to highlight sections.

1w.HBox(
2    [
3        w.VBox([button, radiobuttons]),
4        w.VBox([slider, dropdown]),
5    ]
6)

1box1 = w.VBox([button, radiobuttons])
2box2 = w.VBox([slider, dropdown])
3
4box1.layout = w.Layout(border="solid 1px red", margin="5px", padding="10px")
5box2.layout = w.Layout(border="solid 1px blue", margin="5px", padding="10px")
6
7w.HBox([box1, box2])

Nested ipywidgets in vertical boxes laid out ia a parent horizontal layout

Nested ipywidgets in vertical boxes laid out ia a parent horizontal layout

Rolling average of Deaths from COVID-19

I have previously had a look at the Daily counts and rolling average of COVID-19 numbers, so decided to recreate a chart like this with the addition of widgets. This code retrieves the latest data from Johns Hopkins University Github page, cleans the data and creates the chart for the top five countries with the highest rolling average of deaths.

1# Number of Death Cases - Global: time series data for deaths
2deaths_path = 'https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_time_series/time_series_covid19_deaths_global.csv'
3deaths_df = pd.read_csv(deaths_path)

 1def clean_covid_data(df):
 2    # 1. Drop unwanted columns
 3    df.drop(['Province/State', 'Lat', 'Long'], axis = 1, inplace = True)
 4
 5    # 2. Group by Country
 6    df = df.groupby('Country/Region').sum()
 7
 8    # 3. Convert column headings to datetime
 9    df.columns = pd.to_datetime(df.columns)
10
11    # 4. Remove the column name
12    df.rename_axis(None, axis = 0, inplace = True)
13
14    return df

 1def plot_axis(ax, df, countries, rolling, days):
 2    ax.set_title(f"{rolling}-day rolling average, for the last {days} days")
 3    ax.set_facecolor(plt.cm.Blues(0.2))
 4    rolling_df = df.rolling(rolling, axis=1).mean()
 5    for i, country in enumerate(countries):
 6        ax.plot(
 7            df.columns[-days:],
 8            rolling_df.loc[country].tail(days),
 9            label=country,
10            color=plt.cm.tab10(i),
11        )
12    ax.legend(bbox_to_anchor=(0.99, 1), loc="upper left", facecolor=plt.cm.Blues(0.1))
13    ax.set_xlabel("Date", fontsize="large", fontweight="bold")
14    ax.set_ylabel("Number of Deaths", fontsize="large", fontweight="bold")
15    ax.spines["right"].set_visible(False)
16    ax.spines["top"].set_visible(False)

1def create_rolling_average_chart(df, countries, rolling, days, title):
2    fig, ax = plt.subplots(figsize=(10, 5), facecolor=plt.cm.Blues(0.2))
3    fig.suptitle(title, fontsize="xx-large", fontweight="bold")
4    plot_axis(ax, df, countries, rolling, days)
5    fig.autofmt_xdate()
6    plt.tight_layout()
7    return fig

 1deaths_df = clean_covid_data(deaths_df)
 2daily_deaths_df = deaths_df.diff(axis=1)
 3top_countries = list(
 4    daily_deaths_df.iloc[:, -1].sort_values(ascending=False).head(10).index
 5)
 6create_rolling_average_chart(
 7    daily_deaths_df,
 8    top_countries[:5],
 9    7,
10    100,
11    "Daily rolling average of deaths from COVID-19",
12)

Seven-day-rolling-average-of-deaths-covid-19

Seven-day rolling average of deaths from COVID-19 for five highest countries

Using ipywidgets to modify days for rolling average

The following class is created to put the widgets and the matplotlib chart together. In this example a slider widget is used to select the rolling period from 1 to 10 inclusive. In addition, a dropdown widget is used to set the number of recent days. One of the key aspects to this interaction is the observe method of the widget to register a callback. observe is used to specify a method to call when the widget value changes rolling_slider.observe(self.update_rollout, "value").

  1class average_deaths(widgets.VBox):
  2    df = None
  3    output = widgets.Output()
  4    rolling_days = 7
  5    recent_days = 100
  6
  7    def __init__(self, df):
  8        super().__init__()
  9        self.df = df
 10
 11        # create the chart
 12        self.create_chart()
 13
 14        # create and layout the widget controls
 15        controls = self.create_control_panel()
 16
 17        # add the widgets to the view
 18        self.children = [controls, self.output]
 19
 20    def create_chart(self):
 21        """create the chart in the widget output control"""
 22        with self.output:
 23            self.fig, self.ax = plt.subplots(
 24                figsize=(9.5, 4.5), facecolor=plt.cm.Blues(0.2)
 25            )
 26            self.fig.suptitle(
 27                "Deaths from COVID-19", fontsize="xx-large", fontweight="bold"
 28            )
 29            self.plot_axis()
 30            self.fig.autofmt_xdate()
 31            plt.tight_layout()
 32            self.fig.canvas.toolbar_position = "bottom"
 33
 34    def create_control_panel(self):
 35        """create the slider and dropdown controls"""
 36        rolling_slider = widgets.IntSlider(
 37            value=self.rolling_days,
 38            min=1,
 39            max=10,
 40            step=1,
 41        )
 42        recent_dropdown = widgets.Dropdown(
 43            options=[50, 100, 200, 300, 1000],
 44            value=self.recent_days,
 45            description="",
 46            disabled=False,
 47        )
 48        controls = widgets.HBox(
 49            [
 50                widgets.VBox(
 51                    [
 52                        widgets.Label("Rolling Average Days: "),
 53                        widgets.Label("Recent Number of Days: "),
 54                    ]
 55                ),
 56                widgets.VBox([rolling_slider, recent_dropdown]),
 57            ]
 58        )
 59        controls.layout = self.control_box_layout()
 60
 61        # hook up the widgets to observe changes
 62        rolling_slider.observe(self.update_rollout, "value")
 63        recent_dropdown.observe(self.update_recent, "value")
 64        return controls
 65
 66    def control_box_layout(self):
 67        """create layout for control panel"""
 68        return widgets.Layout(
 69            border="solid 1px Navy",
 70            margin="0px 10px 10px 0px",
 71            padding="5px 5px 5px 5px",
 72        )
 73
 74    def update_rollout(self, change):
 75        """update the chart with changes in rolling frequency"""
 76        self.rolling_days = change.new
 77        self.plot_axis()
 78        self.fig.autofmt_xdate()
 79
 80    def update_recent(self, change):
 81        """update the chart with changes in recent number of days"""
 82        self.recent_days = change.new
 83        self.plot_axis()
 84        self.fig.autofmt_xdate()
 85
 86    def plot_axis(self):
 87        """clear any previous chart and plot the lines for the top countries"""
 88        self.ax.clear()
 89        self.ax.set_title(
 90            f"{self.rolling_days}-day rolling average, for the last {self.recent_days} days"
 91        )
 92        self.ax.set_facecolor(plt.cm.Blues(0.2))
 93        rolling_df = self.df.rolling(self.rolling_days, axis=1).mean()
 94        top_countries = list(
 95            rolling_df.iloc[:, -1].sort_values(ascending=False).head(10).index
 96        )
 97        for i, country in enumerate(top_countries[:5]):
 98            self.ax.plot(
 99                self.df.columns[-self.recent_days :],
100                rolling_df.loc[country].tail(self.recent_days),
101                label=country,
102                color=plt.cm.tab10(i),
103            )
104        self.ax.legend(
105            bbox_to_anchor=(1, 0.8), loc="upper left", facecolor=plt.cm.Blues(0.1)
106        )
107        self.ax.set_xlabel("Date", fontsize="large", fontweight="bold")
108        self.ax.set_ylabel("Number of Deaths", fontsize="large", fontweight="bold")
109        self.ax.spines["right"].set_visible(False)
110        self.ax.spines["top"].set_visible(False)
111
112
113average_deaths(daily_deaths_df)

Interact with matplotlib chart using ipywidgets

Interact with matplotlib chart using ipywidgets

example of using ipywidgets to interact with chart

Example of using ipywidgets to interact with chart

Conclusion

I'm still not sure about the use of ipywidgets. They provide a mechanism of interacting with other user interface in a Jupyter notebook, so they could be used to create a number of variations for charts without having to create the different charts. However, they require a server to deploy or else an interactive Jupyter notebook and if one has the interactive notebook, then the data could be changed and the various charts created. ipywidgets may be more useful when there are easier ways to deploy and interact with the Jupyter notebooks containing them. It would be great if GitHub hosted the Jupyter notebooks in such a way to allow interaction with ipywidgets - Rolling averages of deaths from COVID-19. A work around is to open these notebooks in Binder - Here is a direct link to Binder - Rolling averages of deaths from COVID-19. ipywidgets are interesting, but I'm not seeing the practical application of them yet.

Display world map with country data

Eric Callanan — Sunday, 22 Nov 2020

Geographical maps can be a great way to present a global view of where data applies to different countries and regions. There are a number of third-party tools and libraries that can be used to help create these maps such as Basemap, Cartopy as well as Geopandas and Geoplot. This article will show how to use Geopandas to display a world map showing countries with higher Under Five Mortality Rates in darker colors.

Load the data

Details on downloading and loading the Under Five Mortality Rate data into a dataframe is described in "Pandas - Load data from Excel file and Display Chart". The following loads the data from a local file and filters on the Median values for each country. The ISO.Code field contains the international standard three-letter code for countries in the world.

 1# Load the excel worksheet into a dataframe
 2u5mr_df = pd.read_excel(
 3    "/tmp/data/Under-five-mortality-rate_2020.xlsx",
 4    engine = "openpyxl",
 5    sheet_name = 'Country estimates (both sexes)',
 6    header = 14,
 7    nrows = 585)
 8
 9# Rename 'Uncertainty.Bounds*' column to 'Uncertainty.Bounds'
10u5mr_df = u5mr_df.rename(columns={'Uncertainty.Bounds*': 'Uncertainty.Bounds'})
11
12# Convert year column names to datetime
13u5mr_df.columns = [x[:-2] if x.endswith('.5') else x for x in u5mr_df.columns]# u5mr_df.columns = [pd.to_datetime(f'{x[:-2]}-12-31') if x.endswith('.5') else x for x in u5mr_df.columns]
14
15# Filter to the Median values
16u5mr_med_df = u5mr_df[u5mr_df['Uncertainty.Bounds'] == 'Median']
17
18# Review the data
19u5mr_med_df.iloc[[0,1,2,3,-4,-3,-2,-1], [0,1,2,3,12,-3,-2,-1]]
20
21"""
22(195, 73)
23    ISO.Code Country.Name Uncertainty.Bounds  1950        1959       2017       2018       2019
241        AFG  Afghanistan             Median   NaN         NaN  64.940759  62.541196  60.269399
254        ALB      Albania             Median   NaN         NaN   9.418052   9.525133   9.682407
267        DZA      Algeria             Median   NaN  240.344776  24.319482  23.805926  23.256168
2710       AND      Andorra             Median   NaN         NaN   3.218925   3.085839   2.966929
28574      VNM     Viet Nam             Median   NaN         NaN  20.843125  20.405423  19.935167
29577      YEM        Yemen             Median   NaN         NaN  56.966430  58.460003  58.356138
30580      ZMB       Zambia             Median   NaN  208.929172  64.337901  63.294182  61.663465
31583      ZWE     Zimbabwe             Median   NaN  155.256789  58.234924  55.856832  54.612967
32"""

Load Geopandas

Geopandas and Geoplot are required to create these charts. These can be installed with pip install geopandas and pip install geoplot. Geopandas requires the Geospatial Data Abstraction Library (GDAL) library to be installed - good instructions here on How to install GDAL. Geopandas has a dataset for the contours of all the countries, as well as some information like population estimates, that can be used to plot the world map and colour countries based on population. The following code loads the 'naturalearth_lowres' dataset and plots a map with countries colored based on country population, the darker greens represent higher population. This shows that China and India dominate this map.

 1import geopandas
 2import geoplot
 3import mapclassify
 4
 5world = geopandas.read_file(
 6    geopandas.datasets.get_path('naturalearth_lowres')
 7)
 8fig, ax = plt.subplots(figsize = (10,4), facecolor = plt.cm.Blues(.2))
 9fig.suptitle('Country Populations',
10             fontsize = 'xx-large',  
11             fontweight = 'bold')
12ax.set_facecolor(plt.cm.Blues(.2))
13world.plot(column = 'pop_est',
14           cmap = 'Greens',
15           ax = ax,
16           legend = True)
17
18plt.show()

Map showing country color based on population

Map showing country color based on population

Explore 'naturalearth_lowres' dataset

The 'naturalearth_lowres' dataset is loaded into a Geodataframe, which is a specialised form of a Pandas Dataframe. So all of the functions and properties of Dataframe apply to geodataframe. A geodataframe always contains one geoseries called geometry that holds spatial status.

In the world geodataframe loaded from naturalearth_lowres dataset there are 6 columns and 177 rows.

 1type(world)
 2"""
 3<class 'geopandas.geodataframe.GeoDataFrame'>
 4"""
 5
 6world.shape
 7"""
 8(177, 6)
 9"""
10
11world.columns
12"""
13Index(['pop_est', 'continent', 'name', 'iso_a3', 'gdp_md_est', 'geometry'], dtype='object')
14"""
15
16world.index
17"""
18RangeIndex(start=0, stop=177, step=1)
19"""
20
21world.iloc[[0,1,2,3,-4,-3,-2,-1], :]
22"""
23      pop_est      continent                 name iso_a3 gdp_md_est                                           geometry  
240      920938        Oceania                 Fiji    FJI     8374.0  MULTIPOLYGON (((180.00000 -16.06713, 180.00000...  
251    53950935         Africa             Tanzania    TZA   150600.0  POLYGON ((33.90371 -0.95000, 34.07262 -1.05982...  
262      603253         Africa            W. Sahara    ESH      906.5  POLYGON ((-8.66559 27.65643, -8.66512 27.58948...  
273    35623680  North America               Canada    CAN  1674000.0  MULTIPOLYGON (((-122.84000 49.00000, -122.9742...  
28173    642550         Europe           Montenegro    MNE    10610.0  POLYGON ((20.07070 42.58863, 19.80161 42.50009...  
29174   1895250         Europe               Kosovo    -99    18490.0  POLYGON ((20.59025 41.85541, 20.52295 42.21787...  
30175   1218208  North America  Trinidad and Tobago    TTO    43570.0  POLYGON ((-61.68000 10.76000, -61.10500 10.890...  
31176  13026129         Africa             S. Sudan    SSD    20880.0  POLYGON ((30.83385 3.50917, 29.95350 4.17370, ...  
32"""

Show countries with population greater than 200 million

 1world[(world.pop_est > 200000000)]
 2"""
 3        pop_est      continent                      name iso_a3  gdp_md_est                                           geometry  
 44     326625791  North America  United States of America    USA  18560000.0  MULTIPOLYGON (((-122.84000 49.00000, -120.0000...  
 58     260580739           Asia                 Indonesia    IDN   3028000.0  MULTIPOLYGON (((141.00021 -2.60015, 141.01706 ...  
 629    207353391  South America                    Brazil    BRA   3081000.0  POLYGON ((-53.37366 -33.76838, -53.65054 -33.2...  
 798   1281935911           Asia                     India    IND   8721000.0  POLYGON ((97.32711 28.26158, 97.40256 27.88254...  
 8102   204924861           Asia                  Pakistan    PAK    988200.0  POLYGON ((77.83745 35.49401, 76.87172 34.65354...  
 9139  1379302771           Asia                     China    CHN  21140000.0  MULTIPOLYGON (((109.47521 18.19770, 108.65521 ...  
10"""

Plot the relative population of countries in Africa. This is done by filtering the dataframe on the continent of 'Africa'

 1fig, ax = plt.subplots(figsize = (6,5), facecolor = plt.cm.Blues(.2))
 2fig.suptitle('Africa Populations',
 3             fontsize = 'xx-large',  
 4             fontweight = 'bold')
 5ax.set_facecolor(plt.cm.Blues(.2))
 6ax = world[world.continent == 'Africa'].plot(
 7    column = 'pop_est',
 8    cmap = 'Greens',
 9    ax = ax,
10    legend = True)
11
12plt.show()

Map of Africa showing country color based on population

Map of Africa showing country color based on population

Display country color based on Under Five Mortality Rates in 2019

A global map of the Under Five Mortality Rates can be plotted by merging the Under five mortality rates dataframe with the world geodataframe. The merge is done on the ISO country code. There is a discrepancy in the dataframes in that there are 195 countries in the Under Five Mortality Rates dataframe and only 177 countries in the world geodataframe.

 1# Countries in ufmr dataframe that are not in the world geodataframe
 2u5mr_med_df[~u5mr_med_df['ISO.Code'].isin(list(world['iso_a3']))][['ISO.Code', 'Country.Name']]
 3"""
 4    ISO.Code                      Country.Name
 510       AND                           Andorra
 616       ATG               Antigua and Barbuda
 737       BHR                           Bahrain
 843       BRB                          Barbados
 985       CPV                        Cabo Verde
10112      COM                           Comoros
11118      COK                      Cook Islands
12151      DMA                          Dominica
13187      FRA                            France
14208      GRD                           Grenada
15271      KIR                          Kiribati
16313      MDV                          Maldives
17319      MLT                             Malta
18322      MHL                  Marshall Islands
19328      MUS                         Mauritius
20334      FSM  Micronesia (Federated States of)
21337      MCO                            Monaco
22358      NRU                             Nauru
23379      NIU                              Niue
24382      NOR                            Norway
25391      PLW                             Palau
26436      KNA             Saint Kitts and Nevis
27439      LCA                       Saint Lucia
28442      VCT  Saint Vincent and the Grenadines
29445      WSM                             Samoa
30448      SMR                        San Marino
31451      STP             Sao Tome and Principe
32463      SYC                        Seychelles
33469      SGP                         Singapore
34526      TON                             Tonga
35541      TUV                            Tuvalu
36"""

Show countries that are in the world geodataframe, but not in the Under Five Mortality Rates dataframe.

 1# list all countries in World not in ufmr
 2world[~world['iso_a3'].isin(list(u5mr_med_df['ISO.Code']))][['iso_a3', 'name']]
 3"""
 4    iso_a3                    name
 52      ESH               W. Sahara
 620     FLK            Falkland Is.
 721     -99                  Norway
 822     GRL               Greenland
 923     ATF  Fr. S. Antarctic Lands
1043     -99                  France
1145     PRI             Puerto Rico
12134    NCL           New Caledonia
13140    TWN                  Taiwan
14159    ATA              Antarctica
15160    -99               N. Cyprus
16167    -99              Somaliland
17174    -99                  Kosovo
18"""

There are five countries in the world dataframe with iso_a3 code set to "-99". Three of these are disputed or don't yet have an ISO designation. France and Norway have ISO codes of "FRA" and "NOR" respectively and are updated in the geodataframe with the following.

1# Update ISO codes for France and Norway
2world.loc[world.name == 'France', 'iso_a3'] = 'FRA'
3world.loc[world.name == 'Norway', 'iso_a3'] = 'NOR'

Merge the data from Under Five Mortality Rates with the world geodataframe. The merge is done with an left join on the ISO country code. This reduces the number of countries to 166 that have both mortality information and geometry information. Use of a 'left' join keeps all the countries in the original world geodataframe with the appropriate geometry data. The 11 countries (such as Greenland) that only appear in the world geodataframe will have NaN for all of the under five mortality rate data.

1u5mr_world_df = world.merge(u5mr_med_df,
2                            left_on = 'iso_a3',
3                            right_on = 'ISO.Code',
4                            how = 'left')
5u5mr_world_df.shape
6"""
7(177, 79)
8"""

Create a plot showing the under five mortality rates per country for year 2019. The data in the geopandas.geodataframe can be displayed in a number of ways and it can be confusing to know which parameter to set. This code creates the same plot in four different ways. The first three use the plot function on the geodataframe, which uses matplotlib to generate the plot. The first plot sets a scheme of 'quantiles' for the choropleth classification scheme, which colors the countries based on discrete intervals. When choropleth classification scheme of 'quantiles' is used the legend is of type matplotlib.pyplot.legend so the legend_kwds parameters are different. The default scheme is None, in which case the legend is of type matplotlib.pyplot.colorbar. This is used in chart 2 and 3, with the colorbar being changed from vertical to horizontal. Finally, the fourth plot is rendered using the choropleth function in geoplot module.

 1u5mr_year = u5mr_world_df['2019']
 2fig, axs = plt.subplots(
 3    nrows = 2,
 4    ncols = 2,
 5    figsize = (12,5),
 6    facecolor = plt.cm.Blues(.2))
 7fig.suptitle('National Under Five Mortality Rates in 2019',
 8             fontsize = 'xx-large',  
 9             fontweight = 'bold')
10for ax in axs.flatten():
11    ax.set_facecolor(plt.cm.Blues(.2))
12
13ax1 = axs[0][0]
14u5mr_world_df.plot(
15    ax = ax1,
16    color = 'white',
17    edgecolor = 'black'
18)
19
20u5mr_world_df.plot(
21    column = u5mr_year,
22    scheme = 'quantiles',
23    k = 6,
24    cmap = 'OrRd',
25    ax = ax1,
26    legend = True,
27    legend_kwds = {'title': "UFMR per 1000",
28                   'title_fontsize': 'small',
29                   'frameon': False,
30                   'loc': 'lower center',
31                   'bbox_to_anchor': (-0.2, 0.1, 0.5, 1),
32                   'fontsize': 'xx-small',
33                  },
34)
35[spine.set_visible(False) for spine in ax1.spines.values()]
36ax1.xaxis.set_visible(False)
37ax1.yaxis.set_visible(False)
38
39
40ax2 = axs[0][1]
41u5mr_world_df.plot(
42    ax = ax2,
43    color = 'white',
44    edgecolor = 'black'
45)
46
47u5mr_world_df.plot(
48    column = u5mr_year,
49    cmap = 'OrRd',
50    ax = ax2,
51    legend = True,
52    legend_kwds = {'label': "UFMR per 1000"},
53)
54[spine.set_visible(False) for spine in ax2.spines.values()]
55ax2.xaxis.set_visible(False)
56ax2.yaxis.set_visible(False)
57
58ax3 = axs[1][0]
59u5mr_world_df.plot(
60    ax = ax3,
61    color = 'white',
62    edgecolor = 'black'
63)
64
65u5mr_world_df.plot(
66    column = u5mr_year,
67    cmap = 'OrRd',
68    ax = ax3,
69    legend = True,
70    legend_kwds = {'label': "UFMR per 1000",
71                   'orientation': 'horizontal',
72                   'shrink': 0.7,
73                  },
74)
75[spine.set_visible(False) for spine in ax3.spines.values()]
76ax3.xaxis.set_visible(False)
77ax3.yaxis.set_visible(False)
78
79
80gplt.choropleth(
81    u5mr_world_df,
82    hue = u5mr_year,
83#     scheme = scheme,
84    cmap = 'OrRd',
85    ax = axs[1][1],
86    legend = True
87)
88
89plt.show()

Map showing country Under Five Mortality Rates in 2019

Map showing country Under Five Mortality Rates in 2019

Display country color based on Under Five Mortality Rates in 1985

A function can be created to wrap up the creation of a world map for a particular year based on option 2 above. A color of gray is added to handle missing data using misssing_kwds parameter. This is valuable when dealing with earlier years where there is no data available for many countries and displaying white or no color can be misleading.

 1def create_map_for_year(year, title):
 2    u5mr_year = u5mr_world_df[year]
 3    fig, ax = plt.subplots(
 4        nrows = 1,
 5        ncols = 1,
 6        figsize = (15,6),
 7        facecolor = plt.cm.Blues(.2))
 8    fig.suptitle(title,
 9                 fontsize = 'xx-large',  
10                 fontweight = 'bold')
11    ax.set_facecolor(plt.cm.Blues(.2))
12
13    u5mr_world_df.plot(
14        ax = ax,
15        color = 'white',
16        edgecolor = 'black'
17    )
18
19    u5mr_world_df.plot(
20        column = u5mr_year,
21        cmap = 'OrRd',
22        ax = ax,
23        legend = True,
24        legend_kwds = {'label': "UFMR per 1000",
25                      'shrink': 0.7},
26        missing_kwds = {'facecolor':'Gray'},
27    )
28    [spine.set_visible(False) for spine in ax.spines.values()]
29    ax.xaxis.set_visible(False)
30    ax.yaxis.set_visible(False)
31    return fig

Use the function to create a map for 1985.

1fig = create_map_for_year('1985', 'National Under Five Mortality Rates in 1985')
2plt.show()

Map showing country Under Five Mortality Rates in 1985

Map showing country Under Five Mortality Rates in 1985

Display changes over the decades

The function is modified to plot a map for an axis.

 1def plot_ax_for_year(ax, year):
 2    u5mr_year = u5mr_world_df[year]
 3    ax.set_title(year,
 4                 fontsize = 'xx-large',  
 5                 fontweight = 'bold')
 6    ax.set_facecolor(plt.cm.Blues(.2))
 7
 8    u5mr_world_df.plot(
 9        ax = ax,
10        color = 'white',
11        edgecolor = 'black'
12    )
13
14    u5mr_world_df.plot(
15        column = u5mr_year,
16        cmap = 'OrRd',
17        ax = ax,
18        legend = True,
19        legend_kwds = {'label': "UFMR per 1000",
20                      'shrink': 0.7},
21        missing_kwds = {'facecolor':'Gray'},
22    )
23    [spine.set_visible(False) for spine in ax.spines.values()]
24    ax.xaxis.set_visible(False)
25    ax.yaxis.set_visible(False)
26    return ax

Create a plot with maps through the decades

 1fig, axs = plt.subplots(
 2    nrows = 4,
 3    ncols = 2,
 4    figsize = (15,15),
 5    facecolor = plt.cm.Blues(.2))
 6fig.suptitle('Changes in national Under Five Mortality Rates over time',
 7             fontsize = 'xx-large',  
 8             fontweight = 'bold')
 9years = [f'{x}' for x in range(1950, 2020, 10)] + ['2019']
10for i, ax in enumerate(axs.flatten()):
11    ax = plot_ax_for_year(ax, years[i])
12fig.tight_layout(pad=2)
13plt.show()

Map showing country Under Five Mortality Rates changes from 1950 to 2019

Map showing country Under Five Mortality Rates changes from 1950 to 2019

Conclusion

Geographical maps are a great way to present a global view of where data applies to different countries and regions. Geopandas and Geoplot are used to plot world maps with countries colored based on data of interest. The world maps produced here show the changes over time on Under Five Mortality Rates. This helps visualise particular regions of the world that are consistently doing worse off in addressing child mortality.

Use of Matplotlib creates nice static images, but one drawback of these is precisely that they are static. This data could be more informative if either the maps could be more interactive or displayed as an animation.

Under-five mortality rate:
is the probability of dying between birth and exactly 5 years of age, expressed per 1,000 live births.

Display a line chart with a range

Eric Callanan — Sunday, 08 Nov 2020

Under five mortality rate is an estimate for each particular country for that year based on the available information. The accuracy of this value varies from country to country and is dependent on the availability of quality data. The data produced is composed of the median value as well as upper and lower bound limits for each year. An indication of the accuracy of the data can be shown by displaying the upper and lower bounds on a line chart.

Accurate data over time is not available for some countries and Unicef have a process of applying statistical models to estimate the Under Five Mortality Rates based on available data. The Lower and Upper values refer to the lower bound and upper bound of 90% uncertainty intervals.

This article shows how to display the median as well as the lower and upper bounds.

Load the data

Details on downloading and loading the data into a dataframe is described in "Pandas - Load data from Excel file and Display Chart". The following loads the data from a local file and adds an multi-index on Country and Bounds.

 1# Load the excel worksheet into a dataframe
 2u5mr_df = pd.read_excel(
 3    "/tmp/data/Under-five-mortality-rate_2020.xlsx",
 4    engine = "openpyxl",
 5    sheet_name = 'Country estimates (both sexes)',
 6    header = 14)
 7
 8# Drop the last two rows
 9u5mr_df.drop(u5mr_df.tail(2).index, inplace = True)
10
11# Drop ISO.Code column
12u5mr_df.drop(['ISO.Code'], axis = 1, inplace = True)
13
14# Rename 'Uncertainty.Bounds*' column to 'Uncertainty.Bounds'
15u5mr_df = u5mr_df.rename(columns={'Uncertainty.Bounds*': 'Uncertainty.Bounds'})
16
17# Set multiindex to Country and Bounds
18u5mr_df.set_index(['Country.Name', 'Uncertainty.Bounds'], inplace=True)
19
20# Convert year column names to datetime
21u5mr_df.columns = [pd.to_datetime(f'{x[:-2]}-12-31') for x in u5mr_df.columns]
22
23# Review the data
24print(u5mr_df.shape)
25u5mr_df.iloc[[0,1,2,3,-4,-3,-2,-1], [0,1,2,3,4,5,6,-4,-3,-2,-1]]
26
27"""
28                                 1960-12-31  1961-12-31  1962-12-31  2016-12-31  2017-12-31  2018-12-31  2019-12-31  
29Country.Name Uncertainty.Bounds                                                                                      
30Afghanistan  Lower                      NaN         NaN  299.425519   57.068054   53.744798   50.640922   47.442070  
31             Median                     NaN         NaN  344.629498   67.572190   64.940759   62.541196   60.269399  
32             Upper                      NaN         NaN  399.287713   78.646207   76.832873   75.405564   74.616041  
33Albania      Lower                      NaN         NaN         NaN    9.075629    9.044506    9.067390    9.068154  
34Zambia       Upper               229.757846  224.353213  219.461322   76.133459   75.733787   77.098449   77.575507  
35Zimbabwe     Lower               129.144390  125.263606  121.661598   50.834193   48.083920   44.136918   41.076802  
36             Median              151.084581  146.858820  142.639204   59.538505   58.234924   55.856832   54.612967  
37             Upper               177.879631  173.725419  168.219351   69.425121   69.987570   70.076887   71.709892  
38"""

Display the changes over time for one country

Nigeria has the highest mediam Under Five Mortality Rate in 2019. This code creates a line chart showing the changes over time.

 1country = 'Nigeria'
 2years = u5mr_df.columns
 3median = u5mr_df.loc[(['Nigeria'], ['Median']), :].iloc[0]
 4
 5title = f'Changes in Under 5 Mortality Rate\n\n {c}'
 6
 7fig, ax = plt.subplots(figsize = (8,5), facecolor = plt.cm.Blues(.2))
 8fig.suptitle(title, fontsize = 'xx-large',  fontweight = 'bold')
 9
10ax.set_facecolor(plt.cm.Blues(.2))
11ax.plot(years, median, label = 'Median')
12ax.set_ylabel('Under-five mortality rate', fontsize = 'large')
13ax.set_xlabel('Year', fontsize=14)
14ax.spines['right'].set_visible(False)
15ax.spines['top'].set_visible(False)
16
17plt.show()

Under five mortality rate over time for Nigeria

Under five mortality rate over time for Nigeria

Display upper, lower and median data for a single country

Extract the data for Nigeria into a separate dataframe. Use the data from this dataframe to plot the lower, upper and median bound values over time. This shows that while the median value is going down, the spread between the lower and upper bounds is widening. The 90% confidence interval for Under Five Mortality in Nigeria in 2019 is from 92 to 152. This is an extraordinarily wide range and shows that there is not much confidence in the median value of 117.

This code adds plot lines for upper, median and lower bounds. This displays each result in a separate color by default.

 1country = 'Nigeria'
 2
 3df_c = u5mr_df.loc[([country]), :]
 4years = df_c.columns
 5lower = df_c.loc[(slice(None), 'Lower'), :].iloc[0]
 6median = df_c.loc[(slice(None), 'Median'), :].iloc[0]
 7upper = df_c.loc[(slice(None), 'Upper'), :].iloc[0]
 8
 9title = f'Changes in Under 5 Mortality Rate\n\n {c}'
10
11fig, ax = plt.subplots(figsize = (8,5), facecolor = plt.cm.Blues(.2))
12fig.suptitle(title, fontsize = 'xx-large',  fontweight = 'bold')
13
14ax.set_facecolor(plt.cm.Blues(.2))
15ax.plot(years, upper, label = 'Upper')
16ax.plot(years, median, label = 'Median')
17ax.plot(years, lower, label = 'Lower')
18ax.legend(bbox_to_anchor = (0.9, 0.9),
19          loc = 'upper right',
20          frameon = False,
21          fontsize = 'medium')
22ax.set_ylabel('Under-five mortality rate', fontsize = 'large')
23ax.set_xlabel('Year', fontsize=14)
24# Hide the right and top spines
25ax.spines['right'].set_visible(False)
26ax.spines['top'].set_visible(False)
27
28plt.show()
29
30
31print(df_c.iloc[:, [13,14,15,-3,-2,-1]])
32"""
33                                 1963-12-31  1964-12-31  1965-12-31  2017-12-31  2018-12-31  2019-12-31  
34Country.Name Uncertainty.Bounds                                                                          
35Nigeria      Lower                      NaN  274.178909  274.741658  101.806684   97.303293   92.166854  
36             Median                     NaN  323.283445  316.175804  122.798947  120.037728  117.202078  
37             Upper                      NaN  380.916122  364.765834  150.074233  151.521302  152.451394  
38"""

Under five mortality rate over time for Nigeria with upper and lower bounds

Under five mortality rate over time for Nigeria with upper and lower bounds

Display shading around median data for a single country

This displays the same information as above, but changes the area between the upper and lower bounds to be shaded in the same color as the median values. The functionality to plot the line on the axis is split out into its own function. The axes fill_between method is used color the area between the lower and upper bounds.

 1def update_axis(ax, df, country):
 2    clr = plt.cm.Purples(0.9)
 3    ax.set_facecolor(plt.cm.Blues(.2))
 4    ax.set_title(country, fontsize = 14, fontweight = 'bold')
 5    x = df.columns
 6    y_l = df.loc[(slice(None), 'Lower'), :].iloc[0]
 7    y_m = df.loc[(slice(None), 'Median'), :].iloc[0]
 8    y_u = df.loc[(slice(None), 'Upper'), :].iloc[0]
 9    ax.plot(x, y_m, label = 'Median', color = clr)
10    ax.fill_between(x, y_l, y_u, alpha=0.3, edgecolor=clr, facecolor=clr)
11    ax.set_ylabel('Under-five mortality rate', fontsize = 'medium')
12    ax.set_xlabel('Year', fontsize = 'medium')
13    ax.tick_params(axis='both', labelsize='small')
14    ax.spines['right'].set_visible(False)
15    ax.spines['top'].set_visible(False)
16
17fig, ax1 = plt.subplots(1, 1, figsize = (8, 5), facecolor = plt.cm.Blues(.2))
18title = f'Changes in Under 5 Mortality Rate'
19fig.suptitle(title, fontsize = 'xx-large',  fontweight = 'bold')
20
21country = 'Nigeria'
22df_c = u5mr_df.loc[([country]), :]
23update_axis(ax1, df_c, country)
24
25plt.show()

Under five mortality rate over time for Nigeria with bounds shaded

Under five mortality rate over time for Nigeria with bounds shaded

Display data for four countries

The update_axis function above can be used to display multiple plots, such as for the four countries with highest under five mortality rates in 2019.

 1fig, axs = plt.subplots(2, 2,
 2                        sharex=True, sharey=True,
 3                        figsize = (10, 7),
 4                        facecolor = plt.cm.Blues(.2))
 5fig.tight_layout(pad = 5.0)
 6title = f'Changes in Under 5 Mortality Rate'
 7fig.suptitle(title, fontsize = 'xx-large',  fontweight = 'bold')
 8
 9countries = (u5mr_df.xs('Median', level='Uncertainty.Bounds')
10             .sort_values(by = u5mr_df.columns[-1], ascending = False)
11             .head(4).index)
12
13for i, ax in enumerate(axs.flatten()):
14    df_c = u5mr_df.loc[([countries[i]]), :]
15    update_axis(ax, df_c, countries[i])
16
17plt.show()

Under five mortality rate over time for four highest countries

Under five mortality rate over time for four highest countries

The same data can be extracted and displayed by changing the sort order to ascending. Countries that are doing better and have much lower under five mortality rates tend to haver much better records, so the 90% confidence interval is also much narrower.

Under five mortality rate over time for four lowest countries

Under five mortality rate over time for four lowest countries

Display data for male and female data

List the worksheets in an excel file

Use Pandas.ExcelFile to find out what sheets are in an Excel file. This shows that the file contains six worksheets with the third and fifth containing the data for Females and Males respectively.

 1u5mr_xl = pd.ExcelFile(u5mr_file)
 2u5mr_xl.sheet_names
 3
 4
 5# output
 6'''
 7['Country estimates (both sexes)',
 8 'Regional & global (both sexes)',
 9 'Country estimates (Female)',
10 'Regional & global (Female)',
11 'Country estimates (Male)',
12 'Regional & global (Male)']
13 '''

The use of the multi-index on the dataframe makes is much easier to extract the data for a particular country and then plot the bounds values. The following put the full process into three functions.

Open the specified spreadsheet into a dataframe

 1def load_dataset(file, sheet):
 2    # Load the excel worksheet into a dataframe
 3    df = pd.read_excel(
 4        file,
 5        engine = "openpyxl",
 6        sheet_name = sheet,
 7        header = 14)
 8
 9    # Drop the last two rows
10    df.drop(df.tail(2).index, inplace = True)
11
12    # Drop ISO.Code column
13    df.drop(['ISO.Code'], axis = 1, inplace = True)
14
15    # Rename 'Uncertainty.Bounds*' column to 'Uncertainty.Bounds'
16    df = df.rename(columns={'Uncertainty.Bounds*': 'Uncertainty.Bounds'})
17
18    # Set multiindex to Country and Bounds
19    df.set_index(['Country.Name', 'Uncertainty.Bounds'], inplace=True)
20
21    # Convert year column names to datetime
22    df.columns = [pd.to_datetime(f'{x[:-2]}-12-31') for x in df.columns]
23
24    print(f'df.shape = {df.shape}')
25    return df

Plot the data on an axis

 1def update_axis(ax, df, country, lbl, clr = plt.cm.Purples(0.9)):
 2    ax.set_facecolor(plt.cm.Blues(.2))
 3    ax.set_title(country, fontsize = 'xx-large', fontweight = 'bold')
 4    x = df.columns
 5    y_l = df.loc[(slice(None), 'Lower'), :].iloc[0]
 6    y_m = df.loc[(slice(None), 'Median'), :].iloc[0]
 7    y_u = df.loc[(slice(None), 'Upper'), :].iloc[0]
 8    ax.plot(x, y_m, label = lbl, color = clr)
 9    ax.fill_between(x, y_l, y_u, alpha=0.3, edgecolor=clr, facecolor=clr)
10    ax.set_ylabel('Under-five mortality rate', fontsize = 'medium')
11    ax.set_xlabel('Year', fontsize = 'medium')
12    ax.tick_params(axis='both', labelsize='small')
13    ax.spines['right'].set_visible(False)
14    ax.spines['top'].set_visible(False)
15
16    ax.legend(bbox_to_anchor = (0.9, 0.9),
17          loc = 'upper right',
18          frameon = False,
19          fontsize = 'medium')

Load the data and display the data from different spreadsheets.

 1f_u5mr_df = load_dataset(
 2    "/tmp/data/Under-five-mortality-rate_2020.xlsx",
 3    "Country estimates (Female)")
 4
 5m_u5mr_df = load_dataset(
 6    "/tmp/data/Under-five-mortality-rate_2020.xlsx",
 7    "Country estimates (Male)")
 8
 9fig, (ax1, ax2, ax3) = plt.subplots(1, 3, sharex=True, sharey=True,
10                               figsize = (12, 5),
11                               facecolor = plt.cm.Blues(.2))
12fig.tight_layout(pad = 5.0)
13title = f'Changes in Under 5 Mortality Rate - females'
14fig.suptitle(title, fontsize = 'xx-large',  fontweight = 'bold')
15
16country = 'Nigeria'
17df_c = f_u5mr_df.loc[([country]), :]
18update_axis(ax1, df_c, country, 'Female')
19update_axis(ax3, df_c, country, 'Female')
20
21df_c = m_u5mr_df.loc[([country]), :]
22update_axis(ax2, df_c, country, 'Male', clr = plt.cm.Greens(0.9))
23update_axis(ax3, df_c, country, 'Male', clr = plt.cm.Greens(0.9))
24
25plt.show()

Under five mortality rate for Males and Females in Nigeria

Under five mortality rate for Males and Females in Nigeria

Conclusion

A single line graph does not give a true picture for some information such as Under Five Mortality Rates as the gathering of the data is challenging in many countries. This data contains upper and lower bound data for 90% confidence interval. The addition of a shaded area above and below the median values provides a lot more information on the data. It can be seen that the poorer countries with higher child mortality rates also have a much greater spread from lower to upper bounds. Countries with lower child mortality rates tend to also have a narrower confidence intervals. Data is probably more accurate due to better record keeping.

Under-five mortality rate:
is the probability of dying between birth and exactly 5 years of age, expressed per 1,000 live births.

Create a Bar Chart Race with Matplotlib - part 2

Eric Callanan — Sunday, 25 Oct 2020

A Bar Chart Race is an animation where the bars representing the data change in place on the chart to represent change over time. This is the second article to improve on the bar chart race to include changing categories over time while maintaining the color associated with each category.

How to create a basic Bar Chart Race is described in "How to create a Bar Chart race - part 1". In this article the bar chart race will built upon in the following areas:

Handle a larger range of data where the countries change.
Keep the color for each country constant.
Improve the bar chart race by displaying the current value on the bar.
Wrap up in functions to create other bar chart races.

Load the data

 1# Load the excel worksheet into a dataframe
 2u5mr_df = pd.read_excel(
 3    "/tmp/data/Under-five-mortality-rate_2020.xlsx",
 4    sheet_name = 'Country estimates (both sexes)',
 5    header = 14)
 6
 7# Drop the last two rows
 8u5mr_df.drop(u5mr_df.tail(2).index, inplace = True)
 9
10# Rename the columns to Years
11u5mr_df.columns = [x[:-2] if x.endswith('.5') else x for x in u5mr_df.columns]
12
13# Rename 'Uncertainty.Bounds*' column to 'Uncertainty.Bounds'
14u5mr_df = u5mr_df.rename(columns={'Uncertainty.Bounds*': 'Uncertainty.Bounds'})
15
16# Filter to the Median values
17u5mr_med_df = u5mr_df[u5mr_df['Uncertainty.Bounds'] == 'Median']
18
19# Review the data
20u5mr_med_df.iloc[[0,1,2,3,-4,-3,-2,-1], [0,1,2,3,4,5,6,-4,-3,-2,-1]]
21"""
22    ISO.Code Country.Name Uncertainty.Bounds  1950  1951  1952        1953       2016       2017       2018       2019
231        AFG  Afghanistan             Median   NaN   NaN   NaN         NaN  67.572190  64.940759  62.541196  60.269399
244        ALB      Albania             Median   NaN   NaN   NaN         NaN   9.419110   9.418052   9.525133   9.682407
257        DZA      Algeria             Median   NaN   NaN   NaN         NaN  24.792098  24.319482  23.805926  23.256168
2610       AND      Andorra             Median   NaN   NaN   NaN         NaN   3.369056   3.218925   3.085839   2.966929
27574      VNM     Viet Nam             Median   NaN   NaN   NaN         NaN  21.220796  20.843125  20.405423  19.935167
28577      YEM        Yemen             Median   NaN   NaN   NaN         NaN  56.823614  56.966430  58.460003  58.356138
29580      ZMB       Zambia             Median   NaN   NaN   NaN  234.418232  66.510929  64.337901  63.294182  61.663465
30583      ZWE     Zimbabwe             Median   NaN   NaN   NaN         NaN  59.538505  58.234924  55.856832  54.612967
31"""

Identify the countries of interest

First get the countries that are in the top 10 for all years in the dataset. Use a lambda expression to apply a sort to each row and combining the results of the top 10 in each row. This consists of the following 41 countries.

 1Afghanistan            Angola                             Bangladesh                         
 2Benin                  Burkina Faso                       Cambodia                           
 3Cameroon               Central African Republic           Chad                               
 4Cote d'Ivoire          Democratic Republic of the Congo   Egypt                              
 5Ethiopia               Gambia                             Ghana                              
 6Guinea                 Guinea-Bissau                      Haiti                              
 7Iraq                   Jordan                             Liberia                            
 8Libya                  Malawi                             Mali                               
 9Mauritania             Mozambique                         Nepal                              
10Niger                  Nigeria                            Oman                               
11Pakistan               Peru                               Republic of Korea                  
12Rwanda                 Senegal                            Sierra Leone                       
13Somalia                South Sudan                        Togo                               
14Turkey                 Yemen

1fields = ['Country.Name'] + [str(x) for x in range(1950, 2020)]
2sel_df = u5mr_med_df[fields].set_index('Country.Name')
3countries = list(sel_df.apply(
4    lambda x: x.sort_values(ascending = False).head(10),
5    axis = 0).index)

Create a color map for these countries.

The original bar chart animation showed changes in the same 5 countries from 2015 to 2019. The bar chart race is an animation of a sequence of bar charts in which the color of the bars is associated with the country for the value on the bar. Each bar chart is generated independently of the previous bar charts, so the color of the countries will change as new countries are added to the chart and other countries are removed. This is confusing in the final animated result.

A color-map is created assigning a color to each of the 41 countries to address this issue. There is no predefined color palette for more than 20 colors. The palette is created by joining colors from multiple palettes together.

1# Colours
2cols = plt.cm.tab20.colors + plt.cm.Dark2.colors + plt.cm.Set3.colors + plt.cm.tab20b.colors
3
4m = zip(countries, cols)
5color_dict = {x[0]:x[1] for x in m}
6color_dict

Unique colors for top countries

Unique colors for top countries

Get the data for these 41 countries of interest

Extract the data for the top 41 countries and transpose the data into wide format. Change the index to the last day of the year for each year and convert to datetime format.

 1# Get the data for the countries of interest
 2data_df = (u5mr_med_df.drop(['ISO.Code', 'Uncertainty.Bounds'], axis=1)
 3           [u5mr_med_df['Country.Name'].isin(countries)]).copy()
 4
 5# Set index to "Country.Name" and Transpose the dataframe
 6wide_df = data_df.set_index('Country.Name').T
 7
 8# Remove the column name
 9wide_df.rename_axis(None, axis=1, inplace=True)
10
11# Set index to datetime for end of year
12wide_df.index = pd.to_datetime([f"{x}-12-31" for x in wide_df.index])
13
14wide_df.iloc[[0,1,2,3,-4,-3,-2,-1], [0,1,2,3,-3,-2,-1]]
15"""
16            Afghanistan     Angola  Bangladesh       Benin        Togo      Turkey      Yemen  
171950-12-31          NaN        NaN  345.207382  349.108392  317.200927         NaN        NaN  
181951-12-31          NaN        NaN  335.272222  345.669180  311.993315         NaN        NaN  
191952-12-31          NaN        NaN  325.617440  342.068496  307.047255         NaN        NaN  
201953-12-31          NaN        NaN  316.000124  338.513583  302.123318  299.282646        NaN  
212016-12-31    67.572190  84.211894   35.680942   97.416395   73.515650   12.148032  56.823614  
222017-12-31    64.940759  80.622302   33.921226   95.133079   71.317235   11.396633  56.966430  
232018-12-31    62.541196  77.672320   32.266390   92.773521   69.115785   10.696923  58.460003  
242019-12-31    60.269399  74.686710   30.753860   90.286429   66.904696   10.046388  58.356138  
25"""

Expand the data set for smooth animation

Expand the dataset with rows every two months and creating a ranking dataframe with rankings for the countries for each year. Fill in the newly created rows in the dataframes using interpolate function. Increment any duplicate rankings for a given row to avoid bars disappearing from the bar chart animation.

 1# Expand the dataset
 2expanded_df = wide_df.asfreq('2M')
 3
 4# Create a ranking dataframe
 5rank_df = expanded_df.rank(axis = 1, method = 'first', ascending = False)
 6
 7expanded_df = expanded_df.interpolate()
 8rank_df = rank_df.interpolate()
 9
10# Remove duplicate ranking
11rank_df = rank_df.where(~rank_df.apply(pd.Series.duplicated, axis=1), rank_df*1.01)

Display bar chart for a single year

Displaying a bar chart for a single year showing the ten countries with the highest Under Five Mortality Rate for that year.

 1fig, ax = plt.subplots(nrows = 1,
 2                       ncols = 1,
 3                       figsize = (10, 7),
 4                       facecolor = plt.cm.Blues(.2),
 5                       tight_layout = True)
 6bar_num = 10
 7i = 138
 8sel_df = expanded_df.iloc[:, list(rank_df.iloc[i] <= bar_num)]
 9ax.barh(y = rank_df.iloc[:, list(rank_df.iloc[i] <= bar_num)].iloc[i],
10        tick_label = sel_df.columns,
11        width = sel_df.iloc[i],
12        color = [color_dict[col] for col in sel_df.columns],
13        alpha = 0.8)
14cur_year = expanded_df.index[i].strftime('%Y-%m')
15ax.set_title(f'Under Five Mortality Rate - {cur_year}',
16             fontsize = 'xx-large',
17             fontweight = 'bold')
18ax.set_ylim(10.8, 0.2)
19ax.set_facecolor(plt.cm.Blues(.2))
20ax.tick_params(labelsize = 'medium')
21ax.grid(True, axis = 'x', color=plt.cm.Blues(.1))
22ax.set_axisbelow(True)
23[spine.set_visible(False) for spine in ax.spines.values()]
24plt.show()

Top ten countries with highest Under Five Mortality Rate in 1973

Top ten countries with highest Under Five Mortality Rate in 1973

Display bar chart with mortality rate on the bars

It can be helpful to see the under five mortality rate on the bar chart as the data is changing. This is done by adding an annotation to each of the bars. The mortality rate is displayed inside of the right edge of the bar. This could be displayed anywhere and works quite well outside to the right of the bar.

 1fig, ax = plt.subplots(nrows = 1,
 2                       ncols = 1,
 3                       figsize = (10, 7),
 4                       facecolor = plt.cm.Blues(.2),
 5                       tight_layout = True)
 6bar_num = 10
 7i = 138
 8sel_df = expanded_df.iloc[:, list(rank_df.iloc[i] <= bar_num)]
 9bars = ax.barh(y = rank_df.iloc[:, list(rank_df.iloc[i] <= bar_num)].iloc[i],
10               tick_label = sel_df.columns,
11               width = sel_df.iloc[i],
12               color = [color_dict[col] for col in sel_df.columns],
13               alpha = 0.8)
14
15cur_year = expanded_df.index[i].strftime('%Y-%m')
16ax.set_title(f'Under Five Mortality Rate - {cur_year}',
17             fontsize = 'xx-large',
18             fontweight = 'bold')
19ax.set_ylim(10.8, 0.2)
20ax.set_facecolor(plt.cm.Blues(.2))
21ax.tick_params(labelsize = 'medium')
22ax.grid(True, axis = 'x', color=plt.cm.Blues(.1))
23ax.set_axisbelow(True)
24[spine.set_visible(False) for spine in ax.spines.values()]
25
26for bar in bars:
27    width = bar.get_width()
28    ax.annotate(f'{width:.0F}',
29                xy = (width , bar.get_y() + bar.get_height() / 2),
30                xytext = (-25, 0),
31                textcoords = "offset points",
32                fontsize = 'xx-large',
33                fontweight = 'bold',
34                ha = 'right',
35                va = 'center')

Top ten countries with highest Under Five Mortality Rates displaying rate on bars

Top ten countries with highest Under Five Mortality Rates displaying rate on bars

Display a random sample of bar charts for different year

The expanded dataframe now has 415 rows, the final animation is created by generating 415 bar charts and creating a sequence of these charts. It is helpful to take a random sample of the dataset to validate that the bar charts are displayed as expected. The random rows are selected using the dataframe sample function.

 1sample_num = 5
 2d_df = expanded_df.sample(n=sample_num, random_state=1)
 3r_df = rank_df.loc[d_df.index]
 4
 5fig, axs = plt.subplots(nrows = 1,
 6                        ncols = sample_num,
 7                        figsize = (15, 5),
 8                        facecolor = plt.cm.Blues(.2),
 9                        tight_layout = True)
10
11for i, ax in enumerate(axs.flatten()):
12    sel_df = d_df.iloc[:, list(r_df.iloc[i] <= bar_num)]
13    bars = ax.barh(y = r_df.iloc[:, list(r_df.iloc[i] <= bar_num)].iloc[i],
14                   tick_label = sel_df.columns,
15                   width = sel_df.iloc[i],
16                   color = [color_dict[col] for col in sel_df.columns],
17                   alpha = 0.8)
18
19    cur_year = d_df.index[i].strftime('%Y-%m')
20    ax.set_title(f'{cur_year}',
21                 fontsize = 'large',
22                 fontweight = 'bold')
23    ax.set_ylim(10.8, 0.2)
24    ax.set_facecolor(plt.cm.Blues(.2))
25    ax.tick_params(labelsize = 'medium')
26    ax.grid(True, axis = 'x', color=plt.cm.Blues(.1))
27    ax.set_axisbelow(True)
28    [spine.set_visible(False) for spine in ax.spines.values()]
29
30    for bar in bars:
31        width = bar.get_width()
32        ax.annotate(f'{width:.0F}',
33                    xy = (width , bar.get_y() + bar.get_height() / 2),
34                    xytext = (-5, 0),
35                    textcoords = "offset points",
36                    fontsize = 'medium',
37                    fontweight = 'bold',
38                    ha = 'right',
39                    va = 'center')
40
41plt.show()

Sample bar charts od U5MR from expanded dataset

Sample bar charts od U5MR from expanded dataset

Create bar chart race of highest U5MR over the years

The animation is created by using the FuncAnimation function in Matplotlib. As there are 415 rows in the expladed dataset, this animation can take some time to generate. Note that defining the functions is instantaneous, the time is required when saving the animation to either gif, html or mp4.

 1def update(i):
 2    ax.clear()
 3
 4    p = expanded_df.columns.map(len).max()
 5    bar_num = 10
 6    sel_df = expanded_df.iloc[:, list(rank_df.iloc[i] <= bar_num)]
 7    bars = ax.barh(y = rank_df.iloc[:, list(rank_df.iloc[i] <= bar_num)].iloc[i],
 8                   tick_label = [x.rjust(p, ' ') for x in sel_df.columns],
 9                   width = sel_df.iloc[i],
10                   color = [color_dict[col] for col in sel_df.columns],
11                   alpha = 0.8)
12    plt.setp(ax.get_xticklabels(), fontsize='x-small')
13    plt.setp(ax.get_yticklabels(), fontsize='small', fontfamily = 'monospace')
14
15    cur_year = expanded_df.index[i].strftime('%Y')
16    ax.set_title(f'Under Five Mortality Rate - {cur_year}',
17                 fontsize = 'larger',
18                 fontweight = 'bold',
19                 loc = 'center')
20    ax.set_ylim(10.8, 0.2)
21    ax.set_facecolor(plt.cm.Blues(.2))
22    ax.grid(True, axis = 'x', color=plt.cm.Blues(.1))
23    ax.set_axisbelow(True)
24    [spine.set_visible(False) for spine in ax.spines.values()]
25
26    for bar in bars:
27        width = bar.get_width()
28        ax.annotate(f'{width:.0F}',
29                    xy = (width , bar.get_y() + bar.get_height() / 2),
30                    xytext = (-20, 0),
31                    textcoords = "offset points",
32                    fontsize = 'small',
33                    ha = 'right',
34                    va = 'center')    
35
36fig, ax = plt.subplots(figsize = (8, 5),
37                       facecolor = plt.cm.Blues(.2),
38                       dpi = 150,
39                       tight_layout = True)
40
41u5mr_anim = anim.FuncAnimation(
42    fig = fig,
43    func = update,
44    frames = len(expanded_df),
45    interval = 300)

Generate the gif file.

1u5mr_anim.save('U5MR_bar_chart_race_all_years.gif')

Bar chart race showing top ten countries with highest under five mortality rate from 1950 to 2019

Bar chart race showing top ten countries with highest under five mortality rate from 1950 to 2019"

MP4 file of bar chart race from 1950 to 2019

Wrap up bar chart race creation in functions

There are a number of steps in preparing the data and then expanding, ranking and finally generating the bar chart race. The following wraps this up in a series of functions so these can be used to create a bar chart race from a similar dataset. Three functions are created; one to prepare the data in wide format; one to expand the data; and the final function to create the animation.

prepare_data - Function to get the appropriate data and convert to wide format.

 1def prepare_data(df, highest = True):
 2    # Get all the countries in the top 10 for the years
 3    fields = ['Country.Name'] + [str(x) for x in range(1950, 2020)]
 4    sel_df = df[fields].set_index('Country.Name')
 5    countries = list(sel_df.apply(
 6        lambda x: x.sort_values(ascending = not highest).head(10),
 7        axis = 0).index)
 8
 9    # Create color map for countries
10    cols = plt.cm.tab20.colors + plt.cm.Dark2.colors + plt.cm.Set3.colors + plt.cm.tab20b.colors
11    color_dict = {x[0]:x[1] for x in zip(countries, cols)}
12
13    # Get the data for the countries of interest
14    data_df = (df.drop(['ISO.Code', 'Uncertainty.Bounds'], axis=1)
15               [df['Country.Name'].isin(countries)]).copy()
16
17    # Set index to "Country.Name" and Transpose the dataframe
18    wide_df = data_df.set_index('Country.Name').T
19
20    # Remove the column name
21    wide_df.rename_axis(None, axis=1, inplace=True)
22
23    # Convert index to datetime
24    wide_df.index = pd.to_datetime([f"{x}-12-31" for x in wide_df.index])
25
26    return wide_df, color_dict

expand_data - Function to expand the data and create a ranking dataframe.

 1def expand_data(df, highest = True):
 2    e_df = df.asfreq('2M')
 3
 4    # Create ranking dataset
 5    r_df = e_df.rank(axis = 1, method = 'first', ascending = highest)
 6
 7    # Interpolate
 8    e_df = e_df.interpolate()
 9    r_df = r_df.interpolate()
10
11    # Remove duplicate ranks from the same row
12    r_df = r_df.where(~r_df.apply(pd.Series.duplicated, axis=1), r_df*1.01)
13
14    return e_df, r_df

create_animation - Function to create the animation with the dataframes.

 1def create_animation(expanded_df, rank_df, color_dict, highest = True):
 2    def update2(i):
 3        ax.clear()
 4
 5        p = expanded_df.columns.map(len).max()
 6        bar_num = 10
 7        sel_df = expanded_df.iloc[:, list(rank_df.iloc[i] <= bar_num)]
 8        bars = ax.barh(y = rank_df.iloc[:, list(rank_df.iloc[i] <= bar_num)].iloc[i],
 9                       tick_label = [x.rjust(p, ' ') for x in sel_df.columns],
10                       width = sel_df.iloc[i],
11                       color = [color_dict[col] for col in sel_df.columns],
12                       alpha = 0.8)
13
14        plt.setp(ax.get_xticklabels(), fontsize='small')
15        plt.setp(ax.get_yticklabels(), fontsize='medium', fontfamily = 'monospace')
16
17        cur_year = expanded_df.index[i].strftime('%Y')
18        ax.set_title(f'Under Five Mortality Rate - {cur_year}',
19                     fontsize = 'larger',
20                     fontweight = 'bold',
21                     loc = 'right')
22        ax.set_ylim(10.8, 0.2)
23        if not highest:
24            ax.set_ylim(0.2, 10.8)
25        ax.set_facecolor(plt.cm.Blues(.2))
26        ax.grid(True, axis = 'x', color=plt.cm.Blues(.1))
27        ax.set_axisbelow(True)
28        [spine.set_visible(False) for spine in ax.spines.values()]
29
30
31        h_offset = -20 if highest else 2
32        h_align = 'right' if highest else 'left'
33        for bar in bars:
34            width = bar.get_width()
35            dislpay_value = f'{width:.0F}' if highest else f'{width:.1F}'
36            ax.annotate(dislpay_value,
37                        xy = (width , bar.get_y() + bar.get_height() / 2),
38                        xytext = (h_offset, 0),
39                        textcoords = "offset points",
40                        fontsize = 'small',
41                        ha = h_align,
42                        va = 'center')    
43
44    fig, ax = plt.subplots(figsize = (8, 3),
45                           facecolor = plt.cm.Blues(.2),
46                           dpi = 150,
47                           tight_layout = True)
48
49    data_anim = anim.FuncAnimation(
50        fig = fig,
51        func = update2,
52        frames = len(expanded_df),
53        interval = 200)
54
55    return data_anim

Finally, call these functions to create a bar chart race for countries with either the highest or lowest Under Five Mortality Rates over time. Setting highest = False creates a bar chart race for the countries with the lowest Under Five Mortality Rates from 1950 to 2019.

 1highest = False
 2# 1. Prepare the data
 3df, col_dict = prepare_data(u5mr_med_df, highest)
 4
 5# 2. Expand the data
 6e_df, r_df = expand_data(df)
 7
 8# 3. Create animation
 9data_anim = create_animation(e_df, r_df, col_dict, highest)
10
11# 4. Save animation as gif
12data_anim.save('Bar_chart_race_U5MR_lowest_countries.gif')

Bar chart race showing the countries with lowest under five mortality rate from 1950 to 2019

Bar chart race showing the countries with lowest under five mortality rate from 1950 to 2019

MP4 version available here - MP4 file for bar chart race showing the countries with lowest under five mortality rate from 1950 to 2019

Display U5MR bar chart race for selected countries

The same functions can also be used to compare changes to specific countries over the years.

 1highest = False
 2# 1. Prepare the data
 3df, col_dict = prepare_data(selected_df, highest)
 4
 5# 2. Expand the data
 6e_df, r_df = expand_data(df)
 7
 8# 3. Create animation
 9data_anim = create_animation(e_df, r_df, col_dict, highest)
10
11# 4. Save animation as gif
12data_anim.save('Bar_chart_race_U5MR_selected_countries.gif')

Bar chart race showing under five mortality rate changes for selected countries

Bar chart race showing under five mortality rate changes for selected countries

MP4 version available here - MP4 file for Bar chart race showing under five mortality rate changes for selected countries

Conclusion

The creation of a Bar Chart Race using Matplotlib was improved to handle the countries changing over time. The main challenge is to keep consistent bar colors for each country and this was achieved by creating a color map for all of the countries that appear in the bar chart race. A bar chart race is an animated sequence of bar charts, so the size and number of the charts has an impact on the final size of the gif or mp4 file. The current value for each country is displayed on the bar and this makes it easier to see changes over time.

It was noted that the position of the y-axis changes to accomodate the labels for the countries. This was resolved by right-padding the string for the country name with white space and setting the font to a constant width.

Create a Bar Chart Race with Matplotlib - part 1

Eric Callanan — Sunday, 18 Oct 2020

Bar charts present categories of data as rectangular bars with the heights or widths proportional to the values they represent. Horizontal bar charts are used in this article in which the length of the rectangular represents the value for each category. A standard or static bar chart represents one set of data and a series of bar charts can be used to show the changes over time.

A Bar Chart Race is an animation where the bars representing the data change in place on the chart to represent change over time. It can be a very effective way to visualise the changes in the top section of a data set over time. Bar Chart Race tend to be limited to the top 5 or top 10 to be more effective in communicating a message.

This article will show how to create a Bar Chart race using Matplotlib. It will demonstrate the changes in the countries with the highest Under Five Mortality Rates between 2015 and 2019. This year range is selected as the five countries have remained the same, just their relative position has changed.

The next article will show how to create the Bar Chart race for the ten highest countries since records began. This deals with adding new countries and maintaining the same color for a country throughout the animation.

Outline

Load the data.
Filter to the data concerned.
Display a bar chart.
Display a series of bar charts over time.
Display different colors for each country.
Add Intermediate data for smooth transition.
Create animation with FuncAnimation

Load the data

 1# Load the excel worksheet into a dataframe
 2u5mr_df = pd.read_excel(
 3    "/tmp/data/Under-five-mortality-rate_2020.xlsx",
 4    sheet_name = 'Country estimates (both sexes)',
 5    header = 14)
 6
 7# Drop the last two rows
 8u5mr_df.drop(u5mr_df.tail(2).index, inplace = True)
 9
10# Rename the columns to Years
11u5mr_df.columns = [x[:-2] if x.endswith('.5') else x for x in u5mr_df.columns]
12
13# Rename 'Uncertainty.Bounds*' column to 'Uncertainty.Bounds'
14u5mr_df = u5mr_df.rename(columns={'Uncertainty.Bounds*': 'Uncertainty.Bounds'})
15
16# Filter to the Median values
17u5mr_med_df = u5mr_df[u5mr_df['Uncertainty.Bounds'] == 'Median']
18
19# Review the data
20u5mr_med_df.iloc[[0,1,2,3,-4,-3,-2,-1], [0,1,2,3,4,5,6,-4,-3,-2,-1]]
21"""
22    ISO.Code Country.Name Uncertainty.Bounds  1950  1951  1952        1953       2016       2017       2018       2019
231        AFG  Afghanistan             Median   NaN   NaN   NaN         NaN  67.572190  64.940759  62.541196  60.269399
244        ALB      Albania             Median   NaN   NaN   NaN         NaN   9.419110   9.418052   9.525133   9.682407
257        DZA      Algeria             Median   NaN   NaN   NaN         NaN  24.792098  24.319482  23.805926  23.256168
2610       AND      Andorra             Median   NaN   NaN   NaN         NaN   3.369056   3.218925   3.085839   2.966929
27574      VNM     Viet Nam             Median   NaN   NaN   NaN         NaN  21.220796  20.843125  20.405423  19.935167
28577      YEM        Yemen             Median   NaN   NaN   NaN         NaN  56.823614  56.966430  58.460003  58.356138
29580      ZMB       Zambia             Median   NaN   NaN   NaN  234.418232  66.510929  64.337901  63.294182  61.663465
30583      ZWE     Zimbabwe             Median   NaN   NaN   NaN         NaN  59.538505  58.234924  55.856832  54.612967
31"""

Identify the countries of interest

The countries that are in the top five with the highest Under Five Mortality Rates changes over time. The following function returns a list of countries the top range over a specified period. This shows that the following countries have been the five with the highest Under Five Mortality Rates from 2015 to 2019:

Nigeria
Somalia
Chad
Central African Republic
Sierra Leone

 1def get_countries_for_range(df, start, end, lowest = False, num = 10):
 2    '''Get a list of names of countries that are in the top number
 3    for each of the years over the specified range.
 4
 5    Keyword arguments:
 6    df       -- dateframe of all the mortality rates for all the countries
 7    start    -- First year in the range (inclusive)
 8    end      -- Last year in the range (inclusive)
 9    lowest   -- boolean flag to specify either contries with the lowest rate
10                or highest rate (default is False)
11    num      -- number of countries in the top or bottom data set (default is 10)
12
13    return: A list of unique country names that have been in the top num during the time period
14    '''
15    fields = ['Country.Name'] + [str(x) for x in range(start, end + 1)]
16    sel_df = u5mr_med_df[fields].set_index('Country.Name')
17    countries = list(sel_df.apply(lambda x: x.sort_values(ascending = lowest).head(num), axis = 0).index)
18    return countries
19
20
21get_countries_for_range(u5mr_med_df, 2015, 2019, lowest = False, num = 5)
22"""
23['Central African Republic', 'Nigeria', 'Somalia', 'Chad', 'Sierra Leone']
24"""

Get the data for the top five countries from 2015 to 2019

The function above is used to get the top five countries, which is then used to filter the data down to the year range for these countries.

 1countries = get_countries_for_range(u5mr_med_df, 2015, 2019, lowest = False, num = 5)
 2highest_df = (u5mr_med_df[u5mr_med_df['Country.Name'].isin(countries)]
 3              [['Country.Name', '2015', '2016', '2017', '2018', '2019']])
 4
 5"""
 6                 Country.Name        2015        2016        2017        2018         2019
 797   Central African Republic  126.714463  121.593307  117.470988  114.222380   110.053912
 8100                      Chad  129.406257  125.604878  121.508350  117.664883   113.790418
 9376                   Nigeria  126.833543  125.040170  122.798947  120.037728   117.202078
10466              Sierra Leone  138.342774  123.232842  118.222181  113.544491   109.236528
11481                   Somalia  132.720296  128.428442  124.393442  120.326222   116.972096
12"""

Display a bar chart for 2019

Start with displaying a bar chart for a single year for the top five countries. The following displays a bar chart for 2019.

 1top_df = highest_df[['Country.Name', '2019']].sort_values(by = '2019', ascending = False)
 2
 3fig, ax = plt.subplots(figsize = (10,7), facecolor=plt.cm.Blues(.2))
 4ax.set_facecolor(plt.cm.Blues(.2))
 5
 6fig.suptitle('Highest Under Five Mortality Rate in 2019',
 7             fontsize = 18,  
 8             fontweight = 'bold')
 9
10countries = list(top_df['Country.Name'])
11y_pos = np.arange(len(countries))
12u5mr = list(top_df['2019'])
13
14ax.barh(y_pos, u5mr, align='center')
15ax.set_yticks(y_pos)
16ax.invert_yaxis()
17ax.set_xlabel('Under-five mortality rate', fontsize = 16)
18
19# Display highest on top
20ax.set_yticklabels(countries, fontsize = 16)
21
22# Hide the right and top spines
23ax.spines['right'].set_visible(False)
24ax.spines['top'].set_visible(False)
25
26plt.show

Bar chart for highest Under Five Mortality in 2019

Bar chart for highest Under Five Mortality in 2019

Display a series of bar charts for 2015 to 2019

The data is transposed into wide format to display the data for each year in a sequence. A bar chart race could then be created by displaying the sequence of bar charts over time.

 1wide_df = highest_df.set_index('Country.Name').T
 2# Remove the column name
 3wide_df.rename_axis(None, axis = 1, inplace = True)
 4wide_df
 5
 6"""
 7      Central African Republic        Chad     Nigeria  Sierra Leone     Somalia
 82015                126.714463  129.406257  126.833543    138.342774  132.720296
 92016                121.593307  125.604878  125.040170    123.232842  128.428442
102017                117.470988  121.508350  122.798947    118.222181  124.393442
112018                114.222380  117.664883  120.037728    113.544491  120.326222
122019                110.053912  113.790418  117.202078    109.236528  116.972096
13"""

Display a single bar chart with color for country

Display a single row in a bar chart with a different color for each country.

 1sel_year = wide_df.loc['2019'].sort_values()
 2
 3fig, ax = plt.subplots(figsize = (10, 7), dpi = 144, facecolor = plt.cm.Blues(.2))
 4fig.suptitle('2019', fontsize = 18,  fontweight = 'bold')
 5ax.barh(y = sel_year.index,
 6        width = sel_year.values,
 7        color = plt.cm.Set1(range(5)))
 8ax.set_facecolor(plt.cm.Blues(.2))
 9ax.tick_params(labelsize = 8, length = 0)
10ax.grid(True, axis = 'x', color=plt.cm.Blues(.1))
11ax.set_axisbelow(True)
12[spine.set_visible(False) for spine in ax.spines.values()]
13plt.show()

Bar chart for highest Under Five Mortality in 2019 with different color for country

Bar chart for highest Under Five Mortality in 2019 with different color for country

Display a series of bar charts from 2015 to 2019

Create a series of bar charts for the year range and display the top 5 for each year. The default is for the color to stay with the order in the data series. This is confusing as it is difficult to see the countries change position.

 1fig, axs = plt.subplots(nrows = 1, ncols = 5, figsize = (15, 5), tight_layout = True)
 2dates = list(wide_df.index)
 3for ax, date in zip(axs, dates):
 4    sel_year = wide_df.loc[date].sort_values()
 5    ax.barh(y = sel_year.index,
 6            width = sel_year.values,
 7            color = plt.cm.Set1(range(5)))
 8    ax.set_title(date, fontsize = 'larger')
 9    ax.set_facecolor(plt.cm.Blues(.2))
10    ax.tick_params(labelsize = 8, length = 0)
11    ax.grid(True, axis = 'x', color=plt.cm.Blues(.1))
12    ax.set_axisbelow(True)
13    [spine.set_visible(False) for spine in ax.spines.values()]

Bar chart series highest Under Five Mortality with color for highest

Bar chart series highest Under Five Mortality with color for highest

Display a series of bar charts from 2015 to 2019 with color per country

The dataframe rank function is used to order the results so that the color for each country stays the same. The tick_label needs to be set to display the country name rather than the ranking. In this series of charts the countries retain the same color and it is much easier to see them changing positions.

 1fig, axs = plt.subplots(nrows = 1, ncols = 5, figsize = (15, 5), tight_layout = True)
 2dates = list(wide_df.index)
 3for ax, date in zip(axs, dates):
 4    sel_year = wide_df.loc[date]
 5    ax.barh(y = sel_year.rank(method='first').values,
 6            tick_label = sel_year.index,
 7            width = sel_year.values,
 8            color = plt.cm.Set1(range(5)))
 9    ax.set_title(date, fontsize = 'larger')
10    ax.set_facecolor(plt.cm.Blues(.2))
11    ax.tick_params(labelsize = 8, length = 0)
12    ax.grid(True, axis = 'x', color=plt.cm.Blues(.1))
13    ax.set_axisbelow(True)
14    [spine.set_visible(False) for spine in ax.spines.values()]

Bar chart series highest Under Five Mortality with color per country

Bar chart series highest Under Five Mortality with color per country

Abrupt transition

This sequence of bar charts can be combined in a sequence, but the transition from one year to the next is a bit abrupt.

Initial bar chart race without smooth transition

Initial bar chart race without smooth transition

Smooth transition

Intermediary data points need to be created to create a smooth transition from one year to the next. This is done with the following steps:

Convert the index to datetime
Expand the rows to add intermediary rows
Create ranking dataframe
Fill in the expanded rows with incremental values
Modify duplicate rankings as category data can be hidden
Create an animation with a series of these charts

First convert the year values in the index to datetime values for the last day of the year.

 1wide_df.index = [f"{x}-12-31" for x in wide_df.index]
 2wide_df.index = pd.to_datetime(wide_df.index)
 3
 4"""
 5            Central African Republic        Chad     Nigeria  Sierra Leone     Somalia
 62015-12-31                126.714463  129.406257  126.833543    138.342774  132.720296
 72016-12-31                121.593307  125.604878  125.040170    123.232842  128.428442
 82017-12-31                117.470988  121.508350  122.798947    118.222181  124.393442
 92018-12-31                114.222380  117.664883  120.037728    113.544491  120.326222
102019-12-31                110.053912  113.790418  117.202078    109.236528  116.972096
11"""

The data can be expanded using dataframe asfreq method. The following code inserts new rows at 2-monthly intervals between each data row.

 1expanded_df = wide_df.asfreq('2M')
 2
 3expanded_df.tail(7)
 4"""
 5            Central African Republic        Chad     Nigeria  Sierra Leone     Somalia
 62018-12-31                114.222380  117.664883  120.037728    113.544491  120.326222
 72019-02-28                       NaN         NaN         NaN           NaN         NaN
 82019-04-30                       NaN         NaN         NaN           NaN         NaN
 92019-06-30                       NaN         NaN         NaN           NaN         NaN
102019-08-31                       NaN         NaN         NaN           NaN         NaN
112019-10-31                       NaN         NaN         NaN           NaN         NaN
122019-12-31                110.053912  113.790418  117.202078    109.236528  116.972096
13"""

A dataframe for rank needs to be created for ordering the countries. Use axis = 1 to rank across the row and use ascending = False to rank the countries in reverse order so the country with the highest mortality rate is number 1.

 1rank_df = expanded_df.rank(axis = 1, method = 'first', ascending = False)
 2
 3rank_df.tail(7)
 4"""
 5            Central African Republic  Chad  Nigeria  Sierra Leone  Somalia
 62018-12-31                       4.0   3.0      2.0           5.0      1.0
 72019-02-28                       NaN   NaN      NaN           NaN      NaN
 82019-04-30                       NaN   NaN      NaN           NaN      NaN
 92019-06-30                       NaN   NaN      NaN           NaN      NaN
102019-08-31                       NaN   NaN      NaN           NaN      NaN
112019-10-31                       NaN   NaN      NaN           NaN      NaN
122019-12-31                       4.0   3.0      1.0           5.0      2.0
13"""

The missing data is filled in using dataframe interpolate method.

 1expanded_df = expanded_df.interpolate()
 2
 3expanded_df.tail(7)
 4"""
 5            Central African Republic        Chad     Nigeria  Sierra Leone     Somalia  
 62018-12-31                114.222380  117.664883  120.037728    113.544491  120.326222  
 72019-02-28                113.527635  117.019139  119.565120    112.826497  119.767201  
 82019-04-30                112.832890  116.373395  119.092512    112.108503  119.208180  
 92019-06-30                112.138146  115.727650  118.619903    111.390509  118.649159  
102019-08-31                111.443401  115.081906  118.147295    110.672516  118.090138  
112019-10-31                110.748657  114.436162  117.674686    109.954522  117.531117  
122019-12-31                110.053912  113.790418  117.202078    109.236528  116.972096  
13"""

 1rank_df = rank_df.interpolate()
 2
 3rank_df.tail(7)
 4"""
 5            Central African Republic  Chad   Nigeria  Sierra Leone   Somalia
 62018-12-31                       4.0   3.0  2.000000           5.0  1.000000
 72019-02-28                       4.0   3.0  1.833333           5.0  1.166667
 82019-04-30                       4.0   3.0  1.666667           5.0  1.333333
 92019-06-30                       4.0   3.0  1.500000           5.0  1.500000
102019-08-31                       4.0   3.0  1.333333           5.0  1.666667
112019-10-31                       4.0   3.0  1.166667           5.0  1.833333
122019-12-31                       4.0   3.0  1.000000           5.0  2.000000
13"""

Bar chart series showing transition from 2018 to 2019

Bar chart series showing transition from 2018 to 2019

Missing country in intermediary data

It can be seen in the ranking dataframe that two countries (Nigeria and Somalia) have the same rank in the middle row. Both have a rank of 1.5 as Nigeria is transitioning from 2 to 1 and Somalia is transitioning from 1 to 2. This results in the bar for Somalia over writing the bar for Nigeria so that Nigeria is not displayed at all in the middle transition bar chart.

Bar chart series showing country missing from 2018 to 2019

Bar chart series showing country missing from 2018 to 2019

The duplicates in the rankings dataframe are incremented to ensure all the countries are consistently displayed. The alpha of the color is also set so that the bars have a little transparency so they are seen moving over each other.

 1# Increment duplicate ranks from the same row
 2rank_df = rank_df.where(~rank_df.apply(pd.Series.duplicated, axis=1), rank_df*1.01)
 3
 4"""
 5            Central African Republic  Chad   Nigeria  Sierra Leone   Somalia
 62018-12-31                       4.0   3.0  2.000000           5.0  1.000000
 72019-02-28                       4.0   3.0  1.833333           5.0  1.166667
 82019-04-30                       4.0   3.0  1.666667           5.0  1.333333
 92019-06-30                       4.0   3.0  1.500000           5.0  1.515000
102019-08-31                       4.0   3.0  1.333333           5.0  1.666667
112019-10-31                       4.0   3.0  1.166667           5.0  1.833333
122019-12-31                       4.0   3.0  1.000000           5.0  2.000000
13"""

Bar chart series showing transition from 2018 to 2019 with duplicate offset

Bar chart series showing transition from 2018 to 2019 with duplicate offset

Creating animation

The animation is created by generating a bar chart for each of the rows in the expanded dataframe using the rankings dataframe for bar positions, then combining these images into a sequence. This can be achieved using the FuncAnimation function in Matplotlib. FuncAnimation generates an animation by repeatedly calling a function to clear the previous bars and draw the bars for the next set of data. It is important to clear the bars from the previous chart on each interation of the function being called.

There are a number of parameters required in FuncAnimation:

fig - the figure object.
func = the function to plot the data on the axes.
frames = the number of frames, pass in the number of rows in the expanded dataframe.
interval = the delay in milliseconds between frames, default is 200.

 1def clear_chart():
 2    ax.clear()
 3    ax.set_facecolor(plt.cm.Blues(.2))
 4    ax.tick_params(labelsize = 10) #, length = 0)
 5    ax.grid(True, axis = 'x', color = plt.cm.Blues(.1))
 6    ax.set_axisbelow(True)
 7    [spine.set_visible(False) for spine in ax.spines.values()]
 8    ax.set_ylim(5.8, 0.2)
 9    ax.set_xlim(0, 140)
10
11
12
13def update(i):
14    clear_chart()  
15    ax.barh(y = rank_df.iloc[i],
16            tick_label = expanded_df.iloc[i].index,
17            width = expanded_df.iloc[i].values,
18            height = 0.8,
19            color = plt.cm.Set1(range(5)),
20            alpha = 0.8)
21
22    cur_year = expanded_df.index[i].strftime('%Y')
23    ax.set_title(f'Under Five Mortality Rate - {cur_year}',
24                 fontsize = 22,
25                 fontweight = 'bold')
26
27
28fig, ax = plt.subplots(figsize = (10,7),
29                       facecolor = plt.cm.Blues(.2),
30                       dpi = 150,
31                       tight_layout = True)
32u5mr_anim = anim.FuncAnimation(
33    fig = fig,
34    func = update,
35    frames = len(expanded_df),
36    interval = 300)

Save the animation as a gif file.

1u5mr_anim.save("u5mr_2015_2019_.gif")

Bar chart race with smooth transition

Bar chart race with smooth transition

The animation can also be saved as an MP4 movie file.

1u5mr_anim.save("u5mr_2015_2019_.mp4")

Conclusion

There is a bit of work in creating an animated chart such as a bar chart race. The first step is to format the data into a series of datasets that change over time. This series needs to be expanded to introduce intermediary stages to allow for a smooth transition between states. Once the data is configured and expanded, the Matplotlib FuncAnimation is great for creating the animation. The concept is simply to iterate over the data and use a function to create the same chart for the data at that index. This function can be modified to change the colors, titles, annotations and layout of the chart and these changes will be used to create the animation. FuncAnimation can be used to save the animation as a html video, a gif file or an mp4 movie file.

One issue that was not addressed here is dealing with new categories (countries) being added to the animation and the current countries being removed. This will be covered in the next article, as well as adding the Under Five Mortality Rate numbers to the bars on the chart.

Pandas - Load data from Excel file and Display Chart

Eric Callanan — Sunday, 06 Sep 2020

Pandas is a Python package that provides fast data manipulation and analysis of structured data. It has grown in popularity recently with the rise of data analytics and machine learning. The source code for pandas is available on GitHub, but it is more common to install the binary package using something like pip install pandas. A core component of pandas is the dataframe that provides two-dimensional table-like structure to store and manipulate data.

Pandas makes working with structured data easy using the flexible data structures provided. It has become an essential tool for doing practical data analysis in Python. Using Pandas in Jupyter notebooks is a very efficient way to load data and perform some high-level analysis and then visualize the data using Matplotlib.

Under-five mortality rate

I came across a book from Unicef called "state of the world's children" in 1995. This reports on key issues affecting children in various countries around the world and includes a large set of statistics in tabular format, which I found fascinating. Unicef works to improve child welfare throughout the world and continues to produce numerous reports including the state of the word's children. Unicef makes available a large amount of datasets for public consumption at Unicef Datasets. This article uses Under 5 Mortality Rate dataset.

Under-five mortality rate:
is the probability of dying between birth and exactly 5 years of age, expressed per 1,000 live births.

Pandas functions used in this article
pandas.ExcelFile pandas.read_excel pandas.shape pandas.head pandas.tail pandas.iloc	pandas.drop pandas.columns pandas.rename pandas.sort_values pandas.reset_index pandas.plot

Python modules used in this article

1# import modules
2import matplotlib.pyplot as plt
3import shutil
4import pandas as pd
5import numpy as np
6import requests
7import os

Download the excel file

Use the following function to download any file to a temporary location. It is always better to copy data locally and then work with the local files.

 1def download_file(url):
 2    '''Download a file from url to local temp directory.
 3
 4    Keyword arguments:
 5    url  -- A fully qualified url file path
 6
 7    return: The fully qualified path to the downloaded file
 8    '''
 9    local_dir = '/tmp/data/'
10    if not os.path.exists(local_dir):
11        os.makedirs(local_dir)
12
13    local_file = local_dir + url.split('/')[-1]
14    r = requests.get(url, stream=True)
15    with open(local_file, 'wb') as f:
16        for chunk in r.iter_content(chunk_size=512*1024):
17            if chunk:
18                f.write(chunk)
19    return local_file

This function is used by passing in the url to the Excel file on under-five mortality rate.

1u5mr_url = 'https://data.unicef.org/wp-content/uploads/2020/09/Under-five-mortality-rate_2020.xlsx'
2u5mr_file = download_file(u5mr_url)
3
4print(f"local file = {u5mr_file}")
5# local file = /tmp/data/Under-five-mortality-rate_2020.xlsx

List the worksheets in an excel file

Loading an excel file into a Pandas dataframe is very straight forward with read_excel() and it will default to loading the first worksheet in the Excel file. It is also possible to find out what sheets are in an Excel file with the following code. This shows that the file contains six worksheets and the first one is the country estimates needed.

 1u5mr_xl = pd.ExcelFile(u5mr_file)
 2u5mr_xl.sheet_names
 3
 4
 5# output
 6'''
 7['Country estimates (both sexes)',
 8 'Regional & global (both sexes)',
 9 'Country estimates (Female)',
10 'Regional & global (Female)',
11 'Country estimates (Male)',
12 'Regional & global (Male)']
13 '''

Load excel worksheet into dataframe

First load the first worksheet into a dataframe and examine the data. Examine the data in the dataframe using shape and head. The shape of the data is (601, 73), which is 601 rows of data by 73 columns. The head of the dataframe shows the columns do not have any names and there seems to be a lot of blank rows at the top.

DataFrame.head function returns the first n rows of the dataframe, the default is 5. The first 5 rows appear largely blank except for file information in the first column. The data can be seen to start at row index 13 - when the head function is passed 20 for the first 20 rows.

Displaying all 73 columns can be somewhat unwieldy so the DataFrame.iloc function is used to display the first 20 rows for the first 5 columns. iloc is an integer based indexing mechanism for selecting rows and columns by position. The tail function can be used in a similar way to the head function except to display the last n rows of the dataframe. iloc is used here as well to reduce the number of columns to display.

 1u5mr_df = pd.read_excel(u5mr_file)
 2
 3# OR
 4# u5mr_df = pd.read_excel(u5mr_file, sheet_name = 'Country estimates (both sexes)')
 5
 6print(u5mr_df.shape)
 7"""
 8(601, 73)
 9"""
10
11u5mr_df.head()
12u5mr_df.head(20)
13
14print(u5mr_df.iloc[:20, 0:5])
15"""
16                                           Unnamed: 0    Unnamed: 1  UNICEF GLOBAL DATABASES [data.unicef.org]  Unnamed: 3  Unnamed: 4  
170                                                 NaN           NaN                                        NaN         NaN         NaN  
181                                                 NaN           NaN                                        NaN         NaN         NaN  
192                           Child Mortality Estimates           NaN                                        NaN         NaN         NaN  
203   Country-specific Under-five mortality rate, bo...           NaN                                        NaN         NaN         NaN  
214                                                 NaN           NaN                                        NaN         NaN         NaN  
225   Estimates generated by the UN Inter-agency Gro...           NaN                                        NaN         NaN         NaN  
236              Downloaded from http://data.unicef.org           NaN                                        NaN         NaN         NaN  
247                                                 NaN           NaN                                        NaN         NaN         NaN  
258   Prepared by the Data and Analytics Section; Di...           NaN                                        NaN         NaN         NaN  
269                        Contact us:  data@unicef.org           NaN                                        NaN         NaN         NaN  
2710                                                NaN           NaN                                        NaN         NaN         NaN  
2811                      Last update: 9 September 2020           NaN                                        NaN         NaN         NaN  
2912                                                NaN           NaN                                        NaN         NaN         NaN  
3013                                           ISO.Code  Country.Name                        Uncertainty.Bounds*      1950.5      1951.5  
3114                                                AFG   Afghanistan                                      Lower         NaN         NaN  
3215                                                AFG   Afghanistan                                     Median         NaN         NaN  
3316                                                AFG   Afghanistan                                      Upper         NaN         NaN  
3417                                                ALB       Albania                                      Lower         NaN         NaN  
3518                                                ALB       Albania                                     Median         NaN         NaN  
3619                                                ALB       Albania                                      Upper         NaN         NaN  
37"""
38
39
40u5mr_df.tail()
41print(u5mr_df.iloc[-5:, 0:5])
42"""
43    Unnamed: 0                                         Unnamed: 1   UNICEF GLOBAL DATABASES [data.unicef.org]  Unnamed: 3  Unnamed: 4  
44596        ZWE                                           Zimbabwe                                       Lower         NaN         NaN  
45597        ZWE                                           Zimbabwe                                      Median         NaN         NaN  
46598        ZWE                                           Zimbabwe                                       Upper         NaN         NaN  
47599        NaN                                                NaN                                         NaN         NaN         NaN  
48600        NaN  * Lower and Upper refer to the lower bound and...                                         NaN         NaN         NaN  
49"""

It can be seen from the output above that the header row for the countries start at index 13. The data is reloaded from the excel file with an additional parameter specifying the header row. This could also be done using the skiprows. It also shows that the last two rows are there to contain a footnote.

Reload the excel file with the following code and review the data.

 1u5mr_df = pd.read_excel(
 2    u5mr_file,
 3    sheet_name = 'Country estimates (both sexes)',
 4    header = 14)
 5
 6print(u5mr_df.shape)
 7"""
 8(587, 73)
 9"""
10
11print(u5mr_df.iloc[:10, 0:8])
12"""
13  ISO.Code Country.Name Uncertainty.Bounds*  1950.5  1951.5  1952.5  1953.5      1954.5  
140      AFG  Afghanistan               Lower     NaN     NaN     NaN     NaN         NaN  
151      AFG  Afghanistan              Median     NaN     NaN     NaN     NaN         NaN  
162      AFG  Afghanistan               Upper     NaN     NaN     NaN     NaN         NaN  
173      ALB      Albania               Lower     NaN     NaN     NaN     NaN         NaN  
184      ALB      Albania              Median     NaN     NaN     NaN     NaN         NaN  
195      ALB      Albania               Upper     NaN     NaN     NaN     NaN         NaN  
206      DZA      Algeria               Lower     NaN     NaN     NaN     NaN  181.095341  
217      DZA      Algeria              Median     NaN     NaN     NaN     NaN  242.618134  
228      DZA      Algeria               Upper     NaN     NaN     NaN     NaN  323.057628  
239      AND      Andorra               Lower     NaN     NaN     NaN     NaN         NaN  
24"""
25
26# Review columns on the right
27print(u5mr_df.iloc[:10, -7:])
28"""
29      2013.5     2014.5     2015.5     2016.5     2017.5     2018.5     2019.5
300  68.481212  64.526394  60.716842  57.068054  53.744798  50.640922  47.442070
311  76.830462  73.580267  70.444802  67.572190  64.940759  62.541196  60.269399
322  86.077559  83.378062  80.863431  78.646207  76.832873  75.405564  74.616041
333  10.072106   9.549495   9.225420   9.075629   9.044506   9.067390   9.068154
344  10.430484   9.896397   9.565251   9.419110   9.418052   9.525133   9.682407
355  10.821932  10.272281   9.936220   9.795035   9.826251  10.020056  10.348964
366  25.265117  24.944603  24.613434  24.245765  23.801969  23.262592  22.513380
377  25.784460  25.478560  25.177142  24.792098  24.319482  23.805926  23.256168
388  26.310524  26.023294  25.746094  25.339485  24.843878  24.345820  24.025946
399   1.566198   1.384200   1.232419   1.098862   0.985541   0.910653   0.847791
40"""

Rename the columns to contain the Year

Drop the last two rows that are there to include the footnote.

 1# Drop the last two rows
 2print(u5mr_df.shape)
 3"""
 4(587, 73)
 5"""
 6
 7u5mr_df.drop(u5mr_df.tail(2).index, inplace = True)
 8
 9print(u5mr_df.shape)
10"""
11(585, 73)
12"""
13
14u5mr_df.tail()
15print(u5mr_df.iloc[-5:, 0:7])
16"""
17    ISO.Code Country.Name Uncertainty.Bounds*  1950.5  1951.5  1952.5      1953.5  
18580      ZMB       Zambia              Median     NaN     NaN     NaN  234.418232  
19581      ZMB       Zambia               Upper     NaN     NaN     NaN  308.531403  
20582      ZWE     Zimbabwe               Lower     NaN     NaN     NaN         NaN  
21583      ZWE     Zimbabwe              Median     NaN     NaN     NaN         NaN  
22584      ZWE     Zimbabwe               Upper     NaN     NaN     NaN         NaN  
23"""

The year headings contain 0.5 such as 2019.5 probably to indicate a mean value for the entire year. Use columns field of the dataframe to rename these to just the full year. u5mr_df.columns is a list of all the columns in the dataframe and a list comprehension is used to remove the ".5" if it is there.

The asterisk (*) is also removed from the 'Uncertainty.Bounds*' column name.

 1print(u5mr_df.columns)
 2
 3"""
 4Index(['ISO.Code', 'Country.Name', 'Uncertainty.Bounds*', '1950.5', '1951.5',
 5       '1952.5', '1953.5', '1954.5', '1955.5', '1956.5', '1957.5', '1958.5',
 6       '1959.5', '1960.5', '1961.5', '1962.5', '1963.5', '1964.5', '1965.5',
 7       '1966.5', '1967.5', '1968.5', '1969.5', '1970.5', '1971.5', '1972.5',
 8       '1973.5', '1974.5', '1975.5', '1976.5', '1977.5', '1978.5', '1979.5',
 9       '1980.5', '1981.5', '1982.5', '1983.5', '1984.5', '1985.5', '1986.5',
10       '1987.5', '1988.5', '1989.5', '1990.5', '1991.5', '1992.5', '1993.5',
11       '1994.5', '1995.5', '1996.5', '1997.5', '1998.5', '1999.5', '2000.5',
12       '2001.5', '2002.5', '2003.5', '2004.5', '2005.5', '2006.5', '2007.5',
13       '2008.5', '2009.5', '2010.5', '2011.5', '2012.5', '2013.5', '2014.5',
14       '2015.5', '2016.5', '2017.5', '2018.5', '2019.5'],
15      dtype='object')
16"""
17
18u5mr_df.columns = [x[:-2] if x.endswith('.5') else x for x in u5mr_df.columns]
19
20u5mr_df = u5mr_df.rename(columns={'Uncertainty.Bounds*': 'Uncertainty.Bounds'})
21
22print(u5mr_df.columns)
23"""
24Index(['ISO.Code', 'Country.Name', 'Uncertainty.Bounds', '1950', '1951',
25       '1952', '1953', '1954', '1955', '1956', '1957', '1958', '1959', '1960',
26       '1961', '1962', '1963', '1964', '1965', '1966', '1967', '1968', '1969',
27       '1970', '1971', '1972', '1973', '1974', '1975', '1976', '1977', '1978',
28       '1979', '1980', '1981', '1982', '1983', '1984', '1985', '1986', '1987',
29       '1988', '1989', '1990', '1991', '1992', '1993', '1994', '1995', '1996',
30       '1997', '1998', '1999', '2000', '2001', '2002', '2003', '2004', '2005',
31       '2006', '2007', '2008', '2009', '2010', '2011', '2012', '2013', '2014',
32       '2015', '2016', '2017', '2018', '2019'],
33      dtype='object')
34"""

Filter to the Median values

The data has three values for each country the Lower and Upper bounds and the Median. Create a copy of the dataframe with just the Median values, so there is only one number for each country.

 1u5mr_med_df = u5mr_df[u5mr_df['Uncertainty.Bounds'] == 'Median']
 2print(u5mr_med_df.shape)
 3"""
 4(195, 73)
 5"""
 6
 7u5mr_med_df.head()
 8print(u5mr_med_df.iloc[:10, :10])
 9"""
10   ISO.Code         Country.Name Uncertainty.Bounds        1950        1951        1952        1953        1954        1955        1956  
111       AFG          Afghanistan             Median         NaN         NaN         NaN         NaN         NaN         NaN         NaN  
124       ALB              Albania             Median         NaN         NaN         NaN         NaN         NaN         NaN         NaN  
137       DZA              Algeria             Median         NaN         NaN         NaN         NaN  242.618134  242.066662  241.601453  
1410      AND              Andorra             Median         NaN         NaN         NaN         NaN         NaN         NaN         NaN  
1513      AGO               Angola             Median         NaN         NaN         NaN         NaN         NaN         NaN         NaN  
1616      ATG  Antigua and Barbuda             Median  119.066130  116.624640  114.074405  111.406742  108.602334  105.661011  102.585410  
1719      ARG            Argentina             Median         NaN         NaN         NaN         NaN         NaN         NaN         NaN  
1822      ARM              Armenia             Median         NaN         NaN         NaN         NaN         NaN         NaN         NaN  
1925      AUS            Australia             Median   31.584087   30.966060   30.261701   29.496194   28.711058   27.961205   27.265216  
2028      AUT              Austria             Median   83.482005   73.820239   66.122978   60.263643   56.225511   53.524855   51.536432  
21"""

Get data for the top 10 countries in the latest year

The data is sorted by the values in the last column and the number of rows reduced to ten to reduce the data to the top 10 countries.

 1u5mr_med_df.sort_values(by=u5mr_df.columns[-1], ascending=False).head(10)
 2
 3highest_df = u5mr_med_df.sort_values(by=u5mr_df.columns[-1], ascending=False).head(10)
 4print(highest_df.iloc[:, [0,1,-5,-4,-3,-2,-1]])
 5"""
 6    ISO.Code              Country.Name        2015        2016        2017        2018        2019  
 7376      NGA                   Nigeria  126.833543  125.040170  122.798947  120.037728  117.202078  
 8481      SOM                   Somalia  132.720296  128.428442  124.393442  120.326222  116.972096  
 9100      TCD                      Chad  129.406257  125.604878  121.508350  117.664883  113.790418  
1097       CAF  Central African Republic  126.714463  121.593307  117.470988  114.222380  110.053912  
11466      SLE              Sierra Leone  138.342774  123.232842  118.222181  113.544491  109.236528  
12214      GIN                    Guinea  107.852191  105.241523  103.226528  101.095543   98.802973  
13487      SSD               South Sudan   96.229299   96.229299   96.229299   96.229299   96.229299  
14316      MLI                      Mali  108.586504  104.766458  101.062422   97.410316   94.035418  
1555       BEN                     Benin   99.592931   97.416395   95.133079   92.773521   90.286429  
1679       BFA              Burkina Faso  100.671924   97.034768   93.862909   90.682138   87.542426  
17"""

Create a chart for the top 10 countries in the latest year

Create a line chart using Matplotlib to show the changes in the under-five mortality rate over time. The data needs to be transposed and tidyied up a bit in order to create a line chart using year as the x axis values.

The following steps are done to prepare the data for the chart.

Transpose the data so year can be used as the x-axis
Set the Country.Name as the heading for the columns
Rename the Country.Name column to Year
Drop the rows that do not contain u5mr data

 1# Need to transpose the dataframe to use year as the x-axis
 2top_10_df = u5mr_med_df.sort_values(by=u5mr_df.columns[-1], ascending=False).head(10).T
 3top_10_df.reset_index(drop=False,inplace=True)
 4print(f"shape = {top_10_df.shape}")
 5top_10_df.head()
 6
 7# Set the Country.Name as the heading for the columns
 8top_10_df.columns = top_10_df.iloc[np.where(top_10_df['index'] == 'Country.Name')[0][0]]
 9print(f"shape = {top_10_df.shape}")
10top_10_df.head()
11
12# Rename the Country.Name column to Year
13top_10_df = top_10_df.rename(columns={'Country.Name': 'Year'})
14print(f"shape = {top_10_df.shape}")
15top_10_df.head()
16
17# Drop the rows that do not contain u5mr data
18top_10_df = top_10_df[top_10_df['Year'].isin(['ISO.Code', 'Country.Name', 'Uncertainty.Bounds']) == False]
19top_10_df.reset_index(drop = True, inplace = True)
20print(f"shape = {top_10_df.shape}")
21top_10_df.head()
22
23print(top_10_df.tail())
24"""
251   Year  Nigeria  Somalia     Chad Central African Republic Sierra Leone   Guinea South Sudan     Mali    Benin Burkina Faso  
2665  2015  126.834   132.72  129.406                  126.714      138.343  107.852     96.2293  108.587  99.5929      100.672  
2766  2016   125.04  128.428  125.605                  121.593      123.233  105.242     96.2293  104.766  97.4164      97.0348  
2867  2017  122.799  124.393  121.508                  117.471      118.222  103.227     96.2293  101.062  95.1331      93.8629  
2968  2018  120.038  120.326  117.665                  114.222      113.544  101.096     96.2293  97.4103  92.7735      90.6821  
3069  2019  117.202  116.972   113.79                  110.054      109.237   98.803     96.2293  94.0354  90.2864      87.5424  
31"""

Create a chart for countries with highest U5MR

The code below creates the line chart of the changes to under-five mortality rate in the countries with the current highest rates.

 1current_year = u5mr_df.columns[-1]
 2title = f'Changes in Under 5 Mortality Rate\nfor countries with the highest in {u5mr_df.columns[-1]}'
 3
 4fig, ax1 = plt.subplots(figsize=(10,7), facecolor=plt.cm.Blues(.2))
 5ax1.set_facecolor(plt.cm.Blues(.2))
 6
 7fig.suptitle(title, fontsize=18,  fontweight='bold')
 8
 9top_10_df.plot(kind = "line",
10               x = "Year",
11               y = top_10_df.columns[1:],
12               ax = ax1)
13
14ax1.legend(bbox_to_anchor = (0.5, -0.1),
15           loc='upper center',
16           ncol = 3,
17           frameon = False,
18           fontsize = 'medium')
19
20ax1.set_ylabel('Under-five mortality rate', fontsize=14)
21ax1.set_xlabel('Year', fontsize=14)
22
23# Hide the right and top spines
24ax1.spines['right'].set_visible(False)
25ax1.spines['top'].set_visible(False)
26
27plt.show()

top 10 under-five mortality rate

Changes in Under 5 Mortality Rate for countries with the highest rates

Create a chart for countries with lowest U5MR

A similar chart was created by reversing the sort order of the mortality rates in the latest year after performing the data clean. These two functions were created to make the creation of the charts more consistent and reproducible.

Function to extract the mortality rates for the top countries.

 1def get_top_countries(data_df, lowest = True, num = 10):
 2    '''Extract the mortality rates for the top number of countries
 3    based on rates in the latest year
 4
 5    Keyword arguments:
 6    data_df  -- dateframe of all the mortality rates for all the countries
 7    lowest   -- boolean flag to either contries withe the lowest rate
 8                or highest rate (default is True)
 9    num.     -- number of countries to return (default is 10)
10
11    return: dataframe that has been transposed and filtered to the top n counbtries
12    '''
13    # Need to transpose the dataframe to use year as the x-axis
14    df = data_df.sort_values(by=data_df.columns[-1], ascending=lowest).head(10).T
15    df.reset_index(drop=False,inplace=True)
16
17    # Set the Country.Name as the heading for the columns
18    df.columns = df.iloc[np.where(df['index'] == 'Country.Name')[0][0]]
19
20    # Rename the Country.Name column to Year
21    df = df.rename(columns={'Country.Name': 'Year'})
22
23    # Drop the rows that do not contain u5mr data
24    df = (df[df['Year']
25             .isin(['ISO.Code',
26                    'Country.Name',
27                    'Uncertainty.Bounds']) == False]
28        )
29    df.reset_index(drop = True, inplace = True)
30
31    return df

Function to create the chart of the mortality rates over time.

 1def create_line_chart(df, title):  
 2    '''Create a line chart of the mortality rates over time.
 3    Year is required to be in the first column.
 4
 5    Keyword arguments:
 6    df     -- dataframe of the the countries to render in the chart.
 7    title  -- text to display for the title of the chart
 8
 9    return: figure of the chart
10    '''
11    fig, ax1 = plt.subplots(figsize=(10,7), facecolor=plt.cm.Blues(.2))
12    ax1.set_facecolor(plt.cm.Blues(.2))
13
14    fig.suptitle(title, fontsize=18,  fontweight='bold')
15
16    df.plot(kind = "line",
17            x = "Year",
18            y = df.columns[1:],
19            ax = ax1)
20
21    ax1.legend(bbox_to_anchor = (0.5, -0.1),
22               loc='upper center',
23               ncol = 3,
24               frameon = False,
25               fontsize = 'medium')
26
27    ax1.set_ylabel('Under-five mortality rate', fontsize=14)
28    ax1.set_xlabel('Year', fontsize=14)
29
30    # Hide the right and top spines
31    ax1.spines['right'].set_visible(False)
32    ax1.spines['top'].set_visible(False)
33
34    return fig

lowest 10 under-five mortality rate

Changes in Under 5 Mortality Rate for countries with the lowest rates

Conclusion

Under-Five Mortality Rate

Under-five mortality rate is one of the key development metrics for countries. The results of the current countries with highest continue to decrease year over year. However, some countries such as South Sudan and Somalia seem to have levelled off in recent years. These top ten countries all have U5MR close to 100, that is the expectation that 100 out of every 1,000 live births will die before the age of 5-years of age. This means one in ten people die before the age of five - this is alarming.

All of these ten countries have had high Under-Five Mortality Rate for the last fifty years and they are all improving. The highest rate in this chart is for Mali in 1963 when the Under-Five Mortality Rate was 420. This is an incomprehensible number implying that four out of every ten children are likely to die before the age of 5.

It can be seen that all but one of the top countries with the lowest Under-Five Mortality Rate started at less than 100 in 1950.

Pandas

Pandas is great for loading and manipulating structured data. It is very easy to load data from Excel into a dataframe. Dataframe is also great for filtering, sorting and transposing data as well as creating charts from the data. Pandas can be used in a Jupyter Notebook for interactive exploration of data so that a data processing pipeline can be quickly developed. The final methods can then be used in a python script to repeatedly process similar files and produce consistent output.

How to add a footnote to a chart in Matplotlib

Eric Callanan — Sunday, 30 Aug 2020

It can be nice to add a footnote or a header note to a Matplotlib chart. The easiest way to do this is with annotate function. The exact position, size orientation can be controlled and multiple notes can be added to charts.

Create a simple line chart

Create a simple line chart with three sets of ten random points to show a line chart.

 1# Create the data
 2samples = 10
 3random.seed(42) # seed used to keep the same random points on repeated runs
 4val1 = sorted([random.randint(1,100) for _ in range(samples)])
 5val2 = sorted([random.randint(1,100) for _ in range(samples)])
 6val3 = sorted([random.randint(1,100) for _ in range(samples)])
 7x_val = list(range(samples))
 8
 9# Create a line chart
10fig, ax = plt.subplots(figsize=(10,7), facecolor=plt.cm.Blues(.2))
11ax.set_facecolor(plt.cm.Blues(.2))
12ax.set_title("Sample Line Chart", fontsize=18, fontweight='bold')
13
14ax.plot(x_val, val1, color = plt.cm.cool(0.3), marker='o', markersize=8)
15ax.plot(x_val, val2, color = plt.cm.cool(0.6), marker='*', markersize=8)
16ax.plot(x_val, val3, color = plt.cm.cool(0.9), marker='x', markersize=8)
17
18ax.set_xlabel('X label', fontsize=16)
19ax.set_ylabel('Y label', fontsize=16)
20
21# Hide the right and top spines
22ax.spines['right'].set_visible(False)
23ax.spines['top'].set_visible(False)
24
25# Don't show any ticks on any spine
26ax.yaxis.set_ticks_position('none')
27ax.xaxis.set_ticks_position('none')
28
29ax.tick_params(labelsize = 14)
30
31plt.show()

Sample line chart with three sets of ten random samples

Sample line chart with three sets of ten random samples

Add a note to the chart

Use annotate to add a note to the chart with the addition of the following code. The default for the xy values is to use the units of the chart and this is very useful when adding annotations to data points.

1...
2
3ax.tick_params(labelsize = 14)
4
5# Add a note
6ax.annotate('Note added to chart with minimum parameters', xy = (5, 5))
7
8fig.tight_layout()
9plt.show()

Sample line chart with three sets of ten random samples and note

Sample line chart with three sets of ten random samples and note

Add a note outside the chart

The xycoords parameter is used to specify the coordinate system that xy is given in. xycoords value of 'axes fraction' is used to specify coordinates as a fraction of axes from lower left. This sets the coordinate system for the annotation to use (0,0) to (1,1) for the entire chart, regardless of the chart scales.

The following adds a note at each of the corners of the chart.

 1...
 2
 3ax.tick_params(labelsize = 14)
 4
 5# Add a four notes
 6ax.annotate('Note added at (0,0)', xy = (0, 0), xycoords='axes fraction')
 7ax.annotate('Note added at (0,1)', xy = (0, 1), xycoords='axes fraction')
 8ax.annotate('Note added at (1,0)', xy = (1, 0), xycoords='axes fraction')
 9ax.annotate('Note added at (1,1)', xy = (1, 1), xycoords='axes fraction')
10
11fig.tight_layout()
12plt.show()

Sample line chart with notes at four corners

Sample line chart with notes at four corners

The notes are added at the 4 corners of the chart and the default is that this is where the text in the annotation starts and the text is left-justified. More space is added to the right of the chart to allow for the notes to be displayed.

Add random notes inside and outside the chart

The following code adds fifty notes at random locations from (-1.0, -1.0) to (2.0, 2.0). This demonstrates that annotations can be added outside the chart knowing that the chart is in (0,0) to (1,1). The plot resizes to show these annotations. Each note shows the coordinates at which it was added and these notes can be placed on top of each other as well as other chart elements.

 1...
 2
 3ax.tick_params(labelsize = 14)
 4
 5# Add 50 notes at random locations
 6for i in range(50):
 7    x = random.randint(-10,20)/10
 8    y = random.randint(-10,20)/10
 9    ax.annotate(f'Note added at ({x:.1f}, {y:.1f})',
10                xy = (x, y),
11                xycoords='axes fraction')
12
13fig.tight_layout()
14plt.show()

Line chart with fifty notes in random locations

Line chart with fifty notes at random locations

Add a single note below the chart

Add a note below and to the right of the chart. In addition, the note can be formatted and right-justified.

 1...
 2
 3ax.tick_params(labelsize = 14)
 4
 5# Add a footnote below and to the right side of the chart
 6ax.annotate('Footnote added below the chart with a smaller font',
 7            xy = (1.0, -0.2),
 8            xycoords='axes fraction',
 9            ha='right',
10            va="center",
11            fontsize=10)
12
13
14fig.tight_layout()
15plt.show()

Line chart with footnote at the bottom right of the chart

Line chart with footnote placed below and to the right of the chart

Conclusion

In Matplotlib, annotate can be used to add any note or any number of notes anywhere on or around charts. Annotate defaults to using the coordinate system of the chart and can be used to add text to particular data points. With the use of xycoords set to 'axes fraction' the coordinate system for the annotation sets the chart to a single cell from 0 to 1. This allows exact positioning of annotations inside or outside the chart.

matplotlib on Software Development Notes

Create interactive chart with matplotlib and ipywidgets

Slider with Interact

Widget layout

Rolling average of Deaths from COVID-19

Using ipywidgets to modify days for rolling average

Conclusion

Display world map with country data

Load the data

Load Geopandas

Explore 'naturalearth_lowres' dataset

Display country color based on Under Five Mortality Rates in 2019

Display country color based on Under Five Mortality Rates in 1985

Display changes over the decades

Conclusion

Under-five mortality rate:

Display a line chart with a range

Load the data

Display the changes over time for one country

Display upper, lower and median data for a single country

Display shading around median data for a single country

Display data for four countries

Display data for male and female data

List the worksheets in an excel file

Conclusion

Under-five mortality rate:

Create a Bar Chart Race with Matplotlib - part 2

Load the data

Identify the countries of interest

Create a color map for these countries.

Get the data for these 41 countries of interest

Expand the data set for smooth animation

Display bar chart for a single year

Display bar chart with mortality rate on the bars

Display a random sample of bar charts for different year

Create bar chart race of highest U5MR over the years

Wrap up bar chart race creation in functions

Display U5MR bar chart race for selected countries

Conclusion

Create a Bar Chart Race with Matplotlib - part 1

Outline

Load the data

Identify the countries of interest

Get the data for the top five countries from 2015 to 2019

Display a bar chart for 2019

Display a series of bar charts for 2015 to 2019

Display a single bar chart with color for country

Display a series of bar charts from 2015 to 2019

Display a series of bar charts from 2015 to 2019 with color per country

Abrupt transition

Smooth transition

Missing country in intermediary data

Creating animation

Conclusion

Pandas - Load data from Excel file and Display Chart

Under-five mortality rate

Under-five mortality rate:

Pandas functions used in this article

Python modules used in this article

Download the excel file

List the worksheets in an excel file

Load excel worksheet into dataframe

Rename the columns to contain the Year

Filter to the Median values

Get data for the top 10 countries in the latest year

Create a chart for the top 10 countries in the latest year

Create a chart for countries with highest U5MR

Create a chart for countries with lowest U5MR

Function to extract the mortality rates for the top countries.

Function to create the chart of the mortality rates over time.

Conclusion

Under-Five Mortality Rate

Pandas

How to add a footnote to a chart in Matplotlib

Create a simple line chart

Add a note to the chart

Add a note outside the chart

Add random notes inside and outside the chart

Add a single note below the chart

Conclusion