Quick overview#
Here are some quick examples of what you can do with xarray.DataArray
objects. Everything is explained in much more detail in the rest of the
documentation.
To begin, import numpy, pandas and xarray using their customary abbreviations:
import numpy as np
import pandas as pd
import xarray as xr
Create a DataArray#
You can make a DataArray from scratch by supplying data in the form of a numpy array or list, with optional dimensions and coordinates:
data = xr.DataArray(np.random.randn(2, 3), dims=("x", "y"), coords={"x": [10, 20]})
data
<xarray.DataArray (x: 2, y: 3)> Size: 48B
array([[-1.36328649, 0.87445112, -2.57000833],
[-0.34724935, 0.38542157, -0.3047831 ]])
Coordinates:
* x (x) int64 16B 10 20
Dimensions without coordinates: yIn this case, we have generated a 2D array, assigned the names x and y to the two dimensions respectively and associated two coordinate labels โ10โ and โ20โ with the two locations along the x dimension. If you supply a pandas Series or DataFrame, metadata is copied directly:
xr.DataArray(pd.Series(range(3), index=list("abc"), name="foo"))
<xarray.DataArray 'foo' (dim_0: 3)> Size: 24B array([0, 1, 2]) Coordinates: * dim_0 (dim_0) object 24B 'a' 'b' 'c'
Here are the key properties for a DataArray:
# like in pandas, values is a numpy array that you can modify in-place
data.values
data.dims
data.coords
# you can use this dictionary to store arbitrary metadata
data.attrs
{}
Indexing#
Xarray supports four kinds of indexing. Since we have assigned coordinate labels to the x dimension we can use label-based indexing along that dimension just like pandas. The four examples below all yield the same result (the value at x=10) but at varying levels of convenience and intuitiveness.
# positional and by integer label, like numpy
data[0, :]
# loc or "location": positional and coordinate label, like pandas
data.loc[10]
# isel or "integer select": by dimension name and integer label
data.isel(x=0)
# sel or "select": by dimension name and coordinate label
data.sel(x=10)
<xarray.DataArray (y: 3)> Size: 24B
array([-1.36328649, 0.87445112, -2.57000833])
Coordinates:
x int64 8B 10
Dimensions without coordinates: yUnlike positional indexing, label-based indexing frees us from having to know how our array is organized. All we need to know are the dimension name and the label we wish to index i.e. data.sel(x=10) works regardless of whether x is the first or second dimension of the array and regardless of whether 10 is the first or second element of x. We have already told xarray that x is the first dimension when we created data: xarray keeps track of this so we donโt have to. For more, see Indexing and selecting data.
Attributes#
While youโre setting up your DataArray, itโs often a good idea to set metadata attributes. A useful choice is to set data.attrs['long_name'] and data.attrs['units'] since xarray will use these, if present, to automatically label your plots. These special names were chosen following the NetCDF Climate and Forecast (CF) Metadata Conventions. attrs is just a Python dictionary, so you can assign anything you wish.
data.attrs["long_name"] = "random velocity"
data.attrs["units"] = "metres/sec"
data.attrs["description"] = "A random variable created as an example."
data.attrs["random_attribute"] = 123
data.attrs
# you can add metadata to coordinates too
data.x.attrs["units"] = "x units"
Computation#
Data arrays work very similarly to numpy ndarrays:
data + 10
np.sin(data)
# transpose
data.T
data.sum()
<xarray.DataArray ()> Size: 8B
array(-3.32545457)
Attributes:
long_name: random velocity
units: metres/sec
description: A random variable created as an example.
random_attribute: 123However, aggregation operations can use dimension names instead of axis numbers:
data.mean(dim="x")
<xarray.DataArray (y: 3)> Size: 24B
array([-0.85526792, 0.62993634, -1.43739571])
Dimensions without coordinates: y
Attributes:
long_name: random velocity
units: metres/sec
description: A random variable created as an example.
random_attribute: 123Arithmetic operations broadcast based on dimension name. This means you donโt need to insert dummy dimensions for alignment:
a = xr.DataArray(np.random.randn(3), [data.coords["y"]])
b = xr.DataArray(np.random.randn(4), dims="z")
a
b
a + b
<xarray.DataArray (y: 3, z: 4)> Size: 96B
array([[-4.27177923, -1.14160756, -1.75369532, -1.78577395],
[-1.91781154, 1.21236013, 0.60027237, 0.56819375],
[-3.8848896 , -0.75471793, -1.36680569, -1.39888431]])
Coordinates:
* y (y) int64 24B 0 1 2
Dimensions without coordinates: zIt also means that in most cases you do not need to worry about the order of dimensions:
data - data.T
<xarray.DataArray (x: 2, y: 3)> Size: 48B
array([[0., 0., 0.],
[0., 0., 0.]])
Coordinates:
* x (x) int64 16B 10 20
Dimensions without coordinates: y
Attributes:
long_name: random velocity
units: metres/sec
description: A random variable created as an example.
random_attribute: 123Operations also align based on index labels:
data[:-1] - data[:1]
<xarray.DataArray (x: 1, y: 3)> Size: 24B
array([[0., 0., 0.]])
Coordinates:
* x (x) int64 8B 10
Dimensions without coordinates: y
Attributes:
long_name: random velocity
units: metres/sec
description: A random variable created as an example.
random_attribute: 123For more, see Computation.
GroupBy#
Xarray supports grouped operations using a very similar API to pandas (see GroupBy: Group and Bin Data):
labels = xr.DataArray(["E", "F", "E"], [data.coords["y"]], name="labels")
labels
data.groupby(labels).mean("y")
data.groupby(labels).map(lambda x: x - x.min())
<xarray.DataArray (x: 2, y: 3)> Size: 48B
array([[1.20672184, 0.48902955, 0. ],
[2.22275898, 0. , 2.26522523]])
Coordinates:
* x (x) int64 16B 10 20
Dimensions without coordinates: y
Attributes:
long_name: random velocity
units: metres/sec
description: A random variable created as an example.
random_attribute: 123Plotting#
Visualizing your datasets is quick and convenient:
data.plot()
<matplotlib.collections.QuadMesh at 0x76338b462ba0>
Note the automatic labeling with names and units. Our effort in adding metadata attributes has paid off! Many aspects of these figures are customizable: see Plotting.
pandas#
Xarray objects can be easily converted to and from pandas objects using the to_series(), to_dataframe() and to_xarray() methods:
series = data.to_series()
series
# convert back
series.to_xarray()
<xarray.DataArray (x: 2, y: 3)> Size: 48B
array([[-1.36328649, 0.87445112, -2.57000833],
[-0.34724935, 0.38542157, -0.3047831 ]])
Coordinates:
* x (x) int64 16B 10 20
* y (y) int64 24B 0 1 2Datasets#
xarray.Dataset is a dict-like container of aligned DataArray
objects. You can think of it as a multi-dimensional generalization of the
pandas.DataFrame:
ds = xr.Dataset(dict(foo=data, bar=("x", [1, 2]), baz=np.pi))
ds
<xarray.Dataset> Size: 88B
Dimensions: (x: 2, y: 3)
Coordinates:
* x (x) int64 16B 10 20
Dimensions without coordinates: y
Data variables:
foo (x, y) float64 48B -1.363 0.8745 -2.57 -0.3472 0.3854 -0.3048
bar (x) int64 16B 1 2
baz float64 8B 3.142This creates a dataset with three DataArrays named foo, bar and baz. Use dictionary or dot indexing to pull out Dataset variables as DataArray objects but note that assignment only works with dictionary indexing:
ds["foo"]
ds.foo
<xarray.DataArray 'foo' (x: 2, y: 3)> Size: 48B
array([[-1.36328649, 0.87445112, -2.57000833],
[-0.34724935, 0.38542157, -0.3047831 ]])
Coordinates:
* x (x) int64 16B 10 20
Dimensions without coordinates: y
Attributes:
long_name: random velocity
units: metres/sec
description: A random variable created as an example.
random_attribute: 123When creating ds, we specified that foo is identical to data created earlier, bar is one-dimensional with single dimension x and associated values โ1โ and โ2โ, and baz is a scalar not associated with any dimension in ds. Variables in datasets can have different dtype and even different dimensions, but all dimensions are assumed to refer to points in the same shared coordinate system i.e. if two variables have dimension x, that dimension must be identical in both variables.
For example, when creating ds xarray automatically aligns bar with DataArray foo, i.e., they share the same coordinate system so that ds.bar['x'] == ds.foo['x'] == ds['x']. Consequently, the following works without explicitly specifying the coordinate x when creating ds['bar']:
ds.bar.sel(x=10)
<xarray.DataArray 'bar' ()> Size: 8B
array(1)
Coordinates:
x int64 8B 10You can do almost everything you can do with DataArray objects with
Dataset objects (including indexing and arithmetic) if you prefer to work
with multiple variables at once.
Read & write netCDF files#
NetCDF is the recommended file format for xarray objects. Users
from the geosciences will recognize that the Dataset data
model looks very similar to a netCDF file (which, in fact, inspired it).
You can directly read and write xarray objects to disk using to_netcdf(), open_dataset() and
open_dataarray():
ds.to_netcdf("example.nc")
reopened = xr.open_dataset("example.nc")
reopened
<xarray.Dataset> Size: 88B
Dimensions: (x: 2, y: 3)
Coordinates:
* x (x) int64 16B 10 20
Dimensions without coordinates: y
Data variables:
foo (x, y) float64 48B ...
bar (x) int64 16B ...
baz float64 8B ...It is common for datasets to be distributed across multiple files (commonly one file per timestep). Xarray supports this use-case by providing the open_mfdataset() and the save_mfdataset() methods. For more, see Reading and writing files.
DataTrees#
xarray.DataTree is a tree-like container of DataArray objects, organised into multiple mutually alignable groups. You can think of it like a (recursive) dict of Dataset objects, where coordinate variables and their indexes are inherited down to children.
Letโs first make some example xarray datasets:
import numpy as np
import xarray as xr
data = xr.DataArray(np.random.randn(2, 3), dims=("x", "y"), coords={"x": [10, 20]})
ds = xr.Dataset({"foo": data, "bar": ("x", [1, 2]), "baz": np.pi})
ds
ds2 = ds.interp(coords={"x": [10, 12, 14, 16, 18, 20]})
ds2
ds3 = xr.Dataset(
{"people": ["alice", "bob"], "heights": ("people", [1.57, 1.82])},
coords={"species": "human"},
)
ds3
<xarray.Dataset> Size: 76B
Dimensions: (people: 2)
Coordinates:
* people (people) <U5 40B 'alice' 'bob'
species <U5 20B 'human'
Data variables:
heights (people) float64 16B 1.57 1.82Now weโll put these datasets into a hierarchical DataTree:
dt = xr.DataTree.from_dict(
{"simulation/coarse": ds, "simulation/fine": ds2, "/": ds3}
)
dt
<xarray.DataTree>
Group: /
โ Dimensions: (people: 2)
โ Coordinates:
โ * people (people) <U5 40B 'alice' 'bob'
โ species <U5 20B 'human'
โ Data variables:
โ heights (people) float64 16B 1.57 1.82
โโโ Group: /simulation
โโโ Group: /simulation/coarse
โ Dimensions: (x: 2, y: 3)
โ Coordinates:
โ * x (x) int64 16B 10 20
โ Dimensions without coordinates: y
โ Data variables:
โ foo (x, y) float64 48B -0.05363 -0.3807 0.2609 -1.626 -1.485 -0.1501
โ bar (x) int64 16B 1 2
โ baz float64 8B 3.142
โโโ Group: /simulation/fine
Dimensions: (x: 6, y: 3)
Coordinates:
* x (x) int64 48B 10 12 14 16 18 20
Dimensions without coordinates: y
Data variables:
foo (x, y) float64 144B -0.05363 -0.3807 0.2609 ... -1.485 -0.1501
bar (x) float64 48B 1.0 1.2 1.4 1.6 1.8 2.0
baz float64 8B 3.142This created a DataTree with nested groups. We have one root group, containing information about individual
people. This root group can be named, but here it is unnamed, and is referenced with "/". This structure is similar to a
unix-like filesystem. The root group then has one subgroup simulation, which contains no data itself but does
contain another two subgroups, named fine and coarse.
The (sub)subgroups fine and coarse contain two very similar datasets. They both have an "x"
dimension, but the dimension is of different lengths in each group, which makes the data in each group
unalignable. In the root group we placed some completely unrelated information, in order to show how a tree can
store heterogeneous data.
Remember to keep unalignable dimensions in sibling groups because a DataTree inherits coordinates down through its
child nodes. You can see this inheritance in the above representation of the DataTree. The coordinates
people and species defined in the root / node are shown in the child nodes both
/simulation/coarse and /simulation/fine. All coordinates in parent-descendent lineage must be
alignable to form a DataTree. If your input data is not aligned, you can still get a nested dict of
Dataset objects with open_groups() and then apply any required changes to ensure alignment
before converting to a DataTree.
The constraints on each group are the same as the constraint on DataArrays within a single dataset with the addition of requiring parent-descendent coordinate agreement.
We created the subgroups using a filesystem-like syntax, and accessing groups works the same way. We can access individual DataArrays in a similar fashion.
dt["simulation/coarse/foo"]
<xarray.DataArray 'foo' (x: 2, y: 3)> Size: 48B
array([[-0.05363468, -0.3806731 , 0.26091151],
[-1.62621022, -1.48486736, -0.15014013]])
Coordinates:
* x (x) int64 16B 10 20
Dimensions without coordinates: yWe can also view the data in a particular group as a read-only DatasetView using xarray.Datatree.dataset:
dt["simulation/coarse"].dataset
<xarray.DatasetView> Size: 128B
Dimensions: (x: 2, y: 3, people: 2)
Coordinates:
* x (x) int64 16B 10 20
* people (people) <U5 40B 'alice' 'bob'
Dimensions without coordinates: y
Data variables:
foo (x, y) float64 48B -0.05363 -0.3807 0.2609 -1.626 -1.485 -0.1501
bar (x) int64 16B 1 2
baz float64 8B 3.142We can get a copy of the Dataset including the inherited coordinates by calling the to_dataset method:
ds_inherited = dt["simulation/coarse"].to_dataset()
ds_inherited
<xarray.Dataset> Size: 128B
Dimensions: (x: 2, y: 3, people: 2)
Coordinates:
* x (x) int64 16B 10 20
* people (people) <U5 40B 'alice' 'bob'
Dimensions without coordinates: y
Data variables:
foo (x, y) float64 48B -0.05363 -0.3807 0.2609 -1.626 -1.485 -0.1501
bar (x) int64 16B 1 2
baz float64 8B 3.142And you can get a copy of just the node local values of Dataset by setting the inherit keyword to False:
ds_node_local = dt["simulation/coarse"].to_dataset(inherit=False)
ds_node_local
<xarray.Dataset> Size: 88B
Dimensions: (x: 2, y: 3)
Coordinates:
* x (x) int64 16B 10 20
Dimensions without coordinates: y
Data variables:
foo (x, y) float64 48B -0.05363 -0.3807 0.2609 -1.626 -1.485 -0.1501
bar (x) int64 16B 1 2
baz float64 8B 3.142Note
We intend to eventually implement most Dataset methods
(indexing, aggregation, arithmetic, etc) on DataTree
objects, but many methods have not been implemented yet.
Tip
If all of your variables are mutually alignable (i.e., they live on the same
grid, such that every common dimension name maps to the same length), then
you probably donโt need xarray.DataTree, and should consider
just sticking with xarray.Dataset.