# Loading, processing, and visualizing data

This notebook shows how to load a dataset, use the preprocessor functions, and visualize the result. In this notebook we will plot the annual mean temperature from 1850 till 2100 from one model.

```ipython3
%matplotlib inline

import iris.quickplot
import matplotlib.pyplot as plt

from esmvalcore.config import CFG
from esmvalcore.dataset import Dataset
from esmvalcore.io.esgf import ESGFFile, download
from esmvalcore.preprocessor import annual_statistics, area_statistics
```

Configure ESMValCore so it searches the ESGF for data if it is not available locally. See [data sources](https://docs.esmvaltool.org/projects/esmvalcore/en/latest/quickstart/configure.html#config-data-sources) for more information.

```ipython3
CFG["search_data"] = "quick"
CFG.nested_update(
    {
        "projects": {
            "CMIP5": {
                "data": {
                    "esgf": {
                        "type": "esmvalcore.io.esgf.ESGFDataSource",
                        "download_dir": "~/climate_data",
                        "priority": 10,
                    },
                },
            },
        },
    },
)
```

Define the dataset we are going to use. In this case surface air temperature (`tas`).

```ipython3
tas = Dataset(
    short_name="tas",
    mip="Amon",
    project="CMIP5",
    dataset="MPI-ESM-MR",
    ensemble="r1i1p1",
    exp="historical",
    timerange="1850/2000",
)
tas
```

In order to compute the area average later on, we add the cell areas (`areacella`) as a supplementary dataset. This will append a new dataset to the list of supplementary datasets. Its facets are copied from the `tas` dataset and updated with the provided facets:

```ipython3
tas.add_supplementary(short_name="areacella", mip="fx", ensemble="r0i0p0")
tas.supplementaries
```

ESMValCore can automatically add extra facets based on the `project`, `mip`, `short_name`, and `dataset`. These extra facets are automatically added and used when searching for input files.

```ipython3
tas.augment_facets()
tas
```

Use the `find_files` method to find the files corresponding to the dataset.

```ipython3
tas.find_files()
print(tas.files)
for supplementary_ds in tas.supplementaries:
    print(supplementary_ds.files)
```

```myst-ansi
[ESGFFile:cmip5/output1/MPI-M/MPI-ESM-MR/historical/mon/atmos/Amon/r1i1p1/v20120503/tas_Amon_MPI-ESM-MR_historical_r1i1p1_185001-200512.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf.nci.org.au', 'esgf1.dkrz.de']]
[ESGFFile:cmip5/output1/MPI-M/MPI-ESM-MR/historical/fx/atmos/fx/r0i0p0/v20120503/areacella_fx_MPI-ESM-MR_historical_r0i0p0.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf.nci.org.au', 'esgf1.dkrz.de']]
```

If the files are not available locally, ESMValCore can download them for us.

```ipython3
files = list(tas.files)
for supplementary_ds in tas.supplementaries:
    files.extend(supplementary_ds.files)
files = [file for file in files if isinstance(file, ESGFFile)]
download(files)
tas.find_files()
print(tas.files)
for supplementary_ds in tas.supplementaries:
    print(supplementary_ds.files)
```

```myst-ansi
[LocalFile('~/climate_data/cmip5/output1/MPI-M/MPI-ESM-MR/historical/mon/atmos/Amon/r1i1p1/v20120503/tas_Amon_MPI-ESM-MR_historical_r1i1p1_185001-200512.nc')]
[LocalFile('~/climate_data/cmip5/output1/MPI-M/MPI-ESM-MR/historical/fx/atmos/fx/r0i0p0/v20120503/areacella_fx_MPI-ESM-MR_historical_r0i0p0.nc')]
```

The data in the files can be loaded into an [iris.cube.Cube](https://scitools-iris.readthedocs.io/en/latest/generated/api/iris/cube.html#iris.cube.Cube). ESMValCore will automatically check for and fix problems with the data formatting and attach the cell area.

```ipython3
cube = tas.load()
cube
```

```ipython3
cell_area = cube.cell_measures()[0]
cell_area
```

This code shows how to use some [esmvalcore.preprocessor](https://docs.esmvaltool.org/projects/esmvalcore/en/latest/api/esmvalcore.preprocessor.html) functions to compute the global annual mean temperature in degrees Celsius:

```ipython3
cube = area_statistics(cube, operator="mean")
cube = annual_statistics(cube, operator="mean")
cube.convert_units("degrees_C")
```

The [iris.quickplot](https://scitools-iris.readthedocs.io/en/latest/generated/api/iris/quickplot.html) module has useful functions for quickly plotting the results:

```ipython3
iris.quickplot.plot(cube)
plt.show()
```