.ipynb .pdf

Binder

Contents

 

MÉRA Wind Data for Irish Continental Shelf region

Introduction

Met Éireann Re-Analysis (MÉRA) - Climate Re-analysis is a reanalysis data set developed by Met Éireann, Ireland’s National Meteorological Service. It contains a 35-year very high resolution (2.5 km horizontal grid) regional climate reanalysis for Ireland using the ALADIN-HIRLAM numerical weather prediction system, spanning the period 1981 to August 2019, and includes surface, near-surface and atmospheric parameters. Further details may be found in the following publications:

• Gleeson et al. (2017) - Met Éireann high resolution reanalysis for Ireland

• Whelan et al. (2018) - An Evaluation of MÉRA, a High-Resolution Mesoscale Regional Reanalysis

This notebook provides details of:

1. MÉRA wind data products retrieval.

2. The creation of the MÉRA Zarr wind store that is included in the EOOffshore catalog.

3. A brief look at this Zarr store, including a demonstration of wind speed calculation.

How to cite: O’Callaghan, D. and McBreen, S.: Scalable Offshore Wind Analysis With Pangeo, EGU General Assembly 2022, Vienna, Austria, 23–27 May 2022, EGU22-2746, https://doi.org/10.5194/egusphere-egu22-2746, 2022.

Note: more extensive usage of the EOOffshore MÉRA Zarr store may be found in the following notebooks:

• Offshore Wind in Irish Areas Of Interest

• Comparison of Offshore Wind Speed Extrapolation and Power Density Estimation

---

MÉRA Wind Data Products

MÉRA data products contain wind variables with hourly/3-hourly values at multiple heights (metres above surface level), which means they are particularly useful for the EOOffshore project due to these heights being similar to typical wind turbine hub heights. The following data variables are relevant:

Variable	Unit	Height (metres above sea level)	Description
u	\(m s^{-1}\)	10, 50, 80, 100, 125	U (eastward) wind component
v	\(m s^{-1}\)	10, 50, 80, 100, 125	V (northward) wind component
$$$$	$$$$	$$$$	$$$$

The UCD School of Mathematics and Statistics granted access to their MÉRA data archive, and the following products were retrieved via SSH:

Observation / Models	Reanalysis
Processing level	Level-3
Data type	Gridded (latitude/longitude)
Horizontal coverage	Bounding box [59.6, -20.1, 46.8, 2.8]
Horizontal resolution	2.5 km
Vertical coverage	[10, 50, 80, 100, 125] meters above surface level
Temporal coverage	2001-01-01T00:00:00 to 2016-12-31T21:00:00
Temporal resolution	3-Hourly [00, 03, 06, 09, 12, 15, 18, 21Z]
Update frequency	n/a
File format	GRIB version 1
Total retrieved products	1,920
Total products size	226G

---

MÉRA Wind Zarr Store

As MÉRA u and v variables are provided in separate monthly products for a particular height, pairs of u and v variables were initially merged for all possible month/height combinations. This process involved the selection of coordinates for the Irish Continental Shelf (ICS) regions, and computing the following new variables from u and v using MetPy:

Variable	Unit	Height (meters above sea level)	Description
wind_speed	\(m s^{-1}\)	10, 50, 80, 100, 125	Wind speed calculated from U and V wind components with metpy.calc.wind_speed()
wind_direction	degree	10, 50, 80, 100, 125	Wind direction calculated from U and V wind components with metpy.calc.wind_direction()

Interim products containing only these wind_speed and wind_direction variables were persisted as NetCDF files. These merged products were loaded using xarray.open_mfdataset(), combined by their grid coordinates and concatenated along the time dimension. To reduce computation, the time dimension was restricted to the provided 3-hourly values. The data set was chunked in space (latitude/longitude grid), and persisted to a single, chunked, compressed Zarr store (196G), which is a cloud-optimised format suitable for multi-dimensional arrays. A time chunk size was specified that resulted in a low number of time chunks, as this approach is more suitable for subsequent processing of variables over time for Areas Of Interest (AOIs).

As requested by the MÉRA data set rights notes, the following attribution is declared:

1. Copyright statement: Copyright Met Éireann

2. Source www.met.ie

3. Licence Statement: This data is published under a Creative Commons Attribution 4.0 International (CC BY 4.0). https://creativecommons.org/licenses/by/4.0/

4. Disclaimer: Met Éireann does not accept any liability whatsoever for any error or omission in the data, their availability, or for any loss or damage arising from their use.

5. The generated Zarr store contains modifications of MÉRA data as stated above.

---

MÉRA in EOOffshore Catalog

Open the catalog and view the MÉRA metadata

All EOOffshore data sets, including the MÉRA Zarr store described above, are accessible using the EOOffshore Intake catalog. Each catalog entry provides a description and metadata associated with the corresponding data set, defined in a YAML configuration file. The EOOffshore catalog configuration was originally influenced by the Pangeo Cloud Data Store atmosphere.yaml catalog configuration.

To view the MÉRA metadata (ANALYSIS indicates 3-hourly provided products were used):

from intake import open_catalog

catalog = open_catalog('data/intake-catalogs/eooffshore_ics.yaml')
catalog.eooffshore_ics_mera_wind_ANALYSIS

eooffshore_ics_mera_wind_ANALYSIS:
  args:
    storage_options: null
    urlpath: /meradata/eo/zarr/mera/eooffshore_ics_mera_wind_ANALYSIS.zarr
  description: "EOOffshore Project 2001 - 2016 Concatenated 3-hourly ANALYSIS wind\
    \ variable products from the Met \xC9ireann ReAnalysis (M\xC9RA) data set, for\
    \ Irish Continental Shelf. Wind speed and direction have been calculated from\
    \ the source u and v variables. Copyright Met \xC9ireann. Source www.met.ie. Licence\
    \ Statement - This data is published under a Creative Commons Attribution 4.0\
    \ International (CC BY 4.0). https://creativecommons.org/licenses/by/4.0/ . Disclaimer\
    \ - Met \xC9ireann does not accept any liability whatsoever for any error or omission\
    \ in the data, their availability, or for any loss or damage arising from their\
    \ use."
  driver: intake_xarray.xzarr.ZarrSource
  metadata:
    catalog_dir: /opt/eooffshore/notebooks/datasets/data/intake-catalogs/
    tags:
    - atmosphere
    - wind
    - mera
    - meteireann
    - ocean
    title: "M\xC9RA 3-hourly wind products for Irish Continental Shelf region (2016)"
    url: https://www.met.ie/climate/available-data/mera

---

Load the catalog MÉRA Zarr store

Intake catalog entries typically specify a driver to be used when loading the corresponding data set. The MÉRA entry specifies intake_xarray.xzarr.ZarrSource, a driver implementation provided by the intake-xarray library. This enables NetCDF and Zarr data sets to be loaded using xarray, a library for processing N-D labeled arrays and datasets. As xarray labels take the form of dimensions, coordinates and attributes on top of NumPy-like arrays, it is particularly suited to data sets such as MÉRA whose variables feature latitude/longitude grid coordinates.

This intake driver will load the associated dataset into an xarray.Dataset. To enable support for potentially large data sets, the to_dask() function is used to load the underlying variable arrays with Dask, a parallel, out-of-core computing library. The ZarrSource implementation will load the data set variables into Dask arrays, which will be loaded and processed in parallel as chunks during subsequent computation. As discussed above, variable chunk sizes may be specified during Zarr store creation.

Here is the MÉRA store loaded into an xarray.Dataset:

• All variables have associated coordinate dimensions:

  • height

  • x (longitude) and y (latitude) - the corresponding coordinate grid

• The wind_speed and wind_direction variables have a 3-hourly time coordinate dimension

• A low number of time chunks have been specified, to support subsequent computation across time for smaller AOI grid coordinates.

• Variables with an atlas_ name prefix are precomputed variables used in the MÉRA-based Wind Atlas for Irish Continental Shelf region notebook

ds = catalog.eooffshore_ics_mera_wind_ANALYSIS.to_dask()
ds

<xarray.Dataset>
Dimensions:                        (height: 5, y: 489, x: 361, time: 46752)
Coordinates:
  * height                         (height) float64 10.0 50.0 80.0 100.0 125.0
    latitude                       (y, x) float64 dask.array<chunksize=(32, 32), meta=np.ndarray>
    longitude                      (y, x) float64 dask.array<chunksize=(32, 32), meta=np.ndarray>
  * time                           (time) datetime64[ns] 2001-01-01 ... 2016-...
Dimensions without coordinates: y, x
Data variables:
    atlas_maximum_yield_frequency  (height, y, x) float64 dask.array<chunksize=(2, 123, 181), meta=np.ndarray>
    atlas_mean_power_density       (height, y, x) float32 dask.array<chunksize=(5, 489, 361), meta=np.ndarray>
    atlas_mean_wind_speed          (height, y, x) float32 dask.array<chunksize=(5, 489, 361), meta=np.ndarray>
    atlas_operational_frequency    (height, y, x) float64 dask.array<chunksize=(2, 123, 181), meta=np.ndarray>
    atlas_weibull_scale            (height, y, x) float32 dask.array<chunksize=(5, 489, 361), meta=np.ndarray>
    atlas_weibull_shape            (height, y, x) float32 dask.array<chunksize=(5, 489, 361), meta=np.ndarray>
    wind_direction                 (height, time, y, x) float32 dask.array<chunksize=(1, 10000, 32, 32), meta=np.ndarray>
    wind_speed                     (height, time, y, x) float32 dask.array<chunksize=(1, 10000, 32, 32), meta=np.ndarray>
Attributes:
    Conventions:                    CF-1.7
    GRIB_centre:                    eidb
    GRIB_centreDescription:         Dublin
    GRIB_edition:                   1
    GRIB_subCentre:                 255
    eooffshore_zarr_creation_time:  2022-05-13T19:03:09Z
    eooffshore_zarr_details:        EOOffshore Project: Concatenated 3-hourly...
    geospatial_lat_max:             58.05
    geospatial_lat_min:             45.95
    geospatial_lon_max:             355.15
    geospatial_lon_min:             334.05
    institution:                    Dublin

xarray.Dataset

• Dimensions:
  • height: 5
  • y: 489
  • x: 361
  • time: 46752
• Coordinates: (4)
  • height
    (height)
    float64
    10.0 50.0 80.0 100.0 125.0

    long_name :

    height above the surface

    positive :

    up

    standard_name :

    height

    units :

    m

    array([ 10.,  50.,  80., 100., 125.])

  • latitude
    (y, x)
    float64
    dask.array<chunksize=(32, 32), meta=np.ndarray>

    long_name :

    latitude

    standard_name :

    latitude

    units :

    degrees_north

    Array Chunk Bytes 1.35 MiB 8.00 kiB Shape (489, 361) (32, 32) Count 193 Tasks 192 Chunks Type float64 numpy.ndarray

  • longitude
    (y, x)
    float64
    dask.array<chunksize=(32, 32), meta=np.ndarray>

    long_name :

    longitude

    standard_name :

    longitude

    units :

    degrees_east

    Array Chunk Bytes 1.35 MiB 8.00 kiB Shape (489, 361) (32, 32) Count 193 Tasks 192 Chunks Type float64 numpy.ndarray

  • time
    (time)
    datetime64[ns]
    2001-01-01 ... 2016-12-31T21:00:00

    long_name :

    initial time of forecast

    standard_name :

    forecast_reference_time

    array(['2001-01-01T00:00:00.000000000', '2001-01-01T03:00:00.000000000',
           '2001-01-01T06:00:00.000000000', ..., '2016-12-31T15:00:00.000000000',
           '2016-12-31T18:00:00.000000000', '2016-12-31T21:00:00.000000000'],
          dtype='datetime64[ns]')

• Data variables: (8)
  • atlas_maximum_yield_frequency
    (height, y, x)
    float64
    dask.array<chunksize=(2, 123, 181), meta=np.ndarray>

    long_name :

    Maximum Yield Frequency

    units :

    %

    Array Chunk Bytes 6.73 MiB 347.86 kiB Shape (5, 489, 361) (2, 123, 181) Count 25 Tasks 24 Chunks Type float64 numpy.ndarray

  • atlas_mean_power_density
    (height, y, x)
    float32
    dask.array<chunksize=(5, 489, 361), meta=np.ndarray>

    long_name :

    Power density (estimate)

    units :

    Wm**-2

    Array Chunk Bytes 3.37 MiB 3.37 MiB Shape (5, 489, 361) (5, 489, 361) Count 2 Tasks 1 Chunks Type float32 numpy.ndarray

  • atlas_mean_wind_speed
    (height, y, x)
    float32
    dask.array<chunksize=(5, 489, 361), meta=np.ndarray>

    Array Chunk Bytes 3.37 MiB 3.37 MiB Shape (5, 489, 361) (5, 489, 361) Count 2 Tasks 1 Chunks Type float32 numpy.ndarray

  • atlas_operational_frequency
    (height, y, x)
    float64
    dask.array<chunksize=(2, 123, 181), meta=np.ndarray>

    long_name :

    Operational Frequency

    units :

    %

    Array Chunk Bytes 6.73 MiB 347.86 kiB Shape (5, 489, 361) (2, 123, 181) Count 25 Tasks 24 Chunks Type float64 numpy.ndarray

  • atlas_weibull_scale
    (height, y, x)
    float32
    dask.array<chunksize=(5, 489, 361), meta=np.ndarray>

    long_name :

    Weibull scale (𝜆)

    units :

    m s**-1

    Array Chunk Bytes 3.37 MiB 3.37 MiB Shape (5, 489, 361) (5, 489, 361) Count 2 Tasks 1 Chunks Type float32 numpy.ndarray

  • atlas_weibull_shape
    (height, y, x)
    float32
    dask.array<chunksize=(5, 489, 361), meta=np.ndarray>

    long_name :

    Weibull shape (k)

    units :

    dimensionless

    Array Chunk Bytes 3.37 MiB 3.37 MiB Shape (5, 489, 361) (5, 489, 361) Count 2 Tasks 1 Chunks Type float32 numpy.ndarray

  • wind_direction
    (height, time, y, x)
    float32
    dask.array<chunksize=(1, 10000, 32, 32), meta=np.ndarray>

    coordinates :

    longitude step latitude valid_time

    long_name :

    Wind direction

    units :

    degrees

    Array Chunk Bytes 153.73 GiB 39.06 MiB Shape (5, 46752, 489, 361) (1, 10000, 32, 32) Count 4801 Tasks 4800 Chunks Type float32 numpy.ndarray

  • wind_speed
    (height, time, y, x)
    float32
    dask.array<chunksize=(1, 10000, 32, 32), meta=np.ndarray>

    coordinates :

    longitude step latitude valid_time

    long_name :

    Wind speed

    units :

    m s**-1

    Array Chunk Bytes 153.73 GiB 39.06 MiB Shape (5, 46752, 489, 361) (1, 10000, 32, 32) Count 4801 Tasks 4800 Chunks Type float32 numpy.ndarray

• Attributes: (12)

  Conventions :

  CF-1.7

  GRIB_centre :

  eidb

  GRIB_centreDescription :

  Dublin

  GRIB_edition :

  1

  GRIB_subCentre :

  255

  eooffshore_zarr_creation_time :

  2022-05-13T19:03:09Z

  eooffshore_zarr_details :

  EOOffshore Project: Concatenated 3-hourly ANALYSIS wind variable products from the Met Éireann ReAnalysis (MÉRA) data set, for Irish Continental Shelf. Wind speed and direction have been calculated from the source u and v variables. Copyright Met Éireann. Source www.met.ie. Licence Statement: This data is published under a Creative Commons Attribution 4.0 International (CC BY 4.0). https://creativecommons.org/licenses/by/4.0/ . Disclaimer: Met Éireann does not accept any liability whatsoever for any error or omission in the data, their availability, or for any loss or damage arising from their use.

  geospatial_lat_max :

  58.05

  geospatial_lat_min :

  45.95

  geospatial_lon_max :

  355.15

  geospatial_lon_min :

  334.05

  institution :

  Dublin

---

MÉRA wind speed (2001 - 2016)

Each variable in the MÉRA data set, for example, wind speed, is loaded into an xarray.DataArray:

ds.wind_speed

<xarray.DataArray 'wind_speed' (height: 5, time: 46752, y: 489, x: 361)>
dask.array<open_dataset-eeaa2ce64ff419050a3516ede04d654ewind_speed, shape=(5, 46752, 489, 361), dtype=float32, chunksize=(1, 10000, 32, 32), chunktype=numpy.ndarray>
Coordinates:
  * height     (height) float64 10.0 50.0 80.0 100.0 125.0
    latitude   (y, x) float64 dask.array<chunksize=(32, 32), meta=np.ndarray>
    longitude  (y, x) float64 dask.array<chunksize=(32, 32), meta=np.ndarray>
  * time       (time) datetime64[ns] 2001-01-01 ... 2016-12-31T21:00:00
Dimensions without coordinates: y, x
Attributes:
    coordinates:  longitude step latitude valid_time
    long_name:    Wind speed
    units:        m s**-1

xarray.DataArray

'wind_speed'

• height: 5
• time: 46752
• y: 489
• x: 361

• dask.array<chunksize=(1, 10000, 32, 32), meta=np.ndarray>

  Array Chunk Bytes 153.73 GiB 39.06 MiB Shape (5, 46752, 489, 361) (1, 10000, 32, 32) Count 4801 Tasks 4800 Chunks Type float32 numpy.ndarray

• Coordinates: (4)
  • height
    (height)
    float64
    10.0 50.0 80.0 100.0 125.0

    long_name :

    height above the surface

    positive :

    up

    standard_name :

    height

    units :

    m

    array([ 10.,  50.,  80., 100., 125.])

  • latitude
    (y, x)
    float64
    dask.array<chunksize=(32, 32), meta=np.ndarray>

    long_name :

    latitude

    standard_name :

    latitude

    units :

    degrees_north

    Array Chunk Bytes 1.35 MiB 8.00 kiB Shape (489, 361) (32, 32) Count 193 Tasks 192 Chunks Type float64 numpy.ndarray

  • longitude
    (y, x)
    float64
    dask.array<chunksize=(32, 32), meta=np.ndarray>

    long_name :

    longitude

    standard_name :

    longitude

    units :

    degrees_east

    Array Chunk Bytes 1.35 MiB 8.00 kiB Shape (489, 361) (32, 32) Count 193 Tasks 192 Chunks Type float64 numpy.ndarray

  • time
    (time)
    datetime64[ns]
    2001-01-01 ... 2016-12-31T21:00:00

    long_name :

    initial time of forecast

    standard_name :

    forecast_reference_time

    array(['2001-01-01T00:00:00.000000000', '2001-01-01T03:00:00.000000000',
           '2001-01-01T06:00:00.000000000', ..., '2016-12-31T15:00:00.000000000',
           '2016-12-31T18:00:00.000000000', '2016-12-31T21:00:00.000000000'],
          dtype='datetime64[ns]')

• Attributes: (3)

  coordinates :

  longitude step latitude valid_time

  long_name :

  Wind speed

  units :

  m s**-1

Calculate mean wind speed over time dimension for all heights at AOI grid coordinates

Using Dask, the data set loading process is lazy, where no data is loaded inititally. Instead, data loading is delayed until execution time, where variables will be loaded and processed in parallel according to the corresponding chunks specification. Dask arrays implement a subset of the NumPy ndarray interface using blocked algorithms, and the original variable arrays will be split into smaller chunk arrays, enabling computation on arrays larger than memory using all available cores. The blocked algorithms are coordinated using Dask graphs.

To perform some analysis at known AOI latitude/longitude coordinates, the xarray.DataArray.sel(..., method='nearest') function may be used to select a subset of the data array (or data set) at coordinates nearest to the specified parameters. Here, mean wind speed over the time dimension is determined for the specified coordinates, where Dask graph execution is triggered by calling compute(). The resulting variable values will be contained in a NumPy ndarray.

• Note: xarray.DataArray.sel(..., method='nearest') does not support selection from two-dimensional curvilinear grids as used in the MÉRA data set. Consequently, xarray.DataArray.isel() is used here with the x (longitude) and y (latitude) dimensions for demonstration. A solution enabling two-dimensional grid selection is described in the Offshore Wind in Irish Areas Of Interest notebook.

Graph execution is managed by a task scheduler. The default scheduler (used for executing this notebook) executes computations with local threads. However, execution may also be performed on a distributed cluster without any change to the xarray code used here.

ds.wind_speed.isel(x=250, y=150).mean(dim='time').compute()

<xarray.DataArray 'wind_speed' (height: 5)>
array([ 8.566245,  9.572731,  9.891546, 10.051194, 10.216036],
      dtype=float32)
Coordinates:
  * height     (height) float64 10.0 50.0 80.0 100.0 125.0
    latitude   float64 51.37
    longitude  float64 352.6

xarray.DataArray

'wind_speed'

• height: 5

• 8.566 9.573 9.892 10.05 10.22

  array([ 8.566245,  9.572731,  9.891546, 10.051194, 10.216036],
        dtype=float32)

• Coordinates: (3)
  • height
    (height)
    float64
    10.0 50.0 80.0 100.0 125.0

    long_name :

    height above the surface

    positive :

    up

    standard_name :

    height

    units :

    m

    array([ 10.,  50.,  80., 100., 125.])

  • latitude
    ()
    float64
    51.37

    long_name :

    latitude

    standard_name :

    latitude

    units :

    degrees_north

    array(51.36760684)

  • longitude
    ()
    float64
    352.6

    long_name :

    longitude

    standard_name :

    longitude

    units :

    degrees_east

    array(352.5993649)

• Attributes: (0)

previous

ERA5 Wind Data for Irish Continental Shelf region

next

NEWA Wind Data for Irish Continental Shelf region