API

Documentation of the core API of aerovaldb.

aerovaldb

aerovaldb.open(resource) → AerovalDB[source]

Open an AerovalDB instance, sending args and kwargs directly to the AervoalDB() function

Parameters:: resource – the resource-name for the database. The resource can be - ‘entrypoint:path’, with path being the location where the database should be generated (eg. ‘json_files:.’) - ‘path’, being a json_files dabasase (for example, ‘.’ is equivalent to ‘json_files:.’) - ‘:memory:’ an sqlite in-memory database. Contents are not persistently stored!
Returns:: an implementation-instance of AerovalDB openend to a location

Examples

>>> import aerovaldb
>>> with aerovaldb.open(":memory:") as db:
...     db.put_experiments({'test': 'test'}, 'project')
...     db.get_experiments('project')
{'test': 'test'}

>>> import aerovaldb
>>> db = aerovaldb.open(":memory:")
>>> db.put_experiments({'test': 'test'}, 'project')
>>> db.get_experiments('project')
{'test': 'test'}

class aerovaldb.types.AccessType(value)[source]

Enumeration of access types. Specifies how data will be read and returned.

JSON_STR: Result will be returned as an unparsed json string.
FILE_PATH: Result will be returned as the file path to the file

containing the data.

OBJ: The json will be parsed and returned as a python object.
URI: A string which is a unique identifier of this asset between

implementations of Aerovaldb. Can be used with aerovaldb.AerovalDB.get_by_uuid() and aerovaldb.AerovalDB.put_by_uuid() to read or write respectively.

MTIME: The timestamp for last modification for the resource will be

returned (as datetime.datetime).

CTIME: The creation timestamp for the resource will be returned (as

datetime.datetime)

aerovaldb.AerovalDB - User API

class aerovaldb.aerovaldb.AerovalDB[source]

get_by_uri(uri: str | QueryEntry, /, access_type: str | AccessType, cache: bool = False, default=None)[source]

Gets a stored object by its URI.

Parameters:

uri – URI of the item to fetch.
access_type – See AccessType.
cache – Whether to use the cache.
default – If provided, this value will be returned instead of raising a FileNotFoundError if not file exists. The provided object will be returned as is, and will not be converted to match access_type.

Note:

URI is intended to be consistent between implementations but not between versions of aerovaldb. Using aerovaldb.AerovalDB.get_by_uri() to fetch an identifier which can then be written to another connector using its respective aerovaldb.AerovalDB.put_by_uri() method is a supported use case.

get_config(project: str, experiment: str, /, *args, access_type: str | AccessType = AccessType.OBJ, cache: bool = False, default=None, **kwargs) → int[source]

Fetches a configuration from the db.

Parameters:

project – Project ID.
experiment – Experiment ID.
access_type – How the data is to be retrieved (See AccessType for details)
cache – Whether to use cache for this read.
default – Default value that will be returned instead of raising FileNotFoundError if no data was found (Will be returned as is and not converted to match access_type).

Returns:

The fetched data.

get_contour(project: str, experiment: str, obsvar: str, model: str, /, *args, timestep: str, access_type: str | AccessType = AccessType.OBJ, cache: bool = False, default=None, **kwargs)[source]

Fetch a contour object from the db.

Parameters:

project – Project ID.
experiment – Experiment ID.
obsvar – Observation variable.
model – Model ID.
timestep – Optional timestep. Timestep will be required in the future.
access_type – How the data is to be retrieved (See AccessType for details)
cache – Whether to use cache for this read.
default – Default value that will be returned instead of raising FileNotFoundError if no data was found (Will be returned as is and not converted to match access_type).

Returns:

The fetched data.

get_experiment_mtime(project: str, experiment: str) → datetime[source]

Parameters:

project – Project ID.
experiment – Experiment ID.

get_experiments(project: str, /, *args, access_type: str | AccessType = AccessType.OBJ, cache: bool = False, default=None, **kwargs)[source]

Fetches a list of experiments for a project from the db.

Parameters:

project – Project ID.
access_type – How the data is to be retrieved (See AccessType for details)
cache – Whether to use cache for this read.
default – Default value that will be returned instead of raising FileNotFoundError

if not data was found (Will be returned as is and not converted to match access_type).

Returns:: The fetched data.

get_fairmode(project: str, experiment: str, region: str, network: str, obsvar: str, layer: str, /, *args, access_type: str | AccessType = AccessType.OBJ, cache: bool = False, default=None, **kwargs)[source]

Fetch fairmode.

Parameters:

project – Project ID.
experiment – Experiment ID.
region – Region ID.
network – Observation Network.
obsvar – Observation variable.
layer – Layer.
access_type – How the data is to be retrieved (See AccessType for details)
cache – Whether to use cache for this read.
default – Default value that will be returned instead of raising FileNotFoundError if not data was found (Will be returned as is and not converted to match access_type).

Returns:

The fetched data.

get_forecast(project: str, experiment: str, region: str, network: str, obsvar: str, layer: str, /, *args, access_type: str | AccessType = AccessType.OBJ, cache: bool = False, default=None, **kwargs)[source]

Fetch forecast.

Parameters:

project – Project ID.
experiment – Experiment ID.
region – Region ID.
network – Observation Network.
obsvar – Observation variable.
layer – Layer.
access_type – How the data is to be retrieved (See AccessType for details)
cache – Whether to use cache for this read.
default – Default value that will be returned instead of raising FileNotFoundError if not data was found (Will be returned as is and not converted to match access_type).

Returns:

The fetched data.

get_glob_stats(project: str, experiment: str, frequency: str, /, *args, access_type: str | AccessType = AccessType.OBJ, cache: bool = False, default=None, **kwargs)[source]

Fetches a glob_stats object from the database.

Parameters:

project – The project ID.
experiment – The experiment ID.
frequency – The frequency (eg. ‘monthly’)
access_type – How the data is to be retrieved (See AccessType for details)
cache – Whether to use cache for this read.
default – Default value that will be returned instead of raising FileNotFoundError

if not data was found (Will be returned as is and not converted to match access_type).

Returns:: The fetched data.

get_gridded_map(project: str, experiment: str, obsvar: str, model: str, /, *args, access_type: str | AccessType = AccessType.OBJ, cache: bool = False, default=None, **kwargs)[source]

Fetches gridded map.

Parameters:

project – Project ID.
experiment – Experiment ID.
obsvar – Observation variable.
model – Model ID.
access_type – How the data is to be retrieved (See AccessType for details)
cache – Whether to use cache for this read.
default – Default value that will be returned instead of raising FileNotFoundError if not data was found (Will be returned as is and not converted to match access_type).

Returns:

The fetched data.

get_heatmap(project: str, experiment: str, frequency: str, region: str, time: str, /, *args, access_type: str | AccessType = AccessType.OBJ, cache: bool = False, default=None, **kwargs)[source]

Fetches heatmap data from the database

Parameters:

project – The project ID.
experiment – The experiment ID.
frequency – The frequency.
region – Region.
time – Time.
access_type – How the data is to be retrieved (See AccessType for details)
cache – Whether to use cache for this read.
default – Default value that will be returned instead of raising FileNotFoundError

if not data was found (Will be returned as is and not converted to match access_type).

Returns:: The fetched data.

get_heatmap_timeseries(project: str, experiment: str, region: str, network: str, obsvar: str, layer: str, /, *args, access_type: str | AccessType = AccessType.OBJ, cache: bool = False, default=None, **kwargs)[source]

Fetches heatmap timeseries.

Parameters:

project – Project ID.
experiment – Experiment ID.
region – Region ID.
network – Observation Network.
obsvar – Observation variable.
layer – Layer.
access_type – How the data is to be retrieved (See AccessType for details)
cache – Whether to use cache for this read.
default – Default value that will be returned instead of raising FileNotFoundError if not data was found (Will be returned as is and not converted to match access_type).

Returns:

The fetched data.

get_map(project: str, experiment: str, network: str, obsvar: str, layer: str, model: str, modvar: str, time: str, /, *args, frequency: str | None = None, season: str | None = None, access_type: str | AccessType = AccessType.OBJ, cache: bool = False, default=None, **kwargs)[source]

Fetches map data from db.

Parameters:

project – Project Id
experiment – Experiment ID.
network – Observation network
obsvar – Observation variable.
layer – Layer
model – Model ID
modvar – Model variable.
time – Time parameter.
frequency – Optional frequency (eg. ‘monthly’)
season – Optional season.
access_type – How the data is to be retrieved (See AccessType for details)
cache – Whether to use cache for this read.
default – Default value that will be returned instead of raising FileNotFoundError if not data was found (Will be returned as is and not converted to match access_type).

Returns:

The fetched data.

Note

If either frequency or season are provided, they both must be provided.

get_map_overlay(project: str, experiment: str, source: str, variable: str, date: str, access_type: str | AccessType = AccessType.BLOB)[source]

Getter for map overlay images.

Parameters:

project – Project ID.
experiment – Experiment ID.
source – Data source. Can be either an observation network or a model ID.
variable – Variable name.
date – Date.

get_menu(project: str, experiment: str, /, *args, access_type: str | AccessType = AccessType.OBJ, cache: bool = False, default=None, **kwargs)[source]

Fetches a menu configuartion from the db.

Parameters:

project – Project ID.
experiment – Experiment ID.
access_type – How the data is to be retrieved (See AccessType for details)
cache – Whether to use cache for this read.
default – Default value that will be returned instead of raising FileNotFoundError

if not data was found (Will be returned as is and not converted to match access_type).

Returns:: The fetched data.

get_models_style(project: str, /, experiment: str | None = None, *args, access_type: str | AccessType = AccessType.OBJ, cache: bool = False, default=None, **kwargs)[source]

Fetches model styles from db.

Parameters:

project – Project ID.
(Optional) (experiment) – Experiment ID.
access_type – How the data is to be retrieved (See AccessType for details)
cache – Whether to use cache for this read.
default – Default value that will be returned instead of raising FileNotFoundError if not data was found (Will be returned as is and not converted to match access_type).

Returns:

The fetched data.

get_profiles(project: str, experiment: str, location: str, network: str, obsvar: str, /, *args, access_type: str | AccessType = AccessType.OBJ, cache: bool = False, default=None, **kwargs)[source]

Fetches profiles from db.

Parameters:

project – Project ID.
experiment – Experiment ID.
location – Location.
network – Observation network.
obsvar – Observation variable.
access_type – How the data is to be retrieved (See AccessType for details)
cache – Whether to use cache for this read.
default – Default value that will be returned instead of raising FileNotFoundError if not data was found (Will be returned as is and not converted to match access_type).

Returns:

The fetched data.

get_ranges(project: str, experiment: str, /, *args, access_type: str | AccessType = AccessType.OBJ, cache: bool = False, default=None, **kwargs)[source]

Fetches ranges from the db.

Parameters:

project – Project ID.
experiment – Experiment ID.
access_type – How the data is to be retrieved (See AccessType for details)
cache – Whether to use cache for this read.
default – Default value that will be returned instead of raising FileNotFoundError if not data was found (Will be returned as is and not converted to match access_type).

Returns:

The fetched data.

get_regional_stats(project: str, experiment: str, frequency: str, network: str, variable: str, layer: str, /, *args, access_type: str | AccessType = AccessType.OBJ, cache: bool = False, default=None, **kwargs)[source]

Fetches regional stats from the database.

Parameters:

project – The project ID.
experiment – The experiment ID.
frequency – The frequency.
network – Observation network.
variable – Variable name.

get_regions(project: str, experiment: str, /, *args, access_type: str | AccessType = AccessType.OBJ, cache: bool = False, default=None, **kwargs)[source]

Fetches regions from db.

Parameters:

project – Project ID.
experiment – Experiment ID.
access_type – How the data is to be retrieved (See AccessType for details)
cache – Whether to use cache for this read.
default – Default value that will be returned instead of raising FileNotFoundError if not data was found (Will be returned as is and not converted to match access_type).

Returns:

The fetched data.

get_report(project: str, experiment: str, title: str, /, *args, access_type: str | AccessType = AccessType.OBJ, cache: bool = False, default=None, **kwargs)[source]

Fetch report.

Parameters:

project – Project ID.
experiment – Experiment ID.
title – Report title (ie. filename without extension).
access_type – How the data is to be retrieved (See AccessType for details)
cache – Whether to use cache for this read.
default – Default value that will be returned instead of raising FileNotFoundError if not data was found (Will be returned as is and not converted to match access_type).

Returns:

The fetched data.

get_report_image(project: str, experiment: str, path: str, access_type: str | AccessType = AccessType.BLOB)[source]

Getter for static images that are referenced from the report json files.

Parameters:

project – Project ID.
experiment – Experiment ID.
access_type – One of AccessType.BLOB, AccessType.FILE_PATH

Returns:

Either a string (If file path requested) or a bytes object with the image data

get_scatter(project: str, experiment: str, network: str, obsvar: str, layer: str, model: str, modvar: str, time: str, /, *args, access_type: str | AccessType = AccessType.OBJ, cache: bool = False, default=None, **kwargs)[source]

Get scat.

Parameters:

project – Project ID.
experiment – Experiment ID.
network – Observation network.
obsvar – Observation variable.
layer – Layer.
model – Model ID.
modvar – Model variable.
time – Time parameter.
access_type – How the data is to be retrieved (See AccessType for details)
cache – Whether to use cache for this read.
default – Default value that will be returned instead of raising FileNotFoundError if not data was found (Will be returned as is and not converted to match access_type).

Returns:

The fetched data.

get_statistics(project: str, experiment: str, /, *args, access_type: str | AccessType = AccessType.OBJ, cache: bool = False, default=None, **kwargs)[source]

Fetches statistics for an experiment.

Parameters:

project – Project ID.
experiment – Experiment ID.
access_type – How the data is to be retrieved (See AccessType for details)
cache – Whether to use cache for this read.
default – Default value that will be returned instead of raising FileNotFoundError if not data was found (Will be returned as is and not converted to match access_type).

Returns:

The fetched data.

get_timeseries(project: str, experiment: str, location: str, network: str, obsvar: str, layer: str, /, *args, access_type: str | AccessType = AccessType.OBJ, cache: bool = False, default=None, **kwargs)[source]

Fetches a timeseries from the db.

Parameters:

project – Project id.
experiment – Experiment ID.
location – Location.
network – Observation network.
obsvar – Observation variable.
layer – Layer.
access_type – How the data is to be retrieved (See AccessType for details)
cache – Whether to use cache for this read.
default – Default value that will be returned instead of raising FileNotFoundError if no data was found (Will be returned as is and not converted to match access_type).

Returns:

The fetched data.

get_timeseries_weekly(project: str, experiment: str, location: str, network: str, obsvar: str, layer: str, /, *args, access_type: str | AccessType = AccessType.OBJ, cache: bool = False, default=None, **kwargs)[source]

Fetches a weekly time series from the db.

Parameters:

project – Project ID.
experiment – Experiment ID.
location – Location.
network – Observation network.
obsvar – Observation variable.
layer – Layer.
access_type – How the data is to be retrieved (See AccessType for details)
cache – Whether to use cache for this read.
default – Default value that will be returned instead of raising FileNotFoundError

if not data was found (Will be returned as is and not converted to match access_type).

Returns:: The fetched data.

list_all(access_type: str | AccessType = AccessType.URI)[source]

Returns a list of identifiers for each object stored in the database

Parameters:: access_type – What to return (This is implementation specific, but

in general each implementation should support URI, some may support FILE_PATH). :raises UnsupportedOperation:

For non-supported acces types.

list_glob_stats(project: str, experiment: str, /, access_type: str | AccessType = AccessType.URI) → list[str][source]

Lists the URI for each glob_stats object.

Parameters:

project – str
experiment – str

Returns:

List of URIs.

list_map(project: str, experiment: str) → list[str][source]

Lists all map files for a given project / experiment combination.

Parameters:

project – The project ID.
experiment – The experiment ID.

Returns:

List with the URIs.

list_timeseries(project: str, experiment: str) → list[QueryEntry][source]

Returns a list of URIs of all timeseries files for a given project and experiment id.

Parameters:

project – Project ID.
experiment – Experiment ID.

lock()[source]

Acquires an exclusive advisory lock to coordinate file access between instances of aerovaldb. Intended to be used as a context manager.

Note:

URI is intended to be consistent between implementations but not between versions of aerovaldb. Using aerovaldb.AerovalDB.get_by_uri() to fetch an identifier which can then be written to another connector using its respective aerovaldb.AerovalDB.put_by_uri() method is a supported use case.

put_config(obj, project: str, experiment: str, /, *args, **kwargs)[source]

Stores a configuration to the db.

Paran obj:

The object to be stored.

Parameters:

project – Project ID.
experiment – Experiment ID.

put_contour(obj, project: str, experiment: str, obsvar: str, model: str, /, timestep: str | None = None, *args, **kwargs)[source]

Put a contour object in the db.

Parameters:

obj – The object to be stored.
project – Project ID.
experiment – Experiment ID.
obsvar – Observation variable.
model – Model ID.
timestep – Optional timestep. Will be required in the future.

put_experiments(obj, project: str, /, *args, **kwargs)[source]

Stores a list of experiments for a project to the db.

Parameters:: project – Project ID.

put_fairmode(obj, project: str, experiment: str, region: str, network: str, obsvar: str, layer: str, /, *args, **kwargs)[source]

Store fairmode.

Parameters:

obj – The Object to be stored.
project – Project ID.
experiment – Experiment ID.
region – Region ID.
network – Observation Network.
obsvar – Observation variable.
layer – Layer.

put_forecast(obj, project: str, experiment: str, region: str, network: str, obsvar: str, layer: str, /, *args, **kwargs)[source]

Store forecast.

Parameters:

obj – The Object to be stored.
project – Project ID.
experiment – Experiment ID.
region – Region ID.
network – Observation Network.
obsvar – Observation variable.
layer – Layer.

put_glob_stats(obj, project: str, experiment: str, frequency: str, /, *args, **kwargs)[source]

Saves a glob_stats object to the database.

Parameters:

obj – The object to be stored.
project – The project ID.
experiment – The experiment ID.
frequency – The frequency (eg. ‘monthly’)

put_gridded_map(obj, project: str, experiment: str, obsvar: str, model: str, /, *args, **kwargs)[source]

Store gridded map.

Parameters:

obj – The object to be stored.
project – Project ID.
experiment – Experiment ID.
obsvar – Observation variable.
model – Model ID.

put_heatmap_timeseries(obj, project: str, experiment: str, region: str, network: str, obsvar: str, layer: str, /, *args, **kwargs)[source]

Stores heatmap timeseries.

Parameters:

obj – The object to be stored.
project – Project ID.
experiment – Experiment ID.
region – Region ID.
network – Observation Network.
obsvar – Observation variable.
layer – Layer.

put_map(obj, project: str, experiment: str, network: str, obsvar: str, layer: str, model: str, modvar: str, time: str, /, *args, **kwargs)[source]

Stores map data in db.

Parameters:

obj – The Object to be stored.
project – Project Id
experiment – Experiment ID.
network – Observation network
obsvar – Observation variable.
layer – Layer
model – Model ID
modvar – Model variable.
time – Time parameter.

put_map_overlay(obj, project: str, experiment: str, source: str, variable: str, date: str)[source]

Putter for map overlay images.

Parameters:

obj – The object to be stored.
project – Project ID.
experiment – Experiment ID.
source – Data source. Can be either an observation network or a model ID.
variable – Variable name.
date – Date.

put_menu(obj, project: str, experiment: str, /, *args, **kwargs)[source]

Stores a menu configuration in the db.

Parameters:

obj – The object to be stored.
project – Project ID.
experiment – Experiment ID.

put_models_style(obj, project: str, /, experiment: str | None = None, *args, **kwargs)[source]

Stores model styles config in db.

Parameters:

obj – Object to be stored.
project – Project ID.
(Optional) (experiment) – Experiment ID.

put_profiles(obj, project: str, experiment: str, location: str, network: str, obsvar: str, /, *args, **kwargs)[source]

Stores profiles in db.

Parameters:

obj – Object to be stored.
project – Project ID._
experiment – Experiment ID.
location – Location.
network – Observation network.
obsvar – Observation variable.

put_ranges(obj, project: str, experiment: str, /, *args, **kwargs)[source]

Stores ranges in db.

Parameters:

obj – The object to be stored.
project – Project ID.
experiment – Experiment ID.

put_regions(obj, project: str, experiment: str, /, *args, **kwargs)[source]

Stores regions in db.

Parameters:

obj – Object to be stored.
project – Project ID.
experiment – Experiment ID.

put_report(obj, project: str, experiment: str, title: str, /, *args, **kwargs)[source]

Store report.

Parameters:

obj – The object to be stored.
project – Project ID.
experiment – Experiment ID.
title – Report title (ie. filename without extension).

put_report_image(obj, project: str, experiment: str, path: str)[source]

Putter for static images that are referenced from the report json files.

Parameters:

obj – A bytes object representing the image data to be written.
project – Project ID.
experiment – Experiment ID.

Returns:

Either a string (If file path requested) or a bytes object with the image data

put_scatter(obj, project: str, experiment: str, network: str, obsvar: str, layer: str, model: str, modvar: str, time: str, /, *args, **kwargs)[source]

Stores scat in db.

Parameters:

obj – Object to be stored.
project – Project ID.
experiment – Experiment ID.
network – Observation network.
obsvar – Observation variable.
layer – Layer.
model – Model ID.
modvar – Model variable.
time – Time paramter.

put_statistics(obj, project: str, experiment: str, /, *args, **kwargs)[source]

Stores statistics to the db.

Parameters:

obj – The object to be stored.
project – Project ID.
experiment – Experiment ID.

put_timeseries(obj, project: str, experiment: str, location: str, network: str, obsvar: str, layer: str, /, *args, **kwargs)[source]

Places a timeseries in the db

Parameters:

obj – The object to write into the db.
project – Project ID
experiment – Experiment ID.
location – Location.
network – Observation network.
obsvar – Observation variable.
layer – Layer.

put_timeseries_weekly(obj, project: str, experiment: str, location: str, network: str, obsvar: str, layer: str, /, *args, **kwargs)[source]

Stores a weekly time series in the db.

Parameters:

obj – The object to be stored.
project – Project ID.
experiment – Experiment ID.
location – Location.
network – Observation network.
obsvar – Observation variable.
layer – Layer.

query(asset_type: Route | Iterable[Route] | None = None, **kwargs) → list[QueryEntry][source]

Query function for getting information about assets stored in the db.

Parameters:: asset_type – Enum of the type of asset to query (Can be an iterable of multiple types). By default,

all asset types will be included. :param kwargs: Optional additional filter arguments. Will be matched against QueryEntry.meta. All provided keys must match. For possible keys see function signature of the getter for which you want to match. For instance aerovaldb.AerovalDB.get_experiments() can be filtered on the project key.

Returns:: A list of QueryEntry objects that contains the URI and information about

the queried files.

Example: >>> import tempfile >>> import aerovaldb >>> >>> with tempfile.TemporaryDirectory() as dir: … with aerovaldb.open(f”json_files:{dir}”) as db: … db.put_experiments({}, “project1”) … db.put_experiments({}, “project2”) … db.query(aerovaldb.Route.EXPERIMENTS, project=”project1”)[0].uri … db.query(aerovaldb.Route.EXPERIMENTS, project=”project1”)[0].meta ‘/v0/experiments/project1?version=0.0.1’ {‘project’: ‘project1’}

rm_by_uri(uri: str)[source]

Removes the asset associated with the provided uri. The provided uri must match a whole file (ie. uris corresponding to filtered uris will not work and will raise an UnsupporedOperationError).

Parameters:: uri – URI to remove.
Raises:: UnsupporedOperation – If URI corresponds to a filtered endpoint.

rm_experiment_data(project: str, experiment: str)[source]

Deletes ALL data associated with an experiment.

Parameters:

project – Project ID.
experiment – Experiment ID.