Overview¶
ERDDAPUtil provides additional management features for ERDDAP designed for use cases that focus on automated deployment and update of datasets to an ERDDAP server and for clustering use cases. However, these features can be used with any standard ERDDAP server if desired.
Installation¶
We recommend installation via Docker or another containerized approach, however it is possible to clone the source repository to your server and use the tools manually.
Docker¶
A Docker image is provided for ease of use.
The Quick Start Guide can be used for a quick introduction to using this image via Compose.
ERDDAPUtil Service¶
The ERDDAPUtil service performs all operations related to ERDDAP such as reloading datasets,
compiling datasets.xml, and other tasks. It is organized this way so that only a single
operation is executed at once on the file system to avoid the AMPQ, HTTP, and CLI APIs from
creating race conditions or other conflicts. Each API communicates with the daemon over a
TCP connection (by default on port 9172) to request an operation be performed.
The ERDDAP tools currently included are:
Requesting a dataset be reloaded ASAP at any of the three levels that ERDDAP allows (see Dataset Management)
Recompiling the datasets.xml file from a directory of XML files (see Dataset Management)
Flushing the log files to disk (see Dataset Management)
Listing the datasets currently in datasets.xml (see Dataset Management)
Blocking and unblocking IP addresses from making requests (see Control List Management)
Blocking and unblocking emails from subscribing (see Control List Management)
Adding and removing IP addresses from the allow unlimited requests list (see Control List Management)
Parsing the status.html page and creating Prometheus metrics from it (see ERDDAP Metrics)
Removing old log files from ERDDAP’s logs directory (see Log Management)
Removing decompressed files from ERDDAP’s decompressed directory (see Dataset Management)
Managing statistics from Tomcat on ERDDAP requests (see Tomcat Log Parsing)
The service is launched by calling the ERDDAPUtil Python module and it must be running for any of the other APIs to work.
python -m erddaputil daemon
In Docker, this is the default command when running the container.
Web API¶
The ERDDAPUtil web API is implemented in Flask and provides several useful tools:
A management API that sends commands to the ERDDAPUtil service
A metrics API that allows the ERDDAPUtil service to expose metrics via a Prometheus endpoint as well as metrics about the web API itself.
A health check endpoint
All calls to the management API or to push metrics must be authenticated using HTTP Basic Auth. It runs on port 9173 by default.
The web application can be launched via waitress (using a single thread) with the command:
python -m erddaputil webserver
In Docker, specify command: ['webserver'] to run the webserver in waitress.
See Web API for more details.
CLI API¶
The ERDDAPUtil command line interface is implemented in Click and provides an interface for the ERDDAPUtil service. The CLI can be called using the command:
python -m erddaputil [COMMAND]
In Docker, it can be executed on the daemon’s running container as follows:
docker exec erddaputil_daemon python -m erddaputil [COMMAND]
See Command Line Interface for more details.
AMPQ Integration¶
ERDDAPUtil is designed to support sending the same command to multiple ERDDAP servers at once. The ERDDAPUtil service provides this functionality by sending a message via AMPQ whenever a command is received if configured to do so. Any ERDDAPUtil instance running its AMPQ listener will receive such messages and apply the same action but not rebroadcast it (the local server is configured to ignore its own AMPQ messages so the action is not applied twice).
The AMPQ listener can be run with the following command:
python -m erddaputil ampq
In Docker, specify command: ['ampq'] to run the AMPQ service.
See AMPQ API for more details.