Steve Malone
E-mail: steve@geophys.washington.edu
Geophyics AK-50
University of Washington
Seattle, WA 98195
Phone: (206) 685-3811
Fax: (206) 543-0489

FOREWORD

There are many cases in which a simple solution is developed to solve a specific problem and is potentially generally useful to many. Less often, that solution is recognized and used by others without their reinventing their own way of doing it. The seismic AutoDRM (Automatic Data Request Manager) first developed by Urs Kradolfer of the Swiss Seismological Service is such a solution. This month the "Electronic Seismologist" hosts an article describing the development of this system and some thoughts on its future.

Formalizing a simple way to use electronic mail to request and receive seismic waveform data has solved some of Urs' original problems as well as provided a simple, though highly effective, data-acquisition technique for the seismic community. This system has continued to develop, becoming a standard in the GSE community, and now has a life of its own. It has gone through a major upgrade to a second version which, unfortunately, is not as upward-compatible with the first version as one would hope. Nevertheless, it is a robust, useful player in one's bag of tricks for accessing seismic data from the global community.

AutoDRM: THE FIRST FIVE YEARS

Urs Kradolfer
Swiss Seismological Service
ETH-Hoenggerberg
CH-8093 Zurich
E-mail: urs@seismo.ifg.ethz.ch

Introduction and Background

Looking out through the huge windows in a cafeteria at the United Nations buildings in Geneva, gazing on high Alpine mountains, sipping an orange juice and talking with a seismologist friend--these were the initial steps to the AutoDRM concept. With Manfred Henger, the German delegate in the Group of Scientific Experts (GSE) at the Conference of Disarmament, I was enjoying a break in the meetings, and we were discussing how seismologists might easily access data from other seismological observatories. At that time, March 1989, the Group of Scientific Experts had on its agenda the preparations for the second international data exchange experiment GSETT-2. The German delegation was once again proposing an "Open Station Concept", where one could easily access any observatory's computer and interactively log in and view all data available there. The program that allowed such browsing of the data was called Data Request Manager (DRM) and had already been installed for quite a while at the German National Data Center (NDC).

After many fruitful discussions Manfred Henger and I came to the conclusion that, in the future, it would be nice if one could send electronic mail to an observatory and subsequently get a response in the form of an e-mail message with the requested data.

It took about two years (there is a persistent saying in Switzerland that people originating from Berne are mentally slow) for me to digest this discussion and start to code a new program, called Automatic Data Request Manager (AutoDRM). During this work I was strongly influenced by two other experiences. First, several people used to ask the Swiss Seismological Service for digital waveform data. Negotiating the format and the media for transmission, as well as finally packing the media into a parcel and filling out the customs form, made me increasingly sick and tired, especially if I soon got another request for the same time span but different channels of data. Second, during the GSETT-2 experiment I experienced the tedious undertaking of properly formatting every message to the Experimental International Data Center. It essentially took a computer program to submit a request for data in the proper format. These frustrations motivated the initial version of AutoDRM. The underlying philosophy is to let any user submit a request for data to an observatory using a very simple command language that is, from a syntactical point of view, simple enough so that no program is needed to request data.

Already with the first version of the AutoDRM (presented to the GSE in August 1991) simple requests for waveforms were possible. For example, to request waveform data from two stations with the codes APL and DIX for the interval from 21 June 1991 00:12 to 00:13 UTC one simply sends an e-mail message to autodrm@seismo.ifg.ethz.ch with the following text:
   begin
   date1 199106210012
   date2 199106210013
   wavef apl
   wavef dix
   email user@host.domain
   stop

With this simple syntax it is rather easy to set up and send request messages either by hand or with a computer program. Upon the arrival of such messages our AutoDRM reads and processes the request within one minute and produces a response message by scheduling locally already existing programs to retrieve the desired data. This message is then sent to the requester's e-mail address given in the request. The reason for choosing e-mail as the main communications method was the fact that, at least in those days (early '90's), many European and Asian observatories did have e-mail access but had no direct link to the "real" Internet. The concept of how an AutoDRM works has remained unchanged since then, although other features (e.g., transfer of the requested data via FTP) have been added during the years (see Figure 1).

Developments

During the first few months of operation in 1991, the Swiss AutoDRM typically received about thirty requests per month. They were mostly from European colleagues who requested locations for earthquakes at local, regional, and teleseismic distances. Fewer users requested waveform data, calibration information, or P-phase arrival data. I felt that having written a program that was, on average, used once every day by external users was already a success. I did not imagine that this access rate would increase by two orders of magnitude.

After a presentation of the AutoDRM concept at the XXXIII General Assembly of the European Seismological Commission in 1992 in Prague, Czech Republic, other seismological observatories asked for the AutoDRM code and installed this software on their computers. Compared with today's code, I must apologize to those early recipients for the badly structured code they got. The early versions did not separate the more general parts of the code from the site-specific parts, and so local adaptation was difficult. It is proof of the computer talent of people like Reinoud Sleeman and Ray Buland that they have very successfully adapted the old AutoDRM code to their local environments. It was during this period (1993-1994) that many colleagues around the world made valuable suggestions for improving and extending AutoDRM. Bernard Dost and Tim Ahern encouraged me to add the feature that a response may be automatically ftp'ed to the requester's machine. The status of AutoDRM at that time is described in Kradolfer (1993).

Probably the most important year for the further development and the rapid propagation of AutoDRM to many different places was 1994. During the January 1994 IASPEI meeting in Wellington, New Zealand, the Federation of Digital Seismograph Networks (FDSN) acknowledged that AutoDRM had become a de facto standard for seismological data exchange, and members were encouraged to install AutoDRM's.

Shortly afterward, the GSE, preparing the Third Technical Test (GSETT-3), another large-scale experiment on international seismic data exchange, agreed that AutoDRM would be one of the allowed interfaces for those stations, where the Experimental International Data Center (IDC) would retrieve waveform data segments (today such stations are called auxiliary stations, while primary stations are transmitting a continuous data stream to the IDC) in order to improve both location and characterization of the events detected and preliminarily located using data from the primary stations. However, due to the anticipated large amount of requests to be sent by the IDC to individual stations (or the National Data Centers operating them) it was necessary to develop an extended and more formal AutoDRM command language.

A dedicated format working group was established. Within only a couple of months--partly due to the guidance and great efforts of Jerry Carter at the Center for Monitoring Research (CMR) in Arlington, Virginia--the new protocols and formats for the data exchange were formalized. These are now referred to as GSE2.0 format (Group of Scientific Experts, 1995). An important extension is the possibility that the new message formats allow the exchange--in addition, e.g., to waveform data in the GSE format--of seismic data in other established formats, such as CSS or SEED. Also, the new formats include message identifications in order to enable message tracking, which is an important feature, especially for agencies such as the IDC, where lots of messages are transmitted and received daily.

In the new GSE2.0 format, the sample request message given above would appear as follows:
   begin gse2.0
   msg_type request
   msg_id any_ID_allowed GSE_IDC
   time 1991/06/21 00:12:0.0 to 1991/6/21 00:13
   sta_list apl, dix
   chan_list *Z
   waveform gse2.0
   e-mail user@host.domain
   stop

This example demonstrates that the new format has more flexibility in terms of possible details (e.g., specification of a channel list) and includes better tracking of requests.

Current Status and Documentation

Currently there are about twenty AutoDRM's in operation worldwide that are providing data and supporting the GSE2.0 format (Table 1). There are still other AutoDRM's which still support the old, version 1 format but have not yet switched to the new format. A user may send a request to any of these AutoDRM's with the command HELP and will then automatically receive more information about that specific AutoDRM. This list is updated regularly and can be accessed on the World Wide Web (WWW) at http://seismo.ethz.ch/autodrm.html. It is likely that there are more AutoDRM's already installed and actually providing seismic data. If you wish your AutoDRM to be included in this list, please contact the author of this article (urs@seismo.ifg.ethz.ch). It must be emphasized here that, although each of these AutoDRM's follows the agreed formats, not each of them is based on the Swiss AutoDRM source code. Many people have either written their own code or have heavily modified the Swiss code, and I think they have done a wonderful job. For those who would like to install an AutoDRM, the code of the newest version of the Swiss AutoDRM is available via anonymous FTP at ftp://seismo.ethz.ch (user=ftp, password=ftp) in the directory pub/gse. This version runs on most common UNIX platforms, and a first installation of the software package is only a matter of minutes, since the system-dependent configuration is now done fully automatically.

Information about the AutoDRM and the complete and official GSE2.0 formats may also be obtained through WWW at http://seismo.ethz.ch/autodrm.html, where the documentation may also be directly retrieved in the form of a PostScript file. This documentation also contains a chapter about the minimum requirements for an AutoDRM. I must also clarify that the GSE2.0 format not only applies to digital waveform formats, but also includes the details of many other formats including the request syntax to AutoDRM's.

During an AutoDRM seminar in 1995 at the ETH in Zurich, where eighteen people from twelve countries participated, it was agreed that AutoDRM's should have an e-mail address starting with autodrm@... and should be able to receive and transmit messages in GSE2.0 format. As an absolute minimum, they should understand the command HELP and then return information about the commands understood by that AutoDRM. Today, some AutoDRM's, at least those based on the Swiss code, are backward-compatible so that the original commands such as "date1" and "wavef" are still supported. However, I think users should prepare themselves to switch to GSE2.0 format now in order to be sure that their request is understood by all AutoDRM's installed today.

Outlook

Where will the AutoDRM concept go? These days there are many observatories that are installing new data distribution techniques. The AutoDRM concept still is the only e-mail-based one that is very widespread and relatively well documented. In particular, it is very well suited for a fully automatic data exchange. People often tell me that one of the most enjoyable features of the AutoDRM is that they can quickly submit a request for data and that they then may forget about it, because they know that within a reasonable amount of time they will obtain the requested data. Retrieving data by direct access often takes more of an individual's time and may be tedious or completely unsuccessful in case of heavily overloaded or interrupted Internet links. See, for example, the article by Malone (1996) about his experience with the bottleneck in the jump across the Atlantic. An e-mail-based system is much more reliable in such circumstances, because e-mails are spooled in cases of traffic jams until the link is up again.

Some people have told me that they would never allow someone to access their computers directly because of security considerations. AutoDRM is a comparatively very secure tool because the remote user never directly accesses the computer. Unknown or unauthorized commands are not executed by the program, so the only likely security problems are because the program sendmail is running.

Current developments regarding the future of the AutoDRM concept are being discussed during 1996 by the GSE format working group led by Jerry Carter. An important issue is the structuralization of the AutoDRM commands into core and supplemental keywords. This will facilitate the work of those who are implementing a new, or maintaining an already existing, AutoDRM at their sites.

There is a very promising development at the European-Mediterranean Seismological Center (EMSC) in Paris, France, where Bruno Feignier and Frederic Ramon are testing an AutoDRM that will be capable of sending requests to other AutoDRM's in case the requested data is not available at that center. This would facilitate the work of seismologists, because they could then send a single request to one AutoDRM asking for data stored at different observatories.

Given the increasing number of AutoDRM's, one might ask whether their presence, or rather their usage, might affect the Internet or not. Certainly, the number of requests (and each request usually results in one response) to AutoDRM's have significantly increased during the past five years. During March 1996, the GSE-IDC has sent about 68,000 requests distributed to thirteen distinct AutoDRM's, which gives them access to sixty-two stations. Also during March 1996, our AutoDRM in Zurich received 3,000 requests, and the IDC in Arlington, VA received more than 4,000 requests. The U.S. National Seismograph Network operated by the NEIC in Golden, Colorado, currently receives about 10,000 requests each month. However, I do share the opinion of Steve Malone (1996) that the seismological users will become a smaller and smaller fraction of the total Internet bandwidth compared with other users.

To conclude, I have positive thoughts and would guess that there are many more years of AutoDRM usage ahead of us. While some of the details may evolve and change, the basic concept is sound and will continue to be the basis for the exchange of many seismograms.

References

Group of Scientific Experts (1995). GSETT-3 Documentation, Conference Room Paper/243, Conference on Disarmament, United Nations, Geneva, Switzerland.

Kradolfer, U. (1993). Automating the exchange of earthquake information. Eos, Trans. Am. Geophys. U. 74, 442, 444-445.

Malone, S. (1996). The International Internet--Global connectivity. Seism. Res. Let. 67, 28.

SRL encourages guest columnists to contribute to the "Electronic Seismologist." Please contact Steve Malone with your ideas. His e-mail address is steve@geophys.washington.edu.

Posted: 6 April 1999