Version 2
Copyright © 2004-2012 Delaware Environmental Observing System
Published: September 29th 2004
| Revision History | ||
|---|---|---|
| Revision 1.1 | 2004/09/29 | GEQ |
| Initial version | ||
| Revision 1.2 | 2005/12/16 | GEQ |
| Added configuration file section. | ||
| Revision 1.3 | 2006/07/27 | GEQ |
| Revised for stateful alerting and for version 1.2.2 and 1.3.1 changes. | ||
| Revision 2.1 | 2006/09/26 | GEQ |
| Incremented version number of this document to 2. | ||
| Revision 2.2 | 2006/12/07 | GEQ |
| Added the stream state information and alert information. | ||
| Revision 2.3 | 2007/04/02 | GEQ |
| Added web service user, renamed user to DCF user. | ||
Table of Contents
This document summarizes the design and use of the DEOS Database Control Facility (DCF).
The DCF system utilizes a configuration file for internal settings.
The DCF is the web front-end to the database, and allows the administrator or other regular user to add, change, view and delete items related to the functionality of the DEOS system from their browser.
The DCF has access controls, permitting only a defined set of functionality to be executed by each type of user, either the DEOS administrator, or the regular user.
Each area of control is described in this document as a "class" of functionality, and each class has a narrowly defined range of related capability implemented in the web application.
The following sections detail the specific capabilities assigned to each class of functionality.
This class provides the capability to change DEOS server specific settings.
The data displayed for the information action is the following:
The number of operations settings defined.
The DCF user class includes operations concerning data settings assigned to a single specific individual who is interested in specific data functionality provided by the DEOS system.
The data displayed for the information action is the following:
The number of users defined.
The DCF user class includes operations concerning data settings assigned to a single specific individual
Username: The username is the name by which a DCF user is known to the system.
Locked: This indicates whether the account is allowed to log in to the system.
Administrator: This indicates whether the account is an administrator account having privileges to delete users etc.
Password: These two boxes, if utilized, must contain the same character string, the password for the username.
First Name: The users first name.
Last Name: The users last name.
Description: If filled, provides a description of the DCF user.
Schema Preference: Which schema the DCF user wishes to use for DCF displays and alert contents.
Time Zone: Which time zone/daylight savings setting the DCF user wishes to use for alert contents.
The address information is an assigned street address for the DCF user if available.
Address Name: The address name must be provided.
Description: If filled, provides a description of the address, for example the name of the business or institution.
Street1 and Street2: The street address at this address.
State: The state at this address.
City: The city at this address.
Country: The country at this address.
The contact information is any of an email address, phone number, fax number etc. for the DCF user.
Type: This setting specifies what contact type (Email, phone, mobile or pager) the row of data refers to.
Enabled: This setting indicates whether this particular contact is currently to be used.
Value: This field should contain the contact data itself. [1]
The Web Service (WS) user class includes operations concerning data settings assigned to a single specific individual who is interested in specific WS data functionality provided by the DEOS system.
The data displayed for the information action is the following:
The number of WS users defined.
A table containing rows of data summarizing the individual WS user settings.
The WS user class includes operations concerning data settings assigned to a single specific individual
Organization: The organization the user is affiliated with.
Point of Contact Name: This indicates name of the principal user of the service.
Point of Contact E-mail: This indicates the E-mail address of the principal user of the service.
Description: If filled, provides a description of the WS user.
Enabled: This indicates whether this account is enabled.
Time Span: This indicates the time span, in minutes, over which the user may request data retrieval.
Transform File: This indicates the name of the server-side file of XSLT commands that will be executed for this user.
Key: This users access key.
Account Created: The date and time when the account was created.
Last Data Access: The date and time when the account last accessed web service data.
Web Site Address: The name of the web site (and uses in parentheses) that the user indicated when they signed up.
The alert class includes operations concerning the setting of alerts on data inserted into the database.
Alerts may be used by a single user or a team of users. A user may only modify or delete their own alerts, the team administrator is the only person who can add, modify or delete a team alert.
The data displayed for the information action is the following:
The status of all alerts in the system by the affected stations.
Each alert has a set of parameters required for correct functioning, these are set with the alert class.
Name Of Alert: This is the identifier for this particular alert.
Description: This is the identifier for this particular alert.
Enabled: This indicates if this alert is active.
Station List: Only one list is active for each alert. The station list allows one or more stations to share the same settings.
County List: Only one list is active for each alert. The county list allows one or more stations in the selected county or counties to share the same settings.
Data Type: The data type is the specific meteorological variable to be used in the specified alert, examples might be wind speed or air temperature or precipitation. In the display, the value at the end in parentheses is the native units the database uses to store the data. The entry in the Limit Value is assumed to be in that unit. If values for temperature are entered, use of Fahrenheit or Celsius may be used, as described below.
Comparison: In order for an alert to be fired, a newly measured value must be compared to the alerting value, and also to know whether the alert is to be fired when the new value is more or less than the defined alert value. The comparison provides the latter information. For example, an alert might be set when a measurement is greater than or less than a specific value.
Limit Values: Three alert states are available, one at least needs to be set for a valid alert to be registered.
Timeout: This setting indicates the time that must expire before an alert is sent indicating no state change for problem states.
Duration: For precipitation a duration may be provided. If it is, then the existing data is summed over the duration period when making a comparison with the limit value.
The system events class allows for the display and deletion of system related events.
The data displayed for the information action is the following:
The number of streams defined.
The events class does not have any modifiable settings, but in the view action, the following items describe the fields displayed.
Component: The component of an event is the name of the program generating the event.
Event Type:
Description:
Event Time: The event time is the time that the event was registered and logged.
Process Time: The process time is the measurement time of the data used to generate output.
Run No: The run number is the sequential number of the component run generating the event.
Function: The name of the function in the component registering the event.
The events class allows for the display and deletion of alert events.
The data displayed for the information action is the following:
The number of alert events for each alert ID.
The alert events class does not have any modifiable settings, but in the view action, the following items describe the data provided in the alert information section.
Event Type:
Description:
Time: This field is the time that the event was registered and logged.
The following items describe the data provided in the station alert status section.
Station:
Current State:This will be either OK, Caution, Warning or Critical and color coded.
Last Change:This field is the time that the state last changed.
Last Alert Sent:This field is the time that the user was last alerted.
The team class includes operations concerning data settings assigned to a set of one or more users and those services that each user of the team wishes to subscribe to.
The data displayed for the information action is the following:
The number of teams defined.
The teams class includes operations concerning a team of one or more users.
Team Name:The name by which a team is known to the system. This name must be unique.
Team Type:
Organization:The name of the business or institute comprising the team.
Description:
The address information is an assigned street address for the team if available.
Address Name:The address name must be provided.
Description:If filled, provides a description of the address, for example the name of the business or institution.
Street1 and Street2:The street address at this address.
State:The state at this address.
City:The city at this address.
Country:The country at this address.
Which defined users are members of this team. The team administrator can add or remove users from the team, only the system administrator can change the team administrator.
The three or more geographical points defining the minimum included contiguous bound region.
Location Name:The name of this point.
Latitude:In radians.
Longitude:In radians.
The stations class allows the administrator to configure existing and add new stations to the system.
Until a facility to add a network class is provided, stations may only be added to existing defined streams.
The data displayed for the information action is the following:
Total number of stations defined.
Total number of active stations defined.
A summary table of the station data currently in the database. Columns are:
Station ID
Oldest data available for this station
Newest data available for this station
Number of data items available for this station of all types
Total number of station data items defined.
The stations class includes operations concerning data settings assigned to a single specific meteorological station.
Station Name: The name is the name by which the station is known to the network and will thus appear in data files.
Network: The name of the network this station is a part of.
Description:
Internal ID: For DEOS stations only this value indicates the designated number for this station and is used in the data records generated.
NWS Name: For DEOS stations only this value indicates the name by which the station is known to the NWS and is used in the SHEF files.
Display Name: If a value is provided for this item, the string entered is used in data displays of data from this station, instead of the Station Name.
Retrieve: Indicates whether this station in the network will have data retrieved during stream retrieval.
In Use: Indicates whether this item will provide data for processing.
Commissioned: The commissioning date is the date when the station hardware was placed in situ. On data entry, the format is assumed to be Month/'Day of Month'/Year hour:minute:second in GMT.
Decommissioned: The decommissioning date is the last time a data entry was received from the station. Typically, decommissioning involves removing the in-situ hardware. If the hardware is merely not transmitting data, for example awaiting repairs, then is needn't be given a decommissioned date. On data entry, the format is assumed to be Month/'Day of Month'/Year hour:minute:second in GMT.
Location Name: The location name is required and should contain a unique string describing the location of the station.
Location Description:
Location Type:
Latitude: In radians.
Longitude: In radians.
Elevation: This is the height of the base of the station in meters above mean sea level.
Time Zone Offset: This value is the number of hours that the location is offset from GMT.
Daylight Savings Observed: This value idicates if this station is subject to daylight savings adjustments.
Basin: This entry indicates which watershed the station resides in.
The station data class allows the administrator to review errant data and change its status.
The data displayed for the information action is the following:
The number of flagged data items defined.
Only the "Flagged" item can be changed.
Data Type: The data type is the specific meteorological variable to be used in the specified alert, examples might be wind speed or air temperature or precipitation. In the display, the value at the end in parentheses is the native units the database uses to store the data. The entry in the Limit Value is assumed to be in that unit. If values for temperature are entered, use of Fahrenheit or Celsius may be used, as described below.
Flagged: A flagged data item has a non-zero value in the "Flagged" column signifying that that data item mustn't be used in subsequent data processing.
Station Name: Which station recorded the flagged data item.
Data Value:
Current Maximum: If a measurement is ingested that's larger than the maximum value that data item has the flagged item set on it.
Current Minimum: If a measurement is ingested that's smaller than the minimum value that data item has the flagged item set on it.
The data types class allows the administrator to add new environmental data streams to the database or edit the parameters of existing types.
The data displayed for the information action is the following:
The number of data types defined.
A summary table of pertinent information for all data types containing:
ID
Name
Duration Value (over whattime period the item is generated, 0-instantaneous)
Symbol
Minimum (the value below which the data item is assumed to be physically impossible and is flagged as such)
Maximum (the value above which the data item is assumed to be physically impossible and is flagged as such)
The stations class includes operations concerning data settings assigned to a single specific meteorological station.
Data Type Name: The data type is the specific meteorological variable to be used in the specified alert, examples might be wind speed or air temperature or precipitation. In the display, the value at the end in parentheses is the native units the database uses to store the data. The entry in the Limit Value is assumed to be in that unit. If values for temperature are entered, use of Fahrenheit or Celsius may be used.
Description:
SHEF Code: For DEOS stations only: indicates the code used in the NWS ShEF file for this data type.
USGS Data Descriptor Code: For USGS stations only: indicates the two character data descriptor used in data files.
USGS Data Parameter: For USGS stations only: indicates the five character data parameter used in data files.
Measurement Type: Indicates the nature of the measurement, i.e. instantaneous, time average, etc. more other measurement type or types.
Derived: A derived data type is a measurement that isn't generated by a field deployed instrument but is derived from one or more other measurement type or types.
Alertable: An alertable data type is a measurement that is directed ingested in near real-time and is not derived from one or more other measurement type or types.
Supplementary Data: If a data type has potential additional data ascribed to it, then this flag will be set. An example might be wind gust, where a supplementary datum might be the time the peak gust occurred.
Unit Type: The unit type is the measurement units of the specific meteorological variable. The list indicates the classification, for example a length and a unit, for example mm that the field will be using.
Data Duration: Time span the measurement is applicable for. Instantaneous measurements have duration of zero.
Precision: Indicates the number of decimal places used for this data type/unit type combination.
Scale: Not currently used.
Maximum Value: If a measurement is ingested that's larger than the maximum value that data item has the "Impossible" bit flag set on it.
Minimum Value: If a measurement is ingested that's smaller than the minimum value that data item has the "Impossible" bit flag set on it.
The unit type class allows the administrator to add a new unit type or configure an existing unit type in the system.
The data displayed for the information action is the following:
The number of unit types defined.
The unit types class includes operations concerning data settings assigned to a single specific type of measurement unit.
Unit Type Name: The unit type is the measurement units of the specific meteorological variable. The list indicated the classification, for example a length and a unit, for example mm that the field will be using.
Description:
Schema Type:The schema type is the classification of the units into either the SI or English unit type.
Text-Only Symbol:The symbol to be used in email and other ASCII areas.
Web Symbol:The symbol to be used on web/markup pages.
The counties class allows the administrator to view the counties settings in the database.
The data displayed for the information action is the following:
The total number of counties defined and a table summarizing the number of counties per state.
The cities class allows the administrator to view the cities settings in the database.
The data displayed for the information action is the following:
The total number of cities defined and a table summarizing the number of cities per state.
The field class allows the administrator to add new and alter existing environmental fields to the database.
The data displayed for the information action is the following:
The number of field types defined.
Name: The term field in DEOS refers to a rectangular array of data cells of the same type, for example a percipitation or air temperature field. The field must have a unique name.
Enabled:
Description:
Unit Type: The unit type is the measurement units of the specific meteorological variable. The list indicates the classification, for example a length and a unit, for example mm that the field will be using.
Field Duration: The field duration parameter specifies if the values generated are generated over a span of time. For example a precipitation measurement is usually measured over a time period of anywhere from a few minutes upward.
Precision: Indicates the number of decimal digits used on output for this field/unit type combination.
Scale:Not currently used.
Archive Directory:When the field data files are stored.
Field Generation Settings: What follow are zero or more rows of data settings dependent on the field currently under consideration.
The streams class allows the administrator to add a new stream or configure an existing stream in the system.
The data displayed for the information action is a table containing the following for all in-use networks:
Stream name
Time of last stream state update
Stream state (open or closed)
Time since last state change
If the stream has been open for more than 10 minutes a check box to allow the stream to be reset.
The default installation of the database includes a database job that monitors stream state. If a stream is open for more than 10 minutes an email listing all such streams is sent to all the contacts assigned to the Administrator account in DCF.
The database job ID is stored in the STREAM_STATUS_JOB_ID item in the server settings in case the DBA
wishes to modify or delete this process. The default job runs every five minutes.
Stream Name: A stream is a specific format of data from a network. Each name has to be unique.
Network: The name of the network this station is a part of.
Description:
Retrieve Method:Which data retrieval method this stream uses if any.
In Use: Indicates whether this item will provide data for processing.
Allow Update: Indicates whether this stream allows data already in the database to be overwritten by new data of the same timestamp and data type.
Input Path: The input path is the directory where data files of the specific stream shoudl be placed to allow the ingest program to process them.
Archive Path: The archive path is the directory used to place data files of the specific stream following successful ingestion of the data.
Unprocessed Path: The unprocessed path is the directory used to place data files of the specific stream following unsuccessful ingestion of the data.
Process Time: The process time is the number of minutes past the hour where data processing will occur and therefore when data needs to be stored.
Range Time: The time before and after the stream process time wherein data is to be stored.
The retrieve methods class allows the administrator to add a new retrieval method or configure an existing one.
The data displayed for the information action is the following:
The number of retrieve methods defined.
The retrieve methods class includes operations concerning data settings assigned to a data retrievbal method.
Retrieve Method Name: For data that may be retrieved from an internet source a retrieve method may be assigned from which to get the data.
Protocol: The protocol used in this retrieval method.
Host Name: The machine from which the data is to be retrieved.
User Name: The user on the host machine.
Password: The password of the user on the host machine. Both password entries must be the same.
Remote Directory: The directory on the remote host where the data resides.
Filename: Not currently used.
File Extension: The extension of the file names on the remote host.
The projections class allows the administrator to add a projection to the database or edit a pre-existing projection.
The data displayed for the information action is the following:
The number of projections defined.
The projections class includes operations concerning data settings assigned to an available projection.
Name:A network is a set of one or more physical stations placed in the field. This name must be unique.
Description:
Geographic:Indicates if this projection is based on a rectangular grid in Latitude/Longitude.
Latitude:Center latitude of lower left cell of grid.
Longitude:Center longitude of lower left cell of grid.
Width:Width of the defining cell.
Height:Height of the defining cell.
The network class allows the administrator to add a new network or configure an existing network in the system.
The data displayed for the information action is the following:
The number of networks defined.
The stations class includes operations concerning data settings assigned to a single specific meteorological stat
Network Name: A network is a set of one or more physical stations placed in the field. This name must be unique.
In Use: Whether this network is in use for ingest and retrieval.
Data Feed: Whether data from this network is accessible as a data feed or web service.
Type:
Description:
Report Threshold: Data more recent than this value won't be reported as old in the maintain program data report.
Time Zone Offset:
Daylight Savings Observed:
Data Update Frequency:Indicates the typical data frequency for this network.
Observational Status:If values are provided, colors the appropriate data row in the current network data display.
The protocols class allows the administrator to add a new protocol or configure an existing protocol in the system.
The data displayed for the information action is the following:
The number of protocols defined.
The following table describes the capabilities of the DEOS DCF.
Table 1. DCF Capabilities
| Class | Action [a] | Admin [b] | User [c] | Notes |
|---|---|---|---|---|
| Settings | Add | |||
| View | X | |||
| Modify | X | |||
| Delete | ||||
| DCF Users | Add | X | ||
| View | X | X | For a non-admin user, only that users' settings. | |
| Modify | X | X | For a non-admin user, only that users' settings. | |
| Delete | X | |||
| WS (Web Service) Users | Add | X | X | Goes to a web page external to the DCF for web service subscription information |
| View | X | For a non-admin user, only that users' settings. | ||
| Modify | X | |||
| Delete | X | |||
| Alerts | Add | X | X | The administrator cannot add alerts for other users. |
| View | X | X | The administrator can view alerts for other users. | |
| Modify | X | X | The administrator can modify alerts for other users. | |
| Delete | X | X | The administrator can delete alerts for other users. | |
| System Events (Formerly ELF Events) | Add | N/A | ||
| View | X | X | Users can view all or only error events. | |
| Modify | N/A | |||
| Delete | X | |||
| Alert Events | Add | N/A | ||
| View | X | X | ||
| Modify | N/A | |||
| Delete | X | X | Alerts events for an active alert cannot be deleted by the user. The administrator can delete alert events in this situation. | |
| Teams | Add | X | ||
| View | X | X | ||
| Modify | X | Only the system or team administrators can make modifications. | ||
| Delete | X | Only the system administrator can delete a team. | ||
| Station | Add | X | ||
| View | X | X | ||
| Modify | X | |||
| Delete | X | |||
| Station Data | Add | |||
| View | X | X | ||
| Modify | ||||
| Delete | X | |||
| Data Types | Add | The ID of data types is hardcoded and so addition of new data types is disabled. | ||
| View | X | X | ||
| Modify | X | |||
| Delete | The ID of data types is hardcoded and so deletion of new data types is disabled. | |||
| Unit Types | Add | |||
| View | X | X | ||
| Modify | X | |||
| Delete | ||||
| Counties | Add | |||
| View | X | X | ||
| Modify | ||||
| Delete | ||||
| Cities | Add | |||
| View | X | X | ||
| Modify | ||||
| Delete | ||||
| Field | Add | |||
| View | X | X | ||
| Modify | X | |||
| Delete | ||||
| Streams | Add | X | ||
| View | X | X | ||
| Modify | X | |||
| Delete | X | |||
| Retrieve Methods | Add | X | ||
| View | X | X | ||
| Modify | X | |||
| Delete | X | |||
| Projections | Add | X | ||
| View | X | X | ||
| Modify | X | |||
| Delete | X | |||
| Networks | Add | |||
| View | X | X | ||
| Modify | X | |||
| Delete | ||||
| Protocols | Add | X | ||
| View | X | X | ||
| Modify | X | |||
| Delete | X | |||
[a] The "Default" action (obtained by clicking the class name) for users is a help screen, for an administrator, it is an information screen as described above. Neither capability is described in the table. [b] An 'X' appears for actions that are available to the administrator. [c] An 'X' appears for actions that are available to the DCF user for their account only. | ||||
This section describes the design of the interface that implements the DCF.
This section describes the screen layout in terms of frames making up the display. There are three frames.
The top of the screen consists of a narrow frame across the width of the window giving general information, such as the user name, time and other similar information.
The left side of the screen consists of a frame containing the list of classes and actions available for each class in a scrolling window. The "Information action for each class is implemented by clicking on the class name, all other actions are implemented by clicking on the appropriate action link. The action links will be indented below each class name for easier visual navigation.
[1] Alerts to mobile phones in DEOS are implemented using regular email services. Check with your mobile service provider to determine how to use this type of contact.