Performance Data Reporting v1
Infrastructure Integration Roadmap Task
Task Type(s): Technology
Start by phase: Integration
Complete by phase: Operations
RP role(s): Metrics and performance data contact(s), System administrator(s)
Summary
ACCESS Metrics provides ACCESS XDMoD and various data reporting and analysis services. XDMoD provides reporting on a wide range of CI metrics. This includes CI efficiency reporting that is available to end users, PIs and RP staff.
The main data source for the efficiency reporting is low-level performance data collected from each CI resource. CI resource providers must periodically transfer a copy of the low-level performance data to ACCESS Metrics so that it can be processed and displayed in XDMoD.
This task involves setting up the data transfer of performance data.
Prerequisite tasks
Support Information
For assistance with this task see the Support Information section in the Integration Roadmap Description.
Detailed Instructions
The low-level performance data files must be periodically sent to ACCESS Metrics. The recommended configuration is to use daily increments data transfers. ACCESS Metrics support three types of data transfer design. The choice of data transfer mechanism depends on the RP preferences:
Globus Transfer with scheduled transfers
Data transfer over ssh push from RP to ACCESS Metrics data transfer node (dtn)
Data transfer over ssh pull from RP with login via the ACCESS Metrics Gateway account (Community User xdtas)
Our preferred mechanism is to use Globus Transfer with scheduled and regularly repeating transfers for the data exchange. To setup Globus Transfer, please provide the name or names of the globus accounts that will be used for data transfer. ACCESS Metrics will create a globus share and give write permissions to the user accounts provided by the RP. Instructions for setting up the periodic transfers using the globus tools are either Globus timer cli tool or the web-based Globus Timers.
If option (2) is chosen then the RP should provide an ssh public key and a preferred username that will be used for the data transfers. ACCESS Metrics will provide the name of the data transfer node (dtn). The RP should then use a cronjob (or similar) to transfer the files each day.
Option (3) can only be used if the resource supports ACCESS community accounts and the “Community User xdtas” is authorized. In this case the xdtas account must have read permissions on the low-level performance data files. ACCESS Metrics will manage the periodic transfers.