Troubleshooting breakout with Pegasus Team 4-16-08

From CEDPS

Jump to: navigation, search

Attending

Paul G., Ewa D., Karan V. (Pegasus team); Laura P., KSB, BLT, Dan G. (CEDPS)

Discussion

Current Pegasus: Coarse-grained logging of start/stop of Pegasus [components]: (1) mapping the workflow, (2) executing the workflow. DagMan runs the workflow, with each piece wrapped to do start/stop logging of its own.

Desired for Pegasus:

  • use logs to do "real-time" analysis, etc.
  • currently can load logs into mysql
  • also required by a (not-disclosable) project to do fine-grained logging ~ including periodic CPU usage, memory, etc.
  • Pegasus provenance at application level consists mainly of I/O files and executable names
  • Want to trace all places application "went"
  • Desire a "common ontology"

To-do

Pegasus team will send quickstart logs

CEDPS will load into their DB and imitate current Pegasus queries:

  • how many jobs can run in 1 month
  • how long jobs took start – to – end
  • how man jobs run for a given user (in a time period)
  • which/how-many jobs failed for a given user
Retrieved from "http://www.cedps.net/index.php/Troubleshooting_breakout_with_Pegasus_Team_4-16-08"
Personal tools