Data Quality Monitoring Tools
DQM Suite (Test Beam 2004,
Slice Test 2005)
This xDAQ-integrated EMU
DQM Suite is the first step on the way to developing the ultimate
EMU Data Quality Monitoring Project.
At the moment this software suite must be considered as an early EMU
DQM prototype of very limited functionality:
- EMU Monitor module receives data buffers from FED Module via I2O
protocol.
- Raw data are unpacked into ORCA classes.
- Unpacked data are analyzed and DQM histograms are filled.
- DQM histograms periodically passed to Display Server via SOAP
protocol.
- Display GUIs can be used to access the Display Server and browse
through histograms organized in a tree structure of folders.
DDU Binary Data Examiner
(dduBinExaminer.exe)
This standalone program reads a raw DDU binary data file and checks it
event-by-event for event format integrity, which includes
presence and order of headers/trailers (DDU, DMB, ALCT, TMB, CFEB),
word count (DDU, ALCT, TMB, CFEB), and CRC checks (DDU, ALCT, TMB,
CFEB). It is not intended for identifying dead/hot channels, losses of
synchronization, inefficiency of channels or whole boards, etc.
The primary purpose is to check for possible event format structure
corruptions. A user can request a full list
of failing events, i.e. event numbers, for any selected combination of
errors. A summary table of frequency of all errors is generated once
the end of file is reached. Summary plots can be found in Kotov's
report at the Slice Test Meeting at Rice on Aug 16 (see EMU DQM Reports" link).
DDU Event Dump (dduEvtDump.exe)
This standalone program skims through the raw binary data files and
prints out a requested event in hex form, 16x4=64 bits per line.
Header and trailer markers for all boards are added to the printout to
help a user to navigate through the event. This tool is intended
for experts to examine raw bits of corrupted events.
Monitor Test Program
(MonitorTest.exe)
This is a truncated (simplified) version of the TB2004 DQM Suite (see
above). It does not use xDAQ, FED, and Display Server. The raw file is
read directly by the EMU Monitor Module, an event is unpacked and
analyzed same way as in the TB2004 DQM Suite, and, finally the
histograms are written directly on disk. Browsing of histograms can be
done from ROOT or using the HistDisplay GUI. Note a somewhat irritating
feature of this shortcut option: without Display Server in the loop,
the structure of histogram folders is not preserved in this case, i.e.
all histograms are dumped together in one directory with folder names
reinterpreted as part of histogram names.
Library of DQM Test Functions
This library is a collection of custom functions tailored for various
tests to be performed on histograms. It includes Rule-based tests (not
requiring a reference histogram) and Reference-based tests.
This library is developed by UF for general use with the CMS DQM
framework. Link to the actual repository will be provided
shortly.
FileReaderDDU
FileReaderDDU reads file with DDU data, searches for DDU Headers
and DDU Trailers, and finally returns event buffer. It also handles
data corruptions.
FileReaderDDU continuously reads DDU data in blocks of definite
length. Each time it passes through block, it searches for DDU Header
or DDU Trailer. In same time it composes event buffer. Once it find
DDU Trailer it sets an ``end mark'' in current data block and returns
event buffer with event status flag, which tells that we have
DDU Trailer in this event. On next call it continues passing through
data block starting from ``end mark''. If it finds DDU Header pattern
in first two DDU words it sets status flag accordingly and goes further
until it finds DDU Trailer or another DDU Header.
If DDU Header appeared in a place anywhere after first two DDU words,
program sets Unknown bit in status flag and
returns buffer composed with data in front of this DDU Header.
If program finds DDU Header in a right place and encounters
second DDU Header it returns data between these DDU Headers,
sets DDU Header bit in status flag, and starts from second
DDU Header on next call. If buffer size reaches value of 50000 DDU words,
status flag contains DDUoversize bit set.
Described scheme can be presented by following diagrams of
buffer structure:
| 1) | [H-T] | // Normal event |
| 2) | [H-?]H | // More likely event with corrupted trailer |
| 3) | [H-max] | // DDUoversize condition |
| 4) | [?-T] | // Could be anything |
| 5) | [?-?]H | // Could be anything (depends on previous
event) |
| 6) | [?-max] | // Could be anything |
| 7) | [FFFF] | // Some standard sequence to complete ethernet
package |
One remark to be made here is that DDU Header (as well as
DDU Trailer) consists of three words with identifying pattern in
second word. If two consecutive DDU Headers were found, first
word of second Header goes to both events.
For user convenience additional to read function next
was implemented. Together with select, accept, and
reject functions it allows user to get events of specified type.
Function reject helps to filter out events with any of status flag
bits matches defined criteria. Function accept forces
next
to return events with any of status flag bits matches its
criteria. Function select requires events exactly match its
criteria.
Instruction to run Emu DQM Web Publisher manually
Emu DQM Web Publisher is a set of perl/shell/root scripts on various machines.
Its aim is to proceed Emu Local DAQ data and publish plots on the
web. By default every 5
minutes it starts on emuslice11 machine and checks emuslice02:/data/local/
and emuslice11:~/kotov/ locations for new data.
To run Emu DQM on Global DAQ files download to one of lxplus machines
script and run it with arguments coresponding
to Global DAQ run numbers.
Last time
updated: October 18, 2005
