ISSN 2079-3537      

 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                                                                                             

Scientific Visualization, 2018, volume 10, number 1, pages 77 - 88, DOI: 10.26583/sv.10.1.06

The new approach to monitor the workflow management system ProdSys2/PanDA of the ATLAS experiment at LHC by using methods and techniques of visual analytics

Authors: T.GalkinA,1, M.GrigoryevaB,D,2, A.KlimentovB,C,3, T.KorchuganovaD,4, I.MilmanA,5, S.PadolskiC,6, V.PilyuginA,7, D.PopovA,8, M.TitovB,9

A National Research Nuclear University MEPhI, Russia

B National Research Center Kurchatov Institute, Russia

C Brookhaven National Laboratory, USA

D National Research Tomsk Polytechnic University, Russia

1 ORCID: 0000-0003-2859-6275, TPGalkin@mephi.ru

2 ORCID: 0000-0002-8851-2187, magsend@gmail.com

3 ORCID: 0000-0003-2748-4829, Alexei.Klimentov@cern.ch

4 ORCID: 0000-0001-5792-8182, tatiana.korchuganova@cern.ch

5 ORCID: 0000-0001-9705-9401, igalush@gmail.com

6 ORCID: 0000-0002-6795-7670, spadolski@bnl.gov

7 ORCID: 0000-0001-8648-1690, VVPilyugin@mephi.ru

8 ORCID: 0000-0002-3333-749X, DDPopov@mephi.ru

9 ORCID: 0000-0003-2357-7382, mikhail.titov@cern.ch

 

Abstract

The paper presents the pilot project "Tools and methods of visual analytics as part of the workload management system ProdSys2/PanDA of the ATLAS experiment at the Large Hadron Collider." The project is aimed at expanding the functionality of the existing ATLAS monitoring system using a visual analytics approach to analyze large volumes of multidimensional data of computing tasks and jobs. The functioning of ATLAS workflow management system (WMS) is associated with the processing of multi-petabytes and exabytes data volumes. In this case, the new challenges arise that require the use of multi-level interactive visualization tools to analyze the correlations between individual datasets and their representations. The article contains the description of the proposed approaches to visual analysis of multidimensional data, as well as the definition of areas of use of visual analytics applications for data processing in the ATLAS experiment. The tasks for approbation of the recommended methods using the example of the analysis of statistical data of ATLAS WMS are determined. The final part of the article is devoted to the organizational component of the pilot project.

 

Keywords: visual analytics, workflow management system, big data, LHC.