{"id":12580,"date":"2023-07-12T15:38:06","date_gmt":"2023-07-12T13:38:06","guid":{"rendered":"https:\/\/new.sano.science\/?post_type=research&#038;p=12580"},"modified":"2024-01-05T13:58:30","modified_gmt":"2024-01-05T12:58:30","slug":"declarative-big-data-analysis-for-high-energy-physics-totem-use-case","status":"publish","type":"research","link":"https:\/\/sano.science\/research\/declarative-big-data-analysis-for-high-energy-physics-totem-use-case\/","title":{"rendered":"Declarative Big Data Analysis for High-Energy Physics: TOTEM Use Case\u00a0"},"content":{"rendered":"\n<h2 class=\"wp-block-heading eplus-wrapper\">Avati, Valentina; Blaszkiewicz, Milosz; Bocchi, Enrico; Canali, Luca; Castro, Diogo; Cervantes, Javier; Grzanka, Leszek; Guiraud, Enrico; Kaspar, Jan; Kothuri, Prasanth; Lamanna, Massimo; Malawski, Maciej; Mnich, Aleksandra; Moscicki, Jakub; Murali, Shravan; Piparo, Danilo; Tejedor, Enric<\/h2>\n\n\n\n<div style=\"height:50px\" aria-hidden=\"true\" class=\"wp-block-spacer eplus-wrapper\"><\/div>\n\n\n\n<p class=\" eplus-wrapper\">The High-Energy Physics community faces new data processing challenges caused by the expected growth of data resulting from the upgrade of LHC accelerator. These challenges drive the demand for exploring new approaches for data analysis. In this paper, we present a new declarative programming model extending the popular ROOT data analysis framework, and its distributed processing capability based on Apache Spark. The developed framework enables high-level operations on the data, known from other big data toolkits, while preserving compatibility with existing HEP data files and software. In our experiments with a real analysis of TOTEM experiment data, we evaluate the scalability of this approach and its prospects for interactive processing of such large data sets. Moreover, we show that the analysis code developed with the new model is portable between a production cluster at CERN and an external cluster hosted in the Helix Nebula Science Cloud thanks to the bundle of services of Science Box.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In: Yahyapour, Ramin (Ed.): Euro-Par 2019: Parallel Processing, pp. 241\u2013255, Springer International Publishing, Cham, 2019, ISBN: 978-3-030-29400-7.<\/p>\n","protected":false},"featured_media":0,"template":"","research_type":[8],"research_team":[16],"class_list":["post-12580","research","type-research","status-publish","hentry","research_type-publications","research_team-extreme-scale-data-and-computing"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v27.4 (Yoast SEO v27.4) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Declarative Big Data Analysis for High-Energy Physics: TOTEM Use Case\u00a0 - Centre for Computational Personalized Medicine<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/sano.science\/research\/declarative-big-data-analysis-for-high-energy-physics-totem-use-case\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Declarative Big Data Analysis for High-Energy Physics: TOTEM Use Case\u00a0\" \/>\n<meta property=\"og:description\" content=\"In: Yahyapour, Ramin (Ed.): Euro-Par 2019: Parallel Processing, pp. 241\u2013255, Springer International Publishing, Cham, 2019, ISBN: 978-3-030-29400-7.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/sano.science\/research\/declarative-big-data-analysis-for-high-energy-physics-totem-use-case\/\" \/>\n<meta property=\"og:site_name\" content=\"Centre for Computational Personalized Medicine\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/sano.science\/\" \/>\n<meta property=\"article:modified_time\" content=\"2024-01-05T12:58:30+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@sanoscience\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/sano.science\\\/research\\\/declarative-big-data-analysis-for-high-energy-physics-totem-use-case\\\/\",\"url\":\"https:\\\/\\\/sano.science\\\/research\\\/declarative-big-data-analysis-for-high-energy-physics-totem-use-case\\\/\",\"name\":\"Declarative Big Data Analysis for High-Energy Physics: TOTEM Use Case\u00a0 - Centre for Computational Personalized Medicine\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/sano.science\\\/#website\"},\"datePublished\":\"2023-07-12T13:38:06+00:00\",\"dateModified\":\"2024-01-05T12:58:30+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/sano.science\\\/research\\\/declarative-big-data-analysis-for-high-energy-physics-totem-use-case\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/sano.science\\\/research\\\/declarative-big-data-analysis-for-high-energy-physics-totem-use-case\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/sano.science\\\/research\\\/declarative-big-data-analysis-for-high-energy-physics-totem-use-case\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/sano.science\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Research\",\"item\":\"https:\\\/\\\/sano.science\\\/research\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Publications\",\"item\":\"https:\\\/\\\/sano.science\\\/research-type\\\/publications\\\/\"},{\"@type\":\"ListItem\",\"position\":4,\"name\":\"Declarative Big Data Analysis for High-Energy Physics: TOTEM Use Case\u00a0\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/sano.science\\\/#website\",\"url\":\"https:\\\/\\\/sano.science\\\/\",\"name\":\"Centre for Computational Personalized Medicine\",\"description\":\"Sano \u2013 Centre for Computational Medicine\",\"publisher\":{\"@id\":\"https:\\\/\\\/sano.science\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/sano.science\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/sano.science\\\/#organization\",\"name\":\"Sano \u2013 Centre for Computational Medicine\",\"alternateName\":\"Sano\",\"url\":\"https:\\\/\\\/sano.science\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/sano.science\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/sano.science\\\/wp-content\\\/uploads\\\/2024\\\/05\\\/logo_sano_podstawowe.png\",\"contentUrl\":\"https:\\\/\\\/sano.science\\\/wp-content\\\/uploads\\\/2024\\\/05\\\/logo_sano_podstawowe.png\",\"width\":700,\"height\":265,\"caption\":\"Sano \u2013 Centre for Computational Medicine\"},\"image\":{\"@id\":\"https:\\\/\\\/sano.science\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/sano.science\\\/\",\"https:\\\/\\\/x.com\\\/sanoscience\",\"https:\\\/\\\/www.linkedin.com\\\/company\\\/sanoscience\\\/\",\"https:\\\/\\\/www.youtube.com\\\/channel\\\/UCDZ_8TcjMWUG2ZcgKKgfpwQ\",\"https:\\\/\\\/bsky.app\\\/profile\\\/sanoscience.bsky.social\"]}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Declarative Big Data Analysis for High-Energy Physics: TOTEM Use Case\u00a0 - Centre for Computational Personalized Medicine","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/sano.science\/research\/declarative-big-data-analysis-for-high-energy-physics-totem-use-case\/","og_locale":"en_US","og_type":"article","og_title":"Declarative Big Data Analysis for High-Energy Physics: TOTEM Use Case\u00a0","og_description":"In: Yahyapour, Ramin (Ed.): Euro-Par 2019: Parallel Processing, pp. 241\u2013255, Springer International Publishing, Cham, 2019, ISBN: 978-3-030-29400-7.","og_url":"https:\/\/sano.science\/research\/declarative-big-data-analysis-for-high-energy-physics-totem-use-case\/","og_site_name":"Centre for Computational Personalized Medicine","article_publisher":"https:\/\/www.facebook.com\/sano.science\/","article_modified_time":"2024-01-05T12:58:30+00:00","twitter_card":"summary_large_image","twitter_site":"@sanoscience","twitter_misc":{"Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/sano.science\/research\/declarative-big-data-analysis-for-high-energy-physics-totem-use-case\/","url":"https:\/\/sano.science\/research\/declarative-big-data-analysis-for-high-energy-physics-totem-use-case\/","name":"Declarative Big Data Analysis for High-Energy Physics: TOTEM Use Case\u00a0 - Centre for Computational Personalized Medicine","isPartOf":{"@id":"https:\/\/sano.science\/#website"},"datePublished":"2023-07-12T13:38:06+00:00","dateModified":"2024-01-05T12:58:30+00:00","breadcrumb":{"@id":"https:\/\/sano.science\/research\/declarative-big-data-analysis-for-high-energy-physics-totem-use-case\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/sano.science\/research\/declarative-big-data-analysis-for-high-energy-physics-totem-use-case\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/sano.science\/research\/declarative-big-data-analysis-for-high-energy-physics-totem-use-case\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/sano.science\/"},{"@type":"ListItem","position":2,"name":"Research","item":"https:\/\/sano.science\/research\/"},{"@type":"ListItem","position":3,"name":"Publications","item":"https:\/\/sano.science\/research-type\/publications\/"},{"@type":"ListItem","position":4,"name":"Declarative Big Data Analysis for High-Energy Physics: TOTEM Use Case\u00a0"}]},{"@type":"WebSite","@id":"https:\/\/sano.science\/#website","url":"https:\/\/sano.science\/","name":"Centre for Computational Personalized Medicine","description":"Sano \u2013 Centre for Computational Medicine","publisher":{"@id":"https:\/\/sano.science\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/sano.science\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/sano.science\/#organization","name":"Sano \u2013 Centre for Computational Medicine","alternateName":"Sano","url":"https:\/\/sano.science\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/sano.science\/#\/schema\/logo\/image\/","url":"https:\/\/sano.science\/wp-content\/uploads\/2024\/05\/logo_sano_podstawowe.png","contentUrl":"https:\/\/sano.science\/wp-content\/uploads\/2024\/05\/logo_sano_podstawowe.png","width":700,"height":265,"caption":"Sano \u2013 Centre for Computational Medicine"},"image":{"@id":"https:\/\/sano.science\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/sano.science\/","https:\/\/x.com\/sanoscience","https:\/\/www.linkedin.com\/company\/sanoscience\/","https:\/\/www.youtube.com\/channel\/UCDZ_8TcjMWUG2ZcgKKgfpwQ","https:\/\/bsky.app\/profile\/sanoscience.bsky.social"]}]}},"acf":[],"gutenberg_blocks":[{"blockName":"custom-styles","attrs":{"styles":""}},{"blockName":"core\/heading","attrs":{"epAnimationGeneratedClass":"edplus_anim-oma9Pp","epGeneratedClass":"eplus-wrapper"},"innerBlocks":[],"innerHTML":"\n<h2 class=\"wp-block-heading eplus-wrapper\">Avati, Valentina; Blaszkiewicz, Milosz; Bocchi, Enrico; Canali, Luca; Castro, Diogo; Cervantes, Javier; Grzanka, Leszek; Guiraud, Enrico; Kaspar, Jan; Kothuri, Prasanth; Lamanna, Massimo; Malawski, Maciej; Mnich, Aleksandra; Moscicki, Jakub; Murali, Shravan; Piparo, Danilo; Tejedor, Enric<\/h2>\n","innerContent":["\n<h2 class=\"wp-block-heading eplus-wrapper\">Avati, Valentina; Blaszkiewicz, Milosz; Bocchi, Enrico; Canali, Luca; Castro, Diogo; Cervantes, Javier; Grzanka, Leszek; Guiraud, Enrico; Kaspar, Jan; Kothuri, Prasanth; Lamanna, Massimo; Malawski, Maciej; Mnich, Aleksandra; Moscicki, Jakub; Murali, Shravan; Piparo, Danilo; Tejedor, Enric<\/h2>\n"]},{"blockName":"core\/spacer","attrs":{"height":"50px","epAnimationGeneratedClass":"edplus_anim-ItVSrY","epGeneratedClass":"eplus-wrapper"},"innerBlocks":[],"innerHTML":"\n<div style=\"height:50px\" aria-hidden=\"true\" class=\"wp-block-spacer eplus-wrapper\"><\/div>\n","innerContent":["\n<div style=\"height:50px\" aria-hidden=\"true\" class=\"wp-block-spacer eplus-wrapper\"><\/div>\n"]},{"blockName":"core\/paragraph","attrs":{"epAnimationGeneratedClass":"edplus_anim-ctAHD5","epGeneratedClass":"eplus-wrapper"},"innerBlocks":[],"innerHTML":"\n<p class=\" eplus-wrapper\">The High-Energy Physics community faces new data processing challenges caused by the expected growth of data resulting from the upgrade of LHC accelerator. These challenges drive the demand for exploring new approaches for data analysis. In this paper, we present a new declarative programming model extending the popular ROOT data analysis framework, and its distributed processing capability based on Apache Spark. The developed framework enables high-level operations on the data, known from other big data toolkits, while preserving compatibility with existing HEP data files and software. In our experiments with a real analysis of TOTEM experiment data, we evaluate the scalability of this approach and its prospects for interactive processing of such large data sets. Moreover, we show that the analysis code developed with the new model is portable between a production cluster at CERN and an external cluster hosted in the Helix Nebula Science Cloud thanks to the bundle of services of Science Box.<\/p>\n","innerContent":["\n<p class=\" eplus-wrapper\">The High-Energy Physics community faces new data processing challenges caused by the expected growth of data resulting from the upgrade of LHC accelerator. These challenges drive the demand for exploring new approaches for data analysis. In this paper, we present a new declarative programming model extending the popular ROOT data analysis framework, and its distributed processing capability based on Apache Spark. The developed framework enables high-level operations on the data, known from other big data toolkits, while preserving compatibility with existing HEP data files and software. In our experiments with a real analysis of TOTEM experiment data, we evaluate the scalability of this approach and its prospects for interactive processing of such large data sets. Moreover, we show that the analysis code developed with the new model is portable between a production cluster at CERN and an external cluster hosted in the Helix Nebula Science Cloud thanks to the bundle of services of Science Box.<\/p>\n"]}],"meta_data":{"is_automatically_other_posts":true,"number_of_posts":"3","is_automatically_check_also_posts":true},"_links":{"self":[{"href":"https:\/\/sano.science\/index.php\/wp-json\/wp\/v2\/research\/12580","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sano.science\/index.php\/wp-json\/wp\/v2\/research"}],"about":[{"href":"https:\/\/sano.science\/index.php\/wp-json\/wp\/v2\/types\/research"}],"version-history":[{"count":2,"href":"https:\/\/sano.science\/index.php\/wp-json\/wp\/v2\/research\/12580\/revisions"}],"predecessor-version":[{"id":14733,"href":"https:\/\/sano.science\/index.php\/wp-json\/wp\/v2\/research\/12580\/revisions\/14733"}],"wp:attachment":[{"href":"https:\/\/sano.science\/index.php\/wp-json\/wp\/v2\/media?parent=12580"}],"wp:term":[{"taxonomy":"research_type","embeddable":true,"href":"https:\/\/sano.science\/index.php\/wp-json\/wp\/v2\/research_type?post=12580"},{"taxonomy":"research_team","embeddable":true,"href":"https:\/\/sano.science\/index.php\/wp-json\/wp\/v2\/research_team?post=12580"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}