{"id":10655,"date":"2022-11-17T09:44:00","date_gmt":"2022-11-17T08:44:00","guid":{"rendered":"https:\/\/sano.science\/?p=10655"},"modified":"2024-02-26T15:55:21","modified_gmt":"2024-02-26T14:55:21","slug":"non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa","status":"publish","type":"post","link":"https:\/\/sano.science\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\/","title":{"rendered":"Non-Invasive Dynamic FaaS Infrastructure with HPC-Whisk \u2013 Sano in Dallas, USA"},"content":{"rendered":"\n<p class=\" eplus-wrapper\">Maciej Malawski, the Director of Sano Science and team leader of the Extreme-scale Data and Computing research group in Sano is one of the authors of the paper presented at SC22 conference.&nbsp;<br><br>The presentation was on Wednesday, November 16th, and was titled \u201cNon-Invasive Dynamic FaaS Infrastructure with HPC-Whisk\u201d.&nbsp;<br><br><strong>As we could read in abstract:&nbsp;<\/strong><br><br>Modern HPC workload managers and their careful tuning contribute to the high utilization of HPC clusters. However, due to inevitable uncertainty it is impossible to completely avoid node idleness. Although such idle slots are usually too short for any HPC job, they are too long to ignore them. Function-as-a-Service (FaaS) paradigm promisingly fills this gap, and can be a good match, as typical FaaS functions last seconds, not hours. Here we show how to build a FaaS infrastructure on idle nodes in an HPC cluster in such a way that it does not affect the performance of the HPC jobs significantly. We dynamically adapt to a changing set of idle physical machines, by integrating open-source software Slurm and OpenWhisk.&nbsp;<br><br>We designed and implemented a prototype solution that allowed us to cover up to 90% of the idle time slots on a 50k-core cluster that runs production workloads.&nbsp;<br><br>Congratulations!<\/p>\n\n\n\n<p class=\" eplus-wrapper\"><br>Also Marek Michalewicz, PhD, Sano&#8217;s Visiting Professor was in Dallas at SC Conference Series and he was one of the Wednesday session &#8220;Bridging the HPC\/Data Divide&#8221; leaders.&nbsp;<br><br><strong>More details about this session<\/strong>: <a href=\"https:\/\/lnkd.in\/dV7BVjS7&nbsp;\">https:\/\/lnkd.in\/dV7BVjS7&nbsp;<\/a><\/p>\n\n\n\n<div style=\"height:50px\" aria-hidden=\"true\" class=\"wp-block-spacer eplus-wrapper\"><\/div>\n\n\n\n    <div class=\"grid-of-images grid grid-cols-1 md:grid-cols-2 gap-24px lg:gap-32px auto-rows-auto\">\n\n        \n            \n    \n    <a\n            data-fslightbox=\"gallery\"\n            class=\"lightbox-image rounded-20px overflow-hidden\"\n            href=\"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221114_185026-Male.jpg\"\n            title=\"Press to open photo preview\"\n            aria-label=\"Press to open photo preview\"\n    >\n\n        \n    \n        \n\t\n\t\n\t\t\n\t\t<figure class=\"overflow-hidden h-full\" aria-label=\"\" title=\"\">\n\n\t\t\t<img decoding=\"async\" src=\"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221114_185026-Male.webp\" srcset=\"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221114_185026-Male.jpg 640w, https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221114_185026-Male-300x225.jpg 300w\" alt=\"\" class=\"h-full w-full object-cover\" loading=\"lazy\"\/>\n\n\t\t<\/figure>\n\n\t\n\n    \n\n    <\/a>\n\n\n        \n            \n    \n    <a\n            data-fslightbox=\"gallery\"\n            class=\"lightbox-image rounded-20px overflow-hidden\"\n            href=\"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221115_090149-Male.jpg\"\n            title=\"Press to open photo preview\"\n            aria-label=\"Press to open photo preview\"\n    >\n\n        \n    \n        \n\t\n\t\n\t\t\n\t\t<figure class=\"overflow-hidden h-full\" aria-label=\"\" title=\"\">\n\n\t\t\t<img decoding=\"async\" src=\"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221115_090149-Male.webp\" srcset=\"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221115_090149-Male.jpg 640w, https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221115_090149-Male-300x225.jpg 300w\" alt=\"\" class=\"h-full w-full object-cover\" loading=\"lazy\"\/>\n\n\t\t<\/figure>\n\n\t\n\n    \n\n    <\/a>\n\n\n        \n            \n    \n    <a\n            data-fslightbox=\"gallery\"\n            class=\"lightbox-image rounded-20px overflow-hidden\"\n            href=\"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_121706-Male.jpg\"\n            title=\"Press to open photo preview\"\n            aria-label=\"Press to open photo preview\"\n    >\n\n        \n    \n        \n\t\n\t\n\t\t\n\t\t<figure class=\"overflow-hidden h-full\" aria-label=\"\" title=\"\">\n\n\t\t\t<img decoding=\"async\" src=\"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_121706-Male.webp\" srcset=\"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_121706-Male.jpg 640w, https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_121706-Male-300x225.jpg 300w\" alt=\"\" class=\"h-full w-full object-cover\" loading=\"lazy\"\/>\n\n\t\t<\/figure>\n\n\t\n\n    \n\n    <\/a>\n\n\n        \n            \n    \n    <a\n            data-fslightbox=\"gallery\"\n            class=\"lightbox-image rounded-20px overflow-hidden\"\n            href=\"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122158-Male-2.jpg\"\n            title=\"Press to open photo preview\"\n            aria-label=\"Press to open photo preview\"\n    >\n\n        \n    \n        \n\t\n\t\n\t\t\n\t\t<figure class=\"overflow-hidden h-full\" aria-label=\"\" title=\"\">\n\n\t\t\t<img decoding=\"async\" src=\"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122158-Male-2.webp\" srcset=\"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122158-Male-2.jpg 640w, https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122158-Male-2-300x225.jpg 300w\" alt=\"\" class=\"h-full w-full object-cover\" loading=\"lazy\"\/>\n\n\t\t<\/figure>\n\n\t\n\n    \n\n    <\/a>\n\n\n        \n            \n    \n    <a\n            data-fslightbox=\"gallery\"\n            class=\"lightbox-image rounded-20px overflow-hidden\"\n            href=\"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122328-Male.jpg\"\n            title=\"Press to open photo preview\"\n            aria-label=\"Press to open photo preview\"\n    >\n\n        \n    \n        \n\t\n\t\n\t\t\n\t\t<figure class=\"overflow-hidden h-full\" aria-label=\"\" title=\"\">\n\n\t\t\t<img decoding=\"async\" src=\"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122328-Male.webp\" srcset=\"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122328-Male.jpg 640w, https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122328-Male-300x225.jpg 300w\" alt=\"\" class=\"h-full w-full object-cover\" loading=\"lazy\"\/>\n\n\t\t<\/figure>\n\n\t\n\n    \n\n    <\/a>\n\n\n        \n            \n    \n    <a\n            data-fslightbox=\"gallery\"\n            class=\"lightbox-image rounded-20px overflow-hidden\"\n            href=\"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122609-Male.jpg\"\n            title=\"Press to open photo preview\"\n            aria-label=\"Press to open photo preview\"\n    >\n\n        \n    \n        \n\t\n\t\n\t\t\n\t\t<figure class=\"overflow-hidden h-full\" aria-label=\"\" title=\"\">\n\n\t\t\t<img decoding=\"async\" src=\"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122609-Male.webp\" srcset=\"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122609-Male.jpg 640w, https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122609-Male-300x225.jpg 300w\" alt=\"\" class=\"h-full w-full object-cover\" loading=\"lazy\"\/>\n\n\t\t<\/figure>\n\n\t\n\n    \n\n    <\/a>\n\n\n        \n    <\/div>\n\n","protected":false},"excerpt":"13-18th November 2022 in Dallas, USA had place the International Conference for High-Performance Computing, Networking, Storage, and Analysis. This is one of the most important events in supercomputing field and Sano Science was there!\u00a0","author":7,"featured_media":15524,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"editor_plus_post_options":"{}","editor_plus_copied_stylings":"{}","footnotes":""},"categories":[1],"tags":[],"class_list":["post-10655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v27.3 (Yoast SEO v27.3) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Non-Invasive Dynamic FaaS Infrastructure with HPC-Whisk \u2013 Sano in Dallas, USA - Centre for Computational Personalized Medicine<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/sano.science\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Non-Invasive Dynamic FaaS Infrastructure with HPC-Whisk \u2013 Sano in Dallas, USA\" \/>\n<meta property=\"og:description\" content=\"13-18th November 2022 in Dallas, USA had place the International Conference for High-Performance Computing, Networking, Storage, and Analysis. This is one of the most important events in supercomputing field and Sano Science was there!\u00a0\" \/>\n<meta property=\"og:url\" content=\"https:\/\/sano.science\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\/\" \/>\n<meta property=\"og:site_name\" content=\"Centre for Computational Personalized Medicine\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/sano.science\/\" \/>\n<meta property=\"article:published_time\" content=\"2022-11-17T08:44:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-02-26T14:55:21+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122158-Male.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"640\" \/>\n\t<meta property=\"og:image:height\" content=\"480\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"ict\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@sanoscience\" \/>\n<meta name=\"twitter:site\" content=\"@sanoscience\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"ict\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/sano.science\\\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/sano.science\\\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\\\/\"},\"author\":{\"name\":\"ict\",\"@id\":\"https:\\\/\\\/sano.science\\\/#\\\/schema\\\/person\\\/679dab979bad335501f6f78ea17cffc0\"},\"headline\":\"Non-Invasive Dynamic FaaS Infrastructure with HPC-Whisk \u2013 Sano in Dallas, USA\",\"datePublished\":\"2022-11-17T08:44:00+00:00\",\"dateModified\":\"2024-02-26T14:55:21+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/sano.science\\\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\\\/\"},\"wordCount\":257,\"publisher\":{\"@id\":\"https:\\\/\\\/sano.science\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/sano.science\\\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/sano.science\\\/wp-content\\\/uploads\\\/2022\\\/11\\\/20221116_122158-Male.jpg\",\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/sano.science\\\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\\\/\",\"url\":\"https:\\\/\\\/sano.science\\\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\\\/\",\"name\":\"Non-Invasive Dynamic FaaS Infrastructure with HPC-Whisk \u2013 Sano in Dallas, USA - Centre for Computational Personalized Medicine\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/sano.science\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/sano.science\\\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/sano.science\\\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/sano.science\\\/wp-content\\\/uploads\\\/2022\\\/11\\\/20221116_122158-Male.jpg\",\"datePublished\":\"2022-11-17T08:44:00+00:00\",\"dateModified\":\"2024-02-26T14:55:21+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/sano.science\\\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/sano.science\\\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/sano.science\\\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\\\/#primaryimage\",\"url\":\"https:\\\/\\\/sano.science\\\/wp-content\\\/uploads\\\/2022\\\/11\\\/20221116_122158-Male.jpg\",\"contentUrl\":\"https:\\\/\\\/sano.science\\\/wp-content\\\/uploads\\\/2022\\\/11\\\/20221116_122158-Male.jpg\",\"width\":640,\"height\":480},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/sano.science\\\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/sano.science\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Non-Invasive Dynamic FaaS Infrastructure with HPC-Whisk \u2013 Sano in Dallas, USA\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/sano.science\\\/#website\",\"url\":\"https:\\\/\\\/sano.science\\\/\",\"name\":\"Centre for Computational Personalized Medicine\",\"description\":\"Sano \u2013 Centre for Computational Medicine\",\"publisher\":{\"@id\":\"https:\\\/\\\/sano.science\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/sano.science\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/sano.science\\\/#organization\",\"name\":\"Sano \u2013 Centre for Computational Medicine\",\"alternateName\":\"Sano\",\"url\":\"https:\\\/\\\/sano.science\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/sano.science\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/sano.science\\\/wp-content\\\/uploads\\\/2024\\\/05\\\/logo_sano_podstawowe.png\",\"contentUrl\":\"https:\\\/\\\/sano.science\\\/wp-content\\\/uploads\\\/2024\\\/05\\\/logo_sano_podstawowe.png\",\"width\":700,\"height\":265,\"caption\":\"Sano \u2013 Centre for Computational Medicine\"},\"image\":{\"@id\":\"https:\\\/\\\/sano.science\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/sano.science\\\/\",\"https:\\\/\\\/x.com\\\/sanoscience\",\"https:\\\/\\\/www.linkedin.com\\\/company\\\/sanoscience\\\/\",\"https:\\\/\\\/www.youtube.com\\\/channel\\\/UCDZ_8TcjMWUG2ZcgKKgfpwQ\",\"https:\\\/\\\/bsky.app\\\/profile\\\/sanoscience.bsky.social\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/sano.science\\\/#\\\/schema\\\/person\\\/679dab979bad335501f6f78ea17cffc0\",\"name\":\"ict\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/9facebc387ac05bb345368a389c4a4a073b5c8ef14450135cfdb0c804830324e?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/9facebc387ac05bb345368a389c4a4a073b5c8ef14450135cfdb0c804830324e?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/9facebc387ac05bb345368a389c4a4a073b5c8ef14450135cfdb0c804830324e?s=96&d=mm&r=g\",\"caption\":\"ict\"},\"url\":\"https:\\\/\\\/sano.science\\\/author\\\/ict\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Non-Invasive Dynamic FaaS Infrastructure with HPC-Whisk \u2013 Sano in Dallas, USA - Centre for Computational Personalized Medicine","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/sano.science\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\/","og_locale":"en_US","og_type":"article","og_title":"Non-Invasive Dynamic FaaS Infrastructure with HPC-Whisk \u2013 Sano in Dallas, USA","og_description":"13-18th November 2022 in Dallas, USA had place the International Conference for High-Performance Computing, Networking, Storage, and Analysis. This is one of the most important events in supercomputing field and Sano Science was there!\u00a0","og_url":"https:\/\/sano.science\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\/","og_site_name":"Centre for Computational Personalized Medicine","article_publisher":"https:\/\/www.facebook.com\/sano.science\/","article_published_time":"2022-11-17T08:44:00+00:00","article_modified_time":"2024-02-26T14:55:21+00:00","og_image":[{"width":640,"height":480,"url":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122158-Male.jpg","type":"image\/jpeg"}],"author":"ict","twitter_card":"summary_large_image","twitter_creator":"@sanoscience","twitter_site":"@sanoscience","twitter_misc":{"Written by":"ict","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/sano.science\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\/#article","isPartOf":{"@id":"https:\/\/sano.science\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\/"},"author":{"name":"ict","@id":"https:\/\/sano.science\/#\/schema\/person\/679dab979bad335501f6f78ea17cffc0"},"headline":"Non-Invasive Dynamic FaaS Infrastructure with HPC-Whisk \u2013 Sano in Dallas, USA","datePublished":"2022-11-17T08:44:00+00:00","dateModified":"2024-02-26T14:55:21+00:00","mainEntityOfPage":{"@id":"https:\/\/sano.science\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\/"},"wordCount":257,"publisher":{"@id":"https:\/\/sano.science\/#organization"},"image":{"@id":"https:\/\/sano.science\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\/#primaryimage"},"thumbnailUrl":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122158-Male.jpg","inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/sano.science\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\/","url":"https:\/\/sano.science\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\/","name":"Non-Invasive Dynamic FaaS Infrastructure with HPC-Whisk \u2013 Sano in Dallas, USA - Centre for Computational Personalized Medicine","isPartOf":{"@id":"https:\/\/sano.science\/#website"},"primaryImageOfPage":{"@id":"https:\/\/sano.science\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\/#primaryimage"},"image":{"@id":"https:\/\/sano.science\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\/#primaryimage"},"thumbnailUrl":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122158-Male.jpg","datePublished":"2022-11-17T08:44:00+00:00","dateModified":"2024-02-26T14:55:21+00:00","breadcrumb":{"@id":"https:\/\/sano.science\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/sano.science\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/sano.science\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\/#primaryimage","url":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122158-Male.jpg","contentUrl":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122158-Male.jpg","width":640,"height":480},{"@type":"BreadcrumbList","@id":"https:\/\/sano.science\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/sano.science\/"},{"@type":"ListItem","position":2,"name":"Non-Invasive Dynamic FaaS Infrastructure with HPC-Whisk \u2013 Sano in Dallas, USA"}]},{"@type":"WebSite","@id":"https:\/\/sano.science\/#website","url":"https:\/\/sano.science\/","name":"Centre for Computational Personalized Medicine","description":"Sano \u2013 Centre for Computational Medicine","publisher":{"@id":"https:\/\/sano.science\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/sano.science\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/sano.science\/#organization","name":"Sano \u2013 Centre for Computational Medicine","alternateName":"Sano","url":"https:\/\/sano.science\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/sano.science\/#\/schema\/logo\/image\/","url":"https:\/\/sano.science\/wp-content\/uploads\/2024\/05\/logo_sano_podstawowe.png","contentUrl":"https:\/\/sano.science\/wp-content\/uploads\/2024\/05\/logo_sano_podstawowe.png","width":700,"height":265,"caption":"Sano \u2013 Centre for Computational Medicine"},"image":{"@id":"https:\/\/sano.science\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/sano.science\/","https:\/\/x.com\/sanoscience","https:\/\/www.linkedin.com\/company\/sanoscience\/","https:\/\/www.youtube.com\/channel\/UCDZ_8TcjMWUG2ZcgKKgfpwQ","https:\/\/bsky.app\/profile\/sanoscience.bsky.social"]},{"@type":"Person","@id":"https:\/\/sano.science\/#\/schema\/person\/679dab979bad335501f6f78ea17cffc0","name":"ict","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/9facebc387ac05bb345368a389c4a4a073b5c8ef14450135cfdb0c804830324e?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/9facebc387ac05bb345368a389c4a4a073b5c8ef14450135cfdb0c804830324e?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/9facebc387ac05bb345368a389c4a4a073b5c8ef14450135cfdb0c804830324e?s=96&d=mm&r=g","caption":"ict"},"url":"https:\/\/sano.science\/author\/ict\/"}]}},"acf":[],"gutenberg_blocks":[{"blockName":"custom-styles","attrs":{"styles":""}},{"blockName":"core\/paragraph","attrs":{"epAnimationGeneratedClass":"edplus_anim-mQCCi2","epGeneratedClass":"eplus-wrapper"},"innerBlocks":[],"innerHTML":"\n<p class=\" eplus-wrapper\">Maciej Malawski, the Director of Sano Science and team leader of the Extreme-scale Data and Computing research group in Sano is one of the authors of the paper presented at SC22 conference.&nbsp;<br><br>The presentation was on Wednesday, November 16th, and was titled \u201cNon-Invasive Dynamic FaaS Infrastructure with HPC-Whisk\u201d.&nbsp;<br><br><strong>As we could read in abstract:&nbsp;<\/strong><br><br>Modern HPC workload managers and their careful tuning contribute to the high utilization of HPC clusters. However, due to inevitable uncertainty it is impossible to completely avoid node idleness. Although such idle slots are usually too short for any HPC job, they are too long to ignore them. Function-as-a-Service (FaaS) paradigm promisingly fills this gap, and can be a good match, as typical FaaS functions last seconds, not hours. Here we show how to build a FaaS infrastructure on idle nodes in an HPC cluster in such a way that it does not affect the performance of the HPC jobs significantly. We dynamically adapt to a changing set of idle physical machines, by integrating open-source software Slurm and OpenWhisk.&nbsp;<br><br>We designed and implemented a prototype solution that allowed us to cover up to 90% of the idle time slots on a 50k-core cluster that runs production workloads.&nbsp;<br><br>Congratulations!<\/p>\n","innerContent":["\n<p class=\" eplus-wrapper\">Maciej Malawski, the Director of Sano Science and team leader of the Extreme-scale Data and Computing research group in Sano is one of the authors of the paper presented at SC22 conference.&nbsp;<br><br>The presentation was on Wednesday, November 16th, and was titled \u201cNon-Invasive Dynamic FaaS Infrastructure with HPC-Whisk\u201d.&nbsp;<br><br><strong>As we could read in abstract:&nbsp;<\/strong><br><br>Modern HPC workload managers and their careful tuning contribute to the high utilization of HPC clusters. However, due to inevitable uncertainty it is impossible to completely avoid node idleness. Although such idle slots are usually too short for any HPC job, they are too long to ignore them. Function-as-a-Service (FaaS) paradigm promisingly fills this gap, and can be a good match, as typical FaaS functions last seconds, not hours. Here we show how to build a FaaS infrastructure on idle nodes in an HPC cluster in such a way that it does not affect the performance of the HPC jobs significantly. We dynamically adapt to a changing set of idle physical machines, by integrating open-source software Slurm and OpenWhisk.&nbsp;<br><br>We designed and implemented a prototype solution that allowed us to cover up to 90% of the idle time slots on a 50k-core cluster that runs production workloads.&nbsp;<br><br>Congratulations!<\/p>\n"]},{"blockName":"core\/paragraph","attrs":{"epAnimationGeneratedClass":"edplus_anim-s7CsBj","epGeneratedClass":"eplus-wrapper"},"innerBlocks":[],"innerHTML":"\n<p class=\" eplus-wrapper\"><br>Also Marek Michalewicz, PhD, Sano's Visiting Professor was in Dallas at SC Conference Series and he was one of the Wednesday session \"Bridging the HPC\/Data Divide\" leaders.&nbsp;<br><br><strong>More details about this session<\/strong>: <a href=\"https:\/\/lnkd.in\/dV7BVjS7&nbsp;\">https:\/\/lnkd.in\/dV7BVjS7&nbsp;<\/a><\/p>\n","innerContent":["\n<p class=\" eplus-wrapper\"><br>Also Marek Michalewicz, PhD, Sano's Visiting Professor was in Dallas at SC Conference Series and he was one of the Wednesday session \"Bridging the HPC\/Data Divide\" leaders.&nbsp;<br><br><strong>More details about this session<\/strong>: <a href=\"https:\/\/lnkd.in\/dV7BVjS7&nbsp;\">https:\/\/lnkd.in\/dV7BVjS7&nbsp;<\/a><\/p>\n"]},{"blockName":"core\/spacer","attrs":{"height":"50px","epAnimationGeneratedClass":"edplus_anim-zZkG4z","epGeneratedClass":"eplus-wrapper"},"innerBlocks":[],"innerHTML":"\n<div style=\"height:50px\" aria-hidden=\"true\" class=\"wp-block-spacer eplus-wrapper\"><\/div>\n","innerContent":["\n<div style=\"height:50px\" aria-hidden=\"true\" class=\"wp-block-spacer eplus-wrapper\"><\/div>\n"]},{"blockName":"acf\/grid-of-images","attrs":{"images":[{"image":{"ID":15526,"id":15526,"title":"20221114_185026-Male","filename":"20221114_185026-Male.jpg","filesize":33459,"url":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221114_185026-Male.jpg","link":"https:\/\/sano.science\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\/20221114_185026-male\/","alt":"","author":"5","description":"","caption":"","name":"20221114_185026-male","status":"inherit","uploaded_to":10655,"date":"2024-02-26 14:54:36","modified":"2024-02-26 14:54:36","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/sano.science\/wp-includes\/images\/media\/default.png","width":640,"height":480,"sizes":{"thumbnail":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221114_185026-Male-150x150.jpg","thumbnail-width":150,"thumbnail-height":150,"medium":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221114_185026-Male-300x225.jpg","medium-width":300,"medium-height":225,"medium_large":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221114_185026-Male.jpg","medium_large-width":640,"medium_large-height":480,"large":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221114_185026-Male.jpg","large-width":640,"large-height":480,"1536x1536":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221114_185026-Male.jpg","1536x1536-width":640,"1536x1536-height":480,"2048x2048":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221114_185026-Male.jpg","2048x2048-width":640,"2048x2048-height":480}}},{"image":{"ID":15530,"id":15530,"title":"20221115_090149-Male","filename":"20221115_090149-Male.jpg","filesize":36167,"url":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221115_090149-Male.jpg","link":"https:\/\/sano.science\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\/20221115_090149-male\/","alt":"","author":"5","description":"","caption":"","name":"20221115_090149-male","status":"inherit","uploaded_to":10655,"date":"2024-02-26 14:55:05","modified":"2024-02-26 14:55:05","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/sano.science\/wp-includes\/images\/media\/default.png","width":640,"height":480,"sizes":{"thumbnail":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221115_090149-Male-150x150.jpg","thumbnail-width":150,"thumbnail-height":150,"medium":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221115_090149-Male-300x225.jpg","medium-width":300,"medium-height":225,"medium_large":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221115_090149-Male.jpg","medium_large-width":640,"medium_large-height":480,"large":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221115_090149-Male.jpg","large-width":640,"large-height":480,"1536x1536":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221115_090149-Male.jpg","1536x1536-width":640,"1536x1536-height":480,"2048x2048":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221115_090149-Male.jpg","2048x2048-width":640,"2048x2048-height":480}}},{"image":{"ID":15531,"id":15531,"title":"20221116_121706-Male","filename":"20221116_121706-Male.jpg","filesize":40530,"url":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_121706-Male.jpg","link":"https:\/\/sano.science\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\/20221116_121706-male\/","alt":"","author":"5","description":"","caption":"","name":"20221116_121706-male","status":"inherit","uploaded_to":10655,"date":"2024-02-26 14:55:06","modified":"2024-02-26 14:55:06","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/sano.science\/wp-includes\/images\/media\/default.png","width":640,"height":480,"sizes":{"thumbnail":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_121706-Male-150x150.jpg","thumbnail-width":150,"thumbnail-height":150,"medium":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_121706-Male-300x225.jpg","medium-width":300,"medium-height":225,"medium_large":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_121706-Male.jpg","medium_large-width":640,"medium_large-height":480,"large":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_121706-Male.jpg","large-width":640,"large-height":480,"1536x1536":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_121706-Male.jpg","1536x1536-width":640,"1536x1536-height":480,"2048x2048":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_121706-Male.jpg","2048x2048-width":640,"2048x2048-height":480}}},{"image":{"ID":15532,"id":15532,"title":"20221116_122158-Male-2","filename":"20221116_122158-Male-2.jpg","filesize":40901,"url":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122158-Male-2.jpg","link":"https:\/\/sano.science\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\/20221116_122158-male-2\/","alt":"","author":"5","description":"","caption":"","name":"20221116_122158-male-2","status":"inherit","uploaded_to":10655,"date":"2024-02-26 14:55:07","modified":"2024-02-26 14:55:07","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/sano.science\/wp-includes\/images\/media\/default.png","width":640,"height":480,"sizes":{"thumbnail":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122158-Male-2-150x150.jpg","thumbnail-width":150,"thumbnail-height":150,"medium":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122158-Male-2-300x225.jpg","medium-width":300,"medium-height":225,"medium_large":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122158-Male-2.jpg","medium_large-width":640,"medium_large-height":480,"large":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122158-Male-2.jpg","large-width":640,"large-height":480,"1536x1536":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122158-Male-2.jpg","1536x1536-width":640,"1536x1536-height":480,"2048x2048":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122158-Male-2.jpg","2048x2048-width":640,"2048x2048-height":480}}},{"image":{"ID":15533,"id":15533,"title":"20221116_122328-Male","filename":"20221116_122328-Male.jpg","filesize":44264,"url":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122328-Male.jpg","link":"https:\/\/sano.science\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\/20221116_122328-male\/","alt":"","author":"5","description":"","caption":"","name":"20221116_122328-male","status":"inherit","uploaded_to":10655,"date":"2024-02-26 14:55:07","modified":"2024-02-26 14:55:07","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/sano.science\/wp-includes\/images\/media\/default.png","width":640,"height":480,"sizes":{"thumbnail":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122328-Male-150x150.jpg","thumbnail-width":150,"thumbnail-height":150,"medium":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122328-Male-300x225.jpg","medium-width":300,"medium-height":225,"medium_large":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122328-Male.jpg","medium_large-width":640,"medium_large-height":480,"large":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122328-Male.jpg","large-width":640,"large-height":480,"1536x1536":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122328-Male.jpg","1536x1536-width":640,"1536x1536-height":480,"2048x2048":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122328-Male.jpg","2048x2048-width":640,"2048x2048-height":480}}},{"image":{"ID":15534,"id":15534,"title":"20221116_122609-Male","filename":"20221116_122609-Male.jpg","filesize":26025,"url":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122609-Male.jpg","link":"https:\/\/sano.science\/non-invasive-dynamic-faas-infrastructure-with-hpc-whisk-sano-in-dallas-usa\/20221116_122609-male\/","alt":"","author":"5","description":"","caption":"","name":"20221116_122609-male","status":"inherit","uploaded_to":10655,"date":"2024-02-26 14:55:08","modified":"2024-02-26 14:55:08","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/sano.science\/wp-includes\/images\/media\/default.png","width":640,"height":480,"sizes":{"thumbnail":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122609-Male-150x150.jpg","thumbnail-width":150,"thumbnail-height":150,"medium":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122609-Male-300x225.jpg","medium-width":300,"medium-height":225,"medium_large":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122609-Male.jpg","medium_large-width":640,"medium_large-height":480,"large":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122609-Male.jpg","large-width":640,"large-height":480,"1536x1536":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122609-Male.jpg","1536x1536-width":640,"1536x1536-height":480,"2048x2048":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122609-Male.jpg","2048x2048-width":640,"2048x2048-height":480}}}]},"innerBlocks":[],"innerHTML":"","innerContent":[]}],"meta_data":{"has_thumbnail_pattern":false,"share_on_social_media":{"has_social_media":false}},"featured_image":{"url":"https:\/\/sano.science\/wp-content\/uploads\/2022\/11\/20221116_122158-Male.jpg"},"main_category":{"name":"Uncategorized"},"prev_page":{"slug":"sano-moderated-the-stage-dedicated-to-digital-health-at-lsos-2022"},"next_page":{"slug":"dr-korzeniowski-at-iros-2022-in-kyoto"},"_links":{"self":[{"href":"https:\/\/sano.science\/index.php\/wp-json\/wp\/v2\/posts\/10655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sano.science\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sano.science\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sano.science\/index.php\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/sano.science\/index.php\/wp-json\/wp\/v2\/comments?post=10655"}],"version-history":[{"count":6,"href":"https:\/\/sano.science\/index.php\/wp-json\/wp\/v2\/posts\/10655\/revisions"}],"predecessor-version":[{"id":15537,"href":"https:\/\/sano.science\/index.php\/wp-json\/wp\/v2\/posts\/10655\/revisions\/15537"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/sano.science\/index.php\/wp-json\/wp\/v2\/media\/15524"}],"wp:attachment":[{"href":"https:\/\/sano.science\/index.php\/wp-json\/wp\/v2\/media?parent=10655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sano.science\/index.php\/wp-json\/wp\/v2\/categories?post=10655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sano.science\/index.php\/wp-json\/wp\/v2\/tags?post=10655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}