OCRE Spotlight on the CERN Validation Test Suite

Recently, the OCRE Project team had the opportunity to speak with João Fernandes, Project Leader at CERN to find out more about the CERN Validation Test Suite and how the OCRE community can get involved. 

How did the CERN Validation Test Suite come about and what are its aims?

Currently,  there  is  no  reference  point  for researchers  in  terms  of  the  capabilities and performance of the cloud platforms with regard to research workloads deployment. As part of the lessons learned of HNSciCloud, a technical benchmarking and validation process across the cloud service stacks of commercial providers was identified to be required, to monitor the level of readiness of cloud providers against multi-disciplinary scientific workloads, where such validation would also provide working examples of research deployments. 

Who is the test suite for?

Main target are the Researchers, from multiple disciplines with a need to test/validate cloud offerings from commercial providers. Typical questions that the test-suite will answer to researchers are: "Is this cloud offering designed to the use of my research team?”, “How can I run multiple configurations for my software?”, “Is the cloud setup too time consuming?”, “Do they have GPUs?”, etc.

What is the timeline for the CERN Validation Test Suite?

The test-suite is being used in the scope of many projects, including OCRE. Specifically for OCRE, the plan is to deploy the test-suite over 25 platforms, out of 26 present in the OCRE Framework, before December 2021. The runs have initiated. Vendors are also welcome to run tests themselves before CERN makes ‘OCRE’ runs. Test results to be stored as JSON files on-premise (using the CERN CEPH storage service). Vendors will access only their specific results, via pre-signed S3 URLs. The OCRE project aims to create a certification process for successfully validated platforms. The expectation is to make the testing and validation to be part of the selection criteria for adoption funding. 

How can the OCRE community benefit from the CERN Validation Test Suite?

The research community can use freely the test suite to validate the offers of the contracts being made under the OCRE framework (or any other). CERN has published it under the AGPL open source license: https://github.com/cern-it-efp/EOSC-Testsuite. The benefits for the research community are expected to expand beyond OCRE. A test-suite roadmap for the European Open Science Cloud is being prepared, enhancing the validation process of the activities with commercial providers in the context of the EOSC. Some aspects that will be covered are sovereign data processing, governance, compliance (i.e., data locations, data access levels, etc.), cost optimisation and execution of exit strategies from across cloud providers or between cloud providers and public on-premise data centres. 

How can the OCRE community get involved?

The OCRE community and the research community at large is invited to contribute with tests to be part of the Test Suite (for most of the tests included that is already the case). The process to add tests is the following:

Step 1: Contact the test-suite development team: cloud-test-suite@cern.ch 
Step 2: Assess the integration work to be done and set up requirements
Step 3: Provide Information about the documentation of the given test, contact of the person and the license governing it

To discover more about CERN's Validation Test Suite, more information can also be found via the OCRE Cloud and Earth Observation Procurement, Funding Updates and the OCRE Cloud Catalogue Webinar recording. Watch the video and access the CERN Test Validation Suite slides now!