page 2

news from the EGI communityISSUE 32OCTOBER 2018

TOP STORIES

page 5

MORE

www.egi.eu

EGI Strategic & Innovation Fund: 2nd callpage 1

Inspired

How EGI services support ENVRIplus

The status of the Asian infrastructure

07 DI4R 2018: What to do & where to eat in Lisbon

page 4

EGI & Terradue: a lasting partnership

08 Collaborate on a survey: e-Infrastructure in Europe

page 6

Federation models of the EGI cloud

This edition of the newsletter is focused on updatesfrom the EGI infrastructure, current projects and thebusiness programme.

Your feedback and suggestions are always welcome!

Send an email to Sara & Iulia at:

press@egi.eu

Welcome to issue 32!

The 2nd call for the EGI Strategic and Innovation Fund isnow open: apply by 18 November!

1EGI Inspired Newsletter // October 2018 // www.egi.eu

We launched a 2nd call for theEGI Strategic & Innovation Fund.

The Fund was established by theEGI Council to stimulate targetedactions that bring benefits withinthe EGI Community in the shortto medium term.

The 2nd call of the Strategic andInnovation Fund can be accessedby the EGI Council participants.In addition, the call is open to allorganisations based in countriesrepresented in the EGI Councilwho receive a letter of supportby EGI Council participants.

Project proposals should alignwith the vision, mission andstrategy of the EGI Federationand create changes in the pro-duction environment or validateconcepts that justify investments.

Applications can be submitted by11 November 2018.

The first call of the EGI Strategicand Innovation Fund endedearlier this year and resulted inthe selection of two projects:

1. "Elastic Serverless Platformfor High Throughput ComputingScientific Applications"(prototype category) - a projectdeveloped by the TechnicalUniversity of Valencia, with agoal to create a prototype of anelastic serverless platform forHigh Throughput ComputingScientific Applications on top ofthe EGI Federated Cloud.

2. “Native OpenID ConnectImplementation for OpenStackCloudsc” – (major innovationcategory) - a project developed

by the Spanish National ResearchCouncil (CSIC) that proposes toimplement a native OpenIDConnect plugin for OpenStack-based clouds to be certified bythe OpenID Connect Foundationand available to providers of theEGI Federated Cloud.

Will you be the next winner?

Read more information on theEGI Strategic & Innovation Fundand apply by 11 November 2018.

How EGI services support the ENVRIplus project

Yin Chen reports on the progresses made by four ENVRIplus Science Demonstrators

2EGI Inspired Newsletter // October 2018 // www.egi.eu

The ENVRIplus project is nowentering its final stage, beingdue to end in April 2019.

The objective of ENVRIplus hasbeen to provide solutions toshared challenges for EuropeanEnvironmental and Earth SystemResearch Infrastructures (RIs) intheir efforts to deliver newservices for science and society.

The project now reports sevenScience Demonstratorsshocasing implementationresults of community use cases,and serving as evidence of theadded value brought to RIs.

EGI has been providing fullsupport to the project’s servicedevelopment and use caseimplementations. Four of thescience demonstrators are nowfully integrated with EGI services.

Here is an overview of the workdone so far:

Cross Research InfrastructureimprovementsThe first Science Demonstratoraddresses a common problemfor ENVRIplus RIs: the preparationof data transfer prior to datatransmission is often not yetsufficiently standardised. Thishinders the operation of efficientcross-RI data processing routines,e.g., for data quality checking.

The demonstrator introduces aservice prototype that allows tosubmit and publish raw obser-vational environmental timeseries data in common standardformats (T-SOS XML & SSNO JSON).The EGI Monitoring service(ARGO) is used as a messagingAPI to perform Near Real Time

(NRT) quality control proceduresby an Apache Storm NRT QCTopology deployed on the EGIFederated Cloud, which in turnpublishes the quality controlledand labelled data via amessaging output queue.

Watch demo.

EuroArgo Data SubscriptionServiceThe Euro-Argo ERIC coordinatesthe European contribution to theinternational Argo programmefor marine science observation.The EuroArgo Data SubscriptionService allows researchers tosubscribe to customised views ofArgo data, select specific regionsand time-spans, and choose thefrequency of updates. Tailoredupdates are then provided toresearchers’ private storage.

3EGI Inspired Newsletter // October 2018 // www.egi.eu

The demo shows an integrationsolution that combines theEuroArgo community data portalwith the EGI Federated Cloud forcomputing data product for eachsubscription. The pilot activitywas initiated by the marineresearch community. RIs canbenefit from the subscriptionservices, e.g., to create moreelaborated data products byrequesting data from othersources, and can optimise theirinternal workflows by signing upfor automatic updates.

Watch demo.

Transferable data analyticsThe third Science Demonstratordescribes a service prototypethat supports aerosol scientistsin studying new atmosphericparticle formation events bymoving data analysis from localcomputing environments tointeroperable infrastructures.This results in harmonising dataanalysis and more importantlythe syntax and semantics of dataderived from analysis.

The demonstrator showcases apossible architecture of a socio-technical infrastructure thattransforms data into knowledge.

The service allows researchersto access JupyterLab operatedon the EGI e-Infrastructure toanalyse primary data for thepurpose of new particle forma-tion, event detection & description.

JupyterLab is accessible from thecorresponding D4Science VirtualResearch Environment (VRE).

This approach shows a range ofnovel possibilities, in particularenabling researchers to focus ondata analysis and interpretationwhile leaving data access &transformation from & to systemsto interoperable infrastructure.

The demonstrator contributes toimplementing the global agendaof FAIR data by promoting thenotion of “FAIR by Design”,weaving data FAIRness into thefabric of infrastructures. It buildson the principle not to leavemaking data FAIR to researchersbut to guarantee it by design ofwell-engineered infrastructures.

Watch demo.

LifeWatchLifeWatch is a Research Infra-structure set up to support thefields of ecosystems researchand biodiversity by equippingscientists with access to data,analytical tools and state-of-the-art virtual laboratories. The lastScience Demonstrator illustrateshow a LifeWatch researcher caneasily upload and integrate ananalysis algorithm in D4Science,& share it with other researchersin a VRE. The use case proposedis an inte-gration solution thatlinks the D4Science/gCube VREto the LifeWatch RI and to theEGI e-Infrastructure.

This integration enablesindividual researchers to repeatand reuse algorithms, run trendanalysis, and add newparameters and custom data.

The VRE provides provenanceregistration that improvesreproducibility and also allowsretention of computation resultsin the user’s workspace.

This facilitates the editing andadaptation of algorithms,features that are not provided bythe existing LifeWatch ICT.

Watch demo.

More information

Yin Chen is Senior TechnicalOutreach Expert at the EGIFoundation and working inWP9 of the ENVRIplus project.

ENVRIplus

www.envriplus.eu

4

The e-Science & e-Infrastructuresare having profound impacts inAsia. They are now moving towardsopen science supported by theEOSC-Hub project, which iscoordinated by EGI.

The e-Science infrastructure inAsia was established in 2005 forWLCG collaborations in thebeginning, supported by the AsiaPacific Regional Operation Centrevia the Academia Sinica GridComputing Centre in Taiwan.

Based on the common needs foruser communities in this region,the infrastructure was extendedto support drug discovery (foravian flu for example), seismicwave propagation, tsunami wavepropagation, weather simulation,chemical compound propertyanalysis, long-term data preser-vation and other physics andhigh-energy physics researches.

Disaster mitigation is one of theprimary e-Science collaborationsin Asia. We aim to developknowledge-oriented hazard riskassessment prototypes basedon deeper understanding of rootcauses and drivers of a hazard.

Science gateways of targeteddisaster types which integratehazard metadata, innovativesimulation models, and analysisworkflow from case studies willprovide high-performancesimulation services to allpartners over the regional EGI-compatible e-Infrastructure.

As a result, detailed, quantitativescientific understandings arebecoming possible.

The status of the Asia Regional infrastructure

Eric Yen and Simon Lin on the successes and challenges of the Asian e-infra

In addition to capturing theprecise scientific processes ofthe hazard event, the sciencegateway also permits analysis,reuse, and reproducibility of thecase studies and their data.

Furthermore, through the colla-boration framework composedby the science gateway, casestudies and knowledge base,discoverable and shareable datacan enable collaborations forand cross-disciplinary researchand new discoveries.

The extension and diversity ofthe Asia Pacific region has a lotof advantages. We have engagedcollaborations not only betweenexperts of grids and clouds, butalso among scientists, at aninternational level. Operationtowards a sustainable and easyto use e-Infrastructure has beencoordinated by ASGC and incor-porated with EGI in the pastdecade. In the future, a lot ofbottom-up regional collaborationswill be carried out over the e-infrastructure with EGI's help.

The experiences and requirementsof regional e-Science applicationswill facilitate the advancement oftechnologies, platforms andcollaboration models.

Challenges lying ahead forthe regional collaborationson e-Science include:1) effective collaboration model:the user community has to con-clude the common requirements,priority, workflow and directionsby working with infrastructureand application support teams.

EGI Inspired Newsletter // October 2018 // www.egi.eu

More information

Eric Yen is Associate ResearchScientist at Academia SinicaGrid Computing Centre(ASGC) in Taiwan.

Simon Lin is Project Directorof the Academia Sinica GridComputing Centre (ASGC)in Taiwan.

ASGC

www.sinica.edu.tw

2) make the computing modeladapted to the right technology,new hardware and changingnetworking environment.

3) regional infrastructure hasto support multi-disciplinaryapplications with simplifiedinfrastructure building, inte-lligent middleware & automaticforward scalability.

We are confident that thecontinuous enhancement ofAsia-Global infrastructure willbenefit all worldwide partnersby economies of scale, capabilityof global optimisation andflexibility via collaborations.

EGI’s main goal is to empowerresearchers from all disciplines tocarry out data and computeintensive science. We are alsocommitted to foster relationshipswith industry and SMEs.

SMEs are considered one of thekey drivers for economic growthand innovation. Over the pastyears, the European Commissionhas been putting SMEs in thelead for delivering innovation tothe market.

EGI is supporting this objective byexploring opportunities forstrategic partnerships with SMEs.

For example, during the EGI-Engage project, a dedicated teamwas setup to establish a BusinessEngagement Programme fordeveloping relationships withSMEs to co-develop support solu-tions for their computing needs.This resulted in the accomplishmentof 11 business use cases, withmany more still in progress.

One example is our work withTerradue, a SME based in Rome,Italy. The company specialises onchallenges associated with thehandling of massive and complexdata streams that can supportthematic expertise. TerradueCloud Platform is addressing thistopic with solutions to transferEarth Observation (EO) processingalgorithms to cloud infrastructures.

The platform also provides servi-ces to optimise the connectivityof the data centres with integrateddiscovery & processing methods.

The Terradue & EGI collaborationstarted in 2012-2014 as part ofthe European Commission’ssupport action “Helix Nebula -The Science Cloud”. In September2016, a first agreement betweena group of EGI Resource Centresand Terradue was established.

Terradue has since then beenworking with EGI Cloud resourcesand teams, extending Terradue’soffer to users, by integrating theirservices on the Terradue CloudPlatform for the processing ofsatellite data.

Here are some highlights:• Terradue’s participation in theEC H2020 NextGEOSS encompasses10 Pilots, innovating services aspart of the Group on EarthObservations support to UN’sSustainable Development Goals(e.g. Food Security to achievezero hunger, Solar Energy foraffordable and clean energy).

• Also, Terradue’s leading role inthe ESA Thematic ExploitationPlatform for Geohazards and forHydrology encompasses dataprocessing services contributedto the Committee on EarthObservation Satellites goals (e.g.DLR’s InSAR Browse service at200m resolution covering theworld tectonic areas, and at 50mresolution covering volcanoesworldwide) and to Europeaninternational cooperation (e.g.Niger River).

One key benefit of the renewedMoU is, for Terradue, to further

integrate TEP services with theEGI infrastructure using theallotted resources provided bythe selected EGI cloud providersas described in the relatedOperations Level Agreements(OLAs) and Service LevelAgreement (SLA). Six of them willbe contributing as part of thisMoU a capacity of 538 CPU cores,14TB of RAM and 10TB of storage.

Based on initial proof of concepts,Terradue and EGI will also refinea business model for long-termservice delivery and support, andcontinuous operations that willprovide a direct benefit forTerradue Cloud Platform users.

5EGI Inspired Newsletter // October 2018 // www.egi.eu

EGI & Terradue: a long-lasting partnership

Giuseppe La Rocca outlines the benefits of the agreement

More information

Giuseppe La Rocca isTechnical Outreach Expertat the EGI Foundation.

Terradue

www.terradue.com

6

The EGI Cloud Compute serviceis part of the EGI service catalogueas a federation of Infrastructureas Service (IaaS) cloud providers,where each IaaS is operated bydifferent institutes according toagreed principles & regulations.

These principles and regulationsrequire providers’ IaaS clouds toconnect with central EGI services(e.g. accounting), and exposetheir cloud to users throughcommonly agreed interfaces.

This model has proven to besuccessful to support severaluser communities, but afteryears of production we havefound some limitations:

• Reaching a uniform behaviourand interfaces at multiple, inde-pendently operated IaaS provi-ders is not realistic in practice.Achieving fully distributed, multi-cloud workloads requires higher-level tools or a lot of customisationand site-specific considerationsin the user code.

• Joining the EGI Cloud infras-tructure comes with an extracomplexity in the operation ofIaaS providers. This often causesa decrease in service reliability.

• Only a small fraction of thescientific workloads need accessto multiple cloud sites. Most work-loads fit into a single cloud andnon-federated clouds can bemore reliable, so users don't wantto migrate workloads across sites.

Given these findings, we proposethree alternative implementationsto be considered for inclusion inthe EGI Cloud Compute service:

Application ServicesThis implementation enables‘Software as a Service’ (SaaS)providers to join the service, thiscan be done via the EGI Check-inand Marketplace: the SaaS appli-cation offerings should be inte-grated with Check-In to enableSingle Sign-On and consistentauthorisation across the appli-cations, then registered in theEGI Marketplace for visibility.

IaaS allianceThis model would allow IaaSproviders to expose their servicesin the Marketplace if they meet asimple set of requirements. Theminimum requirement is tointegrate with EGI Check-In andto register in the EGI Marketplace.Integration with additional, cen-trally provided EGI operationalservices & adoption of operationalpractices would be not mandatory.

Application PlatformsThis implementation would allowapplication and IaaS serviceproviders to join if they supportone of the recognised ‘applicationdeployment platforms’, such asKubernetes for container-basedapplications or IM for VM-basedapplications. These technologiesenable applications from providersto be shipped and instantiatedfor/by users at the IaaS sites.

This type of model can nicelysupport cloud-bursting of appli-cations from national clouds tothe federation by using higher-level deployment platforms thathide the complexity of a hybridcloud setup.

EGI Inspired Newsletter // October 2018 // www.egi.eu

These new implementations willintroduce a wider range of options,with advanced functionality thatcan attract new users.

The Application Services andApplication Platforms implemen-tations allow users to focus onresearch instead of managinglow level infrastructure compo-nents, while power users can stillbenefit from the IaaS Alliance toget the full capabilities of an IaaS.

Read more about this proposaland bring your comments in thededicated document (preferablyby 30 November).

More information

Enol Fernández is CloudTechnologist at the EGIFoundation.

EGI Cloud Compute

egi.eu/services/cloud-compute/

Federation models of the EGI Cloud Compute service

Enol Fernandez introduces new ways of engaging with the cloud service

7

The third edition of the annualDI4R conference is just aroundthe corner, and will take place atthe University of Lisbon campusin Lisbon, from 9-11 October.

Apart from an excellent eventvenue, Portugal’s capital is anelegant and vibrant city, so don’tmiss the opportunity to stroll itsstreets and discover its wonders.

Here are a few not-to-missattractions of Lisbon:

What to see:• Tram 28

A true landmark of Lisbon, thewooden tram 28 roars throughLisbon's most historic streetsand passes through the populartourist districts of Graça, Alfama,Baixa and Estrela.

• Castelo São Jorge

The ancient castle dates back tothe ninth century and rules overthe city, being visible fromalmost every street in Lisbon.

• Praça do Comércio

The Praça do Comércio is thelargest of Lisbon's plazas, withtraditional painted buildingsalong the sides and a statue ofKing José I as a centrepiece ofthe ensemble. This was wherethe old royal palace was, beforebeing destroyed in the 1755earthquake, Not to be missed!

• Torre de Belém

A 16th-century fortified towerand a symbol of maritime Lisbon.The tower is located in Belém,very close to two architecturallandmarks built 500 years apart:the Mosteiro of Jerónimos andthe Centro Cultural of Belém.

Where to eat:• The Mercado da Ribeira is acollection of market stalls witheverything from ice cream andpastries, snacks, tapas, fancychef food & vegetarian options.

• Go to a Pastelaria for pastriesand coffee. Every street has oneand everyone has a local (mineis Pastelaria Vitória in Estefânia).The most famous Lisbon pastryis the pastel de nata (custardtarts) but that is just the tip ofthe iceberg. A few of the mosttraditional Pastelarias are theVersailles, the Mexicana, theConfeitaria Nacional.

• Restaurants: you will haveplenty of choice. Here are a few:

- Galeto is a gorgeous placedecorated exactly as it opened inthe '50s. They serve uncomplicatedfood, steak sandwiches, burgersat the counter (no tables!)

- Brasserie de l'Entrecote is afancy place to eat steak, hasn’tchanged since the first time Iwent there with my father (reallynot for vegetarians!)

- Alfaia is a restaurant in BairroAlto that serves traditionalPortuguese food. It’s in a middleof a touristy area so it's best tomake a reservation.

EGI Inspired Newsletter // October 2018 // www.egi.eu

- Santa Clara dos Cogumelos,strictly for mushroom lovers,everything in the menu is cookedwith them. Their crème bruleewith truffles is like nothing else(it's best to make a reservation).

- Honorato is a hipster burgerplace, with a gin menu and all, abit over the top, but the burgersare well worth it.

- Marisqueira do Liz is a perfectplace to eat seafood.

- Portugália is an old school beer-house, where you can have a steak,seafood and beer.

- PSI and Os Tibetanos are twovegetarian restaurants with lotsof vegan options.

More information

Sara Coelho works asCommunications Managerat the EGI Foundation.

DI4R 2018

digitalinfrastructures.eu

DI4R 2018: What to do & where to eat in Lisbon

A few tips & tricks from our Portuguese colleague, Sara Coelho

Collaborate on a short survey: e-Infrastructure inEurope - attributes and metaphors for diffusion

Why are some e-science toolsadopted by a large community,while others are ignored? Whydo some e-science teams colla-borate well while others struggle?How can teams come to a sharedunderstanding of the e-sciencetools they are developing?

These and other questions werediscussed by dr. Kerk Kee (Schoolof Communication, ChapmanUniversity), dr. Christian Burgersand Ellen Droog, MSc (Dept. ofCommunication Science, VUAmsterdam) in a webinar at theEGI Foundation office on 18September 2018 in Amsterdam.

They reported on a first pilot testconducted at two conferences(EOSC-hub in Malaga, April 2018and ENVRIplus in Zandvoort aanZee, May 2018).

The study’s goal was to providecross-cultural validation of anapproach to successful diffusionof e-infrastructure tools andapplication, that was originallydeveloped in a US context (XSEDE).

In the webinar, they provide anoverview of the most importanttool attributes and team charac-teristics to stimulate adoptionbeyond the initial user base.

They now seek to expand thefirst pilot test to a larger sample,and would like to ask you forhelp as a member of the e-infrastructure community.

Do you want to help?Please take a few minutes tocontribute to the short surveyand share your thoughts.

Listen to the webinar:

• Part I: Tool Attributes

• Part II: Attributes of the VirtualOrganisation

• Part III: Metaphors of e-infrastructure

8EGI Inspired Newsletter // October 2018 // www.egi.eu