Advanced Manufacturing Collaboration in a Cloud-basedApp Marketplace

Amit Rama AkulaUniversity of Missouri-Columbia, USA

ar442@mail.missouri.edu

Prasad CalyamUniversity of Missouri-Columbia, USA

calyamp@missouri.edu

Ronny Bazan AntequeraUniversity of Missouri-Columbia, USA

rcb553@mail.missouri.edu

Raymond E. LetoTotalSim, USA

rleto@totalsim.us

ABSTRACTAdvanced Manufacturing Apps that perform complex modeling andsimulation are now becoming available in Marketplaces. App stake-holders face two fundamental challenges in cloud engineering ofthe App Marketplaces: (i) orchestration of service chaining throughan App Runtime, and (ii) finding a suitable cost model to evaluateApp pricing strategies. In this paper, we address these challengesby proposing a new cloud architecture that aims at supporting an‘App Marketplace’ that thrives on agile development, organic collab-oration and scalable sales of next generation Manufacturing Appsrequiring high-performance simulation and modeling. We describehow we are realizing the vision of this architecture through an AppRuntime we have developed that leverages a Resource Brokering Ser-vice to create App chaining mechanisms. We also detail a new costmodel that could be part of an Accounting Service in our proposedarchitecture to address the issues of cost accounting and pricing ofthe Apps faced by the App developers while using cloud infrastruc-tures for hosting their Apps. Lastly, we describe experiments witha real-world implementation of our App Runtime and cost modelin a WheelSim testbed that uses NSF GENI Cloud and Ohio Super-computer Center resources. Our results show benefits to an Appdeveloper in terms of: satisfactory user experience, lower designtime and lower cost/simulation.

KEYWORDSIntegrated Cloud Management, Advanced Manufacturing, App Mar-ketplace, Dynamic App Runtime

ACM Reference format:Amit Rama Akula, Prasad Calyam, Ronny Bazan Antequera, and RaymondE. Leto. 2017. Advanced Manufacturing Collaboration in a Cloud-basedApp Marketplace. In Proceedings of CF’17, Siena, Italy, May 15-17, 2017,10 pages.DOI: http://dx.doi.org/10.1145/3075564.3075582

1 INTRODUCTIONAdvances in the field of cloud computing and high-speed networkinghave transformed the ways enterprises market their expertise and

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish,to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from permissions@acm.org.CF’17, Siena, Italy© 2017 ACM. 978-1-4503-4487-6/17/05. . . $15.00DOI: http://dx.doi.org/10.1145/3075564.3075582

products within domains such as online retail, gaming and evenhealth care. This transformation has led to delivery of expertise andproducts through web and mobile accessible “App” that cater to theneeds of the customers. These advancements have helped in creatingeconomic benefits by transforming investment in infrastructure. En-terprises can now rent infrastructure on-demand without the hassleof frequent maintenance or upgrades. They can also access highperformance computing and elastic resources to collaborate withtheir peers and improve service delivery to customers [1]. An exem-plar use case of cloud services adoption can be seen in a health careindustry study [2], where a secure cloud environment was leveragedto manage information and foster collaboration between emergencyhealth care professionals.

The domain of Cybermanufacturing is still in the early stagesof cloud computing and high-speed networking adoption, and hasnot seen App community vibrancy and economic growth comparedto the other domains. An App in the Cybermanufacturing contextis a data processing workflow which is executed over distributedresources that are managed via cloud APIs, and is accessible tousers on either mobile or other web-based devices. Fortunately,ongoing efforts have led to cloud-transformation of several manufac-turing application contexts, and Apps are now becoming availablein different App Marketplaces such as AweSim [3], Nimbis [4] andAmazon Marketplace [5]. The Apps perform complex modelingand simulations and offer domain expertise to enterprises. TheseApps thus simplify the R&D phase of the manufacturing process andare useful to both experts and novice users. The App Stakeholders(i.e., modeling and simulation engineers, and fabrication engineers)that offer their expertise to their customers, develop and validatetheir applications in their local infrastructure and then scale it in thepublic cloud. Benefits of this approach include the capability to rentinfrastructure on-demand without the hassle of frequent maintenanceor upgrades, and ability to access high-performance computing andelastic resources for peer-collaborations that improve App deliveryto customers.

The App stakeholders face two fundamental challenges in cloudengineering of their App Marketplaces. The first challenge relates tothe App developers’ need to orchestrate cloud resources (i.e., com-putation, networking and storage) to perform chaining of Apps toproduce re-usable designs for customers based on collective exper-tise of multiple designers. The second challenge is to suitably pricethe Apps. In this paper, we address these challenges by proposinga novel cloud architecture shown in Figure 1. The purpose of thiscloud architecture is to support an ‘App Marketplace’ frameworkthat presents an interactive (human collaborations) and interoperable(design interface standardization) ‘Factory-as-a-Service’ solution.

Figure 1: Advanced Manufacturing App Marketplace cloud archi-tecture showing IaaS, PaaS, and SaaS components in a Factory-as-a-Service Solution

We coin the term ‘Factory-as-a-Service’ solution from a Cyberman-ufacturing domain standpoint, which encompasses point solutionsin the cloud computing area such as ‘Software-as-a-Service’ (per-ceived in Cybermanufacturing as App Chaining), ‘Platform-as-a-Service’ (perceived in Cybermanufacturing as App Hosting), and‘Infrastructure-as-a-Service’ (perceived in Cybermanufacturing asApp resources based in a cloud infrastructure).

We envision this framework to have an ‘App Runtime Manage-ment’ capability that allows App developers to use Create/Add/Pub-lish/Delete App functions in the App Marketplace for customers toaccess and purchase. App Monitor function serves to profile cloud re-source usage and assist in billing of on-demand resource reservations.The framework components interface with each other using popu-lar web service mechanisms i.e., RESTful web services, which aresimilar to the Application Programming Interfaces (APIs) supportedby major public cloud providers such as Amazon Web Services,IBM Bluemix and Microsoft Azure. Such a Factory-as-a-Servicesolution would thrive on agile development in Cybermanufactur-ing, and also facilitate organic collaboration and scalable sales ofnext-generation Manufacturing Apps requiring high performancemodeling and simulation, as well as advanced fabrication processes.

Towards realizing the vision of the cloud-based App Marketplaceframework that engenders a Factory-as-a-Service solution for Cyber-manufacturing, this paper describes two main platform services (ourpaper contributions) that we developed that are essential to performApp Marketplace tasks, namely: (i) App Chaining and ResourceBrokering, and (ii) Cost and Performance Monitoring. For the studypurposes of this paper, we integrate these two platform serviceswithin a use case involving a “WheelSim” product App that usesNSF GENI Cloud and Ohio Supercomputer Center (OSC) resourcesin their AweSim App Marketplace. Our study methodology providesApp developers with insights pertaining to estimation of resourcecost, App pricing issues relevant to a manufacturing Marketplace,and the corresponding performance achievable with common Appconfigurations in a cloud environment. Through our testbed experi-ments, we show benefits to an App developer in terms of: satisfactory

user experience, lower design time and lower cost/simulation. Over-all, our work in this paper helps bridge the collaboration gap betweenmanufacturing App developers and the cloud platform engineers andprovides dynamic runtime environment for App developers that al-lows them to build new Apps using iterative development which inturn can foster Agile Manufacturing.

The remainder of this paper is organized as follows: Section 2details related work. Section 3 discusses the research problems.Section 4 discusses our proposed framework solution. Section 5 dis-cusses the experiments conducted and the validation results obtained.Section 6 concludes the paper.

2 RELATED WORK2.1 Cybermanufacturing App Testbed trialsCloud infrastructures have been leveraged extensively to supportconsumer (e.g., retail supply chain or online gaming) App develop-ment and delivery in the form of software-as-a-service. Work in [8]presents a Resource Service Chain composition algorithm that isbased on allowing temporal composition of resources to use cloudresources. Similarly, work presented in [9] features a frameworkto aggregate multiple manufacturing community clouds. However,these earlier trials do not describe how to use cloud infrastructureresources for meeting Cybermanufacturing App design needs. Theexperimental App testbed efforts in our work are built upon ear-lier effort such as [10] where we setup a testbed to explore remotecollaboration and access of advanced manufacturing resources us-ing the GENI infrastructure. Other cloud infrastructures have beenleveraged as development environments in [11] [12] for buildingsoftware applications. There are currently several Marketplaces forManufacturing domain like AweSim [3], Nimbis [4] and AmazonMarketplaces [5] but they host Apps that cannot cooperate with otherApps in an interactive and interoperable manner as we envision andimplement in this paper.

2.2 Cybermanufacturing App PricingIn the context of cost accounting for the use of cloud resources andcorresponding App pricing, few works have explored issues relatedto cost accounting for different cloud resource abstractions. Authorsin [13] - [19] have explored issues related to cost accounting fordifferent cloud resource abstractions. Amongst them, work in [14]and [19] specifically address the issue of cost modeling from thecloud provider perspective. In addition, studies in [15] present a costmodel that can be used by cloud engineers to build their own internalcloud environment i.e. private clouds. Our App Marketplace cloudarchitecture builds upon relevant insights from these existing costmodels, and uses latest pricing information from resource providerssuch as OSC AweSim [16] and Amazon Web Services (i.e., theirTotal Cost of Ownership (TCO) Calculator)[17].

3 PROBLEM DESCRIPTIONIn this section, we discuss and motivate the research problems in-volved in App stakeholder efforts to transition from their sandboxinfrastructure and adopt cloud resources. The first challenge is toorchestrate service chaining within the cloud to allow creation ofdynamic new Apps using existing core Apps. The second challengeis the cost accounting and pricing analysis that is essential for Appstakeholders to evaluate feasibility of revenue generation throughthe use of a cloud infrastructure to sell their Apps.

Figure 2: Workflow of the WheelSimulator Advanced ManufacturingApp

Figure 3: Workflow of modularized Manufacturing Apps that arechained together to create complex workflows

3.1 App Chaining and Resource BrokeringTo design and develop our App Chaining approach and ultimatelyour App Runtime, we look at the workflow of an existing WheelSimApp as a case study. This App performs simulation on a wheel. Theworkflow is shown in Figure 2. This App obtains user input througha web-portal and executes a set of scripts that complete the simula-tion at a remote supercomputer based on the simulation parametersspecified by a Cybermanufacturing Engineer. The processed resultsfrom the remote supercomputer are then displayed back to the userin their web-browser or in a thin-client end-user display.

This App workflow has certain limitations. One limitation is thatthis App works on a fixed geometry of an isolated wheel, which min-imizes the capability of the App. The code developed in this contextworks for a wheel simulation only. Another limitation manifestsif another App that is similar to the existing App has to be devel-oped in future to perform simulation on a related geometry (e.g.,an automobile). In this case, the existing code for the WheelSimApp will be unusable, even though the process and workflow remainthe same. Additionally this App also integrates common workflowsinto a complex workflow without any proper validation. This couldlead to resource wastage in case of a failure of any intermediateworkflow steps. Further, it does not allow for a collaborative ex-change to enhance Apps because of its highly targeted nature andlimited use scope. Hence to overcome these problems, we look atdeveloping a dynamic App Runtime to address these limitations.Our App Runtime would convert this workflow into a modularizedworkflow resembling Figure 3. In this case, a set of core Apps worktogether through an App chaining mechanism to convert existingcomplex workflows into a sequence of simple chained workflows.The challenge here is to identify these core Apps from a catalog ofworkflows.

Identification of the Core Apps (commonly used) and extensionApps (specific to a particular product) is an important step towardsbuilding the App Runtime and hence to this end, we attempt to dis-sect the current WheelSim App that performs user-desired workflowsteps to understand its inner workings and determine the core andextension Apps. In the case of WheelSim, we found that its complexmodeling and simulation workflow can be fundamentally dividedinto: (i) Mesh generation phase, (ii) Simulation phase, (iii) Exportphase, and (iv) Post-processing phase. Upon further reflection, weconcluded that these phases can be abstracted into functional appli-cations that can perform any kind of simulation tasks based on the

Figure 4: Modularized flow of different phases of the Wheel simula-tion

inputs they receive. These Apps are Mesh & Solve, Export, Post-processing and Status/Chaining as shown in Figure 4. For simplicity,we combine the Mesh and Simulation phase into one App. Thesenew Apps would define a configuration that reflects the requirementand running those Apps would best fit the needs of the user. Hence,we design the wheel simulation to encompass all these Apps.

These Apps as a part of an App Runtime (described in detaillater in Section 4.1) can be used to implement new Apps whichcan use other geometries which can drastically reduce the design,development and deployment cycles from several weeks to a fewclicks. Additionally, the App Runtime would allow these Apps to bepublished in an App Marketplace, and benefit both the developers,who build these Apps and customers, who may not have expertisein the field but have a specific use case for advanced manufacturingwhich they find already implemented in the Marketplace.

3.2 Cost And PricingFor developing a suitable pricing strategy to price Apps and generaterevenue, it is first necessary to understand the costs associated withthe utilization of the cloud environment resources. Accountingcosts of a cloud platforms unique environment is however complex.There are different costs that are incurred due to different entities thatinteract with the infrastructure. The Cloud Resource Providers (CRP)provision necessary resources for the Manufacturing Enterprises(MES) who then employ Cybermanufacturing Engineers to developand deploy Apps for their customer. We explain each of these entitiesin detail:

Firstly, we have the CRP who have setup the App Marketplaceenvironment composed of infrastructure and services that bring to-gether other interested parties to engage in sale and purchase ofApps that run on our cloud architecture envisioned in Figure 1. Theprovisioning include setup of the infrastructure components suchas web servers, network connectivity, as well as compute jobs ona supercomputer. The CRP deploys necessary software such asOperating Systems (Windows, Linux etc.), development environ-ment tools such as ParaView/HyperWorks/OpenFOAM and the AppRuntime based on the requirements. The CRP also sets up accessmechanisms to allow MEs and their customers to utilize resourcesprovisioned for this environment. Additionally the CRP deploysnecessary virtualization software such as VMware Horizon [20] toefficiently utilize existing resources. All these components have as-sociated costs and have to be accounted for in the pricing of servicesto generate revenue.

Secondly, we have the MEs that utilize the above environmentsetup. The MEs can utilize this environment to develop innovative

Apps in an App Marketplace environment where they can be mar-keted and sold to various customers. MEs incur costs associatedwith utilizing this environment and they need to be able to generaterevenue from the sales of these Apps.

Finally, we also have the customers who use the Apps devel-oped by the MEs. These customers could be small businesses ina particular manufacturing sub-domain that lack the know-how ofprovisioning necessary resources and building workflows to obtainrelevant results. They could also be large corporations that requirethe functionality of the Apps in the marketplace and find that us-ing the Apps readily available for an affordable price is a betterchoice compared to developing their own flavor. They can henceutilize these Apps that are not only cheaper but provide an interactiveand rich web based interface that increases efficiency and usabilitycompared to traditional approaches.

All these entities incur costs, and thus the effort to model costsfor each of them and efforts to ensure fair pricing are all complexendeavors. We mainly identify the costs for each of these entitiesto be either fixed cost or variable cost or both. Based on our earlierstudy [19], we also have found that it is important to also considerthe virtual machine usage that incurs costs when specific softwarepackages are run by the MEs to develop Apps and view results. Ouraim is thus to develop a cost model to account for the costs incurredby the CRP and the MEs and also address the pricing issues forboth of them. The other expectation from our model is to pricethe resources and the Apps to ensure fairness for the customers.We discuss the cost model along with the Accounting Service weimplemented later in Section 4.2.

4 PROPOSED FRAMEWORK FOR APPMARKETPLACE

In this section, we first detail our Resource Brokering Service imple-mentation that is needed for effective App chaining. Next, we detailour Accounting Service implementation that helps App developersto suitably price their Apps.

4.1 Resource Brokering Service ImplementationThe App chaining is achieved through the Resource Brokering Ser-vice by implementing a dynamic App Runtime environment. TheApp Runtime for the Manufacturing App Marketplace consists oftwo fundamental components: Core Apps and their ManagementFunctions. These components interact with each other to generatenew workflows and execute actions that help incorporate new Appsin the Marketplace. We discuss each of the two components in detail.

4.1.1 Core Apps in the App Runtime. Core Apps are thebasic Apps that can be used in combination to create new Apps. Asthese Apps are related to each other, the output from one App isredirected and served as input to the next thereby simplifying theApp creation process. Based on the type of App, the monitoring ofthe jobs scheduled on a remote supercomputer and the correspondingresult retrieval would be unique. Hence, a new App to monitor theApp status always needs to be part of the App chain, and its purposeis to check the App status and retrieve results in accordance withthe App’s monitoring objectives. Considering the WheelSim Appcontext, we present details of an exemplar set of core Apps in thefollowing:

Mesh & Solve App. This App receives essential informationfrom the user that are necessary to generate a mesh with its specificboundary conditions and also take inputs of the simulation param-eters. The input is obtained in the form of a web form from theuser. User also has the option to choose a default mesh specifica-tion (wheel) or upload a custom mesh specification based on theirrequirement. Based on the input provided, appropriate solvers areselected and the App begins the process of mesh generation. Afterthe mesh generation, the simulation is carried out on the use casescenario i.e., checking the lift and drag forces on the object specified.The output of this App is the mesh and the raw simulation data.

Export App. This App receives input from the Mesh & SolveApp, the input being the generated mesh with the simulation data.The input necessary is pulled from an isolated storage location ofthe remote supercomputer and is subsequently processed. Thislocation is isolated and each user has a unique location. Based onthis information, the Export App begins the process of convertingthe raw simulation data into 3-D models. Upon completion, theApp generates data that is compatible with VTK format and hencecan be viewed in specialized software such as ParaView whichis not commonly distributed and has to be installed in the cloudenvironment.

Post-processing App. This App receives input from the ExportApp. The input in this case are the files that store the simulation data.This input is pulled from the same unique location as in the case ofthe Export App. After gathering the input data, this App performspost processing which essentially converts the files with simulationdata into image data (PNG) which are universally readable from anyoperating system.

Status App. This App is the connecting link between the otherthree Apps. This is the App that is responsible for chaining theapplications. It also monitors the status of the currently runningApp, checks for its completion and then runs the next App. It alsoretrieves the appropriate results from the remote supercomputer afterthe execution of an App.

Additionally, given the unique combination of the three phasesto produce relevant results, there would essentially be three Appconfigurations/workflows to any new App that combines the coreApps to produce results required by the user. The Apps discussedso far are isolated and require a medium to be used. This requiresthe App Runtime to have certain functions in order to successfullyutilize the potential of these Apps. We discuss these functions next.

4.1.2 Management Functions in the App Runtime.

Create. This function gathers inputs from the user about theApp configuration, whether to use custom or default mesh, the Appname description etc. and creates an App with the required setup,isolation and initial information. The relevant data is added to thedatabase. It also reserves storage at both the web server and theremote supercomputer.

Add. This creates a user specific instance of the App throughwhich the customer can run their App in isolation. The users candelete their instance of the App using the Delete function.

Delete. The Agent removes any storage information of the Appfrom the web server. App users i.e., the customers can only deletetheir instance of the App and only the App owner i.e. the Manufac-turing Enterprise developer can delete the actual application.

Figure 5: Workflow of App Runtime Manager

Publish. Based on the input, this function can either publish anunpublished App or hide a published App. The Marketplace wouldthen be updated with the current list of the Apps.

Monitor. This function monitors changes in the resource utiliza-tion. It then invokes the Accounting service for cost computation.

Based on the action performed by the user, the App Runtime usesa general workflow to invoke these functions as shown in Figure5. The user essentially executes one of the actions described abovethrough the web portal. The App Runtime Manager invokes theappropriate function from a library based on the input. The codeinvoked by the manager performs relevant updates to the databaseand updates the information in the App Marketplace.

4.2 Accounting Service and Cost ModelThe Accounting Service is the book keeper of this framework. Ittracks the utilization of the environment by the various entities andcharges them a fair price based on an adopted cost model. Thisservice is invoked each time there is a change in the usage of thesystem by an entity. It monitors usage by both MEs and their cus-tomers. Compute resource usage, VM usage etc. are few of theresources monitored by this service. The service could be invokedby another platform service or it could be invoked by an Agent thatupdates this service about usage information. The invoker carriesessential information about the resource for which the usage is tobe accounted for. Based on the resource, the service queries thecost model which stores the information about the various resourcesand the related costs associated with each of the entities. This infor-mation is processed by the service and existing information aboutthe charges to that entity is updated in a database that is under thecontrol of the service. Based on the preferred cycle, the serviceinvokes related supporting services to generate bills for the entities.The cycle could be weekly, bi-weekly, monthly etc. This serviceensures the continuity of the system by ensuring that the SPs revenuegoals are met from the usage of this environment by the MEs.

There are three main entities that use this infrastructure and wefocus in depth on the costs of the CRPs and the MEs as their costaccounting is more complex compared to the customers.

4.2.1 Cloud Resource Providers Cost & Pricing model.The CRP does all the grunt work in the system and the range of tasksvary from setting up the infrastructure to deploying the software stackthat runs the Marketplace. Hence, there would be an initial cost ofacquisition and then monthly recurring operational cost for computeresources such as servers, networking equipment, etc. Dependingon the contract and provider, the network cost could be an unlimitedusage, fixed monthly cost or variable cost based on /Gb/hour. Inour case, we consider this as a fixed cost i.e., /month recurring

monthly. In our environment, there is also a variable cost associatedwith usage of VMs and usage of remote supercomputer resourcesin terms of Resource Units (RUs) by the MEs. Hence, the total costincurred would be a combination of fixed and variable costs and canbe depicted in the following equation (1)

C total = Cacq +Cfixed monthly ∗T +Cvariable monthly ∗T (1)

Here, Cacq is the cost of acquisition of hardware & software re-sources, Cfixed monthly is the monthly cost incurred by the CRPtoward operations of the system, T is the total months of operationand Cvariable monthly would be the summation of remote supercom-puter resources by all the MEs in terms of Resource Units (RUs)consumed which can further be broken down as:

Cvariable monthly = CRU ∗Total RUs consumed (2)

where, a RU is the metric for the compute capacity used, and To-tal RUs consumed is the total of all the RUs consumed by all theMEs. RU is calculated as follows within OSC AweSim

RU = 0.1 ∗Tt (3)

Here, Tt is the total CPU time in hours for all CPUs

Tt = wall time ∗max ppn ∗ nodes requested (4)

where wall time is the time taken for a simulation job, max ppnis the maximum processors per node and nodes requested are thenumber of nodes requested by the user to be used in the simulationjob.

The service provider essentially invests significant capital into anApp development effort, and hence we now suggest ways for returnon investment. The best approach for this is to recover costs over aperiod of time from the participating MEs. The following equationrepresents the cost incurred by a single ME as a fee for using theservice:

Cmanf =Cacq

(M ∗ N ) +Cfixed monthly

M+Cvariable monthly (5)

where Cmanf represents the price charged to the MEs monthly asa fee, Cacq is the cost of acquisition of the hardware and software,M represents the number of participating MEs using the environ-ment, N is the period in months for return of investment of thecost incurred by the CRPs, Cfixed monthly represents the recurringmonthly charges incurred by the CRPs towards maintenance of theenvironment and Cvariable monthly is the variable cost that includesthe remote supercomputer usage and VM usage by a particular ME.We suggest the value of N to be between 24-to-36 months. At theend of the recovery period, we reach a break-even point and thecharges henceforth are profits towards the first term in the equation.

4.2.2 Manufacturing Enterprise Cost & Pricing model. TheMEs utilize this cloud environment to speed up their App devel-opment cycle. They gain benefits in terms of access to computeresources on-demand, tools to create new Apps and an array of ser-vices to help them in their endeavor to develop Apps. Their costaccounting is simpler compared to CRPs and is done monthly. TheMEs total costs Cmanf total would include two fundamental charges:Fixed monthly Cmanf fixed and Variable monthly costs Cmanf variable.Thus, we have -

Cmanf total = Cmanf fixed +Cmanf variable (6)

where Cmanf fixed is the fixed monthly cost for this ME by theSP for using the infrastructure. This is determined by the Cacq andCfixed monthly costs and can be represented as follows:

Cmanf fixed =Cacq

(M ∗ N ) +Cfixed monthly

M(7)

where the terms on the right-hand side are the first two terms inequation (5) which provide the fixed monthly cost for using this en-vironment as charged the CRPs. The variable cost for manufacturersis the charges for using the remote supercomputer resources and theVMs

Cmanf variable = CRU ∗ RUs consumed +n∑i=1

Bi ∗ t (8)

where CRU is the cost per RU consumed, RUs consumed is thenumber of RUs used for various computations by this ME in thecurrent month, Bi is the Base rate charged for the ith VM used and tis the number of hours used. This equation accounts the computeresources and VMs used by a ME.

We now look at pricing models for MEs. Due to fixed nature ofthe App configurations, we can obtain the cost of a single App rundeveloped by the ME as follows:

Capp usage = CRU ∗ RUs consumed (9)

For a single run of the App, the above equation gives the costincurred. If the ME decides to charge their customers per run, thenwe can calculate a price Papp usage per run of an App such that forN App runs they hit the break-even point with their costs:

Papp usage ∗ N = Cmanf fixed +Cmanf variable (10)

where Cmanf variable accounts for the Capp usage for N runs beyondwhich the App is profitable and any other VM usage charges ap-plicable and Cmanf fixed is the fixed monthly cost as calculated inequation (7). Based on the number of customers and period of re-covery, the MEs can charge an appropriate price for their Apps fromthe customers.

An alternative approach would be to charge the customers accord-ing to their individual usage for an App i.e., the cost for the customerwould depend on the amount of resources or RUs consumed. Inthis case, the ME can charge the customer a price PRU for the RUsconsumed such that -

PRU ∗ RUs consumed = Cmanf fixed +Cmanf variable (11)

Beyond these number of RUs based on PRU, the MEs start gainingprofits.

5 IMPLEMENTATION AND RESULTSIn this section we present the testbed configuration of our WheelSimApp developed by TotalSim Engineers in Dublin, OH in collabora-tion with their remote office in Los Angeles, CA for their customersin North Carolina. Following this, we describe the experimentsconducted to validate the testbed of the App that leverages the GENIpublic cloud infrastructure and OSC resources. Finally, we discussin detail the results of our Cost and Performance model for theWheelSim App context.

5.1 App provisioning resultsWe set up the testbed shown in Figure 6 that included the followingcomponents:

Figure 6: Cloud Testbed setup for a Manufacturing Company

• Public Cloud: A GENI Rack at Metro Data Center whichacts as the resource broker and provides a network overlayinfrastructure

• Elastic HPC backend at OSC to run jobs charged on a run-by-run basis only, thus making it a cost effective computeresource

• Layer-2 connectivity established between the GENI Rack,OSC, TotalSim’s office and remote collaborator at Los An-geles

• A web-portal accessible through Internet is hosted on theGENI Rack to enable customers to run this App that sched-ules batch jobs at OSC

The GENI Rack at Metro Data Center provides the network over-lay infrastructure through a GENI slice. The slice reserved containsa HP bare metal computer with 1 Intel Xeon X5650@2.67GHz pro-cessor, 50 GB of RAM and 1 TB of disk capacity. The slice alongwith a private network and VLAN have been initially reserved forlong-standing experiments over several months timeframe. ESXihypervisor is installed with VMware Horizon as the provisioningmanagement middleware. The installation and administration areperformed remotely with SSH, RDP, and through VMware toolssuch as VMware vSphere web-client.

5.2 App Chaining ImplementationOur App Runtime web framework implementation on this testbedis based on the design presented in Section 4.1. The core Apps aredeveloped with an additional Status App that helps in chaining ofthese Apps based on the configuration. We also developed func-tions for the App Runtime that help in creation, monitoring andmaintenance of these distributed Manufacturing Apps. From theuser’s perspective, this framework appears to be a normal websitewith no apparent distributed architecture. The user is presented withan easy-to-use Web user interface and interactive components thatenable complex workflow orchestrations with few simple mousedrags and clicks. This development was done using PHP, HTML,MySQL, web services and shell scripting. Authentication is per-formed through an Active Directory and login information of eachentity utilizing the environment are added by an Admin. We nowdescribe our App Runtime functions:

Figure 7: A workflow for App execution using our App Marketplace web framework for manufacturing enterprises (MEs) involving - Step 1: theME defines Apps and associated costs; Step 2: a custom workflow is created by specifying configurations for Mesh, Export, Post-processing and Statusmonitoring; Step 3: MEs interact with the execution processes and obtain App information

5.2.1 App creation. The MEs after authentication, are dis-played a dashboard which they can use to easily get started on theApp creation process. The workflow is illustrated in Figure 7. Tocreate a new App, the ME can simply click the ‘Create’ button toinitiate the App creation workflow. The ME is then presented witha web page in which the ME can drag and drop core Apps into aworkflow, the status application is added by default to any App as itis essential in monitoring the progress of the execution of the App.There is also a provision to specify custom geometry for simulation.MEs can upload a template of a geometry file in .STL format and theappropriate Mesh is selected based on the template uploaded. Lastly,the MEs are required to provide a name, description and cost fortheir App. The App is then created by clicking ‘Create App’ button.

5.2.2 App execution. Both the MEs and customers can exe-cute Apps. Depending on the App configuration, Status App eitherdisplays the results or executes another App. The Monitoring func-tion is invoked each time the user runs an App. The App executionvaries with the type of mesh and the configuration of the App. Aworkflow of App execution is shown for a wheel geometry and Post-pro configuration in Figure 8. The user first invokes the App throughthe web interface using the ‘Run App’ sub menu button and thenspecifies the simulation parameters through a form. These param-eters generate a configuration file that is sent to OSC for furtherprocessing. The web service invokes scripts that execute the simula-tion as specified in the configuration file. Each core App has a uniquescript. The progress of the execution can be measured using theStatus App through the sub menu button ‘App Status’ provided. PostApp execution, the Status App validates the execution and analyzesthe App configuration and presents an interface to run the next Appin the workflow. The user is next provided with a button to either‘Run next App’ or ‘View Result’ depending on the App configurationand the last App that was executed. The Status App also retrievesthe result based on the App configuration. The ‘View Results’ submenu button displays the current and past results. Using this page,the user can view the results and analyze the simulation results. Anexpert in the field can judge and make informed decisions about thedesign based on the simulation results presented. In case of Figure 8,we can see the simulation results of the WheelSim that are presentedto the user. Finally, the results can be downloaded as PVTP file orviewed online.

The App Runtime web framework bridges the gap between latestweb technologies and simulation software. The interactive nature of

the App Runtime enables users to drag and drop applications to cre-ate complex workflows. Apps such WheelSim which earlier took amonth of development time, can now be created in just a few mouseclicks. Thus, the App Runtime provides App developers with toolsthat allow them to focus their effort and time in designing and devel-oping complex software using existing utilities to deliver innovativeproducts, versus worrying about hardware/software configurationsin their sandbox environments.

5.3 Cost and Performance Model ResultsWe now describe the cost and performance model analysis we con-ducted on the WheelSim testbed. We use the cost model presentedearlier in Section 4.2 and analyze how the various entities incurcosts based on our assumptions. Specifically, we evaluate costs andundertake a break even study. We also undertake a performancestudy where we observe how the variation in compute resourceseffect the execution and the cost.

5.3.1 Assumptions. We use the proposed cost model for ourframework with some simulated costs to calculate how much itwould cost for MEs to use this environment. This is an essentialrequirement and would shed some light on how they can price theirApps. Hence, we now explore the various costs incurred to themusing the cost model. We make the following assumptions:

• The number of participating MEs are 10 and period ofrecovery is 36 months

• Cost of acquisition is calculated for a single server with 24logical cores, 48 GB RAM, HDD 1 TB

• Operation and acquisition costs are calculated according toAWS’s TCO calculator [17]• We use VMware Horizon environment as the virtualization

software• We do not yet consider the licensing cost for VMs provi-

sioned with Windows operating system

Table 1 provides the breakdown of the acquisition and monthly fixedoperational cost for the environment selected.

Table 2 provides the specifications for the variable costs wherethe RU value is directly taken from OSC and the VM usage cost isobtained from VMware chargeback center.

For a single ME, the fixed monthly cost would be calculatedaccording to equation (6) and is approximately $358. It is difficult topredict how the variable costs will add to the total cost of the ME but

Figure 8: A workflow for App execution using our App Marketplace web framework for customers involving - Step 1: After obtaining App infor-mation, users can input workflow data and execute processes; Step 2: workflow status data is available for analysis and reconfiguration; Step 3: finalresults are available for users in the form of data files and plots.

Table 1: Cost of acquisition and monthly operational cost

Name Specification Cost($)Server cost 24 logical cores, 48GB RAM,1TB HDD 11,879Software cost VMware,Windows OS 10,665Monthly cost Rack, network, power etc. 2954

Table 2: Metrics for the variable cost

Name Metric Cost($)Supercomputer usage RUs 0.4

VM usage usage/VM/hour 0.003

we will make an assumption for a typical usage of the system andthen calculate the price accordingly. Let us assume that ME1 uses100 VMs for 40 hours every week. Let us also assume that total RUsconsumed by ME1 as a result of App execution by their customers is100 RUs. Then, according to the above table the variable cost for amonth with this kind of usage would be $88, and hence the total costwould be $445. This price gives access to a remote supercomputerfor computation, a development environment which provides toolsfor agile development and an App Marketplace for publishing theirApps. If MEs tried to do their own setup of this infrastructure, wenote that it would be several times more expensive, similar to howmuch owning a cluster would cost.

Additionally, based on their requirements, the MEs can run theirApps to complete execution with minimum waiting time or minimumexecution time. Since we run the simulation component of theseApps on a remote supercomputer, we have the capacity to requestrequired number of processors when submitting jobs and these jobswould be placed in a queue. Based on the order and availability ofresources, these jobs get executed at the remote supercomputer. Thenumber of processors requested have a direct effect on the cost andthe duration of the simulation. We performed a cost vs. performanceanalysis where we studied the effect of number of processors on theseparameters. For each App, we measured the total cost by varyingthe number of processors. We also measured the total time takento complete the tasks. With the exception of Post-pro which uses asingle processor to execute, the Mesh & Solve and the Export Appboth use the specified number of processors to execute. Table 3 isthe tabulated data of the results for a simulation on a wheel geometry.

It can be observed that the Export and the Post-pro RUs increaseat a lower rate and are only a fraction of the cost of the Mesh &Solve App. We can observe from Figure 9a that, as the number

(a) Time for different processors

(b) Cost for different processors

Figure 9: Correlation between No. of processors and Cost/Time

of processors increase, the time for the job reduces. We noticethis trend until the number of processors reach a threshold (96),beyond this point, the time actually increases which is an anomaly.Further investigation showed that the overhead from inter-processcommunication causes delays and hence the total time increases.From this experiment we found the optimal number of processorsfor minimum execution time to be 96.

The cost vs. number of processors plot Figure 9b is fairly linearand increases as the number of processors increase. This is verylogical as we are being charged more for the additional computeresources being requested. With this knowledge, we now look at thepricing for the MEs and analyze how they can generate revenue tooffset their costs.

Table 3: Cost and the execution time for running a simulation with different number of processors

# cores Mesh & Solve (RUs) Export (RUs) Post-pro (RUs) Total RUs Total Cost($) Total time(s)24 0.775 0.098 0.080 0.954 0.381 143148 0.996 0.202 0.150 1.349 0.539 101272 0.1.31 0.284 0.204 1.798 0.719 89996 1.690 0.386 0.114 2.192 0.876 822120 2.303 0.466 0.34 3.11 1.244 933144 3.172 0.564 0.408 4.144 1.657 1036

5.3.2 App Pricing Analysis. Based on the discussion aboutthe various considerations of the fixed and variable cost, it becomeseasier now to perform a pricing analysis for the MEs so that theycan generate revenues by having their customers run the Apps inthis environment. The MEs can charge their customers as discussedin 4.2.2 based on the App price per run or vary the price of the RUscharged. We explore both these pricing strategies and undertake abreak-even analysis for both.

Table 4: Cost vs. Revenue based on number of App runs

# runs Cost $) Total cost($) Revenue ($)10 6.76 412.74 10020 13.52 412.74 20020 13.52 419.51 20030 20.28 426.27 30040 27.05 433.03 40050 33.81 439.79 50060 40.57 446.56 60070 47.33 453.32 70080 54.10 460.08 80090 60.86 466.85 900

100 67.62 473.61 1000

5.3.2.1 App Run pricing. In this case, the MEs fix a price foreach run of the App. We consider the basic Mesh & Solve Appthat performs simulation on a wheel geometry as an example forthis analysis. The total monthly cost for the MEs can be computedfrom equation (7). From earlier calculation, we know that the fixedmonthly cost to a ME is $358. We impose a restriction on the VMusage to convert it into a fixed cost and then account for differentApp runs to check the break-even point. For this use case, we assumethat for a month, 100 VMs were used by the ME for 40 hours perweek. At $0.003 hourly usage, this amounts to $48 for the wholemonth. We consider the case of minimum execution time and usethe cost for the case of 96 processors. In this case, the cost for 1 Apprun is $0.67. Let us assume that the ME charges the user $10 foreach App run. We now compute the number of App runs required tohit break-even point between the total cost and the Revenue. Table 4illustrates this data. It can be observed that as the number of runsincrease, the variable cost increases along with it and the revenueincreases accordingly and between 40 and 50 App runs we break-even. We plot a break-even curve between the cost and the revenueFigure 10a. We find that the break-even point for this case is around45 App runs. Beyond this, the MEs earn profits over their monthlycosts. In a month, 45 App runs is easily achievable and the customeralso is paying a very low price for each run. Hence, this pricing isbeneficial for both MEs and customers. This case just considered

(a) Variable No. of App runs

(b) Variable No. of RUs consumed

Figure 10: Breakeven points for Cost/Revenue

one App, if the ME is able to create popular and reusable Apps, theprofit earned will be considerably higher.

5.3.2.2 RUs based pricing. In this case, the MEs charge a dif-ferent price per RU consumed. We follow the same assumptionsas in the previous case. The RUs consumed for a typical Mesh& Solve App is 1.69 RUs for 96 processors. The cost per RU tothe ME is $0.4. The MEs can mark it up and charge a price of $2per RU consumed by the customer. With these assumptions, wevary the number of RUs and compute the varying cost and revenues.Table 5 illustrates this data. It can be observed that the variablecost increases with the number of RUs consumed. The revenue alsoincreases and the break-even point in this case is reached for around253 RUs. We provide a view of the App runs as well to indicateat what number of App runs will the break-even point be reached.We once again plot the break-even curve in Figure 10b. We found

that this is equivalent to 150 App runs. Hence in this case, the costincurred to the customer is less in comparison to the previous case.From the MEs point of view, in the previous case the break-evenwas reached at 45 App runs, but in this case the same is around150. Both these pricing methodologies are almost equivalent butmay change depending on the geometry being simulated and theintegrated workflows present in the App. There are few other pricingapproaches such as Subscription, Freemium etc. which can be usedby the MEs, but their discussion is beyond the scope of this paper.

Table 5: Cost vs. Revenue based on number of RUs consumed

# runs # RUs Cost($) Total cost($) Revenue($)20 33.8 13.52 419.51 67.640 67.6 27.05 433.03 135.260 101.4 40.57 446.56 202.880 135.2 54.10 460.08 270.4

100 169 67.62 473.61 338120 202.8 81.15 487.13 405.6140 236.6 94.67 500.66 473.2160 270.4 108.20 514.18 540.8180 304.2 121.72 527.71 608.4200 338 135.25 541.23 676220 371.8 148.77 554.76 743.6240 405.6 162.30 568.29 811.2

6 CONCLUSIONA cloud architecture for advanced collaborative manufacturing hasbeen presented in this paper that allows agile methods to be adoptedin this domain. This architecture leverages public cloud infrastruc-ture to provide scalable and on-demand resources in a cost-effectivemanner. It encompasses two novel services that we designed and de-veloped at the platform level: (i) Resource Brokering Service and (ii)Accounting Service. We successfully configured a testbed environ-ment for a ‘WheelSim’ product App and conducted a performanceand App pricing study for manufacturing enterprises.

We described an App Runtime framework that is a part of theResource Brokering Service which can be used to aid developersto build new complex applications from a set of core Apps. Wediscussed App Runtime functions that can help in incorporatingthese Apps into an exiting App Marketplace. Using our App Run-time, we showed how new Apps can be created, published, utilizedand monitored. Our work also proposed a cost model for ServiceProviders and Manufacturing Enterprises to account for their costsfor using our environment, and for pricing the Apps.

We also described a cost pricing case study as a part of the Ac-counting Service where we determined the actual cost and projectedrevenue for the Manufacturing Enterprises. Additionally the casestudy provided us with an optimal number of compute resourcesdeployed by the inflection point for minimum execution time.

As part of the future work, we plan to add more diverse Apps tothe current core Apps to increase the diversity of simulations thatcan be performed by our App Runtime. We also plan to diversifyour testbed by adding additional cloud providers and HPC resources.This would lead to evolution of the way Apps are created and de-ployed in an App Marketplace and can allow rapid innovations inmanufacturing products in important sectors such as automobilesand pipes.

ACKNOWLEDGMENTSThis material is based upon work supported by the City of Dublin,Mozilla Foundation and National Science Foundation under awardnumbers CNS-1347889, CNS-0714770. Any opinions, findings, andconclusions or recommendations expressed in this publication arethose of the author(s) and do not necessarily reflect the views of theCity of Dublin, Mozilla Foundation or National Science Foundation.

REFERENCES[1] J. Yu, J. Ni, “Development Strategies for SME E-Commerce Based on Cloud Com-

puting”, Proc. of International Conference on Internet Computing for Engineeringand Science, 2013.

[2] V. Koufi, F. Malamateniou, G. Vassilacopoulos, A. Prentza, “An Android-enabledMobile Framework for Ubiquitous Access to Cloud Emergency Medical Services”,Symposium on Network Cloud Computing and Applications (NCCA), 2012.

[3] AweSim : https://awesim.org/en/[4] Nimbis : https://www.nimbisservices.com/[5] Amazon AWS Marketplace : https://aws.amazon.com/marketplace[6] M. Berman, J. Chase, L. Landweber, A. Nakao, M. Ott, D. Raychaudhuri, R.

Ricci, I. Seskar, “GENI: A federated testbed for innovative network experiments”,Elsevier Computer Networks, 2014.

[7] R. Ricci, E. Eide, “Introducing CloudLab: Scientific Infrastructure for AdvancingCloud Architectures and Applications”, login USENIX Magazine, Vol. 39, no. 6,pp. 3638, 2014.

[8] H. Li, K. C. C. Chan, M. Liang, X. Luo, “Composition of Resource-Service Chainfor Cloud Manufacturing”, IEEE Transactions on Industrial Informatics, Vol. 12,No. 1, pp. 211-219, 2016.

[9] C. Yang, W. Shen, X. Wang, T. Lin, “A Framework for Integrating MultipleManufacturing Clouds”, Proc. of IEEE International Conference on Systems, Man,and Cybernetics (SMC), 2015.

[10] A. Berryman, P. Calyam, J. Cecil, G. Adams, D.Comer, “Advanced Manufacturinguse Cases and Early Results in GENI infrastructure”, Proc. of GENI Research andEducational Experiment Workshop (GREE), 2013.

[11] D. Nasui, V. Sgarciu, A. Cernian, “Cloud-Based Application Development Plat-form for Secure, Intelligent, Interlinked and Interactive Infrastructure”, Proc. ofIEEE Symposium on Applied Computational Intelligence and Informatics, 2013.

[12] L. Cocco, K. Mannaro, G. Concas, “A Model for Global Software Develop-ment with Cloud Platforms”, Euromicro Conference on Software Engineering andAdvanced Applications, 2012.

[13] Y. Zhai, M. Liu, J. Zhai “Cloud Versus In-house Cluster: Evaluating AmazonCluster Compute Instances for Running MPI Application”, IEEE/ACM Conferencefor High Performance Computing, Networking, Storage and Analysis (SC), 2011.

[14] H. Wang, Q. Jing, S. Jiao, R. Chen, B. He, Z. Qian, L. Zhou “Distributed SystemsMeet Economics: Pricing in the Cloud”, Proc. of USENIX HotCloud, 2010.

[15] W. Mach, E. Schikuta, “A consumer-provider cloud cost model consideringvariable cost”, Proc. of IEEE DASC, 2011.

[16] OSC AweSim Resource Price Charging Reference -https://www.osc.edu/supercomputing/software/general#charging

[17] Amazon Total Cost of Ownership (TCO) Calculator Reference -http://aws.amazon.com/tco-calculator

[18] GENI Resource Schema Reference - http://groups.geni.net/geni/attachment/wiki/GEC11RSpec/ComputeOntology-071411.pdf

[19] M. Singhal, J. Ramanathan, P. Calyam, M. Skubic, “In-the-Know: Recommen-dation Framework for City-Supported Hybrid Cloud Service” IEEE/ACM 7thInternational Conference on Utility and Cloud Computing (UCC), 2014.

[20] P. Calyam, S. Rajagopalan, S. Seetharam, A. Selvadhurai, K. Salah, R. Ram-nath, “VDC-Analyst: Design and Verification of Virtual Desktop Cloud ResourceAllocations”, Elsevier Computer Networks Journal (COMNET), 2014.