GINI on a Cloud

GINI on a CloudCloud computing for internet emulator1Professor Muthucumaru Maheswaran

Team MembersMia HocharSimon FoucherDavid El AchkarMarc AtieGINI on a Cloud March 20102David El Achkar, Simon Foucher, Mia Hochar, Marc AtieProfessor Muthucumaru Maheswaran

Mia HocharSimon FoucherDavid El AchkarMarc Atie

DatabaseRecomendationsGINI AlterationsDesign DilemmaEvolution of GINIDispatcherObjectivesGINI on a Cloud March 20103David El Achkar, Simon Foucher, Mia Hochar, Marc AtieSchedulerConclusionEvolution of GINIWhat is GINI?GINIs FeaturesImprovements Overview

GINI on a Cloud March 20104David El Achkar, Simon Foucher, Mia Hochar, Marc AtieGINI DefinedWhat is GINI?Toolkit for creating virtual micro InternetsCreate midsize networksProcess creates virtual instances of network elementsPurpose:Teaching and learning toolSuitable to many levels of knowledgeFuture applications

GINI on a Cloud March 20105David El Achkar, Simon Foucher, Mia Hochar, Marc AtieGINI Snapshot

GINI on a Cloud March 20106David El Achkar, Simon Foucher, Mia Hochar, Marc AtieFeaturesSimple to use GUIFully open-source systemStandard compliant router Ability to implement additional protocols or new network elementsGINI on a Cloud March 20107David El Achkar, Simon Foucher, Mia Hochar, Marc AtieImprovements OverviewGINI on a Cloud March 20108ClientWorkerFront EndBack EndDavid El Achkar, Simon Foucher, Mia Hochar, Marc AtieImprovements OverviewGINI on a Cloud March 20109ClientWorkerFront EndBack EndDavid El Achkar, Simon Foucher, Mia Hochar, Marc AtieImprovements OverviewGINI on a Cloud March 201010Front EndDispatcherDatabaseSchedulerBack EndBack EndBack EndBack EndServerWorkersClientDavid El Achkar, Simon Foucher, Mia Hochar, Marc AtieDrawbacksComplicated Installation process

GINI on a Cloud March 201011David El Achkar, Simon Foucher, Mia Hochar, Marc AtieDrawbacksOS Compatibility issues

GINI on a Cloud March 201012David El Achkar, Simon Foucher, Mia Hochar, Marc AtieGINI Components:Improvements

Basics

DatabaseDatabaseSchedulerSchedulerDispatcherDispatchergRouterGiniLinuxuSwitchWGINIgLoadergBuilderGINIs ComponentsgBuildergLoadergRouterGiniLinuxuSwitchWGINIGINI on a Cloud March 201013David El Achkar, Simon Foucher, Mia Hochar, Marc AtieDispatcherOverviewDispatcher RequestConnection CreationDispatcher LoggingGINI on a Cloud March 201014David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

OverviewRequest/ReplyFront EndDispatcherDatabaseI need a workerWorkers DNSLowest CPU UsageGINI on a Cloud March 201015David El Achkar, Simon Foucher, Mia Hochar, Marc AtieDispatcher RequestRequest/ReplyDispatcherDNSIPuptimeLab2-17134.204.2.560.112Lab2-29134.204.2.370.523Lab2-20134.204.2.1120.938Return worker DNSextractWorkerFromDB()GINI on a Cloud March 201016David El Achkar, Simon Foucher, Mia Hochar, Marc AtieConnection CreationConnection EstablishedGINI on a Cloud March 201017Front EndBack EndSSH TunnelDavid El Achkar, Simon Foucher, Mia Hochar, Marc AtieDispatcher LoggingRecording ConnectionDispatcherDatabaseSend:Client IPWorker DNSConnection Recorded:time stampGINI on a Cloud March 201018David El Achkar, Simon Foucher, Mia Hochar, Marc AtieGINI AlterationsDesign ParadigmConnection to ServerDesign DilemmaGINI on a Cloud March 201019David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Design ParadigmTransparent to userComponent add-on rather than alterationGINI remains independentOnly altered the Configuration file

GINI on a Cloud March 201020David El Achkar, Simon Foucher, Mia Hochar, Marc AtieDesign ParadigmUse of flexible communication library: twistedGINI already in PythonFlexibleAvailable source codeStableHigh-level functions

GINI on a Cloud March 201021David El Achkar, Simon Foucher, Mia Hochar, Marc AtieCurrent Access to Remote ServerGINI on a Cloud March 201022David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Connection to Server23

GINI on a Cloud March 2010David El Achkar, Simon Foucher, Mia Hochar, Marc AtieConnection to server24GINI on a Cloud March 2010David El Achkar, Simon Foucher, Mia Hochar, Marc AtieFront EndBack EndSSH TunnelAddition to GINIGINI on a Cloud March 201025

David El Achkar, Simon Foucher, Mia Hochar, Marc AtieAddition to GINIGINI on a Cloud March 201026David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Gateway to the cloudGINI on a Cloud November 0927

Address can be DNS or IP address.If static, can be hard coded; or at least hard code default addressWhen deployed, choose proper use27Sequence of EventsSend dummy variable

Front EndDispatcherDatabaseNoWorker28GINI on a Cloud March 2010David El Achkar, Simon Foucher, Mia Hochar, Marc AtieSend dummy variable Sequence of EventsDispatcher searches for a worker

Front EndDispatcherDatabase29GINI on a Cloud March 2010David El Achkar, Simon Foucher, Mia Hochar, Marc AtieBest Worker?Send dummy variable Dispatcher searches for a worker

Returns worker or dummy variable

Sequence of EventsFront EndDispatcherDatabase30GINI on a Cloud March 2010David El Achkar, Simon Foucher, Mia Hochar, Marc AtieWorkers DNS nameResult: Unaltered GINI31

GINI on a Cloud March 2010David El Achkar, Simon Foucher, Mia Hochar, Marc AtieConnection to server32Connection remains unalteredUser needs no knowledge of Back End location Front EndBack EndSSH TunnelGINI on a Cloud March 2010David El Achkar, Simon Foucher, Mia Hochar, Marc AtiePurpose of Dummy Variable Need to establish TCP communicationRecycled as an error flagCould be used to convey info in future upgrade

GINI on a Cloud March 201033

David El Achkar, Simon Foucher, Mia Hochar, Marc AtieFlexibilityNon-fixed data types being passedCurrently a string: IP, DNS or anything else

34

GINI on a Cloud March 2010David El Achkar, Simon Foucher, Mia Hochar, Marc AtieDrawbackTemporary TCP connection: Port expiration at timeoutMost Clients dont have DNS

35GINI on a Cloud March 2010David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Design Dilemma36Maintain permanent client-dispatcher connectionResources consumingMight not bring any tangible benefitsRequires alterations to gServer and core ConfigRequires 3 communication events (instead of 1)Drop the connection after delivering the goodsLightweight componentNo extra resources usedImpossible (or difficult) to retrieve clientsGINI on a Cloud March 2010David El Achkar, Simon Foucher, Mia Hochar, Marc AtieMore elaboration on this by Mia and Dave; Second option implemented with DB support for first one if need arises 36DatabaseContentsDispatcher Database RelationshipDesign choicesRecommendationsGINI on a Cloud March 201037David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

DB keeps history of 10 last CPU usages for every workerInitial DesignDatabase Storage: workers TableWorkersUsageHistoryW134%12158045W249%70331028W357%2887865DatabaseGINI on a Cloud March 201038David El Achkar, Simon Foucher, Mia Hochar, Marc AtieDatabase Storage: workers TableUptime is the average load on the system processorworkerIDdnsNameuptime1lab2-420.232lab2-390.183lab2-370.56DatabaseGINI on a Cloud March 201039David El Achkar, Simon Foucher, Mia Hochar, Marc AtieImplementationDatabase Storage: clients TableDB keeps track of all the connections and their start timeDatabaseGINI on a Cloud March 201040David El Achkar, Simon Foucher, Mia Hochar, Marc AtieClientConnected toTime Stamp134.204.2.56134.204.2.561:03168.141.2.9134.204.2.211:21191.24.2.56134.204.2.371:56Initial DesignDatabase Storage: clients TableTime stamp is calculated in minutes starting midnightDatabaseGINI on a Cloud March 201041David El Achkar, Simon Foucher, Mia Hochar, Marc AtieclientIDworkerNameIP addresstimeStamp1Lab2-37134.204.2.567892Lab3-39134.204.2.213003Lab2-42134.204.2.371056ImplementationDatabase - DispatcherGINI on a Cloud March 201042David El Achkar, Simon Foucher, Mia Hochar, Marc AtieDatabaseDispatcherworkerIDdnsNameuptime1lab2-420.232lab2-390.183lab2-370.56getFreeWorker()1Lab2-393Sort workers table by uptime2Database - DispatcherGINI on a Cloud March 201043David El Achkar, Simon Foucher, Mia Hochar, Marc AtieDatabaseDispatcherclientIDworkerNameIP addresstimeStamp1lab2-37134.204.2.567892lab2-39134.204.2.213003lab2-42134.204.2.3710564lab2-39123.45.6.788405Connection between client and lab2-394Client LogGINI on a Cloud March 201044David El Achkar, Simon Foucher, Mia Hochar, Marc AtieCall to the function displayClientsLog()

Design ChoicesDatabase implemented in SQLUsed MySQLdb: wrapper around MySQL Compatible with Python API

Database stored on the SOCS MySQL serverFree, available and provided by McGillDatabase Name: 2010GINIdbMachine Name: mysql.cs.mcgill.ca

Database can be reset at anytime

GINI on a Cloud March 201045David El Achkar, Simon Foucher, Mia Hochar, Marc AtieManaging Client ConnectionsKilling a connectionPolling (time out):Dispatcher periodically polls clientNo response: dispatcher terminates connection ManualClient shuts down connectionSends message to dispatcherToo complex to implement:Needs modification to the core GINI code

GINI on a Cloud March 201046David El Achkar, Simon Foucher, Mia Hochar, Marc AtieManaging Client ConnectionsDatabase functions provided:getTimedOutClients()updateClientTime()removeClientByID()removeClientByIP()

GINI on a Cloud November 0947David El Achkar, Simon Foucher, Mia Hochar, Marc AtieSchedulerPurposeDesign ChoicesMethodologySample OutputRecommendationsGINI on a Cloud March 201048David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

PurposeManage workers and their connectionsExamine CPU usageEnsure all workers are accessibleGINI on a Cloud March 201049David El Achkar, Simon Foucher, Mia Hochar, Marc AtieDesign choicesGINI on a Cloud March 201050ParamikoModule for PythonImplements secure SSH protocol

UptimeLoad average for past 1, 5, and 15 minutesNo need to store each workers history

David El Achkar, Simon Foucher, Mia Hochar, Marc AtieMethodologyGINI on a Cloud March 201051Retrieve workers list from database

Get CPU usage of workersSSH connection to each worker Get uptime informationUpdate database entry

Sleep for one minute, then start again

David El Achkar, Simon Foucher, Mia Hochar, Marc AtieDatabase UpdateGINI on a Cloud March 201052DatabaseSchedulerlab2-42New uptime 0.26

3workerIDdnsNameuptime1lab2-420.262lab2-390.124Request Workers List1 lab2-42, lab2-39, 2lab2-39New uptime 0.12

David El Achkar, Simon Foucher, Mia Hochar, Marc Atie52

Sample OutputGINI on a Cloud March 201053David El Achkar, Simon Foucher, Mia Hochar, Marc AtieDynamic UpdateGINI on a Cloud March 201054SchedulerWorkerWorkerWorker$ gAddWorker$ gRemoveWorkerDavid El Achkar, Simon Foucher, Mia Hochar, Marc AtieDynamic UpdateTwo Bash scripts on server:addWorker.shremoveWorker.sh

Boot configuration and shut down sequence need to be updated.$ /removeworker.sh $ /addworker.sh

GINI on a Cloud March 201055David El Achkar, Simon Foucher, Mia Hochar, Marc AtieWorkersComputers in the Krieble labAll machines could be shut down simultaneouslyNo available workers

Worker could be disconnectedWorkers Log could be used to find new workerInitiate new connection automatically

GINI on a Cloud March 201056David El Achkar, Simon Foucher, Mia Hochar, Marc AtieConfigurationChange passwords:SSH access to SOCS serverDatabase access to SOCS MySql serverGINI on a Cloud March 201057David El Achkar, Simon Foucher, Mia Hochar, Marc AtieConclusionSummaryAdvantagesGINI on a Cloud March 201058David El Achkar, Simon Foucher, Mia Hochar, Marc AtieSummaryInitiating a connection

ClientDispatcherDatabaseWorkerClient contacts dispatcherDispatcher sends IP address of worker to the clientDispatcher queries database for worker, updates database accordinglyClient connects to worker through SSH tunnelGINI on a Cloud March 201059David El Achkar, Simon Foucher, Mia Hochar, Marc AtieAdvantagesNo need to install back end Easier installation (less packages) Saves user trouble of software maintenance / update Cross-platform compatibility Back end can only run on Linux Front end supported by Linux and Windows

GINI on a Cloud March 201060David El Achkar, Simon Foucher, Mia Hochar, Marc AtieAdvantagesMobility Back end runs on remote server Users can run GINI from wherever they are

Resource usage Managed, automated distribution of computing resources among users Easy to monitor and to add more workers

GINI on a Cloud March 201061David El Achkar, Simon Foucher, Mia Hochar, Marc AtieThank you!Questions?

GINI on a Cloud March 201062