DMM Developer’s Guide

This document will explain how to set up a development environment for the Distributed Monitoring and Mining application. It will also explain the use of the DMM command line application to produce training data and train the Google predictive model.

1. Install a suitable IDEYou will need an IDE capable of building and running Microsoft C# .NET. The application has been tested with Microsoft Visual Studio 2012 Professional on Windows and Xamarin Studio on Mac OSX/Linux. Visual Studio 2012 Professional can be obtained for free for students and faculty through Microsoft’s Dreamspark program. The free Visual Studio 2012 Express edition may also work, but it has not been tested. Xamarin Studio can be downloaded here. The remainder of this guide will assume you are using MS Visual studio, although most steps are similar for Xamarin Studio.

2. Install MySQL Server and related toolsThe application uses the free and open source database MySQL for data persistence. You will need to install MySQL on your development machine. It is also recommended to install MySQL Workbench to provide a graphical interface for interacting with the MySQL server. A one-click installer for all popular platforms can be downloaded here. The installer will guide you through all of the necessary steps to get the server up and running. Make a note of the username and password you create to be used later on.

3. Get the application source codeThe code is hosted in a Subversion repository at: https://projects.cecs.pdx.edu/svn/wi2013omse555-dmm. All of the application code is in the “App” directory. Of course you will need a Subversion client to download the code. Recommended for Windows is TortoiseSVN. If using Xamarin Studio on OSX or Linux, it has an SVN client built in.

4. Set up the databaseThere are two scripts available for creating the database. These scripts are in the “Data” directory in the Subversion repository. “CreateDatabase.sql” will simply create the database with empty tables ready for use. “CreateAndPopulate.sql” will create the database and populate it with sample data. The second option is probably the best for setting up a development environment. To create the database simply load the script in the MySQL Workbench application downloaded earlier and execute it.

5. Load the application code in your IDE of choice.Load the application source code into the IDE by opening the DMM.sln file located in the App/DMM folder downloaded from the Subversion repository.

6. Update configuration files with your database settings.Open the Web.Config file under the DMM.Web project. Update the user id and password to match what you configured for your MySQL installation. Make the same change in the App.Config file under the DMM.CLI project.

7. Build the application and run the unit tests.Please note, if you are using Xamarin Studio you will not be able to run the unit tests as they are written with MSTest, which is not supported by Mono. With MS Visual Studio 2012, you can run the tests by opening Test -> Run -> All Tests. This will compile the application and run all of the unit tests. You should see the tests pass in the Test Explorer.

At this point you have a basic development environment all set up.

8. Create an Assembla Developer Account Please note, if you are doing this setup before 6/13/13, this step can be skipped as the account used by the original developers will still be active.

a. If you do not already have one, create an account on Assembla. The free account is fine.

b. Log in to Assembla and navigate to the Profile page. This is available in the dropdown in the upper right corner of the screen.

c. Click on “API Applications and Sessions” in the menu on the left side of the screen.

d. Enter a name for the application, “DMM” works well.e. The “Website” field can be left blank.f. In the “Redirect URI” field enter: http://localhost:49673/Home.aspx. This is the page

that Assembla will redirect to after a user logs in. Please note, this URL is for development only, you will need to change this for any production deployment.

9. Update the configuration files with your Assembla API credentialsa. Once you have created the application on Assembla open the Web.config file in

the DMM.Web project.b. The API application & sessions screen on the Assembla website will contain your

API credentials. Enter these credentials as follows.

Web.Config Key Assembla FieldAssemblaApplicationId API keyAssemblaApplicationSecret API secretAssemblaClientId Application IdentifierAssemblaClientSecret Application secret

c. Make these same changes to the App.config file in the DMM.CLI project.

10. Create a Google developer account and setup a prediction applicationPlease note, if you are doing this setup before 6/13/13, this step can be skipped as the account used by the original developers will still be active.

a. Log in to your Google account and navigate to https://code.google.com/apis/console.

b. In the dropdown in the upper left of the screen choose Create

c. In the next screen, enable “Google Cloud Storage” and “Prediction API”d. For your new API project, click on the API Access menu item

e. In the second screen of the wizard, choose “Service Account”

f. This will create a private key and Google will ask you to download the key. Download the key and save it to a safe place.

g. Add the key you downloaded to the cert directory in both the DMM.Web project and the DMM.CLI project.

11. Update the configuration files with your Google API credentialsa. In the Web.config file, change the “GoogleServiceAccount” key to the email

address of the service account you created above. This is available in the Google API console in the API Access screen.

b. Make the same change to the App.config file in DMM.CLI.c. Finally, change the GoogleServiceKey key in the config files to use your API key

file added above.

12. Create a bucket in Google Cloud Storage to hold your training dataa. From the Google API console for your project, click on the Google Cloud Storage

Service link.

b. In the next screen, click on the “Google Cloud Storage Manager” link.

c. Click on the “New Bucket” button and give your bucket a name.d. Enter the bucket name into the “GoogleBucket” key in the Web.config and

App.config files.

13. CLI Application and training the predictive modelThe DMM.CLI application is a command line application that allows you to download project data from Assembla, create a training file, and train the predictive model.

a. To download data from Assembla, run the application with the –updateAll switch. The compiled application will be located in the bin directory for the DMM.CLI project. You can navigate to this directory and run the command “DMM.CLI.exe –updateAll” to start the download process. If debugging, you can add the command line argument by right clicking on the Project file, selecting properties, clicking on the “Debug” menu item, and entering the switch in the Command line arguments section.

Please note, this entire process takes about 4 hours to complete. If you loaded your local database with the sample data, this step can be skipped for development purposes.

b. To create training data, run the application with the –createTrainData switch. This will create a file called “Metrics.csv” in the bin directory. At this time, the file must be manually uploaded to your Google cloud storage bucket. To upload the file, go to the Google Cloud Storage Manager as detailed in step 12, and click the upload button.

c. Finally, to train the model, run the application with the –trainModel switch. The name of the model can be configured with the “GoogleModel” key in the App.config file. If the name is changed it must also be changed in the Web.config file for the DMM.Web application.