Real Time Connected Vehicle Networking with HDInsight and Apache Storm
-
Upload
our-community-exchange-llc -
Category
Automotive
-
view
272 -
download
0
Transcript of Real Time Connected Vehicle Networking with HDInsight and Apache Storm
![Page 1: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/1.jpg)
![Page 2: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/2.jpg)
Building Client Apps with C# and XamarinWhy Xamarin ? Build Once works universally across platforms on billions of devices..
![Page 3: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/3.jpg)
Basic Hardware –OBD-II Port ConnectivityAn ODB-II Elm 327 type interface (connected to USB, Bluetooth or wi-fi) on your smart phone or tablet connected to the diagnostic port below your steering wheel, or broadband in your newer vehicle. Newer vehicles now have optional broadband in-vehicle.
![Page 4: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/4.jpg)
Visual Studio Community Edition Free IDE (Integrated Development Environment) for PCs Integrates with Xamarin for iOS, Android and Mac Development.. Xamarin is free for students and affordable for Indy Developers
![Page 5: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/5.jpg)
Polling Data From OBD-II Your Xamarin Studio or Visual Studio project will need you to install an OBD manager to communicate with your vehicle’s network of diagnostic codes.. I suggest installing the OBD-II manager from Microsoft’s Coding for Fun Channel 9 Project.. You can integrate this easily in your app using either Xamarin Studio or Visual Studio’s NuGet Package Manager also available from the NuGet gallery..
![Page 6: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/6.jpg)
Microsoft’s Project Detroit
![Page 7: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/7.jpg)
Communicating with the OBD-II PortOBD-II stands for On-Board Diagnostics. Connecting to this port allows us to query for different types of data from the car. We can get the current speed, RPMs, fuel level, etc. for display in the Instrument Cluster and other locations.
For the vehicle, because only one application can open and communicate with a serial port at one time, we created a WCF service that polls the OBD-II data from the car and GPS and returns it to our application when it queries the service.
For the OBD library a manual connection will be used to poll different values at different intervals. For values critical to driving the car—like RPM, speed, etc, it is necessary to poll for the values as quickly as the vehicle can return them. With other values which aren’t critical to driving the car—like the fuel level, engine coolant temperature, can be polled at a 1-2 second interval. For GPS, we can create a code to read the GPS and create a LocationChanged event, which would fire when the GPS values changed.
Rather than creating a new serial port connection for every WCF request for OBD data, it’s important to use a singleton service that is instantiated when the service first runs. Only one object in the WCF service that represents the last OBD and GPS data returned, which is obtained by the continual reading of the latest OBD data using the OBD library as described above. Calls to a WCF service ReadMeasurement method didn’t actually compute anything, but instead serializes the last saved data and returned it via the WCF service.
![Page 8: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/8.jpg)
Instantiating the OBD Service
_service = new ObdService();_service.ObdMeasurementChanged += service_ObdMeasurementChanged;
_service.Start(new TimeSpan(0, 0, 0, 0, 1000), localhost, Protocol.Http, false);
void service_ObdMeasurementChanged(object sender, ObdMeasurementChangedEventArgs e)
{Debug.Writeline("MPH=” + e.Measurement.MilesPerHour);
}
This code creates a new ObdService class and signs up for an event when the measurement has changed. The Start method does the following:
• Set the interval that you want to poll the ObdService, in this case every second (to update the instrument cluster). • Determines what IP address the service is hosted at (localhost), the protocol (HTTP or TCP), and whether to send “demo mode” data.
So we can have some sample data for testing.
To capture the car telemetry data like MPH, RPM, engine load, and throttle (accelerator) position, as well as location data (latitude, longitude, altitude, and course), we used a SQL database with a flat Entity Framework model.
• The primary key, the ObdMeasurementID is a GUID that is returned via the ObdService.• The database logger subscribes to the ObdMeasurementChanged event and receives a new reading at the time interval set in the Start()
method.
![Page 9: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/9.jpg)
Instantiating the OBD Service
public override string RowKey{
get { return new DateTimeOffset(TimeStamp).ToEndOfDays(); }set { }
}
public static class DateTimeExtensions{ public static string ToEndOfDays(this DateTimeOffset source) { TimeSpan timeUntilTheEnd = DateTimeOffset.MaxValue.Subtract(source);
return timeUntilTheEnd.Ticks.ToString();} public static DateTimeOffset FromEndOfDays(this String daysToEnd){ TimeSpan timeFromTheEnd = newTimeSpan(Int64.Parse(daysToEnd));
DateTimeOffset source = DateTimeOffset.MaxValue.Date.Subtract(timeFromTheEnd);return source; }}
The Windows Azure data model uses Azure Table Services instead of SQL Server. The data mapping is essentially the same since both have a flat schema.For Azure Table Storage, in addition to the schema above, you also need a partition key and a row key. For the partition key, we used a custom TripID (GUID) to represent a Trip. When the car is turned on/off a new TripID is created. That way we could group all measurements for that particular trip and do calculations based on that trip, like the average miles per gallon, distance traveled, fastest speed, etc. For the row key, we used a DateTimeOffset and a custom extension method, ToEndOfDays() that provides a unique numerical string (since Azure's row key is a string type) that subtracts the time from the DateTime.Max value. The result is that the earlier a DateTime value, the larger the number.Example:Time=5/11/2012 9:14:09 AM, EndOfDays=2520655479509478223 //larger Time=5/11/2012 9:14:11 AM, EndOfDays=2520655479482804811 //smallerSince they are ordered in reverse order, with the most recent date/time being the first row, we can write an efficient query to pull just the first row to get the current latitude/longitude without needing to scan the entire table for the last measurement.
![Page 10: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/10.jpg)
Instantiating the OBD Service
public ActionResult PostData(){
try
{ StreamReader incomingData = new StreamReader(HttpContext.Request.InputStream);string data = incomingData.ReadToEnd();JavaScriptSerializer oSerializer = new JavaScriptSerializer();
List<MeasurementForTransfer> measurements;
measurements = oSerializer.Deserialize(data, typeof(List<MeasurementForTransfer>)) as List<MeasurementForTransfer>;
if (measurements != null) {
CloudBlob blob = _blob.UploadStringToIncoming(data);_queue.PushMessageToPostQueue(blob.Uri.ToString());
return new HttpStatusCodeResult(200);
}}}
To upload data to Azure, we used a timer-based background uploader that would check to see if there was an internet connection, and then filter and upload all of the local SQL rows that had not been submitted to Azure using the Submitted boolean database field. On the Azure side, we used an ASP.NET MVC controller to submit data. The controller deserializes the data into a List<MeasurementForTransfer> type, it adds the data to a blob, and adds the blob to a queue.
A worker role (or many) will then read items off the queue and the new OBD measurement rows are placed into Azure Table Storage.
![Page 11: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/11.jpg)
Xamarin already has App Store Apps for OBD-II
Xamarin is already popular among App Store Developers
Matt O’Connor from OCTech, and creator of OBD Fusion, shared his app that wirelessly reads onboard vehicle diagnostics from a car. The app enables car enthusiasts to create their own dashboards and access much of the same diagnostic information a master mechanic would use with specialized automotive equipment to diagnose and tune an automobile. With OBD Fusion, users can calculate fuel economy, graph data, and also test emissions in real-time.
Matt originally created his app in C# as a desktop application. Utilizing Xamarin, he was able to convert it to iOS and Android with over 90% code-reuse, and by using app templates he’s created native smartphone and tablet user experiences. In addition to the automobile’s sensor data, he’s also integrated sensors on the mobile device, including the GPS and accelerometer to measure position and vehicle acceleration.
https://www.youtube.com/watch?v=Sd3N8zrD7qs
![Page 13: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/13.jpg)
Creating a OBD-II Client Dashboard
Creating a dashboard is a lot like game programming, to move the needle updates at reasonable speed the code needs to be setup as a game loop as you are seeing here in C Sharp for their XAML interface.. To do this in Xamarin, we can use monogame.
![Page 14: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/14.jpg)
Creating a OBD-II Client Dashboardusing System;using System.Windows.Controls;using System.Windows;using System.Windows.Media;using System.ComponentModel;using System.Windows.Media.Animation;
namespace Coding4Fun.Obd.InstrumentCluster.Classes{
public class Needle : UserControl{
private const double SLOWNESS = 5;
Storyboard uxSB_GameLoop;FrameworkElement uxNeedle;public RotateTransform Needle_RotateTransform = new RotateTransform();public TransformGroup tg = new TransformGroup();private double desiredAngle;
public Needle(){
Loaded += new System.Windows.RoutedEventHandler(Needle_Loaded);Initialized += new EventHandler(Needle_Initialized);
}
void Needle_Initialized(object sender, EventArgs e){
uxNeedle = (FrameworkElement)FindName("uxNeedle");uxSB_GameLoop = (Storyboard)TryFindResource("uxSB_GameLoop");
}
void Needle_Loaded(object sender, System.Windows.RoutedEventArgs e){
if (uxNeedle != null) uxNeedle.RenderTransform = Needle_RotateTransform;if (uxSB_GameLoop != null){
uxSB_GameLoop.Completed += new EventHandler(uxSB_GameLoop_Completed);
uxSB_GameLoop.Begin();}}
void uxSB_GameLoop_Completed(object sender, EventArgs e){
Needle_RotateTransform.Angle += (desiredAngle -Needle_RotateTransform.Angle) / Math.Max(SLOWNESS, 1);
uxSB_GameLoop.Begin();}
public void UpdateNeedle(){
desiredAngle = Maximum == Minimum ? Maximum : MinAngle + (MaxAngle -MinAngle) * (Math.Min(Math.Max(Value, Minimum), Maximum) - Minimum) / (Maximum - Minimum);
if (uxSB_GameLoop == null){
Needle_RotateTransform.Angle = desiredAngle;}
}
private static void OnValuesChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
{Needle ln = (Needle)d;ln.UpdateNeedle();
}
[Category("Common Properties")]public double Value{
get { return (double)GetValue(ValueProperty); }set { SetValue(ValueProperty, value); }
}public static readonly DependencyProperty ValueProperty =
DependencyProperty.Register("Value", typeof(double), typeof(Needle), new PropertyMetadata(new Double(), new PropertyChangedCallback(OnValuesChanged)));
[Category("Common Properties")]public double Minimum{
get { return (double)GetValue(MinimumProperty); }set { SetValue(MinimumProperty, value); }
}
public static readonly DependencyProperty MinimumProperty =DependencyProperty.Register("Minimum", typeof(double), typeof(Needle), new
PropertyMetadata(new Double(), new PropertyChangedCallback(OnValuesChanged)));
[Category("Common Properties")]public double Maximum{
get { return (double)GetValue(MaximumProperty); }set { SetValue(MaximumProperty, value); }
}public static readonly DependencyProperty MaximumProperty =
DependencyProperty.Register("Maximum", typeof(double), typeof(Needle), new PropertyMetadata(new Double(), new PropertyChangedCallback(OnValuesChanged)));
[Category("Common Properties")]public double MinAngle{
get { return (double)GetValue(MinAngleProperty); }set { SetValue(MinAngleProperty, value); }
}public static readonly DependencyProperty MinAngleProperty =
DependencyProperty.Register("MinAngle", typeof(double), typeof(Needle), new PropertyMetadata(new Double(), new PropertyChangedCallback(OnValuesChanged)));
[Category("Common Properties")]public double MaxAngle{
get { return (double)GetValue(MaxAngleProperty); }set { SetValue(MaxAngleProperty, value); }
}public static readonly DependencyProperty MaxAngleProperty =
DependencyProperty.Register("MaxAngle", typeof(double), typeof(Needle), new PropertyMetadata(new Double(), new PropertyChangedCallback(OnValuesChanged)));
}}
![Page 15: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/15.jpg)
Real-time Sensor Data Networks with HDInsight and Apache StormCreating a sensor data network that we can use with our Xamarin App
![Page 16: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/16.jpg)
Apache Storm and Real-Time Sensor NetworksData Capture in Real-Time
![Page 17: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/17.jpg)
What is Apache Storm ?
![Page 18: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/18.jpg)
What you need to get started1. An Azure subscription with HDInsight and Apache
Storm (in beta) Active
2. Visual Studio with the Microsoft Azure SDK for .NET
3. Java and JDK
4. Maven
5. Git
6. Source Code for the Sample Project
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
https://github.com/Blackmist/hdinsight-eventhub-example
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 19: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/19.jpg)
![Page 20: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/20.jpg)
Creating the DashboardThe dashboard is used to display near-real time sensor information. In this case, the dashboard is an ASP.NET application hosted in an Azure Website. The application's primary purpose is to serve as a SignalR hub that receives information from the Storm topology as it processes messages.
The website also contains a static index.html file, which also connects to SignalR, and uses D3.js to graph the data transmitted by the Storm topology.
NOTE: While you could also use raw WebSockets instead of SignalR, WebSockets does not provide a built-in scaling mechanism if you need to scale out the web site. SignalR can be scaled using Azure Service Bus (http://www.asp.net/signalr/overview/performance/scaleout-with-windows-azure-service-bus).
For an example of using a Storm topology to communicate with a Python website using raw WebSockets, see the Storm Tweet Sentiment D3 Visualization project.
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 21: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/21.jpg)
Creating the DashboardThe dashboard is used to display near-real time sensor information. In this case, the dashboard is an ASP.NET application hosted in an Azure Website. The application's primary purpose is to serve as a SignalR hub that receives information from the Storm topology as it processes messages.
The website also contains a static index.html file, which also connects to SignalR, and uses D3.js to graph the data transmitted by the Storm topology.
NOTE: While you could also use raw WebSockets instead of SignalR, WebSockets does not provide a built-in scaling mechanism if you need to scale out the web site. SignalR can be scaled using Azure Service Bus (http://www.asp.net/signalr/overview/performance/scaleout-with-windows-azure-service-bus).
For an example of using a Storm topology to communicate with a Python website using raw WebSockets, see the Storm Tweet Sentiment D3 Visualization project.
1.In Visual Studio, create a new C# application using the ASP.NET Web Application project template. Name the new application Dashboard.
2.In the New ASP.NET Project window, select the Empty application template. In the Windows Azure section, select Host in the cloud and Web site. Finally, click Ok
NOTE: If prompted, sign in to your Azure subscription.
3.In the Configure Windows Azure Site dialog, enter a Site name and Region for your web site, then click OK. This will create the Azure Website that will host the dashboard.
4.In Solution Explorer, right-click the project and then select Add | SignalR Hub Class (v2). Name the class DashHub.csand add it to the project. This will contain the SignalR hub that is used to communicate data between HDInsight and the dashboard web page.
DashHub.cs
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 22: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/22.jpg)
Creating the DashboardStartup.cs
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 23: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/23.jpg)
Creating the DashboardNext Steps
• In Solution Explorer, right-click the project and then click Add | HTML Page. Name the new page index.html. This page will contain the realtimedashboard for this project. It will receive information from DashHub and display a graph using D3.js.
• In Solution Explorer, right-click on index.html and select Set as Start Page.
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 24: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/24.jpg)
Creating the Dashboard
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 25: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/25.jpg)
Creating the DashboardNext Steps
• In Solution Explorer, right-click the project and click Add | HTML Page. • Name the new page test.html. This page can be used to test DashHub and the dashboard
by sending and receiving messages. Save All for the project.• In Solution Explorer, right-click on the Dashboard project and select Publish. Select the
website you created for this project, then click Publish. • Once the site has been published, a web page should open displaying a moving timeline.
Test the dashboardTo verify that SignalR is working, and that the dashboard will display graph lines for data sent to SignalR, open a new browser window to the test.html page on this website. For example, http://sensordash.azurewebsites.net/test.html.The dashboard expects JSON formatted data, with a device id and temperature value. For example {"device":0, "temperature":80}. Enter some test values on the test.html page, using device IDs 0 through 9, while the dashboard is open in another page. Note that the lines for each device ID are drawn using a different color.
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 26: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/26.jpg)
Configuring the Azure Event Hub
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 27: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/27.jpg)
Sending Messages to the Event Hub
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 28: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/28.jpg)
Building and Scaffolding
![Page 29: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/29.jpg)
Program.cs http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 30: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/30.jpg)
Program.cs
For now, you will receive a warning on lines that reference the Event class. Ignore these for now.In the Program.cs file, set the value of the following variables at the beginning of the file to the corresponding values retrieved from your Event Hub in the Azure Management Portal.
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 31: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/31.jpg)
Program.cs
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 32: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/32.jpg)
Program.cs
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 33: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/33.jpg)
Program.cs
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 34: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/34.jpg)
Setting up your cluster and virtual networkCreating your network..
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 35: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/35.jpg)
Setting up your cluster and virtual network
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 36: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/36.jpg)
Setting up your cluster and virtual networkCreating your network..
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 37: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/37.jpg)
Setting up your cluster and virtual networkCreating your network..
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 38: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/38.jpg)
Enabling Real-Time Communication
![Page 39: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/39.jpg)
Enabling Real-Time Communication
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 40: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/40.jpg)
Enabling Real-Time Communication
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 41: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/41.jpg)
EnablingReal-TimeCommunication
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 42: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/42.jpg)
Enabling Real-Time CommunicationThis tells Maven to do the following when building the project:
Include the /conf/Config.propertiesresource file. This file will be created later, but it will contain configuration information for connecting to Azure Event Hub.
Include the /conf/hbase-site.xmlresource file. This file will be created later, but it will contain information on how to connect to HBase
Use the maven-compiler-plugin to compile the application.
Use the maven-shade-plugin to build an uberjar or fat jar, which contains this project and any required dependencies.
Use the exec-maven-plugin, which allows you to run the application locally without a Hadoop cluster.
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 43: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/43.jpg)
Adding Configuration FilesScaffolding
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 44: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/44.jpg)
Adding Configuration FilesScaffolding
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 45: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/45.jpg)
Adding Configuration FilesScaffolding
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 46: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/46.jpg)
Adding Configuration FilesScaffolding
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 47: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/47.jpg)
Adding Configuration Files
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 48: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/48.jpg)
TopologyDataflows
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 49: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/49.jpg)
TopologyDataflows
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 50: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/50.jpg)
TopologyDataflows
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 51: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/51.jpg)
TopologyDataflows
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 52: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/52.jpg)
TopologyDataflows
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 53: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/53.jpg)
TopologyDataflows
SummaryYou have now learned how to use Storm to read data from Event Hub, store data in HBase, and display information from Storm on an external dashboard using SignalR and D3.js.•For more information on Apache Storm, see https://storm.incubator.apache.org/•For more information on HBase with HDInsight, see the HBasewith HDInsight Overview•For more information on SignalR, see ASP.NET SignalR•For more information on D3.js, see D3.js - Data Driven Documents•For information on creating topologies in .NET, see Develop streaming data processing applications with SCP.NET and C# on Storm in HDInsight
http://azure.microsoft.com/en-us/documentation/articles/hdinsight-storm-sensor-data-analysis/
![Page 54: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/54.jpg)
SendEventsPushing Data to Azure HDInsightand Apache StormOur Example running pushing temperature data
![Page 55: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/55.jpg)
CONNECTING TO REAL -TIME VEHICLE DATA AND MODELING IN THE LAB
![Page 56: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/56.jpg)
Azure Mobile Services for Unity 3D
The Azure Mobile Services have a free tier that includes up to 500 devices as well as 500k API calls and you can also use the free tier for up to 10 services. This means that you can test out a few things without having to pay for it.
Azure Mobile Services
Azure Mobile Services is a part of Azure that allows access to a database and has connection and sample code to talk to any mobile system that is out there. This will provide you with the code or library to do the connection to Android, iOS, Windows 8, Windows Phone, HTML5/JavaScript, or even the Xamarin libraries. To get started, if you do not have any Mobile Services defined yet you can click on the Mobile Services tab on the left and then the Create a New Mobile Service link next to it. You can also click on the New option on the bottom the of the screen and select Compute -> Mobile Service -> Create.
From here, you will get a popup to fill in to finish up the creation. The first field is the name of the Mobile Service. This name will also be the address for the service. It must be unique. For this example, I named mine “unityleaderboard”. The next field is the database to use as a back end for the service. You can choice from “Use an existing SQL database“, “Create a free 20 MB SQL database“, or “Create a new SQL database instance“.
![Page 57: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/57.jpg)
Azure Mobile Services for Unity 3D
![Page 58: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/58.jpg)
Azure Mobile Services for Unity 3D
So up to now we have the Mobile Service setup, but there is no data yet. Go into your new Mobile Service and then click on the Data link at the top. You can now add a new Table to the database that was setup earlier.
![Page 59: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/59.jpg)
Azure Mobile Services for Unity 3D
The next step is to add the data fields to the new leaderboard table. This will allow us to save the data for the UserName and the Score that is saved. This is going to be a basic sample and not an optimized database, so I will be adding the UserNameas just a string field to the table. If this was a bigger system supporting multiple games, I would probably make a Player table with all of the players info there and then a leaderboard table that cross referenced that player table. Since this is just a quick and simple leaderboard for a single game, keeping the raw text in the table is not that bad. The Score field is going to be added as a Number so that we do not have to change the numbers of the score into a text field back and forth. After clicking on the table name, you will see and can click on the Columns link to get to add new columns. To add a new column, use the Add Column link at the bottom of the page.
![Page 60: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/60.jpg)
Bitrave Mobile Services Plugin API
https://github.com/bitrave/azure-mobile-services-for-unity3d
![Page 61: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/61.jpg)
Bitrave Mobile Services Plugin APIPut the plugin binaries in your Assets/Plugins folder. These get built into an Output folder in the root of the solution in the right folder structure. And it’s as simple as…
var data = new LevelSaveData() { SaveData = "some data here", Id = 1 };
var azure = new AzureMobileServices(_azureEndPoint, _applicationKey);
azure.Update<LevelSaveData>(data);
or
var azure = new AzureMobileServices(_azureEndPoint, _applicationKey);
azure.Lookup<LevelSaveData>(1, azureResponse =>
{
if (azureResponse.Status == AzureResponseStatus.Success)
{ var ourObject = azureReponse.ResponseData; }}
Data comes back via callbacks and response objects. Unity doesn’t support await/async, but when it does it will move to that model.
![Page 62: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/62.jpg)
Bitrave Mobile Services Plugin APIAPI
Initializeinitialiation is just as simple as you’d expect.var service = new AzureMobileServices(“url”, “token”);
InsertInsert an item into your Azure database in a single line of code from Unity.service.Insert<ToDoItem>(myItem);
UpdateUpdate items in the Azure databsae with just one line of code from Unity.service.Update<ToDoItem>(myItem);
DeleteRemove items from the Azure database in 1 line of code from Unity.service.Delete<ToDoItem>(myItem);
QueryQuery items in your Azure Mobile Services from Unity.service.Where<ToDoItem>(p => p.Category == “Exercise”, azureResponse =>{List<ToDoItem> exerciseItems = azureRepsonse.ResponseData;NOTE: await / async will be available when supported by Unity. Until then we are using callbacks.
LookupLookup items in your Azure Mobile Services from Unity.service.Lookup<ToDoItem>(myItem, azureResponse =>{ToDoItem myToDoItem = azureResponse.ResponseData;NOTE: await / async will be available when supported by Unity. Until then we are using callbacks.
LoginOn supported platforms, LoginAsync can be called for authenticated services.azure.LoginAsync(AuthenticationProvider.Facebook, loginResponse => {var token = loginResponse.ResponseData.MobileServiceAuthenticationToken;});NOTE: await / async will be available when supported by Unity. Until then we are using callbacks.
![Page 63: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/63.jpg)
Bitrave Mobile Services Plugin APIAPI
Initializeinitialiation is just as simple as you’d expect.var service = new AzureMobileServices(“url”, “token”);
InsertInsert an item into your Azure database in a single line of code from Unity.service.Insert<ToDoItem>(myItem);
UpdateUpdate items in the Azure databsae with just one line of code from Unity.service.Update<ToDoItem>(myItem);
DeleteRemove items from the Azure database in 1 line of code from Unity.service.Delete<ToDoItem>(myItem);
QueryQuery items in your Azure Mobile Services from Unity.service.Where<ToDoItem>(p => p.Category == “Exercise”, azureResponse =>{List<ToDoItem> exerciseItems = azureRepsonse.ResponseData;NOTE: await / async will be available when supported by Unity. Until then we are using callbacks.
LookupLookup items in your Azure Mobile Services from Unity.service.Lookup<ToDoItem>(myItem, azureResponse =>{ToDoItem myToDoItem = azureResponse.ResponseData;NOTE: await / async will be available when supported by Unity. Until then we are using callbacks.
LoginOn supported platforms, LoginAsync can be called for authenticated services.azure.LoginAsync(AuthenticationProvider.Facebook, loginResponse => {var token = loginResponse.ResponseData.MobileServiceAuthenticationToken;});NOTE: await / async will be available when supported by Unity. Until then we are using callbacks.
![Page 64: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/64.jpg)
MapNav Plug-In Unity 3D
https://www.assetstore.unity3d.com/en/#!/content/13153
![Page 65: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/65.jpg)
MapNav Plug-In Unity 3D
https://www.assetstore.unity3d.com/en/#!/content/13153
![Page 66: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/66.jpg)
Prime 31 Plug-in for Unity
![Page 67: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/67.jpg)
Prime 31 Plug-in for UnitySign-in to Azure portal. If you don’t have an Azure account yet game developers can register for the Cloud GameDev Offer. Create Azure Mobile Service
![Page 68: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/68.jpg)
Prime 31 Plug-in for UnitySign-in to Azure portal. If you don’t have an Azure account yet game developers can register for the Cloud GameDev Offer. Create Azure Mobile Service
![Page 69: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/69.jpg)
Add a Demo To-Do Table
![Page 72: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/72.jpg)
Create a New Project and Install the Plug-in
![Page 73: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/73.jpg)
Import the Plug-In Scene
![Page 74: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/74.jpg)
Open the MetroAzure Scene
![Page 75: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/75.jpg)
Open MetroAzureDemoUI.cs
![Page 76: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/76.jpg)
Paste in your Azure Connection Strings
![Page 77: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/77.jpg)
Target Windows Store In your program settings
• “Add Current” scene
• Select “Windows Store” and “Switch Platform”
• Select C# Solution and SDK “8.1”
![Page 78: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/78.jpg)
Select Player Settings
Under “Metro Unprocessed Plugins” set:Size: 1Element 0: P31MetroAzure.dllClick Build
![Page 79: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/79.jpg)
Launch Visual Studio Open Windows Store Build
![Page 80: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/80.jpg)
Open Package.AppXManifest.xmlAdd Internet Capabilities to Manifest
![Page 81: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/81.jpg)
Open Configuration Manager Target Current Hardware
![Page 82: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/82.jpg)
Build and RunYou will see your items in Azure
![Page 83: Real Time Connected Vehicle Networking with HDInsight and Apache Storm](https://reader035.fdocuments.net/reader035/viewer/2022062313/55a694a71a28ab67148b45ad/html5/thumbnails/83.jpg)
Connecting to Vehicle Data Near Real-Time in the CloudThe WCC Connected Pontiac G6Modeled and Connected to Real-Time Vehicle Data..