Sjalvstandigt arbete i informationsteknologi16 juni 2018

A Tool for Learning Digital Filters inVirtual Reality

Axel DalbomMarkuss SprogisEmil StolpeNils Soderkvist

Civilingenjorsprogrammet i informationsteknologi

Computer and Information Engineering Programme

Institutionen forinformationsteknologi

Besoksadress:ITC, PolacksbackenLagerhyddsvagen 2

Postadress:Box 337751 05 Uppsala

Hemsida:http:/www.it.uu.se

Abstract

A Tool for Learning Digital Filters in VirtualReality

Axel DalbomMarkuss SprogisEmil StolpeNils Soderkvist

There are various tools used for teaching purposes, be it computers oreven pen and paper. Virtual reality is used to simulate immersive envi-ronments. While it is mostly used in the video game industry we havemade an attempt to use it as a learning tool, in this case for courses re-lated to signal processing by trying to build intuition for digital filterdesign. The end product is a tool in virtual reality that allows the userto alter digital filter specifications and see how these alterations affectit. With this project we show that there is potential for virtual reality tobe used as a teaching assistant. The main drawbacks for tool creationin virtual reality are that there is a limited amount of frameworks thatsupport virtual reality and that they are mainly created for video gamedevelopment.

Handledare: Virginia Grande Castro, Mats Daniels, Anne Peters, Bjorn VictorExaminator: Bjorn Victor

Sammanfattning

Det finns manga olika verktyg for larande andamal, det kan bland annat vara datorereller penna och papper. Virtuell verklighet ar ett verktyg for att skapa datorgenererademiljoer. Det anvands mestadels inom spelindustrin men vi i det har projektet forsokerutoka anvandningsomradet for virtuell verklighet inom utbildning, i detta fall for kurserrelaterade till signalbehandling genom att hjalpa anvandare att bygga intuition for de-signen av digitala filter. Slutprodukten ar ett hjalpmedel i virtuell verklighet som tillateranvandaren att skapa och andra digitala filters specifikationer och sedan se vad dessaforandringar har for konsekvenser for filtren. Med det har projektet visar vi att det finnspotential for virtuell verklighet i larande andamal, aven om det huvudsakliga anvand-ingsomradet ar inom spelindustrin.

ii

Contents

1 Introduction 1

2 Background 1

2.1 Signals and Signal Processing . . . . . . . . . . . . . . . . . . . . . . 1

2.2 Virtual Reality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2.3 External Stakeholder . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

3 Purpose, Aims, and Motivation 3

3.1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

3.2 Aim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

3.3 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

3.4 Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

4 Related Work 5

5 Method 6

6 System Structure 7

6.1 Unity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

6.2 MATLAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

7 Requirements and Evaluation Methods 10

8 Properties of Filters 11

9 Filter Design Interface 12

9.1 Area 1, General Formula . . . . . . . . . . . . . . . . . . . . . . . . . 13

iii

9.2 Area 2, Example Graphs . . . . . . . . . . . . . . . . . . . . . . . . . 14

9.3 Area 3, General Equation . . . . . . . . . . . . . . . . . . . . . . . . . 14

9.4 Sliders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

10 Implementation 14

10.1 MATLAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

10.2 Unity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

11 User Manual 16

11.1 Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

11.2 Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

12 Evaluation results 17

13 Results and Discussion 18

14 Conclusions 18

15 Future work 19

A 24

iv

2 Background

1 Introduction

Virtual reality has been heavily promoted since the kickstarter was made by the Oculusteam in 2012 [14]. The purpose of the Oculus Rift, the virtual reality headset, is to createa truly immersive gaming experience. Since the kickstarter, a number of virtual realityheadsets such as the HTC Vive and the Sony Playstation VR have been engineered withthe very same purpose that the Oculus Rift has [14].

Our goal is to turn away from virtual reality’s original purpose of gaming and movetowards its potential academic applications as a teaching tool. There are many tools thatassist teachers in their teaching such as computer simulations and physical instrumentsthat show theoretical concepts in practice. This project tries to answer the question ifvirtual reality has grown enough to be used as a learning tool.

This virtual reality teaching tool is specifically designed to be used as a learning assis-tant and a teaching assistant for concepts in Embedded Systems and Signal Processingwhich students often find difficult to grasp as is evident from the exam results 2017,see Appendix A. Signal processing requires a lot of practice of signal analysis and isstrongly tied to mathematical transform theory. This mathematical theory is often foundabstract by students. The exact part of signal processing that this tool engulfs is digitalfilter design which is a crucial part of signal processing. There have been many attemptsto create tools for this very purpose to varying degrees of success. The tools in ques-tion are circuits that provide a way to interact with different kinds of filter designs withthe help of a physical object and a web-based educational tool that helps students learnthrough simulations, animations, problems and tests [1, 25, 8].

2 Background

To understand what the project is about we will go through some necessary informationabout signals, signal processing and virtual reality. In addition there will be a briefintroduction and motivation of our external stakeholder.

2.1 Signals and Signal Processing

Signals can be described as a way to convey information about the behavior or attributesof some phenomenon. Signal processing is a field that concerns analysis and modifica-tion of signals [16]. Filtering signals is one useful way to manipulate them. A recorded

1

2 Background

song can be thought of as a signal and if the song includes unwanted noise a filter canbe used to dampen the part of the signal that contains the noise. Signals can be eitheranalog or digital and therefore filters also must be analog or digital [19]. In this projectwe will focus on digital filters and their design.

Embedded Signal Processing Systems is a course taught at Uppsala University, Swe-den, for third year IT-students. The course is about the basics of signals and systemsincluding analysis of simple systems [24]. It also covers the design process and im-plementation of continuous and discrete time filters as well as their realization both ascircuits and using embedded systems.

While the course teaches the fundamentals in terms of math, it also tries to build anintuitive way for the students to see how the math works without the math itself. Twosimple examples of how intuition works are linear growth and exponential growth. Af-ter a few equations of each of these examples it is very easy to determine whether afunction is growing exponentially or linearly. This is the kind of intuition the coursetries to build and it is done using visual aid and practical exercises such as creating fil-ters and then examining how they work with an oscilloscope or through simulations ona computer.

One of the more difficult topics to build intuition for is the design and implementa-tion of digital filters. To see how a specific filter works, usually a plot that shows howthe filter behaves is created. Filters and their properties are explained in more detail inSection 8. It takes time to create a design and to draw a plot of it, and it takes even moretime to do it all over again if any of the filter specifications change. Building an intuitiveway of understanding how the design changes the filters behavior is hard yet necessary.

2.2 Virtual Reality

Virtual reality is a powerful tool used for experiencing immersive three-dimensional(3D) environments. To use virtual reality some sort of virtual reality hardware is oftenbut not always required, for example Google Cardboard only uses a mobile phone anda piece of cardboard to hold the phone [7]. The hardware usually consists of a headsetwhich is a head-mounted display with screens, two controllers and sensors that track theheadset and the controllers. Inside the headset are two screens, one for each eye whichmakes the user be able to perceive the virtual environments in 3D. The controllers areused to interact with objects in the virtual environments. More details about the specifichardware used in this project can be found in Section 5.

2

3 Purpose, Aims, and Motivation

We see a potential for virtual reality as a learning and teaching assistant. As of nowwe think virtual reality has not been used enough in education to show how well itworks in the classroom. This might defer schools from buying expensive gear that costsfrom 400 to 1400 US dollars [3]. So buying a virtual reality headset for every student isprobably not an option but having one per classroom is possible.

People prefer to ingest information in different ways, some prefer learning by doing,some prefer to listen to someone talk about it and some prefer to visualize it. By pro-viding different ways to learn, more students can use their preferred way of learning.Virtual reality combines learning by doing and visualization by placing the user in atailored virtual environment where the user can interact with abstract objects that some-times cannot easily be represented in the real world. An example could be looking at andinteracting with molecules. It is possible to build models in real life but more complexmolecules are much easier to represent in a virtual world. This is what makes virtualreality superior in some ways to other visualization tools such as books or computerswhich have limited intractability.

2.3 External Stakeholder

This project is made possible by one of the teachers of the Embedded Signal ProcessingSystems course and our external stakeholder, Ping Wu. Ping Wu is teaching the practi-cal part of the course which is primarily about digital filters.

While there already are several tools used in the course Ping Wu felt the need for anew tool that takes a different approach. So his idea is to create a tool using virtualreality. Ping Wu has the basic idea and features for the tool but the major design choiceswere left to us. His goal for the application was to make it easier for new students to getan intuitive understanding of different design choices and their consequences in filterbehavior.

3 Purpose, Aims, and Motivation

A summary of the project’s aspirations and objectives are presented in this section, alongwith the motivation behind it.

3

3 Purpose, Aims, and Motivation

3.1 Purpose

The purpose of this project is to determine whether virtual reality is suited to be used asa learning tool and to speculate on how well it would perform. Another purpose for thisproject is to try to increase students’ understanding of concepts relating to digital filterdesign. This is also a potential way to evaluate the usefulness of virtual reality in theclassroom.

3.2 Aim

What we are trying to achieve with this project is to create a 3D environment in virtualreality that acts as a room in which the user then can generate digital filters. Differenttypes of filters can be designed with a variety of parameters which are easily adjustablewith sliders and buttons in the room. They will then be able to move around the plotsof these filters in the virtual room to be able to compare them. Our stakeholder has theintention of implementing the tool in the next iteration of the course Embedded SignalProcessing Systems to aid students in learning the concepts of digital filters. Lastly,the time between entering the filter specifications and receiving the corresponding filtershould be short enough to keep the users interested and attentive.

3.3 Motivation

There will always be a need for new teaching tools that try a new angle on the sameproblems if only to give students and teachers a choice in what tools they can use.With this project we try to determine whether virtual reality is applicable for academicpurposes. It has been tried before (see Section 4) but there is a limited amount of studiesthat explore this use of virtual reality.

The course used to test the utility of virtual reality as a teaching tool is called EmbeddedSignal Processing Systems. This is a tough course and the number of students that failthe course reflects it as seen in appendix A. Moreover studies show that students failto fully understand basic concepts in other similar courses that contain signal process-ing [5, 11, 12, 13].

Ping Wu, the stakeholder of this project is in charge of the practical part of the courseEmbedded Signal Processing Systems. This is the part of the course in which the stu-dents will use signal theory in practice and he is interested in a better way in visualizinghow the filters work for the students.

4

4 Related Work

3.4 Delimitations

The time constraints mean that there is no time to create our own framework for math-ematical calculations to use instead of MATLAB. MATLAB has limited intractabilitysince real time calculations are impossible. This is a big problem in a virtual environ-ment where interaction is important.

Neither of us owns the equipment needed for virtual reality and can therefore only usethe equipment that was given to us by Ping Wu. This limitation means that only oneperson at a time can try it at home. That person also needs both the space and a powerfulenough computer to run our program in virtual reality.

4 Related Work

Students from Netaji Subhas Institute of Technology, India have created a WindowsStore App which is a mobile application that can perform complex signal calculations.The purpose of this application is to teach digital signal processing and related subjectsto students [9]. Their project is similar to this project in the sense that it moves theteaching to a medium other than the classroom, however their focus is more as a calcu-lation tool rather than a pedagogical visualization tool.

A software development company called Medis Media has created a program called3D Organon [10]. This is a program in virtual reality which aims to educate users in theanatomy of the human body. By using virtual reality one can interact with a computergenerated human body to learn more about the human anatomy. Their program is aimedtowards medical students as well as medical educators. 3D Organon is similar to ourproject in regards to both the teaching medium and the target users, the only differenceis the subject.

Several students from Uppsala University, Sweden developed a circuit which by turningswitches to different positions lets users create a filter of their choice [8]. The goal oftheir project was to create a tool for visualizing concepts of digital signal processingto help the lecturers who teach in these courses. The use of their tool makes it easierfor lecturers who are experiencing difficulties in making students understand the funda-mental concepts of digital signal processing. These students’ motivation was also thatsignal processing is hard to learn. Their tool is used to assist students in grasping theconcepts of signal processing. Our project is also trying to help students to understandthe concepts of signal processing. Although in this case the tool is visualized in virtualreality.

5

5 Method

Beijing Bluefocus E-Commerce Co. and Beijing iBokan Wisdom Mobile Internet Tech-nology Training Institutions conducted a case study in Beijing China with the purposeof testing how good virtual reality can be used as a teaching tool [2]. This was achievedby having one group of students learn a specific part of their physics curriculum withtraditional teaching and another group learning by using virtual reality. The resultsshowed an increase in both immediate test results and knowledge retention in the groupthat used virtual reality compared to the group that used traditional teaching. This isvery close to what this project aims to do but instead of physics this project is focusedaround signal processing and digital filters.

5 Method

There exists several programming environments for programming in virtual reality. Twopopular ones are Unity and Unreal Engine 4 [4, 17, 21, 22]. Both of them are game en-gines which means that they were created for video game development.

Both engines have their advantages and disadvantages, for example Unreal Engine isbetter when it comes to graphics and Unity is more dominant in mobile application ar-eas and 2D games [15]. These factors would have mattered if we were creating a mobilegame. None of the differences will actually make an impact on the final version of thisproject so the choice of Unity is based on our own programming language preferences.Unreal Engine uses C++ and Unity uses C# and JavaScript which are languages we haveused before and are therefore familiar to us, this allows us to spend less time to learn awhole new programming language.

MATLAB is both an application and a programming language which has a multitude ofuses, mostly math oriented. These uses are for example mathematical operations, matrixmanipulations and plotting of functions and data [18]. Any other math framework canbe used instead of MATLAB as long as it can compute complex mathematical equationsand is combinable with Unity or any other arbitrary virtual reality development kit. Afree alternative to MATLAB is GNU Octave. GNU Octave is also an application anda programming language, similarly to MATLAB it is intended for mathematical opera-tions [6]. We chose MATLAB since it is available to students for free and is integrablewith Unity which is a must in this project.

6

6 System Structure

Figure 1 HTC Vive with Controllers

The virtual reality system we used is HTC Vive which can be seen in Figure 1. Thisis the gear that was given to us by our stakeholder Ping Wu and since it doesn’t matterwhich type of virtual reality headset is used the project was primarily developed aroundthis system. The picture shows the head-mounted display, the two controllers and thetwo sensors that track the head-mounted display and the controllers.

The programming interface which the virtual reality gear at our disposal uses is SteamVR.SteamVR is the thread that connects the hardware with the software and provides a lowlevel programming interface in C++. SteamVR is encapsulated by another level of ab-straction when integrated with Unity which allows the use of C#.

6 System Structure

The system is composed of two main parts: Unity and MATLAB. The user uses thevirtual reality gear, two controllers and a head-mounted display, to interact with theuser interface (UI) which has been created in Unity. Through this UI the user entersthe desired filter specifications which are sent to MATLAB where a digital filter withthese specifications is created and the corresponding plots are generated. These are thensent back to Unity which then places the plots on a canvas for the user to observe andinteract with. A simplified system structure of how everything communicates can befound below in Figure 2.

7

6 System Structure

Figure 2 System structure

Unity and MATLAB communicate through a local network connection where MATLABacts as the server and Unity as the client.

6.1 Unity

Unity allows its users to access the virtual reality hardware through its interface. Thisleads to it being the tool which is used to build the UI. The traditional UI that one mightthink of is a two dimensional screen with options that affect a system in different ways,for example the big red button used to capture a video with the camera of a phone. AUI in virtual reality is much like its two dimensional counterpart except it must be im-plemented in a 3D environment. The UI created with Unity is a rectangular box withbuttons and sliders which affects how MATLAB interprets the shape of the filter theuser tries to create.

When the filter specifications are decided by the user with the use of the hardwarethey are sent to MATLAB through the use of an Internet protocol called TransmissionControl Protocol, TCP. The use of TCP is arbitrary which means that any other transferprotocol could be used, be it local or remote. User Datagram Protocol, UDP is one ofthe other protocols that could be used instead of TCP. The downside to UDP is its un-reliability which means that information can be lost when it is transfered as well as the

8

6 System Structure

fact that the information could come out of order. These downsides can be overcomeby modifying the UDP protocol however this is time consuming to implement and TCPalready fixes these issues natively.

Figure 3 shows the interface of the game engine Unity that was used to create the en-vironment of this project. In the upper left corner is the virtual room that the user seesand with which the user can interact. In the upper center are the various objects used tocreate the virtual room. The directory of the project is in the lower right corner wheredifferent resources called assets that can be used in the program are located. A consolelog where messages such as error messages and warnings appear can be found in thelower center. Lastly, the right side of the interface is where the user can manage thegame objects.

Figure 3 Unity interface with a prototype of the filter design tool

6.2 MATLAB

In this project MATLAB is used for complex mathematical calculations. MATLAB alsoacts as a server which Unity can then connect to in order to exchange data. After receiv-ing filter specifications from Unity, MATLAB creates the desired filter and generates

9

7 Requirements and Evaluation Methods

the corresponding plots. These plots are then saved in a place that Unity can access andwith the help of a TCP connection notifies Unity that it is done with the calculations.

7 Requirements and Evaluation Methods

There are three ways to evaluate the end product of this project. Firstly how user friendlythe application is, it needs to be user friendly in order for new students to easily pickit up and understand how to use the application. Secondly the performance of the ap-plication, in order for the user to get the best learning experience the application needsto run smoothly and be as non-disruptive as possible. Thirdly the functionality of theapplication, or rather how effective it is as a learning assistant.

To evaluate the user friendliness of the application it must be tested by students thathave taken or are taking the course Embedded Systems and Signal processing and arenot familiar with virtual reality. It makes sure that the testers can focus only on the userfriendliness of the program and not be confused by how filters work. Ideally the morestudents that can test our product the better. The test is a questionnaire which the testersanswer after using the application. Below are the questions which are answered anyway the students wants to be ably to convey as much information as possible.

• Was it easy to understand the controls, did you find them intuitive?

• Is it clear what the buttons and sliders do on the user interface?

• Are the buttons and sliders placed in a logical way?

• What did think about the way the results are displayed?

As far as performance is concerned, the less the user is affected by time lag the better.There are two performance affecting parts to the application. First is the calculating partwhich is triggered whenever a filter is to be calculated and plotted. We think it is accept-able if the calculations and plotting takes at most 2 seconds. This is evaluated simplyby timing a creation of a filter. The second part happens when the plots are constructedas objects in virtual reality. The more objects there are on the screen the more time lagit creates. A test of how many objects can be created without time lag should be made.

To evaluate exactly how pedagogical and educational our project is can be difficult.It would have to be implemented it in a course and use it over several iterations of thatcourse to have enough data for a conclusion. The data should consist of feedback from

10

8 Properties of Filters

students that have used the tool. The feedback should include opinions of usability, userfriendliness. The students could discuss improvements of the application and if it hasimproved their knowledge of the given subject.

8 Properties of Filters

Filters and signals are most often represented in the frequency domain. Depending on itstype a filter can either dampen or pass a signal within a certain frequency range. Cutofffrequencies are the frequencies where the filter begins to either dampen or pass a givensignals strength. There are four basic types of filters, low-pass, high-pass, band-pass andband-stop. These types describe what parts of the frequency range the filter will dampenor pass a signal. For example, a low-pass filter will pass low frequencies and dampenhigh frequencies, where low frequencies are all frequencies lower than the cutoff fre-quency and high frequencies are higher than the cutoff. A high-pass filter functionsalmost the same but it passes high frequencies and dampens low frequencies instead.Band-pass and band-stop filters have two cutoff frequencies, band-pass passes all fre-quencies between the cutoff frequencies and dampens the rest while band-stop does theopposite and dampens all frequencies between the cutoff frequencies and passes theother frequencies.

A filter can be described by a mathematical function called transfer function. Thisconsists of x- and y-terms, each with a b- or a-coefficient respectively. When the filter isgiven an input the transfer function of the filter returns the output. The output is given infrequency domain and is called a frequency response. On one of the generated plots theuser will see a pole-zero plot. Poles and zeros are specific points in the transfer functionof a filter that determines its behavior.

The above is true for both analog and digital filters. To convert an analog filter to adigital filter a sampling frequency is needed. This means that the analog filter is re-peated every sampling frequency. For example if the sampling frequency is 46 000 Hzthe digital filter will look like the analog filter at 0 Hz and then the same at 46 000 Hz,92 000 Hz and so on. Digital filters can either be Finite Impulse Response (FIR) or In-finite Impulse Response (IIR), where the main difference is that IIR filters use feedbackto calculate the output while FIR filters do not.

11

9 Filter Design Interface

9 Filter Design Interface

A whiteboard is used in the virtual room which is where the user sets the desired filterspecifications. It is illustrated in Figure 4 below.

Figure 4 Filter design interface with which the user interacts

The interface consists of four sliders which is used to set the number of coefficients, thesampling frequency, a low cutoff frequency and a high cutoff frequency. These slidersare located on the left hand side of the interface. Above the sliders the user can find twobuttons which are used to select the wanted filter, a digital filter can be one of two types,either an IIR or a FIR. In the upper right corner are four buttons which are used forspecifying which kind of passband the filter should have. The interface also has threeempty areas which are used to show a general filter for pedagogical reasons that willbe explained in greater detail in the next sections. Lastly, the “Generate Filter” -buttongenerates the filter.

12

9 Filter Design Interface

Figure 5 Whiteboard where encircled areas work as previews for pedagogical purposes.

To get a better understanding on what everything is doing on the whiteboard refer toFigure 5 which encircles some areas on the whiteboard which will be explained morethoroughly in the next subsections.

9.1 Area 1, General Formula

This area displays how a general ordinary differential equation (ODE) of the filter lookslike. It is based on the number of coefficients to include and whether the IIR button orFIR button is selected. In both cases another term will be added together with a constant“b” or “a” depending on if the term is an x-term or y-term. It works as follows; whenan x-term is added it will be multiplied by a constant “b” which is numbered based onhow many x-terms are already in the equation. If the ODE for example looks like this:

y(n) = b0x(n) + b1x(n− 1)

13

10 Implementation

and another x-term is added, the equation will look like this:

y(n) = b0x(n) + b1x(n− 1) + b2x(n− 2)

When FIR is selected the formula changes so no y-terms are added.

9.2 Area 2, Example Graphs

This area displays the frequency response depending on what filter type is selected to theleft of the graph i.e. low-pass, high-pass, band-pass or band-stop. The cutoff frequencyor frequencies depending on what filter type is selected can also be seen marked in thegraph. This is to help the user in understanding how the cutoff frequency affects thefilter.

9.3 Area 3, General Equation

This area displays the ODE of the filter the user currently is designing, as mentionedbefore the ODE changes depending how many coefficients the filter has and on whetherthe FIR or the IIR check box is selected. If the formula from area 1 is then applied tothe filter with the selected coefficients the user will get the ODE in area 3.

9.4 Sliders

To change the values of the filters frequencies sliders can be used. The sliders’ valuesare changed by grabbing the circle and dragging it to the right to increase and to the leftto decrease.

10 Implementation

This section covers more specific details on how the two main parts MATLAB and Unitywere implemented. We will also go through what problems we faced while developingthese parts and how we solved them.

14

10 Implementation

10.1 MATLAB

First we had to find a way to establish a connection between MATLAB and Unity. Thiswas done by using MATLAB as a server and Unity as a client with the help of InternetProtocol. There was an attempt to try and encapsulate a MATLAB function into a .NETwrapper. This basically means that we tried to translate MATLAB code into C# whichUnity uses. This did not work because Unity uses an old version of the .NET libraryand there was no way to force MATLAB to use the old one as well.

The second thing we needed was a way for MATLAB to know what filter the userwanted to design and since the specifications are sent as a string from Unity to MAT-LAB a parser for MATLAB was then needed. A parser is a tool used to analyze a stringof characters, in this case to extract the different filter specifications. After the speci-fications are extracted they are used as arguments for another MATLAB function thatcreates the filter. When the filter is created a plot of its frequency response and pole-zeroplot is generated and saved in a local folder on the computer. The contents of this folderis then reachable by Unity.

10.2 Unity

To get HTC Vive to work with Unity a plug-in called SteamVR was needed. This couldbe downloaded for free from Unity Asset Store. Unity has an “asset store” that allows auser to share his or her work with other users, free or for a small price [23]. Another freeplug-in used was Virtual Reality Toolkit (VRTK) [20], which provided several helpfultools such as interaction with objects and movement through the room in the form ofteleportation.

After the initial setup we started with creating the room where the user exists in virtualreality. The goal was to keep the room as simple as possible with only the whiteboardexplained in Section 9 for the user to interact with. The room was made with five boxesthat were formed as floor and walls.

The process of creating the objects that the user is able to see in the room is done bychoosing which object to instantiate from a list of predetermined shapes. Then the useris able to add components to the objects which gives it attributes such as being subjectto gravity and having a solid body. On some objects we needed to add functionalities,actions that occur when interacting with the object. This was done by writing code inC# and then attaching the code to the desired object as a component.

15

11 User Manual

11 User Manual

In this section necessary information regarding how to use this application can be found.This will be the minimum required hardware needed and how the controllers work.

11.1 Prerequisites

To be able to use this application, certain hardware is necessary. An HTC Vive head-mounted display with controllers and sensors are needed and also MATLAB versionR2017a or newer. This the minimum requirements to be able to use this application.Virtual reality is demanding in terms of processing power and therefore a computerwith a powerful graphics card is highly recommended. This is to ensure the user getsthe best virtual reality experience possible.

11.2 Controls

To be able to interact with objects and move around in the room the user uses two con-trollers, the right controller and the left controller. These two controllers look and workthe exact same way, but for this project they have been given different functionalities.To get an insight of how the buttons look on the controller refer to Figure 6 down below.

The left controller is used to move around in the room and consists of three usablebuttons: Pointer, Select and Grip. The user can move in the form of teleportation. Thisis done by pressing the Pointer button which emits a green laser, the user can point itto where they want to go and then release to teleport to that position. The user can alsoslide around the room by touching the Select button in the desired movement direction.Using the Grip button, the user can pick up and drop objects.

The right controller consists of three usable buttons: Select, Grip and Pointer. Theprimary use is to grab and interact with objects in the room.The Pointer button emits a blue laser from the in-game controller which then acts as acursor to give the user more precision when interacting with objects in the game.Select is used to press buttons and grab sliders.Grip is used to grab objects in the game, primarily the generated plots.

16

12 Evaluation results

Figure 6 An HTC Vive controller with the available buttons mapped out

12 Evaluation results

Due to time constraints the user friendliness tests could not be done. Instead the userfriendliness part of the evaluation was done with the help of two students who hadalready completed Embedded Signal Processing Systems and had little previous experi-ence with virtual reality were invited to try the application. The controls were explainedand a task to create filters was given. While performing these tasks they were askedquestions regarding what they thought about the application. Because both studentswere already familiar with digital filter design we were able to get answers more fo-cused on the user friendliness of the product.

Both students thought the filter design interface was easy to understand and had noproblems carrying out their tasks. MATLAB requires at least three to seven secondsto create its plots which is by no means fast. When the students were asked what theythought about the time it took to generate the plots they answered that they had no prob-lem with it.

The system was stress tested to see if it would begin to lag at a large amount of ob-jects on the screen. The amount of objects on the screen, spheres in this case, was set to

17

14 Conclusions

200. The amount of spheres seemed reasonable and no time lag was observed.

13 Results and Discussion

The end result is a prototype of an application used for learning digital filter designin virtual reality. While creating the application all of the functionalities our externalstakeholder wanted could be implemented. The user is able to create various types offilters using the filter designer. When the user has designed and generated their filtera plot of both the frequency response and pole-zero map is instantiated as in Figure 7which can be moved around to be compared with other user-generated filter plots.

Figure 7 A user generated plot in the virtual room

The time it takes to receive a filter after generating it is between three to seven secondswhich is something we believe should be improved. This is a constraint caused byMATLAB and therefore to improve performance of the project an alternative is needed.An alternative as stated in delimitations would be our own written framework in Unityfor calculating and plotting filters.

14 Conclusions

When reflecting on the final product and answers from the students that have testedthis project we can conclude that many of the functionalities in this project could have

18

15 Future work

been done without using virtual reality. In fact it could probably have been made evenbetter if instead of using virtual reality the user is only sitting in front of a computer.This is not true for all subjects, some are more suited for virtual reality than digitalfilter design. Especially where an understand of the structure or anatomy of an object isneeded for example learning about engines or architecture of buildings. Physics couldbe accurately simulated which is difficult to do in real life. This is where virtual realityreally shines when the user can move around, interact and take a look inside of whatis needed to learn. We also believe that since virtual reality gives the user more of animmersive experience than you would get sitting in front of a computer that would boostthe interest of the student.

From our result of using MATLAB together with Unity we realized that a need for moremathematical tools is needed for learning tools in virtual reality. This in order to ease thecreation and to simply make better learning programs in virtual reality. Unity is a gameengine which was evident during the process but if it can be adapted to teaching purposessuch as natively integrated with MATLAB we can definitely see that this project wouldgive a better result.

15 Future work

There is a lot of potential for further development on this project. Some functionalitymight show itself needed after this product has been tested in the next iteration of Em-bedded Systems and Signal processing. However, an example of a future work couldbe a function generator which allows the user to generate their own signals that theycan then send through their created filters. That way they can see how different signalsbehave when their filter is applied. To allow for even more analysis of the signal anoscilloscope could be implemented. Based on our results with using MATLAB, the os-cilloscope would probably have a long delay so a faster tool is needed to achieve closeto real time updates of the output.

A replacement for MATLAB could be made. Virtual reality is all about interactivityand MATLAB offers none when integrated with it. The replacement should be madeon the same platform the virtual reality development takes place on i.e. Unity, UnrealEngine etc. If done correctly graphs could be plotted in 3D and even changed in realtime with no need to redo the calculations.

Another interesting feature would be to have the user be able move the poles and zerosand see how the rest of the filters specifications and filter plot change. We believe thatbeing able to go the other way will help even more in building intuition for digital filter

19

References

design.

References

[1] P. Ashraf, L. Billman, A. Wendelin, Teaching Signals to Students: a Tool for Visu-alizing Signal,Filter and DSP Concepts, Student thesis, Uppsala University, 2016,diVa: diva2:941007.

[2] Beijing Bluefocus E-Commerce Co., Ltd. and Beijing iBokan WisdomMobile Internet Technology Training Institutions 2016, A Case Study- The Impact of VR on Academic Performance, Viewed 12 April 2018https://cdn.uploadvr.com/wp-content/uploads/2016/11/A-Case-Study-The-Impact-of-VR-on-Academic-Performance 20161125.pdf

[3] Best Buy 2018, Viewed 6 May 2018 https://www.bestbuy.com/site/virtual-reality-devices-and-games/virtual-reality-headsets-for-pc/pcmcat1476726957734.c?id=pcmcat1476726957734

[4] Liza Brown, Nov 25,2017 9:32 am filmora Top VR Game Engines Article, Viewed26 April 2018 https://filmora.wondershare.com/virtual-reality/state-of-vr-games-the-game-engines-and-present-convention.html

[5] J. R. Buck, K. E. Wage, M. A. Hjalmarson, and J. K. Nelson, Comparing studentunderstanding of signals and systems using a concept inventory, a traditional examand interviews,. in 2007 37th Annual Frontiers In Education Conference - GlobalEngineering: Knowledge Without Borders, Opportunities Without Passports, Oct2007, pp. S1G–1–S1G–6.

[6] John W. Eaton and many others, 1988 GNU Octave Viewed 3 May 2018 https://www.gnu.org/software/octave/

[7] Google Cardboard 2018, Viewed 25 April 2018 https://vr.google.com/cardboard/

[8] E Hermelin, T Holmberg and E Rick, Teaching Signals to Students: a Tool forVisualizing Concepts in Digital Signal Processing (DSP) Student thesis, UppsalaUniversity, July 2017, diVA: diva2:1118850.

[9] D Jospeh, P Chawla, H Chander, T.K Rawat in ”A Multipurpose Toolkit for Teach-ing DSP in an Undergraduate Course”,Student thesis, Netaji Subhas Institute ofTechnology, New Delhi, Feb 2017. Viewed 12 April 2018 https://onlinelibrary.wiley.com/doi/pdf/10.1002/cae.21819

20

References

[10] Medis Media Pty Ltd. 2017, Medis Media 3D Organon Virtual Anatomy Viewed12 April 2018 https://www.3dorganon.com/

[11] R. Nasr, S. R. Hall, and P. Garik, Student misconceptions in signals and systemsand their origins - part ii, in Proceedings Frontiers in Education 35th Annual Con-ference, Oct 2005, pp. T4E–T4E.

[12] J. K. Nelson, M. A. Hjalmarson, and K. E. Wage, in 2011 Digital Signal Process-ing and Signal Processing Education Meeting (DSP/SPE), Jan 2011, pp. 192–197.

[13] J. K. Nelson, M. A. Hjalmarson, K. E. Wage, and J. R. Buck, in Students’ interpre-tation of the importance and difficulty of concepts in signals and systems, in 2010IEEE Frontiers in Education Conference (FIE), Oct 2010, pp. T3G–1–T3G–6.

[14] Oculus, Oculus: Step Into The Game, Viewed 20 April 2018 https://www.kickstarter.com/projects/1523379957/oculus-rift-step-into-the-game

[15] PluralSight LLC. 2014 PluralSight Unreal Engine 4 vs. Unity: Which Game En-gine Is Best for You? Viewed 3 May 2018 https://www.pluralsight.com/blog/film-games/unreal-engine-4-vs-unity-game-engine-best

[16] Roland Priemer, Introductory Signal Processing, p. 1. ISBN 9971509199.Archived from the original on 2 June 2013.

[17] Slant Community, Apr 19, 2018 What are the best game engines for Virtual Real-ity development? Article, Viewed 26 April 2018 https://www.slant.co/topics/2202/∼game-engines-for-virtual-reality-development

[18] The Mathworks Inc. 1984 The Mathworks MATLAB Viewed 23 April 2018 https://www.mathworks.com/products/matlab.html

[19] The Mathworks Inc. 1984 The Mathworks Practical Introduction to Digital Fil-tering Viewed 28 April 2018 https://www.mathworks.com/help/signal/examples/practical-introduction-to-digital-filtering.html

[20] Thestonefox, 2016 thestonefox VRTK Github repository, Viewed 23 April 2018https://github.com/thestonefox/VRTK

[21] Think Mobile, 2017 Best virtual reality SDKs to build VR apps in 2018 Arti-cle, Viewed 26 April 2018 https://www.slant.co/topics/2202/∼game-engines-for-virtual-reality-development

[22] Unity Technologies, 2018 Unity Technologies Unity3D Viewed 23 April 2018https://unity3d.com/

21

References

[23] Unity Technologies, 2017 Unity Technologies Unity Asset Store Viewed 8 May2018 https://assetstore.unity.com/

[24] Uppsala University, 2012 Viewed 9 May 2018 http://www.uu.se/utbildning/utbildningar/selma/kursplan/?kKod=1TE682

[25] Fahri Vatansever Nedim A. Yalcin, e-Signals and Systems: A web-based ed-ucational tool for signals and systems, Research paper, 27 April 2017, https://onlinelibrary.wiley.com/doi/abs/10.1002/cae.21826

22

References

23

A

A

Embedded Signal Processing Systems, 10.0 cCourse code: 1TE682, Report code: 14412, 33%, DAG, NML,week: 43 - 02 Semester: Autumn 2016week: 03 - 11 Semester: Spring 2017

Course report

Course report for Embedded Signal Processing Systems (1TE682, 10hp) for autumn 2016/spring2017

Registered students: 55Answering frequency: 16/55 (29%)Date: 2017-08-23

Examination results for exam on 2017-01-10

Number of students examined: 44 before compensation: Fail: 35 (80%) Pass: 3 (7%) Pass with Distinction (4 or 5): 6 (13%) after compensation: Fail: 25 (57%) Pass: 12 (27%) Pass with Distinction (4 or 5): 7 (16%)

Brief summary of student viewpoints and suggestions

"Strengths" according to students

video material for lectures

flipped classroom and active involvement of students in lectures

good teachers

"Weaknesses" according to students

problems with lab equipment

a lot (maybe too much) material to cover for given hp

Comments from course director/teachers on the implementation and outcome of the course

The course has developed well during the last years. Flipping the classroom during the last term was verysuccessful. Unfortunately, many students still fail the written exam. Also, a huge amount of the students(approximately 50%) still struggle with mathematical fundamentals like complex numbers and frequency f vsomega. In a feedback session, I discussed possible changes and suggestions with students, who would prefer morecompulsory elements in the theoretical part of the course to force students to study more.

Proposed changes/comments/measures

Due to the very positive feedback, we will continue teaching this course using flipped classroom techniques. The following elements will be made compulsory as a reaction ofrepeated knowledge gaps and recommendations from students:

watching videos or reading the course compendium (and writing chapter summaries) in order to prepare for lecturesessions

solving a homework assignment on mathematical preliminaries (e.g., complex numbers, sinusoids, etc)

solving at least three other homework assignments on different parts of the course (students can choose betweendifferent options)

increasing the awarded hp for the theoretical part to 5hp (from 4hp before)

Steffi Knorn

24