Zakk GiacomettiArizona State UniversityEngineering Portfolio - Fall 2018

Computer (Systems) Engineering B.S.E. (Spring 2019)Computer Engineering (Computer Systems) M.S. (Spring 2020)

1

Contents

Resume 3

Comprehensive Career Plan 4

Interest Paper 12

Project Report (Junior Year) 23

Sample Letter of Recommendation 33

Sample Statement of Purpose 35

2

Zakk GiacomettiH +1 (480) 734 6653B zakkgcm@gmail.com

zakk.giacomettizakkgcm

EducationSpring 2020(Expected)

Master of Science - Computer Engineering 4+1, Arizona State University, Tempe, Arizona.

Spring 2019(Expected)

Bachelor of Science in Engineering - Computer Systems Engineering, Arizona State Uni-versity, GPA 4.0, Tempe, Arizona.Coursework: Data Structures & Algorithms, Design & Synthesis of Digital Hardware, Embedded Micropro-cessor Systems, Operating SystemsIn progress: Intro to Artificial Intelligence, Multimedia Information Systems, Signals & Systems, ComputerNetworks

Technical Skills{ C, C++, Python, Javascript, Java, MATLAB, ROS (Robot Operating System){ GNU/Linux (Debian-based, Gentoo), OpenEmbedded, git, Gazebo, TeX

Experience & AwardsSummer 2018 Software Engineering Intern, Ball Aerospace, Boulder, CO.

Evaluated and prototyped visualization softwares (Unity, rviz, and Gazebo) for a ROS-enabled mechanismand point-cloud data. Contributed estimates on capabilities and time investment toward direction on futureproject decisions.

Jan 2018 -current

Fulton Undergraduate Research Initiative, Arizona State University, Ira A. Fulton Schools of Engineering,Tempe, AZ.Investigated and integrated software algorithms with hardware sensors to develop a robotic campus tourguide. Utilized visual markers (AprilTag) for local path navigation and tracking of those being guided.Evaluated software using Gazebo simulations. Presented work at ASU sponsored symposium. Recievedfunding for proposal to continue improving the work in the Fall semester.

May - Aug2017

Technical Internship, NASA Marshall Space Flight Center, EV41-Control Systems Analysis and Design,Huntsville, AL.Developed software architecture for a ground-based CubeSat controls prototyping platform for use by futurecontrols engineers. Improved work from previous interns by integrating an existing Simulink model withcustom Robot Operating System C++ and Python nodes targeting an embedded ARM computing platform.Produced a solid foundation for implementation of Hardware-in-the-Loop capability and use as a generaldevelopment platform. Presented results of work at intern poster symposium and in a final written technicalreport.

2017-2018 National Science Foundation ASAP-METS Scholar, Arizona State University.(Academic Sucess and Professional Development - Motivated Engineering Transfer Students)

Project ExperienceAugust 2018 Ball Intern Remote Sensing Team (BIRST) (Arduino, C++), Team Project.

Team project to design, build, and launch a high-altitude balloon payload in a 6 week timeframe. Launchedpayload to stamp a pancake with the Ball logo mid-flight and record video, pressure, radiation, andtemperature data. Recovered intact payload, footage, and data. Primarily responsible for mechanism designand electronics integration.

Jul 2017 Pathways Experimental Rocket Compeition (PERC) (Arduino, C++), Team Project.Team competition against intern groups from different NASA centers. Integrated 9-DOF IMU, Pressure,Temperature, and RF transmitter with an Arduino Pro Trinket serving as the telemetry package of a modelrocket for a competition flight. Achieved real-time data stream at a range of roughly 1000ft.

2017 3dboids (Javascript, THREE.js), Solo Project.Implementation of Boids bird flocking behavior simulation. Modelled behavior of boids using simplekinematic equations and localized crowding rules in three dimensions.

Comprehensive Career PlanZakk Giacometti

Computer Systems Engineering, Ira A. Fulton Schools of EngineeringFall 2018

1 Summary of Goals and Objectives

1.1 Technical Area

My grand objectives are to work in the area of multi-robot systems (MRS), especially those operat-ing in a space environment (such as planetary exploration, near-earth, and deep space operations).I believe this is a highly challenging and relevant area, with secondary applications that can bebenefit humanity far into the future and in many aspects of society. This leverages the skills andknowledge base offered to me by my field of study, Computer Systems Engineering, and providesme a path with which to shape my future career and educational plans.

This area spans across many domains including: robotics, control systems, computer science,software engineering, computer networks, mechanical design, and artificial intelligence. Pursuingit will require skills and knowledge in many, if not all, of those domains, as well as a solid feelingfor the application and domain of the specific problems to be pursued. More specific skills I willneed include:

• Strong knowledge of C++ and Python

• Experience with computing systems operating in harsh envrionments (e.g. deep space) andincluding fallback software systems, RAD hardened processors, etc.

• Autonomous decision making (artificial intelligence, machine learning, adaptive control sys-tems)

• Strong experiences working on interdisciplinary teams

These skills will be obtained through my projects (including FURI and capstone), focus of topicsin my senior year and graduate school, internships, and work experiences.

1.2 Goals and Objectives

I will be earning my B.S.E in Computer Systems Engineering in May of 2019. I am set to thenpursue my Masters in Computer Engineering (Computer Systems concentration) with an emphasison Autonomous Systems and Robotics. This will be through the 4+1 accelerated masters at ASU,with my graduation date set for May of 2020. In the long-term, I would like to pursue a PhD inthis area, Computer Systems Engineering, or a complementary field. The knowledge required forthis area, especially when operating with the challenges of space exploration, will likely be besttackled in the research environment of a PhD. My current and future coursework is being carefullyselected to target this specialty. Included are courses in artificial intelligence, linear and multi-variable control systems, and graph theory. For my senior capstone project, I am working with the

1

NASA Psyche mission to develop a system to help classify the bulk composition of iron meteoritesamples, collecting data that will be used later in the mission. I intend to use this experience toenhance my skills working on interdiscipinary teams, C++ programming, and developing systemsthat serve one part of a greater goal.

For the next several years I plan to work in industry after obtaining my masters degree. I plan tofocus my search in the aerospace industry and other companies specializing in robotics. Aerospacehas a strong relationship with my interests with its work on devices such as survey spacecraft,planetary rovers, and planetary science. Possible companies I will focus on include NASA, SpaceX,Ball Aerospace, and Southwest Research Institute. I value companies with challenging work, astrong work-life balance, enriching work environment. I believe that time in industry will provideme with grounded, real-world experience working on problems. Additionally, I think it will helpto provide me greater perspective and domain knowledge that would complement that I have andwould receive in academia.

I eventually plan to return to obtain my PhD and conduct research in MRS. Arizona StateUniversity, Georgia Institute of Technology, UC Berkeley, and University of Pennsylvania are strongcandidates for this endeavor. This long-term plan is contingent on many factors: whether industryhas equally interesting opportunities available to me, money, location, family, and the state ofresearch in the area. I may prefer remain in the industry for longer if circumstances are notfavorable for me to pursue a PhD immediately or I am able to achieve my goals there.

2 Current Status

2.1 Relevant Coursework Completed

My coursework completed to date follows the major map of the Computer (Systems) EngineeringB.S.E. degree at ASU. It spans specialty areas of both computer science and electrical engineering.It has provided me a strong technical and theoretical background in programming, algoirithmdesign, embedded systems, circuit analysis, mathematics, and physics. This background will beessential for more advanced topics in these areas.

• Microeconomic and Macroeconomic Principles

• Calculus: I, II, III

• Discrete Math Structures

• Modern Differential Equations

• Probability and Staticics for Engineering Problem Solving

• Applied Linear Algebra

• Physics: Classical Mechanics, Electricity and Magnetism

• General Chemistry I

• Digital Design Fundamentals

2

• Computer Organization and Assembly Language

• Programming for Computer Engineering

• Data Structures and Algorithms

• Design and Synthesis of Digital Hardware

• Embedded Microprocessor Systems

• Operating Systems

• Intro to Software Engineering

• Circuits: I, II

2.2 Current Relevant Coursework

• CSE423 Systems Capstone Project

• CSE434 Computer Networks

• EEE203 Signals and Systems

• CSE408 Multimedia Information Systems

• CSE471 Introduction to Artificial Intelligence

2.3 Professional Preparation

I have prepared very well in the areas of resume construction, development of my writing andpresentation skills, critical thinking, technical communication, and team communication. Thishas been accomplished through numerous resume review session, team projects in and outside ofclasses, the ASAP FSE394 class, the Fulton Undergraduate Reserach Initiative, and senior capstonedesign. Improvements can be made in the area of planning, mentoring, and leadership. I plan totackle these in the coming years by simply taking proactive steps toward improving them. Thiswill include increasing my volunteering and involvement with clubs and groups on campus, seekingmore mentoring from others, and taking on leadership roles I otherwise would not plan to maketime for.

3 Education

3.1 Future Relevant Coursework

Spring 2019 [noitemsep]

CSE424 Systems Capstone Project

CSE520 Computer Architecture II

MAT416 Introduction to Graph Theory

3

EEE480 Feedback SystemsGraduate

• CSE551 Foundations of Algorithms• EEE554 Random Signal Theory• EEE588 Design of Multivariable Control Systems• CSE574 Planning and Learning Methods in AI• EEE586 Nonlinear Control Systems• EEE686 Adaptive Controls• EEE511 Artificial Neural Computation

4 Research/Project Experience

Research is crucially important because it allows one to explore their interests, direct their ownlearning, and offers another avenue for applying knowledge gained in the classroom in a way thathas positive external effects on society. Moreover, it encourages one to think of the big picturesignificance of the work being done in an area. Continually asking the question: ”What is thesignificance of my research?” greatly stimulates this.

My long-term research focus is in the area of multi-robot systems, particularly in aerospace orspace applications. This is an area I believe is filled with challenges and relevance to a wide range ofindustries and fields. It is well supported by the academic community, with numerous universitiesand professors dedicating resources to mutli-robot systems.

I am currently involved in a FURI project designing an intelligent campus guide robot. Thisresearch utilizes vision processing of markers that mark a path to facilitate navigation betweenlocations on campus. It leverages skills I have gained from classes as well as technologies learnedfrom internship experience. This research serves as a stepping stone to future research, providing abasis on which to expand into a system of multiple robots and/or explore their application in hostileenvironments such as disaster zones and planetary exploration. This research can be continued onas the focus of a masters’ thesis in Computer Systems, as well as the focus of a doctoral thesisin Computer Systems or Planetary Exploration Systems. Several fellowships are relevant to theresearch I am interested in conducting and I will be considering applying to them. Two that I amfamiliar with are the NASA Space Grant and National Science Foundation Fellowship. Many moreexist, and I will be exploring those that are relevant.

5 Industrial/Work Experience

Industrial experience offers a different insight into my technical areas of interest and a differentset of experiences compared to academic research. Industry is more often focused on creatingsolutions that are commercially viable and will have an immediate impact on their customers.Additionally, as with research, the results can still be of benefit to the general public and advancethe state of industry or technology at large. However, the level of advancement in the state oftechnology may not be as great as with more academic research. Opportunities are numerous,with competitive pay, but the quality of such opportunities is difficult to assess. Additionally, adegree of autonomy is lost when working in industry as your interests must work toward those ofthe company and the direction of the projects you work on may not be entirely your own. Overall,certainly early in my career, I believe industry experience is vitally important in discovering andtargeting specific interests. Industry experience in autonomous systems, space science (such assatellites), and embedded systems (such as flight software and flight controllers) would most appeal

4

to me and provide what I hope are exactly the types of experiences that will contribute the mostto my overall goals. Possible industry groups include:

• NASA (Jet Propulsion Laboratory, Marshall Space Flight Center) – Both of these NASAcenters specialize heavily in space robotics. They are two of the leading industry experts inplanetary rovers and deep-space satellite control systems.

• Southwest Research Institute – A non-profit research organization that works in a wide varietyof areas ranging from space sciences, sensors, and instrumentation to autonomous vehiclesand robotics.

5.1 Prior Experience

To date, I have worked as an intern at NASA’s Marshall Space Flight Center for the summerterm of 2017 in its Control Systems Design and Analysis Branch (EV-41). There, I was able togain experience working with the Robot Operating System, a very commonly used and powerfulsoftware framework for robotics. Additionally, I was able to gain a first look into the field of controlsystems, an integral part of robotics. The experience also offered an opportunity to write a finalpaper, create a poster, and present my work.

In the summer of 2018, I secured an internship with Ball Aerospace in Boulder, Colorado. Iwas a part of their Software Engineering division working with a sizeable interdisciplinary teamon a project. There, I worked to help design visualization methods for a Robot Operating Systemenabled complex mechanism. I was able to learn much about complex mechanism design, systemsengineering, as well as how an aerospace contractor operates on these kinds of systems.

5.2 Future Plans

My current future plans are to return to Ball Aerospace this coming summer before the start ofmy final Masters degree semesters. There I hope to gain experience in a different or new area ofsoftware within the aerospace industry

6 Community Service

Community service is an excellent avenue for building one’s network, connecting with causes thatcould be the driving inspiration for research, and assist in developing leadership and teamworkskills.

In the past I have been involved with volunteering for robotics competitions such as FIRSTand Vex as a judge. I would like to be more heavily involved, potentially mentoring a team ortaking on more involved roles in helping to coordinate events. I have also sought relevant volunteeropportunities on campus at ASU. Through my involvement with the Micro Air Vehicles club, I haveparticipated in a few events engaging with younger students and peers. My future involvementwith this club will bring more opportunities. FURI provides the opportunity for me to volunteer inhelping others with research, writing proposals, sharing my experiences, and defining their problemareas. Through the ASAP program I have been afforded the opportunity to connect with andmentor younger students from community colleges and those incoming transfers to ASU. This isdefinitely an area I am interested in developing my leadership skills with.

5

6.1 Leadership Skills

I plan to build leadership skills through continued involvement mentoring others in the ASAPprogram. Continued involvement in clubs on campus such as the MAV club and ASU Linux User’sGroup will also provid me opportunities to do this through leadership roles, knowledge sharing,and campus invovlement. As I continue forward in my career, building leadership skills in the formof mentoring and teaching others and being a driving force behind projects will become even moreimportant.

7 Personal Mentoring Plan

7.1 Importance of Mentoring

Mentoring is essential to helping fully define my future career path and determining how to navigatemy fields of interest and connect them to their utility in society.

I currently turn to Dr. Rodriguez, a professor at ASU with a background in control systemswith interestes in intelligent autonomous systems. I have worked with him on both of my FURIprojects, sought advice about graduate school, and plan to have him involved as part of my Master’sthesis.

I had previously been in contact with a mechanical engineering professor at ASU specializingin Multi-Robot Systems. From the computer science arena, there are other professors that alsoconduct research in the areas of artificial intelligence and multi-robot systems. I plan on contactingthem soon to seek mentorship and discuss my future plans.

8 Economic and Finanical Goals

In five years, my goal is to have finished my education and be several years into my career earninga competitive salary for the industry. More importantly, I hope to have properly invested andhandled those earnings, holding a minimum 6-month emergency fund and significant investments.By that time, I will be fully independent from my parents and hope to be helping them financiallyif necessary. My career prospects in industry are very supportive of these goals, offering reasonablesalaries as mentioned and excellent benefit and investment plans.

9 Graduate School

9.1 Importance of Graduate School

My intention in pursuing graduate school is to gain an extra level of knowledge and experience inmy area that will secure my future, enable me to pursure more exciting opportunities in my career,and further enable life-long learning. I have recieved much advice pushing me to pursue

9.2 Preparation Plans

I have already applied and been accepted to ASU’s 4+1 MS program in Computer Engineering(Computer Systems). My current plans to pay for graduate school are to apply for TeachingAssistant or Research Assistant positions. In addition, I will apply for any relevant scholarship orfellowship opportunities available.

6

9.3 MS Thesis and Committee

I am currently planning to pursue an MS Thesis focusing on task allocation strategies for multi-robot systems in a coordinated construction task. I believe this path will enable me to gain muchmore out of my MS degree. I have several professors in mind that I feel would be good to serve onmy committee. In order to finish my thesis, I plan to begin work in the coming Spring semesterand carry through some work over the summer.

9.4 PhD in Engineering

I have considered pursuing a PhD in engineering, researching a topic similar to that I am pursingfor my MS thesis. My plans have shifted and I have decided to delay that pursuit to later in my life.This is a choice motivated by many factors including my percieved ability to fully carry throughand invest in the program. I intend to keep the possibility of pursuing a PhD open in the long-term.

10 Long-Term Professional Goals

10.1 Job Description

My ideal job is a position designing and implementing software in robotic or intelligent systems.This could include robotic spacecraft, ground vehicles, flight software, or other software. I valuecompany environment very highly. My position must emphasis personal development and flexibilityin choosing responsibilities. My expectations are realistic, as often there will be projects that donot fully align with my interests, but some degree of self-direction is not uncommon in industry.

11 Life-Long Learning

11.1 Professional Development Plan

My professional development plan is simple. I plan to finish my Bachelor of Science in Engineering,then complete my Masters, with thesis option. To develop my professional and mentoring skills, Iplan to work as a Teaching Assistant or tutor during my Master’s degree. I have already workedduring internships in the aerospace industry, building my skills and knowledge there.

In the long term, I will continuously prepare for opportunities in higher positions, such asmanagement and systems engineering, and those positions on projects which are of interest to me.This will ensure I am constantly challenged and moving toward my future goals.

11.2 Personal Development Plan

In the short term I plan to develop myself using the opportunities that will be provided to me bycontinued research during my Masters degree. Working closely with others on research will allowme to further develop my interpersonal skills. I plan to involve myself on campus more than I havebeen during my undergraduate semesters. In the long term, I will continue this type of thinking,involving myself in the community at the company I will be working at and the community I willbe living in.

11.3 Nurturing Specific Skills and Hobbies

As I transition out of school I expect to be drawn into honing a specific set of knowledge and skillsneeded for my work, but plan to supplement those with projects in my spare time. In software,

7

having personal projects is a common way to develop new skills that will be useful in your careeror to keep previously developed software skills sharp.

11.4 Future Education Plans

Aside from continued personal projects, I plan to explore the possibility of obtaining supplementaldegrees and education that will be useful to me in industry. Systems Engineering is often calledfor in industry as a way for an engineer to be beneficial on large projects or in a management role.

12 Family Planning

13 Travel Plans

I have done very little traveling outside of the US and would be interested in having the opportunityto do so in the future. I feel traveling can provide you some perspective on your own life and theissues that you as an engineer are capable of solving. There is plenty of opportunity to do so whenI am working in industry or even while I am completing my education, solo or with my family.

I am not particularly interested in living overseas. However, I know the only way to trulyexperience a country is to live in it and be fully immersed in the culture.

14 Investment Plans

I plan to start seriously planning for my retirement this coming calendar year. My plan is tocontribute to a ROTH IRA and personal investment portfolio. Once I reach industry, I will havemany more options available to me in terms of 401k plans, stock purchase plans, and profit sharingplans.

15 Contingency Plans

There are many opportunities available for me to fall back on if my long term plans fall through.Many of my goals can be delayed, postponed, or modified to accommodate changes in circumstanceor difficulties in execution. I feel an industry job is a highly accessible fallback for not immediatelypursuing a PhD or even a Master’s degree. It will still provide challenges and opportunities whilenot actively preventing advancement in education, as many industry companies are supportive ofemployees obtaining advanced degrees after employment.

16 Philosophy of Life

Contributing to the advancement of mankind is my primary motivator in life. The work or research Ido must have tangible effects on society. This is part of my reasons for studying engineering insteadof a more theoretical field like the hard sciences or math.

17 Other Issues

8

Multi-Robot Systems: Concept to Reality

A Career Shaping Interest Paper

Zakk GiacomettiComputer Systems Engineering, Ira A. Fulton Schools of Engineering

Fall 2018

1 Technical Area Chosen

The area of multi-robot systems, collections of heterogeneous or homogeneous robots exhibitingcollective and cooperative behavior to accomplish a task, is growing at a rapid pace. These systemshave potential applications in medicine, planetary exploration, disaster relief, manufacturing, andmore. Multi-robot systems is inherently a multi-disciplinary field with many overlapping areasof interest and terminologies. It involves specializations from mechanical engineering, industrialengineering, mathematics, control systems, and computer science. Approaches range from relyingon emergent swarming behaviors (such as those of an ant colony) to more centralized coordinationbetween robots.

2 Importance of Chosen Area

Cooperating systems of multiple robots serve as one of the next stages in the development ofrobotics. They take advantage of the increased availability of small form factor computing plat-forms and microcontrollers and rapid small scale manufacturing techniques (such as additive man-ufacturing) to reduce cost. This enables massive numbers of robots to be constructed at low const,in contrast to most current robotics systems which can cost thousands of dollars. Harvard Univer-sity’s KiloBot [1,2] and Arizona State University’s Pheeno robots [3] are two such platforms builtto study swarm robotics and multi-robot systems.

Additionally, the study and development of such systems can serve both as solutions to existingengineering problems and presents its own set of open problems that can advance related scientificfields. There are numerous applications of cooperating robotics being investigated. These are areasthat can see major benefit from the introduction of cooperative robots or are otherwise incrediblydifficult to accomplish without it:

1. Planetary Exploration Current robotic explorations of other planets generally rely on a single,high-cost, heavily outfitted robot. To ensure success of its mission, many of its moves arecareful, deliberate, and as a result, slow. In addition, the availability of a single robot witha single set of instruments restricts the region of a planet that can be safely explored in anygiven amount of time. Using a fleet of coordinated robots would vastly increase the amountof a planet’s surface that could be covered. They would also provide a level of redundancyin the case of failure. Additionally, this redundancy would allow greater risks to be taken

1

Figure 1: Left: Pheeno core module [3] Right: KiloBot, individual references (A,B), and togetheras a swarm (C) [2]

by each individual robot, allowing exploration of areas previously out of reach due to fear oftotal mission failure [4].

2. Construction in Harsh Environments Utilizing robotic swarms may be crucial to buildinglarge-scale structures in environments like space and the surface of the moon. These structuresmay include human habitations, orbiting stations, or space elevators. In these situations, theability for humans to operate is limited and we will undoubtedly rely on robots to performmuch of the work in their construction. Cooperating robots that can perform these taskswith a high level of autonomy will be crucial as human resources and communication will belimited [5,6,7].

3. Search and Rescue Deployable swarms of robots, such as quadrotors, can potentially bequickly deployed to scan, map, and search the site of a disaster for survivors. This wouldserve to both help save human lives and reduce the risk to first responders [8].

Investing knowledge and resources in this area is crucial as robots become more and moreintegrated to our society. Robots will no longer be isolated machines operating in predictableenvironments, but coordinated agents operating and cooperating in uncertain environments withhumans and other robots.

3 Problem to be Pursued

A major area of interest within multi-robot systems is methods of planning and decision making.This is the issue of how to best coordinate a group of robots to accomplish a common task. Solutionsto this problem are will help to bring multi-robot systems out of the laboratory environment andinto real-world scenarios. Coordination is central to the application of multi-robot systems in nearlyevery domain from exploration to search and rescue. My focus will be on robots that will cooperate,with each other and with humans, on collective construction tasks.

Current approaches to this problem range between two extremes of explicit and implicit co-ordination [11]. Explicit coordination mechanisms involve centralized or deliberate coordinationbetween robots over resources. Implicit coordination mechanisms rely on coordinated behaviorsforming from individual actions of robots. Many approaches take inspiration from nature and

2

biology, turning to the collective behaviors of insect colonies such as termites, ants, wasps, andbees [12-15]. These approaches assume that implicit behaviors can efficiently organize robots whilekeeping computational and communication complexity low. Additionally, the element of humanoperators is also considered. In a real-world scenario, a robot may encounter an unforeseen diffi-culty or failure that requires the intervention of a human. Some research is directed toward findingthe optimal balance between the robot’s autonomy and human judgment. Ultimately, some seek todevelop ways to intelligently bridge the cooperation between humans and a multi-robot team [15,16]. In addition to this, there is a great deal of research into specific data structures, algorithms,and strategies for increasing the robustness of multi-robot systems or standardizing the softwareframeworks in use [17-20].

I intend to first proceed with intense study of these various approaches, comparing the benefitsand trade-offs of each. This wide viewing of the field from a computer science angle may providedifferent insights into the solutions presented in the area. The problem is difficult and requirescareful application of mathematical analysis and a novel approach. Approaches that expand oninspiration from nature have proven effective in the past. One of the the biggest hurdles is over-coming computational complexity constraints when increasing the number of robots in a system.This is where simpler more traditional methods, such as the classic search algorithms, fail and moreintelligent, but complex, algorithms prevail [23].

4 Importance of the Problem to be Pursued

The probem of multi-robot coordination and decision making has a wide range of applications tothe areas previously described (planetary exploration, construction, search and rescue). In assem-bly systems involving coordination between multiple robots and humans, these task allocation andplanning problems are especially relevant. For instance, Multi-agent systems in factory environ-ments can improve the efficiency and consistency of product products [25]. For deep space missions,multi-agent systems can allow a smaller team of humans to control larger teams of robotic spacecraft. This has implications in space structure construction and planetary exploration [26].

5 Relevant Career Prospects

As multi-robot systems is still very much in the research and development stage, very few industryentities are active in the area. Kiva Systems (now Amazon Robotics) produces the Kiva robotplatform utilized in Amazon’s warehouse operations. NASA also conducts a wide array of researchin the field related to space and planetary exploration across its centers. NASA Ames and the JetPropulsion Laboratory specialize in robotics and multi-agent robotic system, working on projectssuch as SPHERES and Super Ball Bot [21], and investigating trade-offs of multi-robot systems inspace applications.

Many universities have laboratories dedicated to the study of multi-robot systems and swarmrobotics. Only a portion of those are described here. The Georgia Institute of Technology’s GRITS(The Georgia Robotics and InTelligent Systems) Lab researches the control and coordination ofmultiple robots. They place a heavy emphasis on theoretically provable swarm robotics strategies.The Autonomous Collective Systems lab at Arizona State University focuses on an interdisciplinaryand biologically inspired approaches, incorporating research on both biological and engineeredcollectives. The University of Pennsylvania’s GRASP (General Robotics, Automation, Sensing,and Perception) Lab, and specifically the Kumar Lab group, focus on developing bio-inspiredalgorithms for collective behaviors and robotic swarms.

3

Figure 2: A depiction of a fleet of Super Ball Bot deploying from orbit [21].

6 Approaches: State-Of-The-Art

The cutting edge of this area pulls and combines ideas from both artifical intelligence and controlsystems.

• A system designed by Harvard University, TERMES, is inspired by termites and describesa method for collective construction under heavy idealizations [5]. The TERMES systemuses only local information and on-board sensors to plan a construction sequence, insteadof global tracking methods used by some other approaches. TERMES focuses primarily onthe mechanical design of the robots, utilzing whegs (wheel legs) and uniform building blocksto build 3D structures. It uses relatively simple path planning, ensuring a partial order ofblock sequence and assuming an obstacle free environment. It includes methods for buildingtemporary scaffolding when required.

Figure 3: TERMES robot assembly construction and path planning

4

• An approach by Barros Dos Santos [27,28] looks at decentralized planning utilizing reinforce-ment learning (RL) and heuristic search stratgies, based in automata theory. His work focuseson quadrotors that have available actions of moving, loading, and unloading of materials.

• An IPJR and truss structure being builtA work by Komendera [29] focuses on highly regulartruss structures and heterogeneous robots. This work focuses on the coordination of ”Intel-ligent Precision Jigging Robots” that precisely hold components of a structure in place forwelding by a third agent. This approach is readily applied in a 6-DOF environment suchas the ocean or space, but seems more difficult to make practical use of in a ground-basedenvrionment. Additionally, these agents are limited to truss structures and their algorithmswould need to be adjusted to work for general assembly problems (this is discussed in thepaper).

• Some approaches come from the industrial engineering domain, focusing on process optimiza-tion in assembly lines. These bear resemblence to the classic traveling salesman problem,with one variant being called the assembly sequence problem (ASP). One approach to ASPuses discrete partical swarm optimization to approximate an optimal assembly sequence [30].Another approach considers human-robot collaborative assembly and emphasizes factors suchas operating speed and battery life [31]. This approach utilizes genetic algorithms to find theoptimal allocation of assembly tasks to agents.

These assembly approaches are very specific to fixed robots operating in a mostly linearfashion. This limits their solution domain to industrial settings and applying them to moreunstructured environments may be difficult.

Figure 4: An IPJR and truss structure being built[29]

Figure 5: Comparison of assembly scenarios withgantt charts [31]

5

7 Approach to be Pursued

An approach I may consider is similiar to that ofBarros Dos Santos, pulling from the domain of ar-tificial intelligence to find optimal policies on-linefor a distributed set of quadrotors. I would wantto incorporate humans into the policy search, al-lowing an expert operator to provide input to thepolicy formulation. Simplifying assumptions canbe made, such as the omission of obstacles andlimitations on the structures to be built.

Figure 6: RL architechture proposed bySantos [27]

8 Rationale for Approach

This apprach is primarily valuable in real scenarios where the limitations of perfectly crafted algo-rithms and learning will not be sufficient to handle all given scenarios. In one scenario, an group ofrobotic agents may survey an area prior to beginning construction of a structure. Their assessmentof how to proceed will only be optimal as specified by the input parameters to the algorithmsused to determine it. Even a well trained reinforcement learning system cannot substitute for e.g.a decade of experience in civil engineering and the abstract reasoning of a human. Said agentsmay also lack certain information not provided to them or gathered by their sensors. This couldinclude conditions of the soil, scheduling restrictions, and As such, it is reasonable to propose taskallocation strategies that involve humans in the decision making process when deeemd necessary.

For the same reasoning, being able to involve humans in the task allocation policies that areextracted is also important. General purpose robots are not readily available or reliable, andspecialized robots capable of accomplishing all given tasks in a construction environment are notfeasable for many scenarios.

The use of quadrotors present challenges in the controls domain, but can potentially greatlysimplify policy extraction in the artifical intelligence domain. Having full degrees of motion foragents in the system allows structures to be built without the need for temporary scaffolding oravoidance of other agents in only the X-Y plane.

9 Risk-Reward Assessment

The major risk of pursuing this interest is opportunity cost. Putting effort toward the study ofthe necessary knowledge takes time that could be spent on other relevant pursuits. Since this arelatively sparse field, future specific career opportunities (discussed previously) are slim. There isa (small) risk of becoming too specialized and too engrossed in this one area and having difficultypursuing a career if a related one cannot be found yet. That said, the consideration of that riskis almost eliminated if the general skills and knowledge acquired are kept in mind. Many of thenon-domain specific skills, such as programming, controls, algorithm design, research, and othernon-technical skills are applicable in an incredibly wide array of engineering disciplines. As such, therewards in terms of knowledge and skills gained are well worth any risk of not reaching significantconclusions in this area.

6

Preparing to study multi-robot systems now also prepares me for a future in graduate school onthis topic, providing ample space for a thesis project or paper. As mentioned, the skills gained willprepare me for an industry career, whether in multi-robot systems, robotics, or computer sciencein general. Additionally, it also provides me a solid track to pursue a PhD and to conduct furtherresearch on hard problems in the field if I were to so choose later.

The major obstacles are my current lacking in fundamental knowledge necessary and a need fora larger network of professors and students in the area. Filling the knowledge gap mostly requiresthat I continue my education, both at the undergrad and graduate level. Choosing appropriateelective classes in senior year is critical, as it will be moving into my 4+1 accelerated mastersprogram. Taking note of what appears in relevant research papers will assist greatly in makingthese choices. Building a network requires that I invest time contacting and meeting with professors,attending group meetings, and seeking out fellow students that share my interest.

10 Preparation To Date

I have prepared to pursue this area of interest through my coursework, summer internship pursuits,undergraduate research, connecting with others interested in the topic, and through side projectsto develop technical skills. My field of study, Computer Systems Engineering, is very well suitedto allow me to contribute to the multi-robot systems area. I am gaining a strong theoreticalbackground in both algorithm design and electrical engineering. With the flexibility granted to meby course selections, I intend to focus my studies toward areas relevant to robotics such as controlsystems, machine learning, and artificial intelligence. In the Fall 2018 semester, I have taken anintro to AI course which has already yielded benefits in understanding the fundamental conceptsrequired to proceeed into the advanced understanding of this research topic.

In the summer of 2017, I interned at NASA Marshall Space Flight Center. There, I workedon a project helping to design the software for a CubeSat controls prototyping testbed. Thistestbed would primarily be used for hardware-in-the-loop tests of controls algorithms for proximityoperations involving multiple CubeSats. As part of this, I had to learn an off-the-shelf roboticsframework, Robot Operating System, as well as how to interface this with controls algorithmsdesigned in Simulink. This internship prepared me immensely, introducing me to some of thedifficulties encountered in the design of control systems.

During the summer of 2018, I intered at Ball Aerospace, working on a project with a complexROS enabled mechanism. This offered me the opportunity to gain skills in visualization of complexmechanisms and systems using ROS, as well as learn about the system engineering process inindustry. Along with utilities like rviz and Gazebo commonly found in ROS, I had the opportunityto also connect those with Unity, and see visions for virtual reality enabled systems. All of thoselook to prove valuable in future robotics endeavors.

I am beginning talks with a professor at ASU who specializes in Muti-Agent task allocation,interested in human-aware collaborations. We have begun regular talks on the subject, readingspecific relevant papers in the field and providing guidance on how I can contribute.

11 Special Relevant Skills

The following are special relevant skills that I either posess, am in the process of developing, or willpotentially require to pursue this topic of research.

• Robot Operating System (ROS)

7

• Control Systems (Linear, Non-Linear, Multi-Variable)

• Artificial Intelligence (Filtering/Prediction, Decision Tree Search, Sampling, Monte Carlo)

• Probability Theory

• Theroetical Computer Science (Automata, PSPACE, etc)

In addition, I have strong techincal writing and research skills, developed through the ASAPprogram and FURI. I have had many experiences to improve my presentation skills, which are vitalin any area of interest.

12 Required Resources and Budget

• High Performance ComputingLong-term work will require large computing power to execute simulation and planning al-gorithms in the research phase (before more optimal calculation methods can be developed).ASU hosts a High Performance Computing center that is available for use by students to thisend.

• Robotics HardwareRobotic hardware will required for practical implementation of the methods that are devel-oped. Quadcopters, ground vehicles, or a combination of the two could all be utilized.

• Professional OrganizationsThe Technical Committee on Multi-Robot Systems of the IEEE Robotics and AutomationSociety

13 Timeline

14 Future Plans

My future plans are to contine pursuing research in multi-robot systems. I currently plan to pursuethe thesis option of the ASU 41 Master’s Program in Computer Engineering, focusing my workon multi-robot task allocation. I intend to intertwine this work with experience in the aerospaceindustry. Following the achievment of my M.S degree, I intend to spend a few years working for arelevant aerospace company. This will provide me valuable experience, perspective, and networkingwith people in this area. Additionaly, it will assist me in my long-term financial security. Depending

8

on what opportunities present themselves, I may return to academia to pursue life-long research inthese technical areas.

15 References

(1) Rubenstein, M., Ahler, C., Hoff, N., Cabrera, A., & Nagpal, R. (2014). Kilobot: A low costrobot with scalable operations designed for collective behaviors. Robotics and AutonomousSystems, 62(7), 966-975.

(2) Rubenstein, M., Cornejo, A., & Nagpal, R. (2014). Robotics. Programmable self-assemblyin a thousand-robot swarm. Science (New York, N.Y.), 345(6198), 795-9.

(3) Wilson, S., Gameros, R., Sheely, M., Lin, M., Dover, K., Gevorkyan, R., . . . Berman, S.(2016). Pheeno, A Versatile Swarm Robotic Research and Education Platform. Robotics andAutomation Letters, IEEE, 1(2), 884-891.

(4) Schenker, P. S., Huntsberger, T. L., Pirjanian, P., Baumgartner, E., Aghazarian, H., Trebi-Ollennu, A., ... & Dubowsky, S. (2001). Robotic automation for space: planetary surfaceexploration, terrain-adaptive mobility, and multi-robot cooperative tasks. Intelligent robotsand computer vision XX: algorithms, techniques, and active vision, Boston, MA, USA, 4572,12-28.

(5) Petersen, K., Nagpal, R., & Werfel, J. (2011). TERMES: An Autonomous Robotic System forThree-Dimensional Collective Construction, Petersen, Kirstin, Radhika Nagpal, and Justin K.Werfel. 2011. TERMES: An Autonomous Robotic System for Three-Dimensional CollectiveConstruction. In Robotics: Science and Systems VII, ed. Hugh Durrant-Whyte, NicholasRoy and Pieter Abbeel. Cambridge, MA: MIT Press.

(6) Stroupe, A. W., Huntsberger, T. T., Kennedy, B. A., Aghazarian, H., Baumgartner, E.,Ganino, A., . . . Townsend, J. (2005). Heterogeneous robotic systems for assembly andservicing. European Space Agency, (Special Publication) ESA SP, (603), 625-631.

(7) Matthey, L., Berman, S., & Kumar, V. (2009). Stochastic strategies for a swarm roboticassembly system. Robotics and Automation, 2009. ICRA ’09. IEEE ]nternational Conferenceon, 1953-1958.

(8) Sugiyama, H., Tsujioka, T., & Murata, M. (2013). Real-time exploration of a multi-robotrescue system in disaster areas. Advanced Robotics, 27(17), 1313-1323.

(9) Seungho Lee, Teresa M. Adams, Boong-yeol Ryoo, A fuzzy navigation system for mobile con-struction robots, In Automation in Construction, Volume 6, Issue 2, 1997, Pages 97-107, ISSN0926-5805, https://doi.org/10.1016/S0926-5805(96)00185-9.(http://www.sciencedirect.com/science/article/pii/S0926580596001859)

(10) Xu, D., Zhang, X., Zhu, Z., Chen, C., & Yang, P. (2014). Behavior-Based Formation Controlof Swarm Robots. Mathematical Problems in Engineering, 2014, 1-13.

(11) Jones, C., Shell, D., Mataric, M. J., & Gerkey, B. (2004, September). Principled approachesto the design of multi-robot systems. In Proc. of the Workshop on Networked Robotics,IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2004).

(12) Hsieh, M. A., Halasz, A., Berman, S., & Kumar, V. (2008). Biologically inspired redistributionof a swarm of robots among multiple sites. Swarm Intelligence, 2(2), 121-141.

9

(13) Jevtic, A., Andina, P., Gazi, D., & Jamshidi, M. (2010). Building a swarm of robotic bees.2010 World Automation Congress, WAC 2010, .

(14) Werfel, Justin, Kirsten Petersen, and Radhika Nagpal. 2011. Distributed multi-robot algo-rithms for the TERMES 3D collective construction system. In Proceedings of Robotics: Sci-ence and Systems VII, Los Angeles, CA, September 25-30, 2011, eds. Hugh F. Durrant-Whyte,Nicholas Roy, and Pieter Abbeel. http://www.informatik.uni-trier.de/ ley/db/conf/rss/rss2011.html.

(15) Rosenfeld, A., Agmon, N., Maksimov, O., Azaria, A., & Kraus, S. (2015, July). IntelligentAgent Supporting Human-Multi-Robot Team Collaboration. In IJCAI (pp. 1902-1908).

(16) P. Squire, R. Parasuraman, Effects of automation and task load on task switching during hu-man supervision of multiple semi-autonomous robots in a dynamic environment, Ergonomics53 (8) (2010) 951–961.

(17) Nieto-Granda, C., Rogers, J., & Christensen, H. (2014). Coordination strategies for multi-robot exploration and mapping. The International Journal of Robotics Research, 33(4),519-533.

(18) Colledanchise, M., Marzinotto, A., Dimarogonas, D., & Ogren, P. (2015). Adaptive FaultTolerant Execution of Multi-Robot Missions using Behavior Trees.

(19) Marzinotto, A., Colledanchise, M., Smith, C., Ogren, P. (2014) Towards a Unified BehaviorTrees Framework for Robot Control. In: (pp. 5420-5427). IEEE Robotics and AutomationSociety

(20) Colledanchise, M., Almeida, D., & Ogren, P. (2016). Towards Blended Reactive Planningand Acting using Behavior Trees.

(21) SunSpiral, V., Agogino, A., & Atkinson, D. (2015). Super Ball Bot-Structures for PlanetaryLanding and Exploration, NIAC Phase 2 Final Report.

(22) Cortes, J., Martinez, S., Bullo, F., Cortes, J., Martınez, S., Ebrary, Inc, & MyiLibrary. (2009).Distributed Control of Robotic Networks A Mathematical Approach to Motion CoordinationAlgorithms (Princeton Series in Applied Mathematics). Princeton, NJ: Princeton UniversityPress.

(23) Yi, X., Zhu, A., Yang, S., & Luo, C. (2017). A Bio-Inspired Approach to Task Assignmentof Swarm Robots in 3-D Dynamic Environments. Cybernetics, IEEE Transactions on, 47(4),974-983.

(24) Lv, H., & Lu, C. (2010). An assembly sequence planning approach with a discrete parti-cle swarm optimization algorithm. The International Journal of Advanced ManufacturingTechnology, 50(5), 761-770.

(25) Maoudj, A., Bouzouia, B., Hentout, A., Kouider, A., & Toumi, R. (2017). Distributed multi-agent scheduling and control system for robotic flexible assembly cells. Journal of IntelligentManufacturing, 1-16.

(26) Clark, Christopher & M. Rock, Stephen Latombe, Jean-claude. (2003). Dynamic Networksfor Motion Planning in Multi-Robot Space Systems.

10

(27) Barros Dos Santos, S., Givigi, S., & Nascimento, C. (2015). Autonomous Construction ofMultiple Structures Using Learning Automata: Description and Experimental Validation.Systems Journal, IEEE, 9(4), 1376-1387.

(28) Barros Dos Santos, Dantas, Givigi, Buonocore, Neto, & Nascimento. (2017). A Stochas-tic Learning Approach for Construction of Brick Structures with a Ground Robot. IFACPapersOnLine, 50(1), 5654-5659.

(29) Lv, H., & Lu, C. (2010). An assembly sequence planning approach with a discrete parti-cle swarm optimization algorithm. The International Journal of Advanced ManufacturingTechnology, 50(5), 761-770.

(30) Fei Chen, Sekiyama, Cannella, & Fukuda. (2014). Optimal Subtask Allocation for Humanand Robot Collaboration Within Hybrid Assembly System. Automation Science and Engi-neering, IEEE Transactions on, 11(4), 1065-1075.

11

Systems Development for Robotic Guide Platforms

Zakk Giacometti

Computer Systems Engineering

Spring 2018

Introduction and Motivation

Robotic systems are steadily integrating into our society, working not only for humans, butalongside them. Consequently, this makes research into autonomous systems in humanenvironments important. Research into human-robot interaction is consequently of majorimportance. General robotic tasks such as perception, cognition, and action become much moredifficult when the presence and actions of humans must be considered. Navigation is one suchspecific task that must be conquered for robots to operate outside of more structuredenvironments such as factories [3]. Robotic guides (e.g. tour guides) are one application wherenavigation strategies considering humans are central to their operation.

Robotic tour guides have previously been deployed in museums, fairs, and, expos allowingresearch to focus on making such systems more intelligent and improving integration with theenvironment [4]. Research in this area will contribute to improving the intelligence of robots inless structured human environments, like a college campus, and help to pave the way for moresophisticated interactions between humans and robotic systems. These sophisticated systemscould involve multiple guide robots operating in extreme conditions such as disaster zones [20].

Additionally, the proposed work will be thoroughly documented. As such, each of the algorithmsand hardware/software used can be readily incorporated in relevant classes on robotics, planning,systems, programming, sensing and control. These include introduction to engineering as well assenior design. Moreover, the developed vehicle(s) can be used for outreach purposes (e.g.visiting community colleges, high schools, middle and elementary schools, visits to ASU).

Literature Survey

Visual navigation has been studied as a means to combat position estimation caused by deadreckoning error by introducing cameras to obtain fixed points of reference. A variety oftechniques have been explored to accomplish this. Some utilize static markers that designatefixed points of reference [1], [5]. Others utilize mobile markers [2].

Human-aware navigation is another point of interest in our study. In addition to accounting forstatic obstacles, guide robots must account for dynamic obstacles, such as crowds of students thatwould frequently appear on a college campus. They must also work in cooperation with humansin the environment. In most cases, they must make intelligent decisions about those humans (e.gto wait for, avoid, or alert them of the robot’s presence) [3].

Toward this point, obstacle avoidance is also of great concern.

General Description of Approach

1

We seek to develop a campus guide robot platform that can operate in an unstructured, uncertainenvironment that works in cooperation with a human being guided by it. Our aim is to utilizevisual navigation techniques, minimize hardware costs and requirements, and develop portablesoftware.

Our robot seeks to utilize visual markers placed in the environment for two purposes: local pathplanning between landmarks (buildings), and correction of dead-reckoning errors in positionestimation.

Contributions of Work

This work seeks to define and explore the answers to the following critical questiions:

1. What course of actions are best taken by a robot in a guide role when needing to, forexample, follow a path to a destination while avoiding obstacles?

2. How can we best manage the computationally heavy tasks described given limitedcomputing power?

In that process, it also seeks to address the following specific technical questions:

1. How do we quickly detect markers in the image plane (i.e. way point indicators on theground or on a building)?

2. How do different markers change the results (e.g. big dot, collection of dots, etc.)?

3. How do we accurately and swiftly move from marker to marker while maintaining theguided person at a pre-specified range (e.g. 5-8 feet) and avoiding obstacles (e.g. personcrossing path or standing directly in path to the next way point)?

4. How can the robot achieve the objective of getting from one building to another with aminimal set of pre-set markers? no pre-set markers? The latter requires forming andstoring an internal map and then using it to set waypoints and then move from accuratelyand swiftly waypoint to waypoint while avoiding obstacles along the path. This requiresfull use of GPS, cameras, lidar, and ultrasonic sensors.

5. When will remote intervention be necessary (e.g. kill switch, tentative human remotecontrol)?

6. For each of these, how do we systematically tradeoff computational speed and accuracy?Eventually, we can consider exploiting distributed computational resources (e.g. desktopssituated in buildings along the path) to augment the very limited on-board computationalresources.

To these ends, we have demonstrated one such set of actions that a guide robot can take infollowing a path. We have also tested some of the computational load required, and limitsencountered, when executing this particular method. This provides another data point and set ofsoftware from which to build on when designing more advanced systems.

2

Detailed Description of Approach

Simplified System Overview

Major Hardware

Initial tests were conducted using an Arduino Mega 2650 and Raspberry Pi 3 single boardcomputer configured with Ubuntu 16.04 and ROS Kinetic. The aims of this setup were to obtaina baseline of performance for low cost, easily available hardware while simultaneouslydeveloping software on the ROS platform that could be easily be made to run on futureconfigurations.

Moving forward, a robot platform similar to the TurtleBot 2 family of robots will be utilized. Onit will be the following major components:

• NVIDIA Jetson TX2 Module (New Target)

• Arduino Mega 2560 (inner loop control)

• RPLidar 2

• RGB-D Camera

• Dual Magnetic Motor (Wheel) Encoders

3

Figure 1: Simplified Block Diagram of Overall System

Software

• Robot Operating System (ROS) (C++ and Python)

• Gazebo

• OpenCV

• AprilTags (Fiduciary Marker System)

Algorithms

Navigation Algorithm

A simple, but effective outer-loop navigation algorithm was used for movement from waypointto waypoint.

4

Figure 2: In-Progress "TurtleBot" Platform

To simplify the algorithm, it was assumed that only one guided person would be tracked and thepath to be followed would be kept relatively straight. The use of visual markers as waypointsallowed even further simplification of the system, but would require these physical markers to bepre-placed along the path where the robot would be operating.

This algorithm and the code supporting it make no assumptions about the underlying hardware.It only requires a robot capable of maintaining angular and linear velocity and a source of markerdetection data. In other words, it is nearly completely decoupled from the underlying platform,allowing complexity to be added without much consideration for lower-level aspects of thesystem.

Results

Observed ROS Publish Rates (Raspberry Pi, Arduino Mega 2650, BNO055 9-Axis IMU)

Readings from the ROS topic stream were taken (using rostopic hz) as a metric of processingspeed of the system. Those readings are as follows:

imu_data: 5hz with on-board Odometry estimation, 17Hz without

odom: 5hz

tag_detections: 0.38hz

camera: 10hz (fixed framerate of 10fps to reduce load)

As can be seen, only the raw camera input data stream performed as desired. The rates observedfrom Arduino message publishing suggest that a better approach would be to provide only rawsensor values via Arduino, allowing any processing to occur on a more powerful device.

The AprilTag detections performed at an abysmal 0.38Hz. This equates to 1 tag being processedroughly every 3 seconds. For very slow-speed movement or navigation this may be acceptable,but operation at human speeds (particularly human walking speed) requires drastically higher tagdetection rates.

5

Figure 3: Simplified Local Navigation Algorithm

Current State of Software

ROS Nodes

Gazebo Simulation

The Gazebo robot simulation platform is used in conjunction with ROS for closed-loop softwaresimulation of the robot. A model of the robot is defined in a xacro-based URDF (Unified RobotDescription Format) file and loaded, through ROS, into Gazebo. This allows us to simulatesensor inputs (e.g cameras) and send control commands to the robot (using a simple differentialdrive controller) in the Gazebo environment.

Camera sensor inputs are generated by the libgazebo_ros_camera.so plugin and a genericdifferential drive controller is provided by the libgazebo_ros_diff_drive.so plugin.

6

Figure 4: Graph of ROS nodes showing the Raspberry Pi Camera, and Apriltags detectors

ROSSerial

Testing was conducted with the ROS Serial library to facilitate communication between theArduino handling inner-loop control and the outer-loop ROS system. This enables us to decouplethe high-level decision software from the lower-level controls. This in turn allows easierreplacement of the underlying robot platform, including its methods of locomotion if desired.

The Arduino Mega was configured to publish both Imu (IMU) and primitive integratedOdometry messages.

Summary and Directions for Future Work

Summary

A foundation for future research into robotic guides has been developed. The collection ofcomponents (ROS, Gazebo, hardware) and simple navigation algorithm combining themprovides a starting point for future work. In addition, the brief characterization of hardware forthe system provides a baseline for what guide robot platforms utilizing visual navigation requirein terms of computational complexity and sensor quality.

Future Work

Improved Hardware

7

Figure 5: Gazebo simulator and Rviz showing real time robot pose and camera data

Raspberry Pi Camera -> RGB-D Camera: The Raspberry Pi Camera module initially select foruse has a limited field of view, image resolution, and bandwidth. Stable framerates are possible,but the limited optical qualities of the camera make it unsuitable. To improve the visual sensingcapabilities of the robot, an RGB-D camera with a wider field of view, higher frame rate, andbetter resolution will be used.

Raspberry Pi -> NVIDIA TX2: The Raspberry Pi 3 was initially selected for its low cost,ubiquity, and ample support. However, its limited computing performance, especially lack ofhigh-speed image processing capability, proved unsuitable for our purpose.

While it is able to handle the specified capabilities, it cannot accomplish them in human time-frames. To ease development of these more advanced capabilities, an NVIDIA Jetson TX2 boardwill be targeted as the main outer-loop control platform.

Algorithm Design

Additional future work can also focus on introducing optimizations from research in computervision, reducing the computational costs (both processor demands and monetary). This may takethe form of other marker systems (e.g. AprilTag [18], ChromaTag [19]).

Detection of both stationary and moving obstacles can be implemented into the system. This is acrucial feature for a guide robot operating in any environment. Ideas from [6], [10], and [17]will be explored. Thesee algorithms must be incorporated into the local path planning algorithm,allowing a temporary deviation from the path laid by the markers to avoid detected obstacles.

Global path planning between high-level landmarks (such as buildings) can be developed on topof the local planning algorithm described.

Markers can be replaced with more advanced vision techniques such as those described in [8].This is especially helpful when needing to track guided persons as it relieves their requirement towear a special marking tag.

8

References[1] Lamberti, F., Sanna, A., Paravati, G., Montuschi, P., Gatteschi, V., & Demartini, C.

(2013). Mixed Marker-Based/Marker-Less Visual Odometry System for Mobile Robots.International Journal of Advanced Robotic Systems, 10(5), International Journal ofAdvanced Robotic Systems, 2013, Vol.10(5).

[2] Acuna, R., Li, Z., & Willert, V. (2017). MOMA: Visual Mobile Marker Odometry.

[3] Kruse, Pandey, Alami, & Kirsch. (2013). Human-aware robot navigation: A survey.Robotics and Autonomous Systems, 61(12), 1726-1743.

[4] López, Joaquín. (2013). GuideBot. A Tour Guide System Based on Mobile Robots.International Journal of Advanced Robotic Systems, 10(11), International Journal ofAdvanced Robotic Systems, 2013, Vol.10 (11).

[5] Gomez, C., Hernandez, A., Crespo, J., & Barber, R. (2016). A topological navigationsystem for indoor environments based on perception events. International Journal ofAdvanced Robotic Systems, 14(1), International Journal of Advanced Robotic Systems,2016, Vol.14(1).

[6] Cherubini, A., Spindler, F., & Chaumette, F. (2014). Autonomous Visual Navigation andLaser-Based Moving Obstacle Avoidance. Intelligent Transportation Systems, IEEETransactions on, 15(5), 2101-2110.

[7] Kanda, Arai, Suzuki, Kobayashi, & Kuno. (2014). Recognizing groups of visitors for arobot museum guide tour. Human System Interactions (HSI), 2014 7th InternationalConference on, 123-128.

[8] Gupta, M., Kumar, S., Behera, L., & Subramanian, V. (2017). A Novel Vision-BasedTracking Algorithm for a Human-Following Mobile Robot. Systems, Man, andCybernetics: Systems, IEEE Transactions on, 47(7), 1415-1427.

[9] Olson, E. (2011). AprilTag: A robust and flexible visual fiducial system. Robotics andAutomation (ICRA), 2011 IEEE International Conference on, 3400-3407.

[10] A. A. Rodriguez, K. Puttannaiah, et. al. (2017). “Modeling, Design and Control of Low-Cost Differential-Drive Robotic Ground Vehicles: Part I - Multiple Vehicle Study,” IEEE,Conference on Control Technology and Applications.

[11] A. A. Rodriguez, K. Puttannaiah, Z. Lin et. al. (2017). “Modeling, Design and Control ofLow-Cost Differential-Drive Robotic Ground Vehicles: Part I - Multiple Vehicle Study,”IEEE, Conference on Control Technology and Applications.

[12] Z. Li. (2013). “Modeling and Control of a Longitudinal Platoon of Ground RoboticVehicles," Master Thesis, ECEE, ASU. Committee: A. A. Rodriguez, A. Panagiotis, S.Berman.

[13] Z. Lin. (2015). “Modeling, Design and Control of Multiple Low Cost Robotic Vehicles,”Master Thesis, ECEE, ASU. Committee: A. A. Rodriguez, J. Si, S. Berman.

9

[14] X. Lu. (2016). “Modeling and Control for Vision based Rear Wheel Drive Robot andSolving Indoor SLAM problem using LIDAR,” Master Thesis, ECEE, ASU. Committee:A. A. Rodriguez, A. Panagiotis, S. Berman.

[15] J. Lopez. (2016). “Image Processing Based control of Mobile Robots,” Master Thesis,ECEE, ASU. Committee: A. A. Rodriguez, A. Panagiotis, S. Berman.

[16] I. Anvari. (2013). “Non-Holonomic Differential Drive Mobile Robot Control and Design:Critical Dynamics and Coupling Constraints,” Master Thesis, ECEE, ASU. Committee:A. A. Rodriguez, K Tsakalis, J Si.

[17] D. Chopra. (2013). “Feedback Control and Obstacle Avoidance for Non- HolonomicDifferential Drive Robots,” Master Thesis, ECEE, ASU. Committee: A. A. Rodriguez, KTsakalis, J Si

[18] Wang, J., & Olson, E. (2016). AprilTag 2: Efficient and robust fiducial detection. IEEEInternational Conference on Intelligent Robots and Systems, 2016, 4193-4198.

[19] DeGol, J., Bretl, T., & Hoiem, D. (2017). ChromaTag: A Colored Marker and FastDetection Algorithm.

[20] Sugiyama, H., Tsujioka, T., & Murata, M. (2013). Real-time exploration of a multi-robotrescue system in disaster areas. Advanced Robotics, 27(17), 1313-1323.

10

Letter of Recommendation (Computer Engineering (Systems) Graduate Program, Arizona State

University) (DRAFT)Zakk Giacometti - Computer (Systems) Engineering Undergraduate

Relevant BackgroundZakk and I first met through his involvement with my ASAP METS project-

centric scholarship program and related FSE394 class. He has been an excellentscholar in the program, following many of its directives and approaches to makeimpacts on his success.

Zakk has shown incredible enthusiasm, passion, and initiative in ourconversations on the subject of robotics. His willingness to be involved with FURIdespite being a first semester transfer student shows a great deal of dedication.Additionally, his prior leadership experience at the Mesa Community CollegeEngineering Club reflects an ability to step up as a leader. As a scholar of the ASAPMETS program, Zakk has taken advantage of the opportunity to continuedeveloping those professional and leadership skills. His initiative, effort, and ideaswhen working through the Fall FURI only reinforces that.

Relevant PreparationZakk’s academic record shows strong time management skills and an excellent

ability to learn and apply new information quickly. Achieving an maintaining a 4.0GPA through community college, with engineering specific classes, is difficult andrequires hard work and dedication. Carrying that 4.0 GPA through his first andsecond semesters at ASU proves that he had taken the steps to prepare to continuethat achievement. He has also shown strong technical writing ability, a key still ingraduate studies. This was demonstrated in both his ten page technical interestpaper for FSE394 and his two funded FURI proposals.

His technical abilities are more than adequate to prepare him for the work to beconducted. His prior internship experience at NASA Marshall Space Flight Centerhas developed both his hard engineering design and soft project skills. From thisinternship, he gained experience with the Robot Operating System (ROS), andgained a basic working knowledge of control systems. This continued onto FURI,working again with C++ and ROS at a more algorithmic level. Combined with hiscoursework, he has developed very strong programming, algorithm design, andsystem integration skills. He has receptive in both our technical and non-technicalconversations, showing a strong interest in being a lifelong learner and expandinghis knowledge in developing intelligent systems.

Zakk's funded FURI proposal to continue work on his intelligent campus guiderobot for the upcoming fall semester will strongly direct his future researchinterests. During the summer and fall, he will be developing algorithms for globaland local path planning and obstacle avoidance. The knowledge and skills gainedwill provide a strong foundation for planned graduate-level work with controlsystems and multi-robot coordination and control.

This past summer, Zakk was accepted to a highly competitive internship at BallAerospace. The internship allowed him to further apply and expand his knowledgeof ROS in an aerospace and defense context, preparing him for future work in

intelligent systems. There, he had the opportunity to learn from experiencedsoftware and systems engineers, giving him powerful insight into the engineeringprocess.

Additional Planned PreparationZakk is currently beginning the 4+1 MS program at ASU in Computer

Engineering, with a thesis option. This again shows a strong desire to learn andwill help to prepare him for advanced work in his areas of interest. His focus willbe on multi-agent systems, taking classes in control theory, artificial intelligence,and robotics.

Zakk has immense potential to succeed in <name of program> as evidenced byhis personal, professional, and academic achievements.

Zakk GiacomettiArizona State University - Computer Systems Engineering

Statement of PurposeRobotics has captured my interest for many years and I have always found that I orient my goals

toward this area. My plans have been to focus on systems of cooperating robots. I feel this area is one thatoffers the greatest potential and range of applications. Advancements in multi-robot systems are a majorpart of the hastening introduction of intelligent systems in society. This may take the form of groundrobots, space and satellite systems, or planetary exploration vehicles. I seek to work at the cutting edge ofthe field in challenging, unstructured, environments like the scenes of disasters and the vacuum of space.The breadth of faculty and resources available at Arizona State University in robotics and space sciencemake this endeavor especially possible. My ultimate goals are to make significant contributions in theseareas, advancing the knowledge of intelligent systems.

I have been deeply influenced by the opportunities afforded to me by Mesa Community College(MCC). I was able to engage with motivated peers, and seek close guidance from my professors. Throughthe MCC Engineering Club I was able to explore the different disciplines of engineering outside theclassroom environment, engaging with those in industry to find where my interests aligned. I also had theopportunity to be selected for the NASA Community College Aerospace Scholars program (Fall 2017).This was a summer long learning experience ending with a four day team-based project experience withNASA engineers and fellow participants from around the country. I was inspired by those from NASAwho came to speak and work with us. I was equally inspired by those community college students whowere there alongside me.

An internship at Marshall Space Flight Center (MSFC) (May-August 2017) in the Control SystemsDesign and Analysis branch deeply immersed me in an engineering environment. I worked extensivelywith a robotics software framework, the Robot Operating System (ROS), contributing to a platform thatwould be used by engineers in prototyping controls systems for complex CubeSat maneuvers. While there,I had the opportunity to interact with not only MSFC, but tour other NASA facilities in the region, seeingthe sheer scale of infrastructure and expertise required to support space science. I was introduced to anarea of engineering previously unknown to me (control systems) and experienced a completely newperspective on what my small project was part of. Working closely with true experts in their fieldreinforced my motivation to continue learning and pushing my limits. During the Summer of 2018, I willfurther enhance my understanding of robotics in space applications during an internship at BallAerospace.

In pursuit of my goals, I have submitted and been awarded a Fulton Undergraduate Research Initiative(FURI) project for both the previous spring and upcoming fall 2018 semesters. This FURI seeks toresearch control and software algorithms for the development of an intelligent campus guide robot. As partof this, I am expanding my knowledge of ROS and working with a professor and graduate studentsspecializing in control systems. This experience will continue to enhance my understanding of controlsystems and robotics, connecting with my coursework in embedded systems and artificial intelligence. Italso serves as a foundation for future research in the area, expanding to more complex scenarios andexpanding to fleets of robots. The ideas and understanding I develop here will directly translate to andreinforce my future work.

My ultimate direction is clear. I wish to use the skills and knowledge I gain to contribute to society,academically and practically, growing professionally in the process. My experiences interacting with peersand professionals in and outside of school have shaped my motivations. The actions I have taken andfuture plans I have in place prepare me to gain those skills and knowledge that are vital to advancedresearch in robotics. This includes my internships in the aerospace industry and research and studies incomputer systems. This is an exciting time for my field of study, and I am well poised and taking the stepsto be a part of it.