AppceleratorTEI_final_July 30

A Forrester Total Economic

Impact™ Study

Commissioned By

Appcelerator

Project Director:

Liz Witherspoon

July 2014

The Total Economic

Impact™ of The

Appcelerator Platform

Table Of Contents

Executive Summary .................................................................................... 3

Homes.com Case Study: Appcelerator Platform for External App

Delivery ......................................................................................................... 4

Analysis ........................................................................................................ 7

Virgin Money Case Study: Appcelerator Platform for Internal App

Delivery ....................................................................................................... 15

Financial Summary ................................................................................... 19

The Appcelerator Platform: Overview .................................................... 20

TEI Framework And Methodology .......................................................... 21

Disclosures ................................................................................................ 22

Appendix A: Total Economic Impact™ Overview ................................. 23

Appendix B: Glossary ............................................................................... 24

Appendix C: Supplemental Material ....................................................... 25

ABOUT FORRESTER CONSULTING

Forrester Consulting provides independent and objective research-based

consulting to help leaders succeed in their organizations. Ranging in scope from a

short strategy session to custom projects, Forrester’s Consulting services connect

you directly with research analysts who apply expert insight to your specific

business challenges. For more information, visit forrester.com/consulting.

© 2014, Forrester Research, Inc. All rights reserved. Unauthorized reproduction is strictly prohibited.

Information is based on best available resources. Opinions reflect judgment at the time and are subject to

change. Forrester®, Technographics

®, Forrester Wave, RoleView, TechRadar, and Total Economic Impact

are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective

companies. For additional information, go to www.forrester.com.

3

Executive Summary

Appcelerator commissioned Forrester Consulting to conduct a Total Economic Impact™ (TEI) study and examine the

potential return on investment (ROI) enterprises may realize by deploying the Appcelerator Platform for mobile app

development, management and analytics. The purpose of this study is to provide readers with a framework to evaluate the

potential financial impact of this platform on their organizations.

To better understand the benefits, costs, and risks associated with using the Appcelerator Platform, Forrester interviewed

two existing customers, Homes.com and Virgin Money. Homes.com has used the Platform for two years, primarily to deliver

external customer-facing apps to increase leads to the business. Virgin Money, a more recent customer, is developing

internal apps with the aim of improving business processes and increasing productivity. The financial analysis featured in this

study reflects the cost savings and productivity gains experienced by Homes.com. However, both use cases have the

objective of improving time-to-market for apps while decreasing development, testing, integration, and delivery costs.

Furthermore, Forrester Research predicts that in the next few years, most organizations will be required to develop apps for

external and internal customers to meet business demand. To compete, they must embrace the “Mobile Mindshift” - the

expectation that your customer (whether consumer, partner or employee) can get what they want in their immediate context

and moments of need. However, not all organizations have the strategy and resources in place to meet the demand for

quality apps.

Prior to investing in Appcelerator, Homes.com and Virgin Money developed native apps and used agency partners for app

delivery. This experience led both companies to explore cross-platform mobile development and delivery solutions. Upon

evaluating the competition, a few important differences stood out that led to their Appcelerator investment:

• A comprehensive, easy-to-use platform for app delivery. Both Homes.com and Virgin Money cited the ease of

use of the platform and its cross-platform capabilities, analytics, and APIs as key selling points. They were able to

develop and test new fully-native cross platform apps in weeks instead of months using lower-cost labor.

Furthermore, analytics functionality gave developers the means to pinpoint errors before they impacted user ratings

and Appcelerator APIs made it easier to integrate apps with existing systems, further reducing time-to-market and

costs.

• Support from the Appcelerator team. The Appcelerator team trained both Homes.com and Virgin Money

developers to use the Platform. This partnership was valued by both companies. The Appcelerator team went above

and beyond the agreement by attending Homes.com’s weekly developer team meetings and responded quickly to

requests.

• Scalable platform for future growth. Both organizations expect to meet increasing demand from customers and

business units to develop and update apps. Developing more apps means an exponential increase in updates and

maintenance. Being able to reuse code and update apps quickly and cost-effectively is critical to managing this

growth. Investing in Appcelerator now positioned Homes.com and Virgin Money well to contain those future costs.

By using the Appcelerator Platform, Homes.com was able to:

• Decrease the time-to-market for new mobile apps by 40%,thereby increasing leads and revenue.

• Lower the cost of development by over 50% using JavaScriptskillsets rather than expensive iOS and Android developers.

• Decrease the time-to-resolution for app problems by 75%.

“It allows us to be more cost-efficient and do more with less, with

the best impact possible. It’s the cost of doing business to stay

competitive.”

– Dan Gaertner, VP of Technology, Homes.com

By implementing the Appcelerator Platform, Virgin Money

was able to:

• Speed the release of prototypes, new internal apps and updatesthrough an iterative two-week development cycle.

• Lower infrastructure spend and eliminate the lead time forgetting projects initiated.

• Reduce spend to an external agency for app development.

“The idea is to drive efficiency in the business and the ultimate aim

is for the mobile group to become self-funding, by delivering enough

efficiency savings in the business to cover the cost of development.”

– Chris Edwards, IT Mgr., Digital Development Virgin Money

4

Homes.com Case Study: Appcelerator Platform for External App Delivery

Homes.com, a division of Dominion Enterprises, has provided advertising and marketing services to help real estate

professionals connect with homeowners, buyers, and sellers since 1973. Homes.com powers many of the apps that are

used by consumers to locate and view home purchases and rentals, including Remax, ERA and others. Homes.com sought

out a solution to maintain its competitiveness in an increasingly mobile world and put apps in the hands of prospective home

owners and renters who are, by nature, on the move while hunting.

Prior to licensing the Appcelerator Platform, Homes.com was developing native mobile apps for two operating systems:

Apple’s iOS and Google’s Android. These platforms required high-priced software development talent who were difficult to

find and retain due to market demand for mobile skills. Furthermore, each mobile app codebase had to be written

independently for both platforms, doubling the time and expense to develop and test the code, and update it once the app

launched. More challenging still was the lack of visibility into the apps’ performance, including crashes and exceptions, which

made Homes.com and its sister companies vulnerable to the public spotlight with app store user ratings and reviews. This

exposure and unnecessary expense, coupled with the scarcity and high turnover rate of mobile developers led Dan

Gaertner, VP of Technology, to take a different approach.

With the Appcelerator Platform, Homes.com has been able to create high-quality fully native mobile apps at a significantly

lower cost and with more speed and ease. This has enabled the business to maintain its competitiveness, appear higher in

user ratings and reviews, generate more leads, reduce costs, and retain its developers. Said Gaertner, “It allows us to be

more cost-efficient and do more with less, with the best impact possible. It allows us to be in the game and it’s the cost of

doing business to stay competitive.”

Our interview with Homes.com and subsequent financial analysis found that they experienced the risk-adjusted ROI1,

benefits, and costs shown in Figure 1.

FIGURE 1

Financial Summary Showing Three-Year Risk-Adjusted Results

ROI: 491%

Benefits PV: $1,336,047

Costs PV: $226,221

NPV: $1,109,826

Time-To-Market:

40%

Source: Forrester Research, Inc.

5

Benefits. Homes.com experienced the following risk-adjusted benefits:

• Reduced time to develop new mobile apps from 20 weeks to 12 weeks while decreasing costs by 52% -

saving $420,032. The average time to develop new fully native mobile apps decreased from 20 weeks to 12 weeks,

increasing the velocity of the development while decreasing the costs. The two primary contributors to this cost

savings include lower-cost developers (33% less annual salary) and only needing to develop a single codebase in

one language (JavaScript) rather than two (in Objective C for Apple’s iOS and in Java for Google’s Android). Note:

Native developers in most large cities would command higher salaries than in Norfolk, thus increasing the cost

savings for app delivery when applying this model to organizations in urban areas.

• Reduced time to update existing mobile apps by 25% while decreasing costs by 40% - saving $753,647.

Investment in the Appcelerator Platform enabled Homes.com to update existing mobile apps 25% faster than native

app development with a development cost savings of 40% per app. Most significantly, the benefits realized from

updates to existing apps compounds over time as more net new apps are developed.

• Reduced time-to-resolution for app problems by 75%, improving user reviews – saving $162,369. The

Platform’s performance management capabilities enabled Homes.com to pinpoint and resolve app crashes faster,

which is linked to improved app use and user ratings. The company saw an increase in positive user reviews,

leading its apps to appear in the top 15 of the Lifestyle category of Google Play, a digital distribution platform for

apps on Android and an online electronics and digital media store developed and maintained by Google.

• Reduced testing time for new app development and updates to existing apps by 40%. Homes.com was able

to reduce its unit, integration and functional testing time in parallel with its development savings. This resulted from

being able to deliver for two platforms using a single codebase which is tested with automation. Although automated

testers command a higher hourly salary, the reduction in testing time makes up for that increase while still yielding

cost savings. This benefit is accounted for in the first two benefits highlighted above.

• Increased reuse of code leading to further reduction in development time from 12 weeks to 8 weeks. The

Appcelerator Platform made is easier to reuse code across different mobile platforms. Dan estimates that apps built

with similar functionality and business rules could shorten time-to-develop by another four weeks, from 12 to 8

weeks. This yields an additional $20,000 in savings per new app developed. Note: this was not included in the total

benefits calculated, but could be incorporated in the model based on an individual company’s development strategy.

• Increased monthly leads to the business through high-quality, high-performing apps. Since using the

Appcelerator Platform to develop and update mobile apps, Homes.com has seen an increase in leads generated for

the company. Although not accounted for in the financial model, it is a significant benefit to the organization and

could be tracked to determine an individual app’s contribution to revenue. To reach customers in a highly-

competitive app arena, an app must achieve high ratings. Consumers and employees alike expect mobile apps to

serve as an equal (or better) channel for doing business. Dan Gaertner has been able to benefit from this mobile

mind shift and has seen that impact lead growth: “Now that we have multiple apps, we can reach a large audience,

and that draws more leads – that trend has been obvious.”

• Decreased employee turnover by an estimated 20%. Although not captured in the financial model, the ability to

hire, train, and retain JavaScript developers was a significant benefit realized by Homes.com that led to additional

impact. Said Dan, “The learning curve to get someone up-to-speed used to set me back. Now, we have a low

turnover team and we’ve been 2-3 times faster to market as a result.” Forrester Research’s mobile analysts estimate

a 20% reduction in turnover when a company eliminates the need for highly-sought iOS and Android developer

talent. Not only do companies compete with high-tech companies in urban tech meccas, they also compete with

agencies serving clients who need to accelerate mobile development to maintain competitiveness.

6

› Costs. Homes.com experienced the following risk-adjusted costs:

• Software licensing fees of $80,000. The Appcelerator Platform requires an annual licensing agreement based on

the number of user licenses. (Platform users are generally app developers, testers, managers and the like.) This fee

recurs annually and increases as more users adopt the platform. Note: An open source version of the platform,

Titanium, is also available.

• Training and implementation costs of $30,000. To accelerate the internal enablement, Homes.com paid $20,000

for an on-site developer training for about 25 developers across Dominion Enterprises. This was an initial, one-time

fee paid to Appcelerator. In addition, there was a ramp-up time when the Homes.com team tested the build of an

app in the platform while continuing to build in its former model, resulting in extra expense in the first few months of

implementation (estimated at $10,000).

• API calls. The Appcelerator license includes a very high threshold of API calls per month (up to 10 million). The

available calls generally serve the needs of an organization with significant room to grow. If the APIs are used

extensively, an organization will license additional capacity.

7

Analysis

CASE STUDY ORGANIZATION

To develop the financial model for this study, Forrester conducted

an in-depth interview with Dan Gaertner, VP of Technology from

Homes.com at Dominion Enterprises. Homes.com is a privately-

held US-based company and has about 3,000 employees across

the country with offices in San Diego, CA, Boca Raton, FL and

Norfolk, VA. Homes.com is the technology provider and partner of

many familiar brands in the home purchase and rental space,

including Remax, ERA and others. Aware of the market imperative

to produce high-quality mobile apps quickly to meet consumer

demand, Dan saw an opportunity to achieve the desired business

results while saving time and money. Previously, he led a team of

native iOS and Android developers, but was confronted with

challenges many in his position face: native developers are in

demand (read: high costs and high turnover). Compounding this

was the fact that his organization is headquartered in Norfolk,

Virginia, where it can be difficult to attract top technical talent and

recent college graduates. Meanwhile, the demand to create mobile

apps was hitting a breaking point for the business. Homes.com saw

an opportunity to meet the business imperative of developing native

mobile apps quickly and cost-effectively by leveraging a more

available skillset of JavaScript developers who could be retained

more easily.

INTERVIEW HIGHLIGHTS

Homes.com sought out a mobile platform that relied on more

available skillsets to solve its business challenges.

› Homes.com needed a unified mobile platform that could help the

company maintain its competitiveness. With Web-based upstarts

such as Trulia and Redfin in the mix, the organization was

competing against companies with four times the development

budget. Said Dan, “It allows us to be more cost-efficient and do

more with less, with best impact possible. It allows us to be in the

game and it’s the cost of doing business to stay competitive.”

› The biggest driver for Homes.com to seek a solution was that its

apps’ consumer reviews in public stores such as Google Play and

the Apple Store were reporting quality issues and concerns with

crashes. Because the number of downloads and ratings drive

rankings inside those app stores, the team needed to solve its crash-use issues. Furthermore, a customer frustrated by a

poor user experience will abandon the app. This ultimately impacts lead generation and revenue.

› Homes.com was looking for a platform that could leverage more widely-available skillsets in the Norfolk, VA region. Said

Dan, “It’s a lot easier to develop in a JavaScript-centric platform like Appcelerator versus going out and finding someone

“We had so many different priorities in

mobile for ERA, Remax, and

Homes.com that it was becoming a

challenge to develop in both Android

and iOS natively, so we needed a

platform that gave us more flexibility

to get more done. We ultimately

landed on Appcelerator and it gave us

flexibility and allowed us to do more

with less.”

~Dan Gaertner, VP of Technology, Homes Media

“People don’t download your app if you

have 2 star ratings. Since

implementing the Appcelerator

Platform, we’ve hopped up in the

ratings and have been a featured app

in the top 15 of the Lifestyle category.”

~ Dan Gaertner, VP of Technology, Homes

Media

8

who is just dedicated to being an iOS or Android developer; those people are hard to find and expensive. It’s definitely

more difficult to pick up those types of languages.”

› Homes.com wanted to improve the quality of their mobile apps to encourage high adoption and better user reviews. Said

Dan, “Appcelerator’s testing and reporting capabilities have really given us a lot more insight – we can get crash reports

down to the line of code; we know exactly when and where things are crashing; before it was all a guessing game. It

definitely put a focus on quality – you can see it in our reviews and ratings.”

SOLUTION

After evaluating multiple vendors, Homes.com selected Appcelerator for its alignment with the widely available skillset in his

region and the tech company’s support for getting his team started with the Platform. Furthermore, the Appcelerator

Platform’s ability to transform JavaScript into fully native iOS and Android apps supported Homes.com’s aim of being able to

create native apps without the cost and complexity of traditional iOS and Java development in multiple codebases. They

then began deployment:

› Implementation started in 2012. Although not required, his developers took part in an on-site training offered by

Appcelerator that included other Dominion Enterprises developers. Dan commented that the Appcelerator team was like

an extension of his own, “They attended weekly development meetings to answer any challenges we had. They were open

to getting on the phone with us when we hit a roadblock.”

› Homes.com took small steps at first. They built one app while still maintaining the original native app to see and compare

the differences. The first phase focused on the baseline apps for home purchase that already had a mobile app,

Homes.com.

› Once Homes.com saw the ease and speed of development using Appcelerator, they were off and running. In fact, the

development team took on the challenge of building a major app right before a high-profile Remax conference and it took

them only three months from start to finish. The Appcelerator Platform now powers six apps that have a substantial

number of users and sessions per month/year. His team has developed five additional apps and has been able to update

them quickly and with higher frequency.

The interview revealed that:

› The velocity of getting new mobile apps to market has made

a big business impact. The most significant benefit

experienced was the ability to build a new fully-native mobile app

in one language and deploy it across multiple platforms quickly

and cost-effectively. Not only does this velocity maintain the

company’s competitiveness, it also put their home and rental

market apps in the hands of prospective home owners and

renters who are, by nature, on the move while hunting.

› The ease of creating and improving apps has improved their

user ratings, which generates more leads. Dan cites the ease

of use of the Appcelerator Platform as enabling his developers to

develop more quickly, but also more carefully and creatively. The

results can be seen in the apps’ user ratings and reviews.

Although they vary over time (a real estate app has higher usage

and rankings on the weekends but sees lower standing on

weekdays when users return to their desktops) Dan says that

“with multiple apps we can reach a larger audience and draw

more leads.” He’s seen a positive trend in lead growth over time

“If you are reading a book and

I told you to go to page 200

and it didn’t have any page

numbers, it would take you a

while to find it. You’re just

flying blind. Now we’re able to

pinpoint exactly where in the

line of the code that error is

occurring so you can go and

fix it.”

~Dan Gaertner, VP of Technology, Homes Media

9

since deploying Appcelerator. Positive user reviews in the Apple Store abound, such as “Perfectly does what a real estate

app should do. If you are looking for a highly customizable search app you need not look further!” Another app reviewer

commented, “So easy to use that it makes it easy to look for houses that are inside our price range and close by. Very

much recommended!”

› The ability to detect and fix errors before they reach user review “critical mass” has saved them face (and

money). Because mobile apps are created more quickly to meet changing consumer demand, they require multiple

iterations to get them right. Generally speaking, companies favor speed-to-market over perfection out of the gate, using

subsequent iterations to refine the app. So being able to pinpoint errors and potential crashes as they occur and before

they impact user reviews and hinder business is critical. Homes.com cites the Appcelerator Platform’s performance

management capabilities as a key element that has reduced the mean time to resolution by an estimated 75%.

“You’d see reviews saying ‘I’m getting crashes every time I open the app.’ Now

we know about it before they put that in a review – it has made it easier for us

to get out in front of it.”

~ Dan Gaertner, VP of Technology, Homes Media

› The speed to update existing apps is equally (and potentially more) valuable than the ability to develop new apps

quickly. Mobile apps are, by nature, reflective of the users who adopt them and will evolve as those users’ needs change.

Because Homes.com can make updates to existing apps more quickly and cost-effectively, the effect compounds over

time – the more apps added to the arsenal (by some predictions this will be in the hundreds for many organizations) – the

more the productivity of the updates matters. When looking at Homes.com’s three-year time horizon, the cost savings from

updating existing apps can outstrip the benefits gained from initial development.

› Their testing productivity has improved by 40% alongside the development productivity. Testing is a critical and

often lengthy part of creating mobile apps. The unit, integration, and functional testing performed only needs to be done

once for Homes.com’s two chosen operation systems, shaving many weeks off of the delivery timeline. Said Dan, “We’re

able to do automated testing with the Appcelerator testing software on a new build…before it was an all manual effort.

We’d have people running through on different devices and simulators, playing around with it as a consumer would. Now

it’s automated testing across all these different permutations. It improves the quality and makes sure you don’t miss

anything.”

› The potential for reusing code for future apps could further yield more long-term benefits. Said Dan,

“Some of the sites we’ve developed – Remax, Homes Rental, etc. – are powered off of the same codebase. It can be as

simple as the fact that every app has a calculator to calculate a monthly payment of a house or rental. This is all reusable

code.” The same is true for search functionality and any other feature that can be easily reused once its components have

been written and tested in JavaScript. This would further shorten development time and lead to increased savings. Dan

estimates that his team could cut another four weeks off of time-to-market for apps that reuse previously-built code.

10

BENEFITS

Homes.com experienced a number of quantified benefits in this case study:

› Reduced time to develop and test new mobile apps

› Reduced time to update and test existing mobile apps

› Faster time-to-resolution for underperforming apps

Other important benefits seen but not quantified included improved app quality and improved reusability of code. Both of

these benefits lead to higher-quality, higher-performance apps that customers are more likely to download and use. Better

apps mean better user reviews, which translate to more downloads and, ultimately, revenue-generating leads for

Homes.com.

Reduced time to develop new mobile apps

Homes.com indicated that a key benefit from the Appcelerator Platform was a reduction in time to develop and deliver new

apps. Investment in the platform enabled Homes.com to develop new mobile apps 40% faster than native app development

with a development cost savings of more than 50% per app. Prior to using the platform, the development process took 20

weeks with highly-paid iOS and Android developers. With the Appcelerator Platform, the development team could do the

same work in 12 weeks, using JavaScript.

FIGURE 3

Reduced Time-To-Market – New Mobile Apps.


Homes.com mentioned that there is a natural attrition that occurs as a development skillset moves from iOS and Android to

JavaScript. Therefore, the full development cost savings would not be realized in the first year because of the overlap of

developer resources and the transition process that must take place. To compensate, this benefit was risk-adjusted by 25%

in the first year only. The risk-adjusted total benefit resulting from reduced time to develop new mobile apps was $420,032.

See the section on Risk for more detail.

11

TABLE 1

Reduced Time and Cost To Develop New Apps: Reducing the Number of Platform Codebases and Using Lower-Cost Developer Talent Leads to Cost Savings and Faster Time-To-Market.

Ref. Metric Calculation Year 1 Year 2 Year 3

A1 Cost to develop new app before 135,000 135,000 135,000

A2 Cost to develop new app after 63,600 63,600 63,600

A4 Number of net new mobile apps 4 2 2

At Incremental cost savings (A1-A2)*A4 $285,600 $142,800 $142,800

At Risk Adjustment 25% First Yr. Only

Atr Incremental cost savings (risk-adjusted) $214,200 $142,800 $142,800


Reduced time to update existing mobile apps

Homes.com indicated another key benefit that drove cost savings for them was the reduced time to update existing apps.

Prior to using the Appcelerator Platform, the iOS and Android developers had to update two distinct codebases and test on

both platforms before releasing. They relied on manual testing which was more time consuming and error-prone. After

implementing Appcelerator, the team reduced it’s time to update an app by 25% and was able to benefit from using lower-

cost JavaScript developer talent at 33% hourly savings and more sophisticated automated testing techniques. Most

significantly, the benefit associated with making updates to existing apps increases over time as more net new apps are

developed.

Table 2

Reduced Time to Update Existing Apps: Faster Turnaround and Lower-Cost Talent Drove Savings


B1 Cost to update app before 10,800 10,800 10,800

B2 Cost to update app after 6,500 6,500 6,500

B3 Number of apps 4 6 8

B4 Number of updates per app (monthly) 12 12 12

Bt Incremental cost savings (non-risk adjusted) (B1-B2)*B3*B4 $206,400 $309,600 $412,800

Risk Adjustment 0%

Btr Annual revenue from additional new business initiation (risk adjusted)

$206,400 $309,600 $412,800


Faster time-to-resolution for underperforming apps

Homes.com was able to cut down time-to-resolution of app problems by 75% using Appcelerator's Performance

Management capabilities, a core element of the platform. A key indicator of quality is crash rate. Prior to using the

Appcelerator Platform, it took the development team hours to pinpoint and fix an error. That time has been reduced by 75%,

12

leading to additional cost savings. Most importantly, crashes as a result of defects cause users to get frustrated and either

abandon the app or, worse still, post a negative user review in a public place. Using Forrester Research’s mobile analysts’

industry benchmark for error rates, the study estimates that about 125 defects will be found in each mobile codebase when

releasing a new app. However, different organizations will have unique defect rates. To account for this, the benefit was risk-

adjusted down by 15%. See the section on Risk for more detail.

TABLE 3

Faster Time-To-Resolution of app problems: Pinpointing Errors before They Lead To Negative User Reviews


C1 Number of hours to pinpoint error

before Appcelerator 4 4 4

C2 Average number of defects per mobile

app code base 125 125 125

C3 Average number of new apps

released/year 4 2 2

C4 Number of mobile platforms for which

apps are developed (before) 2 2 2

C5 Hourly salary of tester 37.5 37.5 37.5

C6 % reduction in time to pinpoint errors 75% 75% 75%

Incremental output per worker C1*C2*C3*C4*C5*C6 $112,500 $56,250 $56,250

Risk Adjustment 15%

Ctr Annual revenue from additional new business initiation (risk adjusted)

$95,625 $47,813 $47,813


Total Benefits

Table 4 shows the total of all benefits across the three areas listed above, as well as present values (PVs) discounted at

10%. Over three years, Homes.com expects risk-adjusted total benefits to be a PV of more than $1,336,047.

13

TABLE 4

Total Benefits (Risk-Adjusted)

Ref. Benefit Initial Year 1 Year 2 Year 3 Total

Present

Value

Atr

Reduced Time to

Develop New Mobile

Apps $0

$214,200 $142,800 $142,800 $499,800 $420,032

Btr

Reduced Time to

Udpate Existing

Mobile Apps $0

$206,400 $309,600 $412,800 $928,800 $753,647

Ctr

Faster Time-to-

Resolution for App

Problems $0

$95,625 $47,813 $47,813 $191,250 $162,369

Total Benefits $0 $516,225 $500,213 $603,413 $1,619,850 $1,336,047


14

COSTS

Homes.com experienced two primary costs associated with the Appcelerator Platform:

› Cost No. 1. Software subscription fee.

› Cost No. 2. One-time training cost.

The Homes.com team received a good deal of guidance from the Appcelerator support team, including having them attend

their regular development team meetings. This level of support was at no additional charge to Homes.com. Said Dan, “The

Appcelerator team attended weekly development meetings to answer any challenges we had. They were open to getting on

the phone with us when we hit a roadblock.”

Cost No. 1. Software Licensing Fees

Software licensing fees for the Appcelerator Platform were incurred during the initial implementation period and recur on an

annual basis. They are primarily based on the number of users (e.g. developer/tester) of the platform. The cost increases

when user seats are added, something that the Homes.com team has not had to do so far.

TABLE 5

Annual platform license


D1 License Cost (5-10 Users) 80,000 80,000 80,000

Dt Total Software Costs (not risk adjusted)

80,000 80,000 80,000


Cost No. 2. Training and Implementation Costs

Training costs are optional and on a one-time basis and benefitted more developers across the entire Dominion Enterprise

and not just Homes.com. The implementation cost accounts for the overlap of resources when developing the first app.

TABLE 6

Annual Training and Implementation Costs


E1 Training and Implementation Costs $30,000 $0 $0

Et Total Costs $30,000 $0 $0


Total Costs

Table 7 shows the total of all costs as well as associated present values, discounted at 10%. Over three years, Homes.com

expects total costs to total a net present value of a little more than $226,221.

15

TABLE 7

Total Costs (Risk-Adjusted)

Ref. Cost Initial Year 1 Year 2 Year 3 Total

Present

Value

Dtr Annual Platform

License $0

$80,000 $80,000 $80,000 $240,000 $198,948

Etr Training and

Implementation $0

$30,000 $0 $0 $30,000 $27,273

Total Costs $0 $110,000 $80,000 $80,000 $270,000 $226,221


Virgin Money Case Study: Appcelerator Platform for Internal App Delivery

Forrester Research predicts that the average organization could realize incredible momentum and cost-savings if it develops

apps for both internal and external uses (see Figure 4). To capture the benefits of using the Appcelerator Platform for internal

development, Forrester Research conducted an interview with Virgin Money’s IT Manager of Digital Development, Chris

Edwards.

BACKGROUND

Virgin Money is a UK-based retail bank that provides savings, mortgages, credit cards, pensions, investment and protection

products. Virgin Money knew it was time to invest in a mobile delivery platform when they saw Web traffic to their site coming

from tablets and mobile devices growing significantly year-on-year. Virgin Money sought a platform that could deliver mobile

apps inexpensively with its team of JavaScript developers. In addition, they had the goal of using a solution that could be

supplemented easily with outside resources when necessary and didn’t wholly rely on internal infrastructure resources. After

a thorough selection process, Virgin Money determined that Appcelerator fit the bill. Virgin Money is now taking a phased

approach to leveraging the Appcelerator Platform, first identifying internal business processes that can be improved easily

with simple mobile functionality and later move towards external B2B and B2C app development.

After testing two internal prototype apps using Appcelerator’s open source Titanium tool, Virgin Money won support from the

executive team to license the Appcelerator Platform for enterprise use. It was an easy justification because the platform

enabled them to deliver apps for a fraction of the cost and in very short order. Furthermore, their use of Appcelerator’s multi-

tenant public cloud meant that they could initiate the project without up-front investment in infrastructure. Reducing the

initiation costs meant this R&D effort was successfully prioritized against wider business priorities given the potential

efficiency gains it could offer the organization. Said Chris, “The idea is to drive efficiency in the business and the ultimate aim

is for the mobile group to become self-funding, by delivering enough efficiency savings in the business to cover the cost of

development.”

Virgin Money developed two prototypes that “ignited the imagination” of the senior stakeholders and won their support:

› Prototype 1 - Rate-My-Store App: Increase efficiency of order entry for store managers by going from manual

means to touchpad. Virgin Money has 75 branches in the UK. Every day, the store manager has to perform activities to

ensure standard protocols are being followed. The previous mechanism for this data capture was a manual task. The

Virgin Money team built a prototype app “Rate My Store” quickly with an experienced JavaScript developer who’d never

used the Appcelerator product before. The iPad app allows store managers to do a geo-location lookup for the nearest

store, complete a questionnaire and submit their ratings. Said Chris, “The reaction was very positive – it made a quick and

easy impact.” The team plans to integrate the app with their management information systems dashboard.

16

› Prototype 2 - IT Service Mgmt. Incident App: Consolidate system alerts for the CIO. Impact to service is a critical

issue for the CIO and it is monitored constantly. Being able to deliver timely communications through a consistent channel

is key to incident resolution. So the team developed “Incident App” that immediately alerts IT management to a service

impact by allowing incident managers to subscribe to it, log errors, and receive push notifications about updates on error

status. The incident managers can follow a conversation around a particular service issue and receive those updates

through a single channel. Said Chris, “The app was incredibly well-received, has become an integral part of the Incident

Management framework and was a key driver behind making an investment in the Appcelerator Platform.”

“The Appcelerator Platform has made the ability to deliver the apps much

quicker and cheaper …we see the MBaaS solution as part of our strategic

architecture going forward.”

~ Chris Edwards, IT Manager, Digital Development, Virgin Money

Virgin Money expects to continue to develop internal apps in the next year with a relatively small team of developers who will

be responsible for both development and testing.

Key Benefits Experienced by Virgin Money Implementing the Appcelerator Platform:

• Speed of prototyping and development. The ability to develop fully native apps quickly (including testing) and

iterate on those apps every two weeks is a key reason Virgin Money invested in the Appcelerator Platform. They

have a mobile app board that reviews suggestions for new apps or updates to existing apps. The team views the

Analytics portion of the platform as key to their “test and learn” strategy for rapid app development. They plan to put

analytical “hooks” into all of their apps to understand how they’re being used and improve their performance.

• Availability of existing code and APIs. The login and geo-location functionalities are just a couple examples of

how the easy reuse of code and the library of APIs will play a key role in Virgin Money’s expanded use of the

Appcelerator Platform. In addition to integrating with their content management and management information

systems, Appcelerator offers pre-built connectors to enterprise data sources and public data sources that enable fast

integration with popular applications, including SAP, Oracle, Salesforce.com, Microsoft SharePoint and Microsoft

Dynamics.

• Security of the virtual private cloud. The Enterprise MBaaS portion of the Appcelerator Platform is available via

highly secure and scalable public, virtual private, or private deployment options. Virgin Money will evaluate the move

from a public multi-tenant environment to a dedicated virtual private cloud as they accelerate app development.

Now that the Virgin Money team has licensed and trained on the Appcelerator Platform, they expect to track the bottom-line

savings that result from delivering apps through it. Being able to develop, deliver and support mobile apps with a small team

of JavaScript developers will undoubtedly prove the business case all over again for them. The efficiencies gained in terms

of business productivity, the independence from having to rely on infrastructure resources, as well as not having to rely on a

third party for app delivery will enable Virgin Money to achieve its goal of becoming a self-funded team.

17

FLEXBILITY: THE POWER OF USING A MOBILE DELIVERY PLATFORM FOR INTERNAL AND EXTERNAL APPS

Flexibility, as defined by Forrester’s TEI methodology, represents an investment in additional capacity or capability that could

be turned into business benefit for some future additional investment. This provides an organization with the “right” or the

ability to engage in future initiatives but not the obligation to do so. There are multiple scenarios in which a customer might

choose to implement the Appcelerator Platform and later realize additional uses and business opportunities. Flexibility would

also be quantified when evaluated as part of a specific project (described in more detail in Appendix A).

Homes.com developed customer-facing apps to have a direct impact on the core business of driving revenue-generating

leads. Virgin Money is focusing initially on building internal apps for efficiency and productivity gains with future plans to

expand to customer-facing apps.

Forrester Research mobile analysts estimate that the average $1B+

organization can expect an upward growth trend for both its internal and

external app development. They predict that the average organization could

realize incredible momentum and cost-savings if it develops apps efficiently for

both internal and external uses.

~ Forrester Research

Below is a figure that models the cost savings with more aggressive app development plans over three years.

FIGURE 4

Cost Savings with Additional Usage of the Appcelerator Platform for Internal and External App Creation

This figure is based on an assumption that an average $1B + organization average number of apps and updates.

Assumptions

Costs To Develop New Apps And Update Existing Apps

381,600

810,000699,600

1,485,000

1,081,200

39,000

64,800110,500

183,600

221,000

367,200

With Without With Without With Without

Year 1

1 2

Year

3

Year 2 Year 3

Updates

New apps

This chart models the cost difference between

delivering apps with the Appcelerator Platform

and without it. An average $1B + organization

that uses the Appcelerator Platform for app

delivery will incur substantially lower costs

over time for both new app creation and

updates to existing apps.2,295,000

New apps

Updates to existing apps

1 3 5

5 8 12

6 11 17

Number of internal- andexternal-facing apps for update

6 17 34

Total number of new appsyear-over-year

Number of newexternal-facing apps

Number of newinternal-facing apps

Appcelerator Appcelerator Appcelerator

18


RISKS

Forrester defines two types of risk associated with this analysis: “implementation risk” and “impact risk.” “Implementation risk”

is the risk that a proposed investment in the Appcelerator Platform may deviate from the original or expected requirements,

resulting in higher costs than anticipated. “Impact risk” refers to the risk that the business or technology needs of the

organization may not be met by the investment in the platform, resulting in lower overall total benefits. The greater the

uncertainty, the wider the potential range of outcomes for cost and benefit estimates.

If risk-adjusted NPV of costs and benefits still demonstrate a compelling business case, it raises confidence that the

investment is likely to succeed because the risks that threaten the project have been taken into consideration and quantified.

The risk-adjusted numbers should be taken as “realistic” expectations, as they represent the expected value considering risk.

Assuming normal success at mitigating risk, the risk-adjusted numbers should more closely reflect the expected outcome of

the investment.

TABLE 8

Benefit And Cost Risk Adjustments

Benefits Adjustment

Years 2 and 3

Adjustment

Reduced Time To Develop New Apps (1st Year Only) 25% 0%

Faster Time-To-Resolution for App Problems 15% 15%

Costs Adjustment

(Costs were not risk-adjusted) 0%


Quantitatively capturing implementation risk and impact risk by directly adjusting the financial estimates results provides

more meaningful and accurate estimates and a more accurate projection of the ROI. In general, risks affect costs by raising

the original estimates, and they affect benefits by reducing the original estimates. The risk-adjusted numbers should be taken

as “realistic” expectations since they represent the expected values considering risk.

The following impact risks that affect benefits are identified as part of the analysis:

› Because development teams typically have resources in place, they are not able to immediately benefit from the cost

savings on developer salaries. Moving from a native development team to a JavaScript team is a transition process that

takes place in the first year of the implementation. By years 2 and 3, the organization will realize the full benefit.

› All mobile apps have defects when they are released and the number varies by company, by team, and by individual app.

An industry average was used for this calculation and assumes a defect rate of between 1-4 defects per KSLOC

(thousand lines of code). Mobile apps have about a maximum of 50KSLOCs per codebase. In an average app you’d

expect 50-200 defects when first released. For the purpose of this model, 125 defects were used for the calculation.

However, it was risk-adjusted down by 15% to account for the possibility that not all companies and apps have that

number of defects. Table 8 above shows the values used to adjust for risk and uncertainty in the cost and benefit

estimates. Readers are urged to apply their own risk ranges based on their own degree of confidence in the cost and

benefit estimates.

19

Financial Summary

The financial results calculated in the Benefits and Costs sections can be used to determine the ROI, NPV, and payback

period for the organization’s investment in the Appcelerator Platform.

Table 9 below shows the risk adjusted ROI, NPV, and payback period values. These values are determined by applying the

risk-adjustment values from Table 8 in the Risk section to the un-adjusted results in each relevant cost and benefit section.

FIGURE 5

Cash Flow Chart (Risk-Adjusted)


TABLE 9

Cash Flow: Risk-Adjusted

Initial Year 1 Year 2 Year 3 Total Present value

Costs

$100,000 $80,000 $80,000 $260,000 $217,130

Benefits

$516,225 $500,213 $603,413 $1,619,850 $1,336,047

Net benefits

$416,225 $420,213 $523,413 $1,359,850 $1,118,917

ROI

515%

Payback period

5 months


20

The Appcelerator Platform: Overview

The following information is provided by Appcelerator. Forrester has not validated any claims and does not endorse

Appcelerator or its offerings.

In response to the Mobile Mindshift – the latest technology shift to impact organizations today with consumers and

employees expected access anywhere, anytime to information on their smartphones and tablets– Appcelerator developed a

unified platform that enables companies to deliver great apps that run on a range of devices, and connect to an exploding set

of backend data. The platform has three primary “legs” – Apps, for building and testing native, cross-platform apps, APIs that

enable mobile-optimized connection to backend data sources, and Analytics, which provide a real-time view into app usage

and adoption, quality and performance, and API consumption.

› Apps – The Apps component of the Platform enables teams to design, prototype and build fully native, cross-platform

apps, all from a single JavaScript codebase – one of the simplest, most popular and widest-used web languages. Unlike

other mobile platforms, Appcelerator converts the code into a native version for each operating system. It also includes full

test automation to decrease testing time and improve app quality.

› APIs – connection to public and enterprise data sources is provided by a true, enterprise mobile backend-as-a-service

(MBaaS), allowing companies to extend their enterprise architecture with mobile-optimized APIs backed by a scalable

cloud architecture.

› Analytics – The Analytics portion of the platform offers a lifecycle dashboard that provides a comprehensive real-time view

of the end-to-end mobile lifecycle, from development through production. It also includes a real-time, tablet-based app

dashboard for business owners that measures and manages key metrics around app adoption, usage, penetration and

portfolio ROI.

With a large mobile ecosystem that includes hundreds of partners, over a thousand customers, hundreds of thousands of

mobile developers, millions of devices and monthly cloud API calls, Appcelerator brings a vast ecosystem of mobile

innovation to customers.

21

TEI Framework And Methodology

INTRODUCTION

From the information provided in the interviews, Forrester constructed a Total Economic Impact™ (TEI) framework for those

organizations considering implementing the Appcelerator Platform. The objective of the framework is to identify the cost,

benefit, flexibility, and risk factors that affect the investment decision.

APPROACH AND METHODOLOGY

Forrester took a multistep approach to evaluate the impact the Appcelerator Platform can have on an organization (see

Figure 2). Specifically, we:

› Interviewed Appcelerator marketing, product, and sales personnel, along with Forrester analysts, to gather data relative to

Appcelerator and the marketplace for its platform.

› Interviewed two organizations currently using the Appcelerator Platform to obtain data with respect to costs, benefits, and

risks.

› Constructed a financial model representative of the interviews using the TEI methodology. The financial model is

populated with the cost and benefit data obtained from Homes.com because they have been using the tool for two years,

whereas Virgin Money is in its initial stage of use.

› Risk adjustment is a key part of the TEI methodology. While interviewed organizations provided cost and benefit estimates,

some categories included a broad range of responses or had a number of outside forces that might have affected the

results. For that reason, some cost and benefit totals have been risk-adjusted, and is detailed in each relevant section.

Forrester employed four fundamental elements of TEI in modeling the Appcelerator Platform’s: benefits, costs, flexibility, and

risks.

Given the increasing sophistication that enterprises have regarding ROI analyses related to IT investments, Forrester’s TEI

methodology serves to provide a complete picture of the total economic impact of purchase decisions. Please see Appendix

[B] for additional information on the TEI methodology.

FIGURE 2

TEI Approach


22

Disclosures

The reader should be aware of the following:

› The study is commissioned by Appcelerator and delivered by Forrester Consulting. It is not meant to be used as a

competitive analysis.

› Forrester makes no assumptions as to the potential ROI that other organizations will receive. Forrester strongly advises

that readers use their own estimates within the framework provided in the report to determine the appropriateness of an

investment in the Appcelerator Platform.

› Appcelerator reviewed and provided feedback to Forrester, but Forrester maintains editorial control over the study and its

findings and does not accept changes to the study that contradict Forrester's findings or obscure the meaning of the study.

› Appcelerator provided the customer names for the interviews but did not participate in the interviews.

FRAMEWORK ASSUMPTIONS

Table 10 provides the model assumptions that Forrester used in this analysis.

The discount rate used in the PV and NPV calculations is 10% and time horizon used for the financial modeling is 3 years.

Organizations typically use discount rates between 8% and 16% based on their current environment. Readers are urged to

consult with their respective company’s finance department to determine the most appropriate discount rate to use within

their own organizations.

TABLE 10

Model Assumptions (Note: Salary Assumptions are based on Norfolk, VA pay scales. Expect a higher hourly rate for developers and testers if your organization is located in a urban area with competition for tech talent.)

Ref. Metric Calculation Value

C1 Hours per week 40

C2 Weeks per year 52

C3 Hours per year (M-F, 9-5) 2,080

C4 Hours per year (24x7) 8,736

C5 Hourly salary of a native developer (fully

loaded) $75.00

C6 Hourly salary of a JavaScript developer (fully

loaded) $50.00

C7 Hourly salary of a manual tester (fully loaded) $37.50

C8 Hourly salary of an automated tester (fully

loaded) $62.50

C9 Average number of defects per app released 125


23

Appendix A: Total Economic Impact™ Overview

Total Economic Impact is a methodology developed by Forrester Research that enhances a company’s technology decision-

making processes and assists vendors in communicating the value proposition of their products and services to clients. The

TEI methodology helps companies demonstrate, justify, and realize the tangible value of IT initiatives to both senior

management and other key business stakeholders.

The TEI methodology consists of four components to evaluate investment value: benefits, costs, flexibility, and risks.

BENEFITS

Benefits represent the value delivered to the user organization — IT and/or business units — by the proposed product or

project. Often, product or project justification exercises focus just on IT cost and cost reduction, leaving little room to analyze

the effect of the technology on the entire organization. The TEI methodology and the resulting financial model place equal

weight on the measure of benefits and the measure of costs, allowing for a full examination of the effect of the technology on

the entire organization. Calculation of benefit estimates involves a clear dialogue with the user organization to understand

the specific value that is created. In addition, Forrester also requires that there be a clear line of accountability established

between the measurement and justification of benefit estimates after the project has been completed. This ensures that

benefit estimates tie back directly to the bottom line.

COSTS

Costs represent the investment necessary to capture the value, or benefits, of the proposed project. IT or the business units

may incur costs in the form of fully burdened labor, subcontractors, or materials. Costs consider all the investments and

expenses necessary to deliver the proposed value. In addition, the cost category within TEI captures any incremental costs

over the existing environment for ongoing costs associated with the solution. All costs must be tied to the benefits that are

created.

FLEXIBILITY

Within the TEI methodology, direct benefits represent one part of the investment value. While direct benefits can typically be

the primary way to justify a project, Forrester believes that organizations should be able to measure the strategic value of an

investment. Flexibility represents the value that can be obtained for some future additional investment building on top of the

initial investment already made. For instance, an investment in an enterprisewide upgrade of an office productivity suite can

potentially increase standardization (to increase efficiency) and reduce licensing costs. However, an embedded collaboration

feature may translate to greater worker productivity if activated. The collaboration can only be used with additional

investment in training at some future point. However, having the ability to capture that benefit has a PV that can be

estimated. The flexibility component of TEI captures that value.

RISKS

Risks measure the uncertainty of benefit and cost estimates contained within the investment. Uncertainty is measured in two

ways: 1) the likelihood that the cost and benefit estimates will meet the original projections, and 2) the likelihood that the

estimates will be measured and tracked over time. TEI applies a probability density function known as “triangular distribution”

to the values entered. At a minimum, three values are calculated to estimate the underlying range around each cost and

benefit.

24

Appendix B: Glossary

Discount rate: The interest rate used in cash flow analysis to take into account the time value of money. Companies set

their own a discount rate based on their business and investment environment. Forrester assumes a yearly discount rate of

10% for this analysis. Organizations typically use discount rates between 8% and 16% based on their current environment.

Readers are urged to consult their respective organizations to determine the most appropriate discount rate to use in their

own environment.

Net present value (NPV): The present or current value of (discounted) future net cash flows given an interest rate (the

discount rate). A positive project NPV normally indicates that the investment should be made, unless other projects have

higher NPVs.

Present value (PV): The present or current value of (discounted) cost and benefit estimates given at an interest rate (the

discount rate). The PV of costs and benefits feed into the total NPV of cash flows.

Payback period: The breakeven point for an investment. This is the point in time at which net benefits (benefits minus costs)

equal initial investment or cost.

Return on investment (ROI): A measure of a project’s expected return in percentage terms. ROI is calculated by dividing

net benefits (benefits minus costs) by costs.

A NOTE ON CASH FLOW TABLES

The following is a note on the cash flow tables used in this study (see the example table below). The initial investment

column contains costs incurred at “time 0” or at the beginning of Year 1. Those costs are not discounted. All other cash flows

in years 1 through 3 are discounted using the discount rate (shown in Framework Assumptions section) at the end of the

year. PV calculations are calculated for each total cost and benefit estimate. NPV calculations are not calculated until the

summary tables are the sum of the initial investment and the discounted cash flows in each year.

Sums and present value calculations the Total Benefits, Total Costs and Cash Flow tables may not exactly add up, as some

rounding may occur.

TABLE [EXAMPLE]

Example Table



25

Appendix C: Supplemental Material

Related Forrester Research

The Mobile Mindshift: Engineer Your Business to Win in the Mobile Moment, Ted Schadler, Josh Bernoff, Julie Ask,

Groundswell Press, 2014

“Build Five-Star Mobile Apps,” Jeffrey Hammond, Forrester Research, Inc., November 7, 2012

“Build Great Apps That Drive Engagement,” Jeffrey Hammond, Forrester Research, Inc., June 28, 2012

“Measuring Mobile Apps,” Jeffrey Hammond, Forrester Research, Inc., November 18, 2013

“The Engagement Platform’s Aggregation Tier,” Michael Facemire and Jeffrey Hammond, Forrester Research, Inc., May 27,

2014

1 Forrester risk-adjusts the summary financial metrics to take into account the potential uncertainty of the cost and benefit

estimates. For more information see the section on Risk.

AppceleratorTEI_final_July 30

Documents

Transcript of AppceleratorTEI_final_July 30