Project Management and Technology don't matter
-
Upload
andrew-patricio -
Category
Software
-
view
29 -
download
0
Transcript of Project Management and Technology don't matter
District of Columbia Public Schools | 1200 First Street, NE | Washington, DC 20002 | T 202.442.5885 | F 202.442.5026 | www.dcps.dc.gov
Project Management &Technology don’t matter
Council of Great City Schools CIO Conference3 June 2015
Project Management and Technology Don't Matter.
The District of Columbia Public Schools successfully completed a major effort to replace its Student Information System (SIS). Phase 1 for 60 elementary schools was completed in only 4 months from vendor start to first go-live for SY14-15. Phase 2 for the remaining schools was completed for the 2015=2016 school year.
Key Factors:• Managing failure rather than success• DCPS in house technical expertise and leadership• Strong focus on people rather than technology
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 3
About Me – Andrew Patricio Deputy Chief for Data Systems• Reporting to Chief of Office of Data and Strategy• Worked as contractor for DCPS 2009- 2010• Started as FTE fall of 2010• Most senior technical resource at DCPS
(No official DCPS CIO)
Personal background• BS in Electrical Engineering• Previously worked in Management and
and IT Consulting• DCPS is my first stint in government• Two adorable kids
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 4
“Project Management and Technology don’t matter”
Project Management and Technology DO matter….
...but not as much as we think
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 6
“Project Management doesn’t matter”
Good project management governs how well you succeed not if you succeedMany IT projects have been “well-managed failures”• Tracked all milestones• Had strong technical team• Still resulted in cost-overruns, delays,
poorly utilized systems
Some kind of project management is important, but details don’t matter• Project manager should understand school context, not just manage deliverables• Approach should be compatible with organization management style
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 7
“Office Space” 1999
“Technology doesn’t matter”
Technology choice matters, implementation less so• Implementation governs how well you succeed not if you succeed• Can’t fix non-scalable, non-customizable system after the fact• Many effective options available for base technology so choose
what is most compatible with skill set available to your team• Procurement is therefore critical step for technology aspect
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 8
Ensure Competent Technology – RFPCritical technological step is effective RFP processTrue competitive solicitation process resulted in multiple potential vendors with system that met requirements for DCPS• Released RFP to the street for 2 months• Vendors had opportunity to ask questions• Evaluated written proposals• Invited vendors in for demos
• 1 full day for vendor to demo SIS features to school staff focus group• 1 half day to demo specifically how system can be customized by district
• Evaluated demos and “Best and Final Offer” proposals
Decision made based on total points awarded for proposal, demos, and cost
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 9
RFP – Scope of WorkProposal requirements were intended to get a sense of vendor’s implementation skills as well as system feature set. Three areas of requirements:Vendor background and project approach • Project implementation plan• Vendor transition plan – how to transfer development over to DCPS• Vendor qualifications – company size, stability/longevity of product ,etc
Feature set out of the box• Attendance, Mark Entry, Scheduling,
Enrollment, Gradebook, etc
Customization• We were very specific about what we meant by customization: specific data model
modifications, user interface customizations, etc
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 10
RFP – Prioritized RequirementsEach detailed technical requirement was assigned a rating within a range of necessity rather than simply “optional” or “required”Provided way to convey more nuanced view of DCPS thinking
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 11
Rating DescriptionMandatory Any solution which does not offer this feature will not be considered.
Critical Solutions lacking this feature will still be considered but will be evaluated negatively.
Necessary All solutions should offer this feature. DCPS is looking for solve a specific issue that is not being addressed well or at all with current SIS. Solutions lacking this feature will be evaluated negatively but can make up for that with greatly enhanced functionality in other areas. Missing support for too many necessary features will make it much less likely that the vendor will win this contract.
Expected Best solution should offer this feature. DCPS views this as being a part of an effective system but can make do without it. Solutions lacking this feature will not be evaluated as positively but will not necessarily be evaluated negatively. Supporting these features will help make up for any lacking support for "Necessary" features.
Desired Ideal solution should offer this feature. DCPS views this as something that would help differentiate between two equally good solutions. Supporting these features will not make up for lack of support for any "Necessary" features.
Helpful Wishlist feature. DCPS is not currently encountering an issue addressed by this feature but recognizes its usefulness. These features are mentioned for completeness but will have very little bearing on the overall qualifications of a particular solution.
RFP – Prioritized Requirements – Examples
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 12
Ref # Requirement Priority Weighting
GC1 Solution must be a web-based application with an interface using the native rendering capabilities (ie HTML, CSS, Javascript, etc) of standard web browsers without requiring any plugins. Specifically any solutions requiring installation or updates of Java, Flash/Shockwave, Silverlight, or similar plugins will be rejected.
Mandatory
GC10 Every student record must have a unique, global, permanent ID generated by system
Critical
CF13.2 API to pull dynamically pull any data from backend database for use in front end customizations
Necessary
GU12.1 validates each field as it is entered rather than after an entire screen or transaction is entered
Expected
AG21 allows for the printing of attendance or hall passes to identify late arrivals and early departures
Desired
GU12.6 provides “spell checking” ability to all narrative fields Helpful
RFP – Vendor Requirements CoverageSimilarly all vendors were required to be specific in how they proposed to meet each requirement.
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 13
Rating Description
Y This feature can be demonstrated in system immediately
N System does not provide this feature
C1, C2, C3 Feature can be implemented by DCPS via increasing level of effort to customize system (C1 – few days, to C3 – few months)
UD Feature is being actively developed by vendor for next release
P Feature is currently planned to be included in future release
3 Feature provided via 3rd party software or add-on
RFP – Vendor Demo – System Features
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 14
1.5 days for each vendorFirst full day – feature demo• Ran through all features of system with school
based staff focus group• DCPS required presence of actual assigned vendor
project manager to evaluate people skillsSecond half day – customization/technical demo• Purely technical, ran through example
customizations for DCPS technical resources• Required vendor deep technical experts to be
present (not just sales) to get into details
RFP – Vendor Demo – People factors
HumilityVendor team is willing to take direction and are not wedded to their own ideas
HearingVendor team listens to what we are actually saying instead of just assuming what we are going to say
HumorVendor team is personable and easy to work with
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 15
Intangibles are just as important as system features
RFP – Technology Chosen
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 16
Result was that we were as confident about choice of technology as we could be before implementing
• Did not encounter any big surprises during technical implementation
• Minimal “no worse than STARS” worst case scenario was assured
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 17
So if not that, then what does matter?
Limit scope
Renovated house on my own• Scope creep• Kept assuming best case
scenario• Project ended up taking
almost 3 years
Lesson:• Be pragmatic• Walk before you can run
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 19
“Manage Failure”
Failure is inevitable
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 20
Manage FailureCan’t do everything well. Some things are going to drop
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 21
Consciously decide what to do well and what to skimp on instead of falling into it • Things need to be done right but good enough is usually good enough• First focused on school needs then central office reporting
• Data starts at the schools: garbage data in = garbage out for reporting
Goal was being perfectly clear, not being perfect• Clarity has a certain correctness all its own• Everyone should agree on what decision was, even if they disagree with it
Facts Not OpinionsMade conscious decision to prioritize timeline over perfecting process• No business process improvement meant that requirements were stable
target• The “right process” is an opinion, the “existing process” is a fact
Manage Failure
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 22
Phase 1 – Elementary Schools onlyElementary Schools as de facto “Pilot” for Aspen at DCPSThey do everything that Middle and High schools do but much simpler• Enrollment the same• Daily instead of period attendance/scheduling• Simple rubric report card: no course marks, GPA, or graduation requirements
Our mantra for Phase 1 was “If STARS didn’t do it, Aspen wouldn’t either”• DCPS did not have the time or capacity
for deep process improvement• Familiar process in new system
= easier transition for schools
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 23
Phase 2 – All remaining schoolsScheduling was exception to “no process improvement”5 high schools with veteran staff scheduled SY15-16 courses entirely in Aspen• Hired dedicated contractors who were experts in scheduling to support pilot• Developing standard process for DCPS to use to schedule SY16-17
Remaining schools started scheduling SY15-16 in DC STARS, finished in Aspen• Scheduling from scratch in Aspen would have been overwhelming• Most schools normally copy and manually tweak schedules from year to year
Implemented remaining phase 2 requirements also using STARS as baseline• No replacement of other systems initially, eg Student Behavior Tracker• No new functionality initially, eg parent portal, student schedule requests
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 24
Reporting Platform - Phase 1SY14-15 reporting from two systemsReporting platform combines student level data from both STARS and Aspen to provide district-wide reports for certain data
Started off with elements common to ES. Data which was strictly MS/HS was deprioritized since it was still in STARS so could continue to use existing reporting
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 25
ETL
STARS
Aspen
Reporting Platform
HS/MS only reporting:Courses, GPA, etc
Elementary / District-wide reporting: Attendance, Enrollment, etc
(ETL = Extraction, Transformation, Load)
Reporting Platform - Phase 2 & beyondSY15-16 and beyond reporting Reporting platform is independent from transactional systems and thus data model/processing can be optimized for reporting.• More efficient reporting
• Create pre-calculated datasets to report from instead of re-implementing business rules in every report so not redoing work
• Correlate data from separate databases, including manual pulls• Provide more self-service access to data instead of everything is a custom query
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 26
ETLSp Ed
Aspen
Reporting Platform
All reporting
LMS
Secondary & Tertiary Processing
Implementation - District Expertise
District of Columbia Public Schools | 2015 CGCS CIO Conference 27
In House expertise and project leadership1983 Mercedes diesel • Easy to understand• Can direct repairs on anything I
can’t fix myself• Much cheaper to run than
previous modern (2011) vehicle
Lesson:• Importance of understanding
things myself when working with vendors
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 28
Toxic cycle for IT projects
Well-meaning vendors, passionate about technology
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 29
Well-meaning school district personnel, passionate about education
+ =Disaster
Why?
Lost in translation
District of Columbia Public Schools | 2015 CGCS CIO Conference 30
DCPS concept
I wantGrey skinned animalMust have a trunkMust have large earsMust have tusks
“What do you want?” Delivered by external resource lacking DCPS context
…
After multiple expensive iterations, still not quite right
DCPS Program Office Vendor Technical Team
Vendor’s interpretation matches requirements but does not meet need and frustration ensues. “What do you want?” is the wrong question because DCPS program client is not a technical expert and vendor is not a business expert:
No one translating what programs need.
Speaking the Same Language
District of Columbia Public Schools | 2015 CGCS CIO Conference 31
…modulated by DCPS technical team familiar with DCPS context (minimal iterations)
Initial interpretation…
DCPS Technical
Actually, they probably mean an elephant….
DCPS concept
I want Grey skinned animalMust have a trunkMust have large earsMust have tusks I need to have an animal for my circus
“What do you need?”
DCPS Subject Matter Expert
In house technical experts more aware of DCPS context and can interpret requirements based on true “need” rather than stated “want”:
Programs get what they expected even if they didn’t quite know how to articulate that initially
Vendor driven projects
Student Information System (SIS) Replacement
32District of Columbia Public Schools | 2015 CGCS CIO Conference
Vendor Business analysts,
trainers, etc
Vendor PM
Vendor Executive
Leadership
Vendor Technical
Team
DCPS Executive Leadership
DCPS SIS Program Owner
DCPS Subject Matter Experts
• Vendor does not have to live with long term consequences of decisions
• Accountable to their leadership directly
• Often not dedicated to one client• Different motivations/point of view
No dedicated DCPS resources resulted in over-reliance on vendor in previous IT projects
Single path for Accountability
Student Information System (SIS) Replacement
33District of Columbia Public Schools | 2015 CGCS CIO Conference
DCPS Business
analysts, etc
DCPS PM
Vendor Executive
Leadership
Vendor PM and Implementation
Support
DCPS Executive Leadership
DCPS SIS Program Owner
DCPS Subject Matter Experts
• DCPS directed project manager and business analyst
• Worked directly with vendor implementation team
• Independent from vendor team• Dedicated to nothing other than
Aspen implementation• Vendor PM acted more as project
coordinator for vendor resources
Dedicated DCPS resources leading project meant that DCPS was in the drivers seat.
Implementation – People not Technology
District of Columbia Public Schools | 2015 CGCS CIO Conference 34
First real computer: 386 PC • Got just before freshman year
of college• Even typing “dir” at dos
prompt was scary
Lesson:• Anything new, especially new
technology, is intimidating• Fear is the problem, not lack
of knowledge
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 35
Manage change not technologySuccess is all about managing fear:• Fear in your implementation team that they are going to fail• Fear in your user base that they won't be able to do their work• Fear in your vendor that the project won't go well and they will get blamed
But push through that and in the end it is almost always:
”It wasn’t as bad as I thought it would be”
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 36
“Fear of change” cycleWhy do people fear change? Because they are rational and rational thinking seeks to solve problems.
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 37
Encounter problem
Rational mind begins to determine solution to problem
Can I solve the problem? No need to fear
Know exactly how much to fear (bounded)
yes
no
Fear is bounded even if problem cannot be solved because you know how bad it is
“Fear of change” cycleWhat happens with the unknown? Rational mind looks for problems, when it can’t find any, imagination kicks in
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 38
Encounter unknown
Rational mind begins to determine solution to problem
But what if it’s a worse problem than that?
yes
no
Whether or not you can deal with the problem you’ve imagined, you can always imagine something worse, adding more fear each turn of the cycle so that it increases without bound until it takes on a life of its own
No data = no problem
Problem is imagined
Can I solve this problem?
Worse problem is imagined Fear increases
Break cycle with informationAdding information gives the rational mind something to chew on, cycle is broken and fear becomes bounded once more
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 39
Encounter unknown
Rational mind begins to determine solution to problem
But what if it’s a worse problem than that?
No data = no problem
Problem is imagined
Problem becomes specific
Project team gives information Fear increases
No need to fear
Know exactly how much to fear
yes
no
Can I solve this problem?
Familiar process and trusted advisorsLimited fear of change in rolling out system by limiting amount of change• “If STARS didn’t do it, Aspen wouldn’t either”
• Process was familiar so just had to train on new way to enter the same data• Initially addressed fear about how Aspen wouldn’t be worse before trying to sell
how it would be better
• Trainers assigned to “their” set of schools built relationship and trust• Biggest project expense was personnel providing direct school support (training,
helpdesk etc)• Provided human face to project:
hearing “I understand” is powerful as it counters one emotion (fear) with another (fellowship)
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 40
Reduce Fear - Focus on school support - Phase 112 out of 18 resources dedicated to direct school support
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 41
Role PeopleSIS implementation 1 PM 1 BA 1 SME School Support 9 trainer , 3 helpdeskReporting 1 DBA, 2 devsVendor 0.5 PM (2 x .25), 2 FTE, ….Existing SIS support 2 helpdesk , 1 trainer, 4 technical contractor
FY14 – 6 months (April to September 2014)Roughly two thirds of overall budget was allocated to personnel costs and most of that was for directly supporting school staff
Focus on school support – Phase 224 out of 32 resources dedicated to direct school support
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 42
Role PeopleSIS implementation 1 PM 1 BA 1 SME 1.5 devSchool Support 12 trainers , 8 helpdesk, 3 scheduling consultants, 1 mgrReporting 1 DBA, 2 devsVendor 0.5 PM (2 x .25), 2 FTE (4 x .25)
FY15 (1 Oct 2014 to 30 sept 2015) Again, almost two thirds of overall budget was allocated for personnel costs and 70% of that was for directly supporting school staff
School Staff InvolvementMade sure schools felt a part of the process• This reduced fear and increased buy-in• Schools took part in vendor demos as part of RFP process• Got feedback on system configuration from selected school staff before rolling out• Reached out to experts at the schools to help develop scheduling process
• Original plan was to have all schools start scheduling SY15-16 in Aspen from the beginning• Feedback from school schedulers resulted in changing that plan to just 5 pilot schools
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 43
+ =
Do what’s best for the project but force yourself to be in the shoes of your users/vendors/team members Don’t imagine yourself doing things, imagine them doing things• Think of a time when you did something for the first time rather than on something you
are now comfortable with• Compare actual reality vs your own initial intimidation
Be selfish but not self-centered
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 44
+ =
TakeawaysHuge range between complete failure and perfect success
What needs to be not done wrong• Project management• Technology choice
What needs to be done right• Manage failure not success (ie manage scope)• Ensure that technical expertise is available to district and project leadership is
accountable to district and not just vendor• Focus on people, view IT system implementations as change management
projects not technology projects
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 46
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 47
Original Leading Slides(Changed the order to get to the less DCPS specific slides more quickly on slideshare)
District of Columbia Public Schools | 2015 CGCS CIO Conference 48
DCPS FactsLed by Chancellor Kaya Henderson since 201048000+ students (steady increase for the past 5 years) in 111 schools• 60 Elementary Schools• 11 Middle Schools• 18 Education Campuses (usually Preschool through 8th grade)• 15 High Schools• 7 other (Special Ed etc)
6800+ school staff (3600+ teachers)600+ central office staff
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 49
DC STARSCurrent SIS, called DC STARS, had been running since 2003. • Outdated technology (Java applet front end)• Difficult to customize• No longer supported by vendor
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 51
By 2013, DC STARS rigidity meant that data was fragmented among a patchwork of applications put in place to cover gaps in functionality
Made the decision early on to replace with new system rather than reinventing the wheel by developing custom solution from scratch
FlexibilityDCPS has an extremely high-caliber workforce and entrepreneurial culture• System features must keep pace with velocity of reform• System must be customizable by DCPS staff
StabilityTimeline was very short for initial Phase 1 launch but SIS is critical to operations• System cannot lose functionality from what STARS provides for schools• System vendor must have history of successful deployments to K12 districts
Replacement SIS Context
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 52
New SIS: Follett School Systems ASPEN
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 53
DC STARS
Follett ASPEN
Project TimelinePhased rollout• Phase 1 – Elementary Schools only (not Education Campuses)• Phase 2a – Schedule for upcoming year in Aspen with 5 pilot high schools • Phase 2b – All schools in Aspen for everything
Student Information System (SIS) Replacement
District of Columbia Public Schools | 2015 CGCS CIO Conference 54
Milestone Date
RFP released November 2013
Vendor demos January 2014
Vendor start April 2014
Go live for Elementary Schools only August 2014 (SY14-15)
Phase 2 scheduling pilot February 2015 (pre SY15-16)
Go live for all remaining schools July 2015 (SY15-16)
SY 2013 -2014 SY 2014-2015 SY 2015-2016
ES Design & Implementation
DC STARS for SY14-15 MS / HS Operations
MS / HS OperationsRollout
DC STARS Available for Read Only in HS MS
SY15-16 Scheduling Pilot in Aspen
High Level rollout schedule (Phased Approach)
SchoolStat: PARCC/eSIS
ES Rollout
SIS Vendor Selection
Oct Nov DecJan 2014 Feb Mar Apr May Jun Jul Aug Sep Oct Nov DecJan Feb Mar Apr May Jun Jul Aug Sep
Vendor Starts
DC STARS For ES Operations
DC STARS Available for Read Only in ES
Jan Feb Mar
ES Operations in Aspen →
MS/HS Operations in Aspen →
MS / HS Design & Implementation
MS / HS Scheduling PilotPro
ject
Rol
lout
Pha
se
and
Mile
ston
es
Out
com
e fo
r S
choo
ls &
C
entr
al
ODS/Vendor Work Streams Activities Before Vendor Selected
Key DC STARS Usage by Schools
Aspen Usage by Schools DC STARS Data Availability
SY15-16 Scheduling begins
SY15-16 Begins
PM & BA Start
55