Post on 13-Nov-2014
description
1
Mastering the Hard WaySafe custom ABAP code
Implementing Virtual Forge CodeProfiler for ABAP security and quality in a grown application landscape
Markus Theilen, EWE AG
22
Agenda
About EWE
Your presenter
Motivation
Looking back
Approach with CodeProfiler
Lessons learned
3
Comprehensive solutions in three key sectors
EWE brings together energy, telecommunications
and information technology, and thereby possesses all the key expertise
for sustainable, intelligent energy supply systems
4
EWE – one of the largest companies in northwest Germany
Sales of €8.9 billionNet profit of €57.2 million
Average number of employees 9,162
5
Our strengths are our excellent service and advice as well as the proximity to our customers
2013
1.4 million electricity customers
1.6 million gas customers
680,000 telecommunications customers
6
EWE’s regions in Germany, Poland and Turkey
77
Introducing your presenter: Markus Theilen
Enterprise Architect
2001 – 2012: software developer and architect working in the easy+ ABAP development at BTC AG
o Created and established development guidelineso Introduced automated checks of development objects
Since 2012: IT coordinator in the E-IT group “Billing and Market Communication”o Responsible for coordination of development and operation of easy+
Since 2009: associate speaker of DSAG working group Development
Co-Author: Best Practice Guidelines for Development – Practical tips on the ABAP Development
Author of one of the most popular presentations about ABAP code analysis:http://www.slideshare.net/therealtier/static-abap-code-analyzers
88
About easy+
o 100% custom-developed, based on SAP ERP 6.0
o Includes components for meter reading, accounting, invoicing, claims
management, market communication
o Reports and controls EWE group energy services
o Comparable with the functionality of SAP IS-U
o Entirely written in ABAP
o In productive use since 1995
o Approximately 100 people in development
99
Why “mastering the hard way?”
o Easy+ developed over decades
o Involved more than 100 developers with very different skill levels
o No distinct encapsulation of internal modules ultra dependent monolith
extremely difficult to maintain
o Rare checks for compliance with developer guidelines
o Completely manual regression testing (high efforts), including for purely
technical changes (even higher efforts)
1010
Technical view on easy+
o 8Tb data volume
o 10 Mil. lines of code
o 1,600 packages
o 11,000 programs
o 8,600 classes and 1,500 interfaces
o 1,500 function groups
o 4,400 tables
1111
Far too much code for manual reviews
o The more code, the higher the complexity
(exponential growth)
o Code might look OK upon manual review, but
it can have a severe impact in the context of
the call hierarchy
o It’s impossible to check complex code
manually
Our paradigm:
We do not allow development requirement that we can’t check automatically!
Risky Statement
1212
Looking back
Before 2009:
o No static analysis tools
o No regular code reviews
o No meaningful reporting about code quality was possible
However, we found bad code in old programs and expected better in new developments
1313
Starting with static analysis tools
In 2009:
o Introduction of a ABAP code scanning product in easy+ development at BTC
o Focus: ABAP and risk reporting for management
o Scanned non-ABAP, including Java, C/C++, .NET, and other languages
1414
The Good …
o Some limited reporting on the quality of code
o Developers got “used to” code analysis
o Knowledge transfer about good and bad coding was spread among developers informally
o From a 10,000 feet point of view, many dashboards were available for a management target group, revealing insightful results
1515
… and the bad
o The tool was expensive, hard to use, and error-prone
o Many false positives or entirely wrong checking rules decreasing acceptance by developers
o Developers started to do a “benchmark optimization” with negative impact they tried to satisfy the tool and stopped thinking
o No integration in ABAP development process possible (workbench, TMS) no simple way to bring “their” results to the developer’s desk
o Distinction between legacy coding (not-changed) and recently changed coding almost impossible corrections could only be made in context of complete code, which significantly increased the manual testing effort
1616
A (last) word on the competitive product
o The vendor has much in-house expertise in languages other than ABAP. In a non-ABAP context, it’s much more stable and mature.
o The product offers interesting and informative analysis tools based on a “development object database” with meta data and manifold relations between objects
o The products focus aims at the management level
o It was only after using the tool that we realized we should follow a different direction we need a different tool for this
1717
Changeover to CodeProfiler
End of 2012: start of a productive pilot
Since early 2013: in full productive use
Initial focus was more on feedback for developers, not on dashboards
1818
Positive experience and potential
o Faster analysis
o Much lower false positive rate
o TMS integration for automatic checking of code changes
o Targeted checking of existing code base
o Best coding practices documentation for ABAP
Vendor:
• Applies feedback into future development
• Fast and accurate response
• Semi-annual releases
1919
Impact of Scrum introduction on development
By end of 2012, we started to switch to a Scrum-based development process in the easy+ environment
Key Scrum principle: feedback early and often!
o The shorter the feedback cycles, the quicker and easier the right target can be reached again
o In parallel, agile development practices to improve quality were introduced
The principle of quick feedback should be applied to the compliance of developer guidelines
Change of behavior necessary
2020
The behavioral change
o Feedback
Regularly inform all involved parties in a factual and objective way about all issues
o Penalties and rewards
• Impact of “bad” behavior must be tangible for those who can change something in their day-to-day work
• Positive behavior must be reinforced by appropriate rewarding mechanisms
2121
Competitive product feedback
o Infrequent feedback from the beginning
o No direct relation between developer’s day-to-day work and abstract “management figures”
o No direct “pain” or a feeling of inconvenience from possible penalties
No incentive for behavioral change
2222
CodeProfiler feedback
o Feedback about security and quality is possible any time through the tight integration of CodeProfiler in the ABAP development process
o Developers get instant feedback by transport release several times per day or week Much closer to the time of adding issues in newly developed code
o Immediate “pain” by penalty:
• Violation of known and accepted requirements leads to rejection of transports and an approval process with potentially unpleasant inquiries
We are still working on providing a rewarding option gamification
approach
2323
Rollout of CodeProfiler
o Step 1: integrate CodeProfiler in development environment
o Step 2: activate selected, recognized, expert-developed rules that would block transport release
o Step 3: establish approval instance for deciding about exceptions (architecture team)
o Step 4: activate entire rule set in waves:
• Approx. every 3 months
• Within 12 months, all rules could be activated
2424
Current status of transports and approvals per month
Transports without critical findings: 499
w. Findings: 50Rejected, and then cor-
rected44%
approvals56% (*)
∑ Findings: 87
(*) Reasons for approvals: - Code that we cannot touch- Technical follow-up transports (system copy)- False positives (only in rare cases)
After ~12 months: All new code tested 95% of transports “clean:”
90% good code by developer 5% after rejection and correction
2525
Criteria for testing rules selection
A board of key developers discussed a selection of rules that are activated and can also stop a transport according to the following criteria:
o Effort for correction
o Occurrences of findings in existing code
o Impact of findings
o Personal opinion / experience
2626
How we deal with legacy code
Legacy code often violates more rules than new coding because the rules were not in place when the code was written.
Approach 1: handle legacy code like new code
o Fixes old code when you touch it
o Provokes an “outcry of horror”
o Works if you softly roll out the rules in waves
Approach 2: check code only after a certain creation date
o Makes rollout easier
o Risk: old issues will not be fixed (yet)
We use approach 1
2727
Next steps
o Implement CodeProfiler BW components for management reporting
o Roll out ABAP development tools (ABAP in Eclipse) with CodeProfiler integration allows the earliest feedback (interactively while you write the code)
• ABAP development tools allow seamless integration of further (in-house) tools in the development environment
2828
Evolution of feedback cycles
14 days
x times per week
any time
Competitivetool
CodeProfiler
ABAP Development
Tools withCodeProfiler
2929
Lessons Learned
o Not everyone is happy about the new code quality transparency, but this is a “must” if you want to successfully change and improve
o The work council was involved quickly to address possible employee concerns
o We observed “benchmark optimization” in order to avoid penalties
3030
Lessons Learned
o Start with a small, piecemeal extension of scope:
• Roll out rules in waves
• More and more code will be part of tool-based scans
o Integrate testing tools as efficiently and early as possible in the development lifecycle
o Involve developers to decide about the set of rules
High acceptance in overall developer team
32
Contact:
Email Markus.Theilen@ewe.de
Twitter @therealtier
33
Thank you for your attention.
EWE Aktiengesellschaft
Tirpitzstrasse 3926122 Oldenburg, Germany
T +49 441 4805 - 0
www.ewe.com