11. Deficient Impact analysis and regression testing

Symptoms: Production end users call the help desk to report errors and problems with their system functionality. The SAP production maintenance support group has the responsibility for addressing and resolving the reported help desk problems. The project’s change control board and integration manager do not consider and assess the merits of the production problem, how many man-hours will be needed to address the problem, whether the production problem is scope-creep or consistent with documented requirements, and the total dollar cost to implement a resolution to the problem.  

Furthermore the problem is exacerbated when the SAP production team with limited or no documented analysis implements the software fix and moves it into production without considering how the transported fix will impact previously working system functionality. No end to end regression testing affecting other modules is conducted to ensure that nothing is broken after a fix is implemented even when the fix requires configuration changes.  

Suggestions: Establish a change control board (CCB) to analyze the impact of all system changes triggered as a result of problem reported to the production help desk. The stakeholders from the CCB need to review all proposed changes no matter how trivial or how “minor” for scope, conflict with existing requirements, priority, level of effort, testing needs, release notes, training notes, availability of resources, cost to the project, impact to the project schedule and necessity.  

As an example of impact analysis, after reviewing a reported production problem the CCB may decide that it is not cost-effective to spend 200 billable hours at an average rate of $180/hr to resolve a production problem that requires configuration changes, user exits, etc when the problem is expected to be resolved in a future SAP version that will be released within the next 6 months. 

If the CCB on the other hand decides to implement a solution to a reported problem then the need arises to identify a list of “sunny day” scenarios that should always execute successfully after a configuration change is made. Automate the identified sunny day scenarios and execute these scenarios whenever a production change affecting the system configuration is scheduled to be released into production.