Journal of Knowledge Management Practice, December 2003

Legacy Systems: Must History Repeat Itself?

Dennis Kulonda, University of Central Florida, Mohammed Arif, Carthage College, & Captain Jennifer Luce, United States Air Force

---

ABSTRACT:

Legacy software systems are notorious for repeating the mistakes of the past. However, it is entirely possible that the solutions to problems might be available in their past. Maintaining a Lessons Learned (LL) database and consulting it regularly might result in avoiding potential problems. This research uses surveys as a means to obtain LL and corrective actions taken in legacy systems. It also presents three case studies to demonstrate the utility of LL in legacy systems. This research found that LL are used in legacy systems, however their use is not widespread. When employed, the LL process improves a legacy system’s situation and validates the lessons learned tool utilized by hardware engineering projects. In addition, the research shows that problems facing legacy systems are not isolated, but are replicated throughout industry. Preventive Measures as identified in LL, and applied to new efforts can dramatically affect system life cycle costs but this remains to be verified in subsequent research.

---

1. Introduction

History repeats itself in war, economics, love, and software development projects (Whitten, 1996). Apparently, the fact that mistakes are often repeated in software projects is well known (Luce, 2000). It is also well known that those engineering projects’ mistakes and failures provide invaluable information and insight into management actions that can detrimentally impact a project (Kharbanda and Pinto, 1996). These Lessons Learned (LL) from past projects can be used to avoid future mistakes. The need to use lessons learned has also increased due to growing complexity as well as operational life of software systems (Schwartz, 1996). Documenting LL avoids the “same mistake twice” syndrome. Luce (2000) attributes this syndrome to increasing maintenance costs of software systems. This paper will concentrate on legacy type software systems.

A legacy is an object handed down by an ancestor or predecessor from the past (Myerson, 1996). Refining that definition, legacy software can be defined as a pre-existing software code or an algorithm that is modified and/or extended to meet customer or user requirements rather than using newly developed software (Jarvis and Hayes, 1999). A system that takes into account legacy software along with people, expertise, hardware, data, business processes, and approaches to software maintenance is called a legacy system (Brooke and Ramage, 2001). However, for this research the terms legacy system and legacy software will be used interchangeably.

Maintaining legacy systems could be very expensive. Many experts attribute two-thirds of a legacy system’s life cycle cost to maintenance activities (Schach, 1990). The frustration associated with maintaining legacy systems is apparent from this definition, “A legacy system can be described as any or all of the following: large, old, heavily modified, difficult to maintain, and old fashioned” (Brooke and Ramage, 2001). However, legacy systems are a necessary evil. Defense, government, and space program etc. types of sectors rely on legacy systems due to the specialized nature of their operations. Therefore, research is needed to facilitate the maintenance of legacy systems. This paper documents the results of a survey conducted to investigate major lessons learned and corrective actions taken in the past for maintaining legacy systems. Section 2 presents a review of literature. Section 3 describes the survey methodology for this research. Section 4 summarizes the survey result, section 5 documents three case studies in order to demonstrate the use of LL in legacy systems, and section 6 summarizes this research.

2. Literature Review

Society has become increasingly reliant on software (Luce, 2000). Not only are the number of software systems growing, but they are becoming increasingly complex, with many complicated tasks and functions to perform. There are three main reasons for this increasing complexity of software (Schach, 1990). First of all, there are many different states or modes of operation for each software program. In many cases, all the states cannot be tested individually without dramatically increasing the cost and schedule of software projects. Secondly, the software code in software systems can often be very difficult to understand, especially since many software systems contain more than one software language. Finally, a physical representation of a software product is very difficult to develop and understand, since software is nonphysical and inherently conceptual. Simple flowcharts cannot accurately capture the fluctuations of large software systems. Some of those complex software systems in both the private and government sectors are expected to have a life span between five and ten years (Schwartz, 1996).

Despite thorough testing performed during system development, today’s operational software systems are highly susceptible to bugs, glitches, and viruses that can briefly interrupt or shut down a software system’s mission (Schwartz, 1996). Bugs and glitches otherwise known as errors are often introduced during the system’s development, but not uncovered for many years until specific circumstances leads to activating those errors. As a result, maintaining software-intensive systems can be very expensive and time-consuming (Luce, 2000).

There are three types of systems development processes for software systems (Sneed, 1989). They are:

1. Throwaway system which has an operational lifetime of less than two years and is not upgraded or maintained past user acceptance

2. Static system which receives minimal maintenance and upgrade, usually less than 10% of the system’s size

3. Evolutionary systems which are maintained until they are obsolete and either replaced or discarded

This research concentrates on evolutionary systems that by their very age fit the classification of legacy systems. These evolutionary systems go through iterative repetitions of the software development life cycle with many intermediate states and versions due to maintenance and upgrades. According to Sneed (1989) and Schach (1990) the primary reasons for these continuous upgrades are:

¨ need to correct problems

¨ adapt to new environments

¨ meet new requirements

¨ enhance performance

While performing these upgrades, attention has to be paid in ensuring that installing the enhancement does not negatively impact, or regress, the system’s current operational impact (Luce, 2000). Overlooking this requirement, results in a maintenance nightmare.

Maintenance of legacy systems faces another challenge from upgrades and patches offered by the vendors (Luce, 2000). Some of these upgrades are forced on users due to maintenance agreements. Apart from the constant upgrades some other document issues with legacy systems are (McConnell, 1998):

¨ old technology

¨ changing operational environments

¨ use of older generation languages

¨ criticality to organizational operations

¨ huge initial system development cost

¨ huge system replacement cost

¨ inadequate or no maintenance documentation

¨ lack of skilled maintenance support environment

¨ resource limitations

¨ inadequate software support tools

¨ difficulty in accessing and maintaining system databases

Owing to all the challenges mentioned in the previous paragraph, there is a move away from legacy systems to other alternate environments (Cimitile et. al., 1999; Serrano, Carver and Montes de Oca, 2002). But we will never be able to completely do away with legacy systems. Therefore, we have to identify frequently occurring maintenance issues with legacy systems. Use of LL is one alternative. A “lesson learned is a specific experience and knowledge gained from an analysis of a project’s failure or mistakes with intent to apply the lesson to future projects (Kharbanda and Pinto, 1996). Applying LL to other projects can minimize the chance of repeating the exact same or similar mistake, which impacts schedule and the cost of the project (Whitten, 1990).

However, implementing a LL program can be labor intensive (Luce, 2000). Therefore, many companies rely on experienced team members to contribute to their LL program. But this approach can mean disaster for the company if this team member leaves. As far as formal LL programs are concerned there are computerized LL systems available (Luce, 2000). Some of the new software development methods have incorporated LL and made the systems development process dynamic or never-ending, constantly improvising based on LL (Davies and Williams, 2003). But despite all these efforts, there is no research that documents major issues and root causes of LL. No data is available on the extent of usage for LL systems in industry either. This paper documents that information based on the survey results of legacy system users and maintainers. The following section describes the methodology followed to perform the survey.

3. Proposed Research

The first sub-section below describes the initial intended methodology for this research. The second section describes some risks associated with this research and potential alterations in the methodology done in order to avoid these risks.

3.1. Research Method

For the purpose of this research, professionals with experience in developing, supporting, and maintaining legacy software systems will be interviewed. The professionals interviewed can either perform engineering or managerial functions. The professionals interviewed will consist of configuration managers, logistics managers, test engineers, software engineers, systems engineers, and hardware engineers. Since this research is an attempt to validate a management tool that can be applied to legacy systems, the interview questions will not require in-depth technical explanations. Instead, the survey will attempt to gather enough information to get the general context and situation surrounding the LL, as well as the methodology for obtaining them. The interview will consist of eight questions summarized in Table 1. The interview will be conducted individually, one subject at a time. There will be no time limit on the interview.

Table 1: Interview Questions for the Survey (Luce, 2000)

Question #	Question
1	Have you ever been involved in program oversight or management of a legacy software system or of a program utilizing legacy software? If the answer is “yes” please answer the following questions:
2	What were the types(s), if any problem(s) did that system face? Please be as specific as possible about the details surrounding the problem.
3	Could any actions have been taken to prevent the problem(s) identified in question 2?
4	Was experience gained or lessons learned from another program applied to resolving the problem(s) identified in question 2?
5	If the answer to question 4 is “yes” how were the lessons learned used, obtained?
6	If the answer to question 4 is “yes” what were the impacts of applying the lessons learned? Please be as specific as possible.
7	Did your legacy system have a process for identifying, analyzing, and disseminating, capturing lessons learned?
8	If so, please specify the process and methodologies of the lessons learned system.

Apart from the interview result this research will also examine the advantages of incorporating LL through short case studies of systems, where different levels of LL have been implemented. However, this process of collecting data is not without risks. The following section summarizes the risks associated with the data collection process and proposes some mitigating actions taken to avoid problems.

3.2. Risks And Mitigating Actions

The main issues, or risks associated with this research consist of the wide range of management processes. The success of management methodologies depends in large part upon situation. Leadership styles, technical expertise, and team interaction can vary widely across industry and even within individual organizations. Some other risks associated with this research include subjects not remembering all the details, facts, and circumstances surrounding problems that impacted their legacy systems, personal bias, etc. All the identifiable risks were addressed in this research using a list of mitigating actions tabulated in Table 2.

Table 2: Identified Risks and Potential Mitigating Actions (Luce 2000)

Risk	Potential Mitigating Actions
“One-sided view” of legacy systems	Ensure as many functional areas involved with legacy systems are addressed
Not asking the right questions	Perform interviews on 2 test subjects and then adjust questions as necessary for the rest of the subjects
Incomplete interview data	Let subjects take time to answer questions
	Request back-up data if needed
	Utilize follow-up interview process in needed
Inaccurate interview data	Request back-up data
	Provide questions in advance in order to let the subject recollect what happened in the past
	Utilize follow-up interview process
Subject has limited experience with legacy systems	Do not complete the interview and reject the data
Limited sample size	Interview at least 25 people
Misunderstanding of data provided	Ask subject of understandable details
	Let subjects take time to answer questions
	Utilize follow-up interview process
	Perform thorough review of the interview results

The end product of this research will be a list of activities a manager can utilize when facing a particular situation in a legacy system. This list can serve as an initial checklist for project planning and provide a general sense of the activities required for problem resolution. However, due to the subjective nature of project management, strict adherence to this list might not be possible but it still can be used as a starting point. In addition this research will use case studies to demonstrate the usefulness of utilizing LL. The following section presents an analysis of the data obtained from this survey

4. Data Analysis

This section is further divided into three sub-sections. The first sub-section summarizes the list of problems facing the legacy system maintainers. The second sub-section contains the list of actions taken to address these problems. The third sub-section summarizes the data on the extent of usage of LL in today’s legacy systems.

4.1. Problems Facing Today’s Legacy Systems

Thirty nine professionals were interviewed for the purpose of this research. The interview subjects identified 139 problem instances facing legacy systems. In almost all cases, multiple responses by the subjects were provided. The problems were divided into ten logical categories: configuration management (CM), cost, documentation, personnel issues, customer requirements, system design, systems engineering (SE), test, and vendor products. Appendix A lists the 139 problems and their respective number of occurrences identified in this research. Table 3 summarizes the top three most frequently encountered problems and Figure 1 summarizes information from Appendix A.

Table 3: Top 3 Problems in Legacy Systems

Problems	# of Occurrences
Component obsolescence	11
Inadequate documentation	9
Hard to fix system problems	8

Figure 1: Legacy System Problems by Category

As documented in Table 3, the most frequently occurring problem was component obsolescence. As the legacy system evolves, system components face obsolescence due to emerging technologies or simply a vendor going out of business. The second most frequently occurring problem was inadequate documentation. Discipline is needed at the time of development in order to ensure that processes, steps, system designs, user instructions, and maintenance instructions are completely documented. If documentation is not complete or accurate, future upgrades of the system will be affected because either people will not be able to figure out some aspect of the system or end up making a wrong assumption somewhere. The third most frequently discussed problem in our survey was hard to fix system problems. There is definitely a possibility that some of these problems become hard to fix or even hard to identify due to inadequate documentation.

As depicted in Figure 1, the highest number of responses suggested system design to be an issue. Owing to the evolutionary nature of the legacy systems, this response is not surprising. Since a legacy system is developed over a long period of time as and when the need is identified, issues with design creep into the system. This problem is further compounded by lack of documentation, which also happens to be the third largest category. The second highest causes of problems are the vendors. Sometimes the customer is forced to upgrade the system even though the current version meets all the requirements. Another major issue with vendors is them going out of business. In that case, the customer may be forced to replace the vendor product since the product is no longer supported. The following sub-section summarizes the recommended corrective actions suggested by subjects during their interview.

4.2. Recommended Actions To Correct Those Problems

The interview subjects identified a total of 103 actions that when carefully implemented help mitigate or reduce the impact of the problems identified in the previous section. The complete list of actions is documented in Appendix B. Table 4 identifies the most frequently recommended efforts that were identified by the subjects, while Figure 2 displays the suggested activities within the ten individual categories identified in the previous subsection.

Table 4: Top 3 Common Recommended Actions

Action	# of Occurrences
Perform upgrades on a regular basis	9
Disciplined SE process	7
Develop/manage a clear system design	6
Document a clear design	6

Figure 2: Recommended Actions by Category

The most common recommended action was the performance of regular upgrades. Subjects were of the view that systems should be upgraded regularly in order to avoid a situation such as component obsolescence when an upgrade is forced on the company. Regular upgrades will also ensure use of the latest and most efficient technology in the system. The second most frequently recommended action was a disciplined software engineering process. A disciplined process includes actions like proper documentation, maintenance input during design, management’s input on technical issues, and proper interface definition between components of the system. There were two actions tied for third place. The first action was the development of a clear system design. This recommendation means that more effort has to be invested into designing the system and visualizing future upgrades. This vision has to be properly documented in order for future system users and maintainers to follow. The last recommended action was clear design documentation. A major effort and expense at the time of upgrade or for the purposes of maintenance of the system is needed in order to understand the current system. This effort can be significantly reduced if the initial documentation is thorough and clearly written, and any future upgrades are also properly documented.

Figure 2 displays the suggested activities within the ten individual categories identified in the previous subsection. The highest numbers of actions were recommended for the system design category. This category included actions like development and management of clear system design, utilization and enforcement of a disciplined design process etc. The second place was shared by SE, vendor products, and documentation. SE included actions like utilization and enforcement of a disciplined SE process, ensuring that maintainability is a part of the SE process etc. The recommendations on vendor product included, performance of version upgrades on a regular basis, addressing component obsolescence regularly, etc. Recommendations on documentation included clear design documentation, updating documents regularly, and defining a minimum document set. The third place was taken by personnel-related actions. The recommendations in this area included making the user a key player in development and maintenance activities, ensuring the availability of system experts late in the life cycle of the legacy system, etc. The next subsection summarizes the data on the extent of usage of LL in today’s legacy systems.

4.3. Lessons Learned And Today’s Legacy Systems

Five out of the eight interview questions deal with LL and their usage in the development, management, and support of today’s legacy systems. The questions asked were if LL were used to resolve problems, where they were obtained, the impact seen after using those LL, whether those legacy systems utilized a LL process, and details about that process. Table 5 summarizes the interview results in regards to those subjects who used LL to resolve problems and those who employed a LL process.

Table 5: LL Summary

	LL Process	No LL Process	Subtotals
LL not utilized to resolve problems	7	12	19
LL utilized to resolve problems	16	4	20
Subtotals	23	16	39

Twenty respondents indicated that LL helped address problems facing their systems, while nineteen subjects denied the use of LL process. In all twenty positive cases, the subjects declared that the situation facing the legacy systems improved. The problems were either resolved or became more manageable after the application of the LL. Only twenty three of the interview subjects said that their legacy systems used a process to identify, analyze, and disseminate LL, while sixteen respondents said no LL process was even implemented. Early in the interviews, the subjects described two types of LL processes: formal and informal. For the formal process, the program’s leadership or customer dictates and leads the LL process. LL are documented and updated on a regular basis, as mandated by the customer or management. The informal process arises spontaneously to resolve problems. A team often led the latter process and the results were sometimes briefed to the program’s leadership. However, this LL may not be updated on a regular basis. Seventeen respondents participated in a formal process, while seven stated that an informal process was used. The next section describes three case studies, each describing a different level of use of the LL process and the impact on the system.

5. Case Studies

There are three case studies presented in this section taken from Luce (2000). The first case provides an example of a legacy system that did not use LL from other programs to mitigate a potential problem. The second case demonstrates a legacy system that utilized a LL process from initial program development throughout its life cycle. The last description shows that a LL process can be successfully incorporated into the management processes of legacy systems.

5.1. Case 1

This case was mentioned by one of the subjects during the interview. The case deals with United States Air Force’s (USAF) Level C trainer for the Joint Surveillance Target and Attack Radar System (JSTARS) based on the Boeing 707. A contractor was awarded the contract to build a JSTARS trainer with one constraint, which was developed as a result of system’s acquisition strategy. The contractor was required to model JSATRS by reusing KC-135 trainer software. The contactor was provided both the KC-135 legacy code, written in an old version of Fortran, and the hardware as government furnished equipment. USAF gave the approval to modify hardware in order to accurately emulate JSATRS aircraft. The documentation provided to the contractor included draft versions of drawings that were later changed during the aircraft’s development phase. Final drawings or revisions to the draft documents were not provided to the contactor in a timely fashion.

However, the acquisition strategy of the USAF was based on a flawed trade study performed by a contractor. The reuse of KC-135 legacy code was recommended because both the KC-135 and Boeing 707 appeared to be similar planes. But in this project the code proved to be non-portable to the new hardware. In addition, the software did not mirror the performance of the JSTARS plane, potentially impacting the ability to get Federal Aviation Administration certification of the trainer. The contractor was forced to rewrite the legacy code. This resulted in schedule and cost overruns. Eventually the contractor was fired and a new contractor was hired to complete the project.

For this case, the underlying assumption of the portability of the legacy code was wrong. The consultant who wrote the acquisition strategy did not review the KC-135 trainer software for feasibility and practicality of reuse, nor did he identify the configuration as well as the operating differences between the JSATRS and KC-135. In addition, the dismal rate of hardware design documentation was not clearly communicated to the strategy decision makers. LL from similar projects were not reviewed during the system strategy development phase. This could have also prevented some of the problems encountered. The subject who described this case did mention that towards the end of the project a formal LL process was instituted.

5.2. Case 2

The UHF Follow-On (UFO) Satellite program consisted of the design, development, fielding, and phasing out of ten United States Navy communication satellites. Hughes Space and Communications was required to design and build all aspects of the two billion dollar program including the payloads, the bus, and the ground system. The software used was written in C++ and Ada. Originally a new development, the UFO program became a legacy system due to the program’s long procurement cycle.

The contractor instituted a LL process early in the design stage of the program for several reasons. First of all, the design and development phases for ten individual satellites can be quite long and non-concurrent, depending upon funding and approved schedules. The other major issue was team members leaving the project before its end. Dynamic movement of program personnel can cause problems as experience with and reasons behind design decisions are forgotten and lost to the remaining team members. In addition, each satellite was designed, built, and tested by different teams. Those individual teams took great pride in not making the same mistakes as another team.

For the UFO project, LL were analyzed and compiled for every major review during each phase of satellite’s life cycle using a Microsoft Access database. The database was set in order to facilitate querying using the program phase or keyword search and a quality assurance person was designated to ensure people used as well as updated this database. Early into the UFO program, the contractor teams discovered that reviewing LL at the beginning of each program phase for each satellite significantly reduced mistakes made in that phase. At the end of each phase, LL were once again identified and placed into the database.

The LL process used on the UFO program helped resolve a problem facing the program’s legacy software. In order to eliminate and reduce the impacts of component obsolescence, the contractor was forced to upgrade the operating system and database capabilities on a regular basis, potentially causing modifications to the legacy system. If the vendor products were not updated, software tools and support would no longer be available to support required application software upgrades. If the contractor let the vendor products become obsolete, the entire system might have to be thrown away or require very expensive overhaul. Therefore, four steps were identified to deal with component obsolescence in the LL process. The steps were: 1] Perform yearly review of all software releases; 2] Verify that the software release can run on the latest version of the operating system and database; 3] Verify the source code can be edited and compiled upon the latest release of software development tools; and 4] Verify that the software performs on the latest hardware releases.

The program has been a success and the contractor was able to incorporate evolving requirements to meet the changing needs of the customer over the life of the program. In summary, the LL process employed by the UFO contractor allowed for problem resolution across all satellite development teams. Issues could be easily tracked, prevented, and resolved across multiple satellites, reducing the chances of repeating the same mistake twice.

5.3. Case 3

The third case is about Boeing’s Payload Ground Operations Contract (PGOC). PGOC is tasked with sustaining all facility systems as ground equipment for the National Aeronautics and Space Administration (NASA) at Kennedy Space Center (KSC), Florida and Vandenberg Air Force Base, California. Annually, PGOC’s thirty million-dollar budget funds approximately 180 different projects. Many of PGOC’s projects involve legacy software systems, and its overall mission is to successfully manage sustaining programs to meet user requirements, schedules, and budgets. PGOC’s sustaining engineering program provides three basic categories of on-going support to NASA systems. First of all, sustaining engineering must maintain a design that fulfills its original design intent and is compatible with intended operational use. Second, product improvement efforts intending to create cost-effective operations may require upgrades in operational performance capabilities. The majority of the work on the contract falls into that category. Finally PGOC’s sustaining engineers may be tasked to incorporate approved requirement changes to existing support equipment. Contractual products and services include engineering cost estimates for engineering support requests, design and development support along with corresponding documentation updates, configuration management for all support equipment, troubleshooting, problem analysis, problem resolution, and operations personnel training.

Most of the equipment sustained by PGOC can be considered legacy equipment. Over a six-year period, the PGOC team learned that sustaining and incorporating changes to the legacy and non-legacy equipment required a disciplined process, which incorporated the collection, analysis, and dissemination of LL across the program. This lesson is enforced by the fact that many of PGOC’s customers set schedule dates that have to be met at any cost. Since all the projects encountered by the PGOC team are different, repetitive mistakes are quite few. Therefore, the contactor team concentrates its effort on improving processes. A major priority is to get the job done right at the right time. The PGOC team has not yet missed a deadline. The disciplined process enforced across all PGOC projects enhances Boeing’s ability to successfully address legacy software system issues in the required timeframe.

Two problems do face the PGOC program on a regular basis. Vacillating requirements are the first problem. Customers tend to think in terms of solutions, not specific requirements. In addition, some requirements originally not provided to the team but are needed to complete the task uncovered on a regular basis. One methodology “learned,” carefully defining a project’s scope and requirements hand-in-hand with the user, reduces the occurrences of both those problems.

LL are gathered at each project’s review. The PGOC team welcomes the customer’s input during the review. After the project review, the team’s processes may require adjustments in order to optimize the contract’s performance. Metrics, tracking measures like cost performance, are also reviewed for potential process improvements.

6. Summary and Conclusions

This research was intended to study LL for legacy systems. On performing the literature review abundant information on LL in traditional engineering projects was found. However, for LL in legacy software systems very little research results were found. Therefore, data was collected by surveying managers and engineers dealing with legacy systems. The survey was designed in order to give subjects the freedom to express themselves in their own words. During the survey some of the subjects described in detail the situation in their organizations. The three case studies presented earlier are the result of description provided by those subjects.

Originally, this research focused on legacy software systems in the maintenance phase of its life cycle. However, legacy systems are not always in the maintenance phase. The reuse of a portion of legacy software can be quite challenging as Case 1 showed. According to the initial strategy, Case 1 was a legacy system in a modified development phase. However, as problems arose from the reuse of the legacy code, the program eventually transitioned to a new development effort. Another lesson that became obvious early in this research is that mistakes made in the development phase, such as contracting and funding problems, can haunt legacy systems in the maintenance phase. For example, funding cuts may force the cancellation of documentation development in the design phase of a software-intensive system. In many cases, maintenance of that system can be quite difficult without that documentation, especially if the maintainers are not the system developers. Therefore, system maintainers are often forced to deal with problems they didn’t cause.

Based on the results of the interviews, LL are being used on some legacy systems. However, their use is not widespread. The results of the interviews show that a little more than half of the systems discussed employed LL and obtained visible improvements in system’s situation. PGOC and UFO case studies show that LL can help lead projects supporting legacy systems, to succeed on a consistent basis.

Case study 3 provides evidence that legacy systems that are developed and maintained by the same organization through the use of a long-term contract obtain the greatest benefit from using LL as a tool. This can be attributed to the fact that this organization will have to live with the consequences of its own work.

In addition, 139 problems were identified and placed into ten major categories which are: CM, cost, documentation, personnel issues, requirements, system design, SE, system upgrades and updates, test, and vendor products. 103 recommended activities were listed to address the identified problems. For example, disciplined SE and design practices can be used to guard against system design problems, the number one problem area identified. The lists provided in Appendices A and B respectively, can be used by a lower level manager of a legacy system as a starting point for problems. However, this list should not be considered as exhaustive. The subjectivity of managing projects as well as the sample size dictates that the lists cannot be taken literally. Those using the list, like all project managers, will have to review their situation carefully prior to implementing them. Like the research subjects who utilized LL on a regular basis state, reviews of previous experiences should only be considered as starting points, or baselines for initial action. Baselines are changeable and should be adapted as improvements are sought and uncovered.

The problems identified and handled by the research subjects were not just minor problems. Some problems listed such as lack of documentation or product obsolescence can bring the project to a halt. Those two problems, alone or together, can make maintenance difficult or impossible. The subjects took creative measures, in most cases, to address or correct those problems. The subjects gained valuable experience that they passed on to the surveyor for this research. This is precisely the transfer of knowledge across stages of the system life cycle that captures the benefits of LL. After all, today’s new system projects are tomorrow’s legacy systems. However, documenting the economic value of problem avoidance is like cost avoidance, easy to assert, difficult to prove. Yet, the scope of the issues identified in this research suggests that the relatively small cost of information collection and dissemination in the LL database is a worthwhile use of a company’s time and resources.

As shown in the case studies, a LL process combined with a disciplined design and engineering process can help resolve problems facing legacy systems. In fact, some problems occur only once or not at all, as seen in last two cases. The two subjects who described the cases 2 and 3 respectively took great pride in the fact that their legacy systems were success stories. They both recognized that mistakes and problems still could happen, but that their project team is ready to face and overcome them.

7. References

Brooke, C., and Ramage, M., (2001), “Organizational Scenarios and Legacy Systems”, International Journal of Information Management, 21, 365-384.

Cimitile, A., Lucia, A.D., Di Lucca, G.A., and Fasolino, A.R., (1999), “Identifying Objects in Legacy Systems Using Design Metrics”, The Journal of Systems and Software, 44, 199-211.

Davies, P.B., and Williams, M.D., (2003), “The Diffusion of Information Systems Development Methods”, Journal of Strategic Information Systems, 12, 29-46.

Jarvis, A., and Hayes, L., (1999), “Dare to Be Excellent”, Prentice Hall, Upper Saddle River, NJ.

Kharbanda, O.P., and Pinto, J.K., (1996), “What Made Gertie Gallop? Lessons from Project Failures”, Van Nostrand Reinhold, New York, NY.

Luce, J.L., (2000), “Application of Lessons Learned Into Legacy Software Systems”, Unpublished Masters Thesis, Department of Industrial Engineering and Management Systems, University of Central Florida, Orlando, FL.

Myerson, M., (1996), “Risk Management Processes for Software Engineering Models”, Artech House, Boston, MA.

McConnell, S., (1998), “Software Project Survival Guide”, Microsoft Press, Redmond, WA.

Schach, S.R., (1990), “Software Engineering”, Irwin, Homewood, IL.

Serrano, M.A., Carver, D.L., and Montes de Oca, C., (2002), “Reengineering Legacy Systems for Distributed Environments”, The Journal of Systems and Software, 64, 37-55.

Shwartz, E.I., (1996), “Trust Me, I’m Your Software”, Discover, 17(5), 78-81.

Sneed, H., (1989), “Software Engineering Management”, Ellis Harwood Limited, Chichester, England.

Whitten, N., (1996), “Managing Software Development Projects: Formula for Success”, John Wiley & Sons, Inc., New York, NY.

8. Appendices

Appendix A: Legacy System Problems Identified by Major Categories (Luce, 2000)

Category	# of Problems
1. Configuration Management (CM)	11
Poor version control of software development files	1
Poor CM	5
Initial CM system not up-to-date or understood	1
Problems implementing CM	1
No CM	2
Lack of CM for vendor products	1
2. Cost	4
Low budgets	3
High upgrade costs	1
3. Documentation	17
Lack of documentation	5
Inadequate documentation	9
Out-of-date documentation	1
Paperwork intensive to make changes to the systems	2
4. Personnel	16
Multi-players involved in decision making	3
Lack of managerial involvement in technical planning	1
Poor transference of organizational responsibility	1
Poor training for maintainers and operators	1
Lack of adequately trained support personnel	4
Poor project management	1
User “tweaking” of the system	2
No/Limited maintainers viewpoint during development	2
Single knowledgeable person	1
5. Requirements	12
Poor understanding of system requirements	1
Requirements creep	6
Lack of well-defined requirements	3
Legacy code no longer meets requirements of the system	1
Difficulty in responding to changing requirements	1
6. System Design	38
Poor system design	5
Errors inherent in original software code	3
System instability	2
Lack of consistency within the software performing similar functions	2
Lack of software portability	3
System designed with a prototype mentality	1
Computer/component resource limitations	4
No upward compatible software components	1
Database issues	3
Poor understanding of system functions	3
User unfriendly	1
Category	# of Problems
Hard to fix system problems	8
Poor system integration	1
Inadequate security measures	1
System not Y2K compliant	2
Interface compatibility difficult	1
Multiple languages	1
7. Software Engineering (SE)	2
Poor SE process	1
Poor interface definition process	1
8. System upgrades	12
Lack of planned upgrades	3
Problems introduced post-upgrade	6
Difficulty in installation of upgrades	3
9. Test	6
Improper testing	1
Poor testing control and processes	1
Inadequate test articles	2
Rigid acceptance test procedures	2
10. Vendor Products	21
Component obsolescence	11
Minimal backward compatibility	1
Requires numerous upgrades to product/system software	2
Proprietary components	2
Lack of vendor support	4
Total number of problems identified	139

Appendix B: List of Actions Suggested by Subjects (Luce, 2000)

Categories	# of Occurrences
1. CM	9
Enforce CM within the project	5
Clearly define CM procedures and processes	4
2. Cost	3
Preplan for life cycle costs	2
Perform trade-offs when budgets are reduced	1
3. Documentation	15
Document clear design	6
Keep documentation up-to-date	5
Make user documentation available to supporting contractors	1
Define a minimum documentation set	3
4. Personnel	11
Educate management	1
User key player in development and maintenance activities	4
Hire experienced programmers	1
Allow legacy system employees to participate in development of replacement systems	1
Ensure some system experts are available late in the life cycle for critical functions	2
Improve training process	1
Do not allow users to write code	1
5. Requirements	7
Perform a thorough requirements analysis process	2
Ensure requirements are validated by the user	1
Take the time to write well-defined, meaningful requirements	2
Focus on mission priorities	2
6. System Design	17
Develop and manage a clear system design	6
Utilize and enforce a disciplined design process	5
Design for system robustness	1
Document “tribal” design knowledge of maintainers on a regular basis	1
Ensure potential for system migration is included in design decisions	2
Encapsulate legacy components into a more modern design	1
7. SE	14
Utilize and enforce is disciplined SE process	7
Ensure maintainability is part of the SE process	3
Ensure SE includes risk management, technical performance monitoring, and concurrency management	2
Define a sound interface management process early in system development	1
Define a sound failure analysis process for problem identification	1
8. System Upgrades/Updates	4
Ensure upgrades/updates are properly tested	1
Perform annual reviews of planned releases to ensure compatibility	1
Validate requirement for system upgrades/updates	1
Develop pre-contingency plans	1
9. Test	8
Define a complete and realistic test process	3
Ensure test equipment accurately emulates the system	2
Utilize a validated data test set	2
Test release thoroughly before installing	1
10. Vendor Products	15
Address component obsolescence regularly	1
Perform version upgrades on a regular basis	9
Continue with maintenance agreements	1
Don’t buy state of the art product	1
Tie system to standards or practices, not specific vendor products	1
Make vendor part of the team for critical items	1
Total number of preventative actions	103

---

About the Authors:

Dr. Dennis J. Kulonda is an Associate Professor of Engineering Management at the University of Central Florida. He is a former industry consultant from Arista Manufacturing Systems and a former partner with Operations Associate …

Dr. Dennis J. Kulonda, Associate Professor, Department of Industrial Engineering and Management Systems, University of Central Florida, 4000 Central Florida Blvd, Orlando, FL (USA) 32816; Phone: (407) 823 5568; Fax: (407) 823 3413; Email: dkulonda@mail.ucf.edu

Dr. Mohammed Arif is an Assistant Professor of Business Administration at Carthage College, Kenosha, Wisconsin. He obtained his Ph.D. and M.S from the University of Central Florida (UCF) in Industrial Engineering and his B.S from Jamia Millia Islamia, New Delhi, India in Mechanical Engineering …

Dr. Mohammed Arif, Assistant Professor of Business Administration, Carthage College, 2001 Alford Park Drive, Kenosha, WI (USA) 53140; Phone: (262) 551 5848; Fax: (262) 551 6208; E-mail: marif@carthage.edu

Captain Jennifer Luce serves as a National Systems Plans and Programs Officer for the United States Air Force (USAF) in the Washington D.C. area. She received a B.S. in Mathematics from Baylor University in 1993 and a M.S. in Engineering Management from the University of Central Florida in 2000 …

Captain Jennifer Luce, United States Air Force; Email: jennifer.luce@nro.mil

---