Journal of Knowledge Management Practice, Vol. 10, No. 1, March 2009

Lessons Learned: A Practical Approach

Ofer Pharhi, PMP

ABSTRACT:

The purpose of this paper is is to give a practical outline of the Lessons Learned process: How it should be applied and when; what should be included in the LL process; organizational and personal issues, and the rules of storing and retrieving the lessons that were learned in a practical and “use it” manner

Keywords: Lessons learned, Process application, Issues, Storage, Retrieval


1.         Introduction

The study of human knowledge has always fascinated curious amateurs and professional researchers alike. The significance of knowledge has been emphasized by professionals in many a field, from classical philosophy, to the exact sciences world, to pedagogy and education.

In recent years, a new field of research has emerged around improving our human knowledge in respect to repetitive missions, projects that we undertake: Knowledge Management. Organizations and corporations are beginning to realize that managing their information is a key factor to success in all dimensions – financial, business, operational and organizationally. Thus arose a variety of methods, systems and curricula regarding this emerging topic, and many organizations are making every effort to transform themselves into a learning entity that uses its knowledge efficiently.

In project-oriented organizations, knowledge management consists mainly of taking information from past projects and putting it to use in upcoming projects. In the methodology of project management, this procedure is defined as the “Lessons Learned” process (LL). This is done by extracting the relevant information (or epistemologically    speaking – turning hidden knowledge to available knowledge) and storing it for future use and analysis.

The purpose of this paper is not merely academic. Its goal is not to give a theoretical view of the process, but to give a practical outline of the lessons learned process: How it should be applied and when; what should be included in the LL process; organizational and personal issues, and the rules of storing and retrieving the lessons that were learned in a practical and “use it” manner.

While the context of this paper is project management, it should be noted that the process described here can be applied to other fields as well. At times, it will be productive to give the process a different name (such as "event analysis" or "guided learning"). But as we shall see later, “lessons learned” is the most appropriate term when dealing with project management.

2.         Lessons Learned As A Concept

As a concept lesson learned might well be defined as learning gained from the process of performing a project; however, some additional clarifications should be added in order to really understand what the lessons learning process is all about:

¨      It should be noted that the learning process covers both positive events (a milestone completed, products delivered to customers etc.), and negative events (delays in schedule, failing to meet product specifications etc.).

¨      Every project has end-products which define the nature of the project. However, projects also have “internal products” which are defined by the performing organization itself, such as project documentation, process optimization, historical data etc. An organization whose managers are committed to learning and improvement will also have the lessons learned summary document as an internal product of the project.

¨      A formal and orderly LL process is an essential part of releasing the team when the project is completed. Those who take part in such a process, often see it as the final wrap-up of their work on the project.

¨      Projects which terminate prior to completion are not exempt from the LL process. Indeed, in such projects the LL process may be even more important than in “regular” ones.

¨      The responsibility for the LL process lies on the project manager. However, the commitment of the organization's management is a critical and predominant factor to this process and to ensuring that it will lead to effective results.

3.         Lessons Learned As A Process

The main goal of the LL process is to provide information that will enhance the efficiency of future projects. The information gathered during the project is held, in bits and pieces, in the hands of the project's team members. These people, as individuals, cannot transform the information they have into the desired useful knowledge needed by the process. In many cases, especially in the software development world, the team members leave the organization upon the completion of the project, leading to possible loss of critical information when the project is complete.

It is known that people tend to feel uneasy towards the LL process. The terminology is also a factor here; for example, the term “Investigation” would make people even more uneasy in being part of the LL process. Here, the organization should pick a term which is well accepted organization-wide.

Never-the-less, and regardless of its name, there are several factors which contribute to this uneasy feeling:

¨      Psychological reasons: People don't want to be a part of the LL process, because they fear of being blamed for mistakes and errors which occurred during the work on the project.

¨      Political reasons: People have a tendency to avoid situations which might damage their position (or their team's) in the organization. Nobody wants to have his weaknesses brought up to the spotlight, freely shared and recorded for all to see, as this may damage the person's reputation and impair the ability to deal with the political reality within the organization.

¨      Cultural reasons: There are certain cultures which heed to the motto of “what's done is done”. Such people are not accustomed to speak of past failures or to learn from them.

An organization which systematizes the LL process and realizes that the focus of the process is learning rather than blaming, may reduce this resistance by educating and preparing all involved, to include full support and commitment of management. It's particularly important to explain that mistakes are an opportunity to learn, and that the LL process is not about pointing a finger to one person or another for these mistakes.

It is very important to formalize the LL process by creating an orderly organizational methodology, appointing the people who'll supervise the process, tracking recommendations and their application, and ensuring the commitment of the management to the process. An informal and disorderly process will result in an inefficient use of the organization's learning ability and future use of the knowledge. The process itself, as a process, may improve and empower the organization's general ability to learn.

4.         The Lessons Learned Process

As stated earlier, the goal of this paper is to provide a practical outline for the structure of the LL process. Such a process is built upon the following steps:

1.      Planning the LL process

2.      Creating and distributing LL questionnaires

3.      Collecting the LL questionnaires and preparing a lessons learned meeting

4.      The lessons learned meeting

5.      The lessons learned summary document

6.      Monitoring

4.1.      Planning The Lessons Learned Process

The LL process is one of the roles and responsibilities of the project manager. Planning the LL process should include setting goals for the process, appointing a moderator, notifying the team members, and allocating tasks for the next steps of the LL process into the project's master schedule.

The goals of the LL process will be usually organization-wide in nature. Such goals may include statements as:

¨      Documenting the duration/resources that was needed to accomplish various tasks in the project.

¨      Analyzing events that occurred during the project.

¨      Providing knowledge tools for future projects in the organization

¨      Finding the cause of errors and setbacks that occurred during the project.

¨      etc.

Choosing a moderator from outside the project team is recommended for the purpose of leading the LL meeting. Such a moderator is someone with no personal interest in the outcomes of the project and should be uninvolved with the project's team in order to provide an unbiased point-of-view that will help the team get the most from the process. It is common to appoint another project's manager (from the same organization) as the moderator. Another common choice for moderator is a PMO representative, when there is such office. The project manager will build a process frame for the moderator, and will provide information regarding the process' goals and the schedule set for next steps.

In the opening meeting of the project team, typically at project kickoff, the project manager will inform the team about the LL process, and will explain the importance and goals of this process. The project manager will ask everyone to carefully document – while the project is running – any event which they deem useful as input to the LL process.

When planning the project's schedule, the project manager will allocate time for filling in the questionnaires, planning and executing the lessons learned meeting, and creating the summary document for the process.

4.2.      Creating And Distributing LL Questionnaire

At the heart of the lessons learned meeting, is the analysis of questionnaires filled by the project's team members. These questionnaires will cover all the phases in the life cycle of the project (initiation, planning, execution, monitoring and closure), as well as the knowledge areas of the project (integration, scope, time, risks, communications, etc.). There should also be questions regarding the technical details of the specific project at hand.

For an organization which is already well-trained in the LL process, there should be templates of such questionnaires. At the very least, there will be example questionnaires from previous projects. Constructing the questionnaire is the responsibility of the project manager, although it is perfectly acceptable to assign other team members to assist with this task. The final version of the questionnaire will be distributed only after the moderator approves its content.

The project manager will distribute the questionnaires during the late phases of the project. It should be done before the official closure phase, yet after most of the project deliverables are completed, with the memory of the major project events still fresh in the team's mind. The project manager will ask his team to completed the questionnaires, and emphasize that they are to be filled anonymously. This should decrease the natural reluctance most people have towards the LL process. Finally, a definite deadline for handing in the questionnaires should be set. Without such a deadline, it is likely that most team members simply won't fill them at all.

A complete example of such a questionnaire is outside the scope of this paper. However, example questions and a proposed structure for such a questionnaire are provided in Figure 1 below:

Text Box:    Part I – The Planning Phase
	Agree                      Disagree
 Was the project scope clear to the team members?						
 Did the team members actively participate in the estimation process?						
 Did you feel that your opinion was heard in the planning phase?						
 Did you state your opinion regarding the construction of the WBS?						
.
.
.
   Part II – The Execution Phase
	Agree                      Disagree
 Were the project meetings held in adequate intervals?						
 Were progress indicator models used?						
 Were changes in the project adequately controlled?						
 Was the acceptance process after each milestone handled properly?						
.
.
.
    Part VI – Miscellaneous
     Please describe any additional events that occurred during the project, which you deem worthy of being  
     discussed in the lessons learned meeting:

     ________________________________________________________________________

     ________________________________________________________________________

     ________________________________________________________________________
Figure 1: Example Of Questionnaire

4.3.      Collecting The Questionnaires And Preparing The Meeting

It is the project manager's responsibility to schedule the date and time of the lessons learned meeting. Before this meeting is held, the project manager will make sure that all team members had adequately completed the questionnaires, and will start the process of gathering the data into a coherent workable document.

Most of the task of analyzing the completed questionnaires should be the responsibility of the project manager. In some organizations, however, it is acceptable for the LL moderator to help the project manager.

The task of assembling and sorting the data from the questionnaires includes:

¨      Creating a list of the most common issues that arose from the questionnaires

¨      A primary analysis of the reasons for these issues, according to the questionnaires

¨      Extracting additional discussion topics from the answers given to open questions in the questionnaires

¨      Gathering all the information which may help the evaluation process of future projects

¨      Looking for bottlenecks in the execution processes of the project

¨      Gathering data which may give clues to proper risk mitigation strategies in future projects

After gathering the relevant information the project manager will create the schedule for the meeting itself. For example, it may be desirable to allocate time for:

¨      Discussing how the project products were accepted by the customer

¨      Analyzing progress indicators (the most common one being the EV)

¨      Analyzing the project management methodology and the actual execution of the project

¨      Discussing interpersonal, team and organizational issuesAny other topic which the project manager or the moderator finds worthy of discussing

The LL meeting should occur after the customer officially accepted the project's final product. This will allow for the reaction of the customer, as well as the closing processes of the project, to be taken into account. All team members should be invited to this meeting and notified when it will take place. The time scheduled for the meeting, of-course, should be allocated beforehand, as an inseparable part of the project's schedule.

The project manager and the moderator may wish to (and usually do) invite people outside the core project team to the LL meeting. It is common practice to invite people such as the project's sponsor, managers of other projects within the organization, technical support staff and PMO representatives. Such participants can have the status of passive observers, but they are usually asked to fulfill a more active role and state their opinions about the project events discussed in the meeting.

There are three main reasons, cultural and psychological in nature, which create a reluctance to set up lessons learned meetings:

1.      The feeling that it's all about finding someone to blame.

2.      The feeling of “what's done is done” which is especially strong when the project is close to completion.

3.      The feeling that such a meeting would be a waste of time, and that nobody is going to bother with retaining or implementing the learned lessons in the future.

There are ways to deal with these factors and reduce the resistance, but nothing can come in place of a true commitment to the process by the organization's managers: A clear statement that the LL process is a learning process rather then a finger-pointing act; embedding the process as an integral part of the project, rather than treating it like some “appendix” attached to the end of the project; setting up guidelines for storage, retrieval and proper use of the lessons learned. These actions will reduce the resistance to the process and will help the team to become more actively involved in it.

If the project was terminated before completion, the LL meeting should be held immediately after the decision to terminate the project prematurely. Such a meeting will obviously be more psychologically difficult then a regular lessons learned meeting, but it is of crucial importance, especially if the project was aborted due to factors which the organization has some control over.

4.4.      The Lessons Learned Meeting

At the LL Meeting the moderator will present the agenda of the meeting, as planned by the project manager. It is usually a good idea to start the meeting with the project manager (or an appointed representative) congratulating the team members for their achievements during the project. This is true for projects that were terminated before completion as well. Even when the project as a whole has failed, people still put effort in their work and they deserve to be commended for it.

At the core of the meeting, will be the discussion of project events which were most important or most frequent, as shown by the questionnaires. These are, usually, central events in the project, such as milestone completion, product quality, and communication between shareholders. The moderator will lead the discussion towards the goal of finding what factors caused the said events, rather then focusing on the event itself (or the person responsible for it). It should be clear that this procedure applies equally to positive and negative events: Finding the factors leading to success is no less important then finding those who lead to failure. The goal of this part of the meeting is to produce a list of causes and their effects, which will help in preventing failures, which will help in ensuring repetitive success, in decision making at future projects.

Other topics which may be relevant to the lessons learned meeting are:

¨      Discussing the assumptions on which the project was based on. In retrospect, were they accurate? Was the process of making these assumptions reasonable? Were there any important assumptions left-out? If so, what are they?

¨      Project change management analysis: Were the changes in the project managed properly? What was the method of authorizing changes during the project?

¨      Did the project products meet the desired specifications? Discuss the feedback received from the customer after every milestone, and the feedback received at the end of the project.

¨      Discussing progress indicators: Was the progress adequately measured? Were the chosen indicators a good choice? What was the effect, if any, of these measurements on the execution of the project?

¨      Analyzing the effect of QA and QC components on project events.

¨      Were the management and execution methodologies consistent with the type of the project?

¨      Were lessons learned from previous projects put to use, were they put to use in a successful and operational manner in the planning phase of the current project? If so, how?

It is imperative to avoid mentioning names of people involved in the events discussed, unless they've voluntarily taken the responsibility and clarify their position about what happened. This, of-course, does not apply to positive events.

It is also imperative that the discussion will be held candidly and openly, and the participants will discuss the topic at hand without hiding anything. An open and honest discussion is the only way to efficiently utilize the knowledge and information held by the participants.

4.5.      The Lessons Learned Summary Document

The primary product of the LL meeting is a document which will be distributed among the people attending the meeting and other shareholders, including the organization's management. This document will be included in the official list of project documents, and its content will be stored in an organizational database.

Creating this document is the responsibility of the project manager, but the moderator or other team members may also take part with this task.

The document will open with the goals of the LL process (as defined in section 1 above) and a description of the method used to manage the process before the meeting. The document will then detail the main points which were reached during the meeting itself (as demonstrated by the points in section 4 above).

An important part of every analysis of every event is providing an organizational context which will serve as a reference for future projects. For example, it may prove useful to list the name of the person or the department who can provide more detailed information regarding the event in question. It is also important to write down the final conclusions drawn (in terms of causing factors and consequences for future projects) for every event being discussed.

The next part of the document will be a list of practical recommendations for future projects. Examples for such recommendations are:

¨      Increasing the estimation range on certain specific tasks

¨      Appointing a single person to communicate with shareholders

¨      Appointing a single person to be in charge of managing project changes

¨      Involving the management whenever there's an escalating conflict with a customer

¨      Managing the LL process itself: Was it done properly? In what ways can the process be improved?

¨      etc.

Once the first draft of this document is complete, the project manager will distribute it to all the participants of the lessons learned meeting, for their approval. This is especially important when analyzing failure events, due to the personal and emotional sensitivities that tend to accompany such events.

Once the document is approved by all participants, the project manager will make sure that the document summary is stored in an organizational database. The nature of this database varies from organization to organization. It can be an application built specifically for lessons learned, a query-based database, or even a simple spreadsheet. Regardless of its technical form, appropriate steps must be taken to ensure that the database contains all the relevant lessons learned information, and that this information is retrievable.

The final version of this document will be distributed among the participants of the lessons learned meeting as well as all other stakeholders (excluding the customer) and the organization's management.

4.6.      Monitoring

The distribution of the Lessons Learned Summary Document marks the end of the last phase of the project. Therefore the job of monitoring the implementation of the lessons learned recommendations does not rest on the shoulders of the project manager. Instead, it becomes a responsibility of the PMO office (if it exists) or the organization's management.

The monitoring stage includes three main components:

1.      The Summary Document should include practical recommendations to those who are managing other projects in the organization. It is therefore necessary to check that these recommendations are actually followed through.

2.      It is necessary to check that all the conclusions of the LL process are properly stored in the organizational database, and that the organization's employees were notified about the availability of this new information.

3.      It is necessary to check that the information in question can be retrieved efficiently (keywords, categories, queries, technical classifications etc.). It is also important to ensure that other project managers in the organizations are applying the lessons learned when they begin planning their own projects.

5.         Conclusion

Knowledge management is an important tool for success in the modern business world, especially in project-oriented organizations. The ability to utilize previous knowledge is especially useful in organizations in which the projects are similar in nature to one another, where this process of learning from past mistakes and past successes can help save resources, boost efficiency and mitigate risks.

However, it should be noted that such ability does not come on its own. Without the necessary commitment from the management, the data gathered from previous projects will remain in its unprocessed form. Only a conscious and formal implementation of the lessons learned process, will serve the organization's future success.