Journal of Knowledge Management Practice, Vol. 7, No. 3, September 2006

How Measuring Learning May Limit New Knowledge Creation

Deborah Blackman, University of Canberra, School of Business and Governance

ABSTRACT:

This paper demonstrates that the desire found within many organisations to measure the learning being undertaken can undermine the ability to develop new knowledge. The reasons for this are cited as the potential impact of mental models as closing mechanisms, coupled with the likely affect that this will have upon the dominant coalition. This propensity is seen to become more likely because of the way that organisations routinely support learning via targets, appraisals and objective setting which will all be framed from within the current mental models. The paper concludes by considering future ways of measuring that might not close down knowledge development, but might even encourage it. The most important implication of this research is that the more passionate an organisation is about knowledge development, the more it may prevent or adversely affect it because of the way that it monitors its own performance.

Keywords: Learning, Knowledge, Measurement, Mental Models, Paradox


1.         Introduction

It is a widely held view that the most important contributor to corporate competitiveness will be the ability of one company to learn faster than others (Altman and Iles, 1998; Pemberton and Stonehouse, 2000); as a result, companies may seek to become learning organisations in order to increase new knowledge development and knowledge utilisation which will lead to competitive advantage (de Geus, 1997) and transformational change (Pedler et al, 1989; Senge, 1990). It follows that, in order to benefit from the knowledge being developed within it, an organisation must know that learning is occurring and that the knowledge it is looking for is emerging. Without this ability to use the knowledge, there can be no change. However, few of the writers on organisational learning or the learning organisation models discuss how they will ensure that they are creating and utilising useful knowledge.

An overview of the learning organisation literature demonstrates that some models have elements of ongoing feedback and review which might give this confidence (see for example Pedler et al, 1989 and Garvin, 1993) but, because the review is triggered by a problem or behaviour linked to current policies and past experiences, the certainty of usefulness of knowledge becomes problematic (Blackman et al, 2004). The models encourage organisations to learn from experience, but how they are to ensure that the meanings created will reflect the right perspective and, therefore, provide the acquisition of useful knowledge is not obvious. However, what is made clear is that successful companies will be those who ‘actively manage the learning process to ensure that it occurs by design rather than by chance’ (Garvin, 1993; pp. 81).

It is this act of managing the learning process that will be the focus of this paper. In order to ensure that learning is occurring, many organisations will want to identify ways of measuring potential changes (Watkins and Marsick, 1993; Dobson and Tosh, 1998). Such measurement encourages the assessment and reward of desired learning behaviours and, it is argued, ensures that such new behaviours will develop in an ongoing way. It could be posited, however, that as rewarding a behaviour will ensure the repetition of such a behaviour (Thorpe and Homan, 2000; Le Boeuf, 1986), if the wrong thing is measured it will encourage inappropriate learning behaviours. This paper will discuss ways which are used to measure potential learning, thereby considering whether the impact of such measurement systems achieves the desired output of increased organisational knowledge development and utilisation. It will be posited that measuring learning actively reduces the likelihood of it adding value to the organisation and that, the more enthusiastic a company is to encourage learning, the more likely it is that it will reduce new knowledge development.

2.         Measurement As An Element Of A Learning Organisation

Learning organisations continually strive to improve and transform themselves, not merely make incremental adjustments (Pedler et al, 1989; McGill et al, 1992). They are organisations which seek to improve their effectiveness through reflection, innovation, continuous evaluation, quality improvement and timely responses to their internal and external environments (Pearn et al, 1995). Their advantage has been seen to be the development and harnessing of new knowledge, developed via managed learning processes (Pedler et al, 1989; Carr, 1997; Bierly et al, 2000; Flood, 1999). Pedler et al (1989) defined ten key steps to becoming a learning company and Senge (1990) outlined five key disciplines which were the underpinning principles of developing a learning organisation. What has been written since (see for example: Goh, 1998; Holt et al, 2000; Dobson and Tosh, 1998; Dowd, 2000; Watkins and Marsick, 1993; Wishart et al, 1996) is largely based upon these initial writings.

An analysis of the literature identifies a set of theoretical elements which are argued as being needed to become a learning organisation (Blackman, 2002). Two types of input changes can be identified as being needed to develop the desired output changes: firstly, organisational inputs designed to develop individual learning and, secondly, specific learning organisation behaviours that will support the sharing of the individual developments.

The learning organisation literature was reviewed and a summary model was developed of the inputs which included the element ‘Enable Continuous, Monitored Learning Opportunities’ (sourced from, amongst others: Clegg, 1999; Popper and Lipshitz, 2000; Grieves, 2000; Griego et al, 2000; Snell, 2001) see figure 1. This is seen as a process whereby learning systems can be planned in such a way as to ensure learning development. Effort is spent on defining and measuring key factors when venturing into new areas, which leads to organisations seeking quantifiable measures of learning in order to support systems and reward learning. These inputs and outputs can be summarised as a model that will lead to new knowledge creating transformational change and competitive advantage (figure 1).

FIGURE 1: An Input/Output Model Of The Learning Organisation (Blackman & Henderson, 2005)

 
 

 

 

 

 

 

 

 

 

 

 

 

 

 


Many of these inputs are about ensuring that learning is developed, supported and communicated and are specifically designed to try to overcome some of the potential problems identified as preventing organisational learning. It is widely accepted that organisational learning is limited by a lack of shared language, values, knowledge and understanding (Easterby-Smith et al, 1998). Consequently, sharing mental models is recommended for both organisations and individuals to create shared meaning, thereby enabling common understandings and a development of knowledge (Hayes and Allison, 1998; Bell et al, 2002; Senge, 1990). Dixon states that ‘it is this joint construction of meaning that is organizational learning’ (1997; pp. 25).

Organisational mental models provide collective frameworks of value and belief systems which act as the basis for analysis, policy and procedural and cultural development (Caldwell et al, 2002); they are cited as providing the link between collectives and individuals as they proffer a context for the interpretation and understanding of new information (Kim, 1993; Doyle Corner et al, 1994; Dixon, 1997; Swaab et al, 2002). Consequently, all new knowledge develops from the basis of the mental models in place. For these reasons the development and sharing of mental models is seen as a very important element in developing a learning organisation (Senge, 1990; Pedler et al, 1989; Dovey, 1997; Clute, 1999).  Mental models become the bounded rationality of individuals or, collectively, teams, communities of practice or organisations (Simon, 1991). Actions emerge as a result of the mental models currently held. The role of mental models is, therefore, significant; their advantage is held to be an element of predictability which can develop as a result of the models held (Wetzel and Buch, 2000), but this can become a problem as flexibility and adaptability can be reduced (Hill and Levenhagen, 1995). 

When one analyses the potential results of sharing mental models it is possible that, instead of aiding organisational learning, they may in fact hinder it. The concept of a mental model as a framework for new knowledge means that it could also act as a closing mechanism. Blackman (2001) and Coopey (1996) have demonstrated that, through dominant logic and coalitions, new knowledge creation can become self-referential in nature, leading to only some ideas (those related to the current mental model) being acceptable to the system for consideration. The stronger the mental model, the greater the influence it has upon the state of openness of the system. In extreme cases, the flow of knowledge acquisition can be completely turned around with only ideas which have been actively sought out by members of the system being recognised and encouraged (Blackman, 2002).  It can then be seen that this prospect of organisational closure may have the potential to affect the success of measurement within an organisation.

Garvin (1993) stressed the need for measurement designed to ensure that an organisation could see when its move towards being a learning organisation was working since, by measuring the learning inputs it should be possible to recognise when desired outputs are being achieved. This need to track achievement is important within organisations in general, thus many organisations, when trying to become learning organisations, set about targeting new ideas and setting up development systems which will enable a more group feel and culture to emerge (Arkin, 1993; Nevis et al, 1995; Dobson and Tosh, 1998; Tosey and Smith, 1999; Buckler, 1998): for example ‘Desired outcomes from the workshop were articulated as interlinked themes’ (Attwood and Beer, 1988; pp. 205). There is also considerable focus on management by objectives in many schemes (Arkin, 1993). The need for measurement drives the learning organisation towards monitoring and evaluating the learning process.

Kock et al (1996) feel that the focus on organisational learning produces the myth that all organisations should be learning systems and, consequently, organisations seek to learn even when they do not need to. One of the mistaken targets may actually be to have more learning, not to use more knowledge. How you categorise and measure something depends on how you look at it and what you are differentiating it from. Since there are so many different ontologies of organizational learning, the more one sets out to measure precisely its nature and extent, the more one is likely to fall into what Ryle (1949) calls a ‘category mistake’.  ‘…there is much danger of methods based on one set of assumptions being applied to a model of organizational learning based on a wholly different set of assumptions’ (Easterby-Smith et al, 1998; pp. 267.)  For example, when Iles (1994) suggests the use of competences, there seem to be two potential problems. Firstly, who defines the competences and do they know what they are looking for? Secondly, as they are framed by the current mental models and are, therefore, couched in the same language and understanding as other current organisational processes, how can they be as radical and different as some others imply processes must be, if they are to transform the organisation? It is this potential for the way that learning is measured to actively reduce both organisational flexibility and the effectiveness of organisational learning and knowledge development, that will be explored within this paper.

3.         Methodology

There are many different views of what organisational learning is and how it comes about, but most writers are in agreement that, as a result of the learning, something within the organisation changes (Blackman, 2001). It is this change that the learning organisation is attempting to harness in order to achieve competitive advantage. The methodology chosen for this research set out to explore differences created by learning organisation processes, focusing upon identifying knowledge output differences between learning and non-learning organisations. The objective was to consider the elements that were being used to develop and sustain learning (including the element of measurement) in each case and to analyse differences, in order to demonstrate how developing learning systems was leading to improved performance and competitive advantage. Such research clearly needed a depth of data and analysis rather than breadth. This could only be achieved with methods that would permit exploration and theory building (Creswell, 1994), rather than looking for confirmatory data and so four qualitative case studies were developed in the UK. The profile of the four was two companies that self-assed themselves to be learning organisations and two which did not. The first two companies, who were widely acclaimed for their learning organisation status, focused internally upon identifying and sustaining this status. For the two that were non-learning organisations, one had heard of the concept but made no claims to even try to achieve such a status. The other had never heard of such an idea and, when it was discussed, was very clear that such definitions did not reflect their processes and practices. All four were Public Limited Companies of a similar size and all were based on single sites. The sample was seen to be of two pairs of companies. One pair were production companies, one a learning organisation (Company 4) and one not (Company 1), and the other pair were service companies, again one a learning organisation (Company 2) and one not (Company 3). All four are considered to be successful – this is measured as making a profit and achieving company goals.           

Data was collected in three ways. Firstly, via samples of organisational literature and documentation seeking to understand the organisations’ own views of their learning and knowledge development practices. Secondly, via open-ended questionnaires exploring differences in perceptions and understanding between the organisations pertaining to knowledge, learning, culture, the nature of organisations and the role of information. A return rate of between 20 and 34 percent for each company was achieved giving 82 questionnaires in total There were equal returns for learning and non-learning organisations. Thirdly, via qualitative interviewing of 36 individuals throughout the organisations (senior managers, supervisors and line employees) which was 10 percent of all staff for each case. The questions sought participant’s views on what is knowledge, what is learning and explored views on the status of inputs outlined in figure 1.

Subsequently, a second study has been undertaken seeking to explore how learning and knowledge was being encouraged in companies within the Western Sydney area. The objectives of the study were: to understand how organisations prepared for and met the challenges of an increasingly complex, competitive and globalised world; to understand how organisations prepared their members for these challenges and to compile an inventory of key enablers and barriers to learning organisation development. Because it was determined that there was a need to understand the nature of the problem being researched and the cognitive structures within the organisations, a qualitative approach was adopted (Creswell, 1994).

 Data was collected from eleven case companies ranging in size from 5 to 4000 employees (although this large company was split into divisions and only one product and area was researched), of which some were owner run and managed, whilst others were major corporations. It was the location of the companies that was of initial interest in order to consider how learning and knowledge were being developed in Western Sydney, Australia. Accordingly the sample was mixed, as it was thought that different patterns might emerge in different sizes and types of company and that such differences could then be explored. Interestingly, however, the patterns were very similar across all the organisations. In order to get as broad an understanding as possible of a range of voices, the method was designed to get a picture of the views held throughout each organisation. Semi-structured interviews were undertaken with employees from differing levels within the companies and some focus groups were also undertaken in order to consider if discourse changed when employees were in groups rather than being interviewed individually. The data was then entered into NVIVO and coded for themes.

4.         Findings

There was a clear set of common themes running within all four companies from the first study when it came to enabling continuous monitored learning opportunities. These were categorised into appraisal, objective setting and targeting knowledge. When the data from the second study was analysed, it became clear that the companies in this study were also focusing upon appraisal and management development as their methods for learning and knowledge development. Despite a lapse in time (4 years) and a difference in location (UK versus Australia) the methods being used to support and enable learning had no noticeable differences between them.

5.         Appraisal

All the companies had a form of performance appraisal, which led to the output of a training and development schedule. What was of interest was that, whilst all the companies were trying to develop knowledge and skill enhancements via appraisal, all were also using it to enable the measurement of learning within the companies. The learning organisations had more complex and quantified methods for measuring the learning taking place, but, there was little evidence that this was being translated into new, different knowledge. This seemed to be because of the way that the appraisals were being managed, which was very much about objective setting of learning inputs rather than the measurement of new knowledge outputs (see below): there was an implication that by measuring something such as a course this would ensure a knowledge output:  “Need to measure the output of training – how done best – measurement of qualifications is a good way[company 2, project 1].

Another concern was that the use of appraisals was leading to the organisation determining what should be learnt. Both company 2 and company 4 (those considering themselves to be learning organisations) were very focused upon linking the appraisals and personal development plans to the organisational objectives. This led to comments such as “I will get support to go on training but I will always have to show how it will be directly related to my work”[company 4, project 1]; “It is all competency based – assess needs first and then find out what is available ... As needs are assessed against the competencies it gives focus to progress and is good for discussion” [company 2, project 1]; “I like the idea of everyone having learning targets [as a part of appraisal review] - we are nowhere near this yet but it's on the agenda. People will have ownership of their own jobs and, therefore, their own learning. We have a long way to go” [company 3, project 1].

If we consider the concern of self-referentiality indicated earlier it can be seen that the organisation may be pre-determining what is being learnt, which could, in turn, lead to new ideas in the environment being lost or ignored (Blackman, 2001). Issues of power may emphasise these concerns, as power can be seen potentially to detract from the free and open exchange of ideas (Coopey, 1995; Blackman, 2001). The concern is that the organisation is more powerful than the individual, thus what the organisation determines as important will take precedence over any other issues the individual may notice. There is a strong possibility that the organisational mental models will pre-empt all new learning

 
 

 

 

 

 

 

 

 

 

 

 


FIGURE 2: How Organisational Mental Models Frame Knowledge Generation And Sharing

Figure 2 indicates a turning around of the traditional knowledge acquisition flow and what can be seen is that the current mental models will act as the filters determining what will need to be learnt. Everything about the learning processes set up within the companies studied is designed to ensure that the organisations are seen to learn, but not to ensure that the knowledge will be useful, challenging or develop new mental models. The effectiveness of the processes will always depend upon what are the mental models held; if the leaders hold a truly visionary perspective and this drives the learning, then there will be changes. If the leaders do not set up the necessary tensions, since current operational issues are driving the shared mental models, really new ideas and knowledge will not develop.

6.         Objective Setting

The use of measurements, including objective setting which incorporated fixed numbers of training days and encouragement to attend regular staff development leading to agreed qualifications, were limiting the range of new ideas entering the organisational decision-making frameworks. Problems with measuring development have been documented previously (Kirkpatrick, 1998; Phillips, 1999), showing that unless measurement is specifically designed to encourage higher level learning, it tends merely to audit lower level knowledge and this was what was occurring. The levels of learning were analysed across all four companies and, despite the companies considering themselves to be learning organisations being very focused upon clear objectives, the levels of learning being demonstrated were not very different.

Pffeffer and Sutton (1999) also stress this need to develop further than know-how, as the possession of apparently tangible knowledge leads to a belief that it is used, and will be used, appropriately and efficiently. They say that ‘why’ before ‘how’ is crucial to the success of any knowledge development within organisations. Clegg also stresses the need for a process to develop knowledge as more important than clear outputs (1999). Edmondson and Moingeon (1996) also differentiate between ‘how’ and ‘why’ and, whilst they state that both are important to the long-term success of an organisation, they stress that competitive advantage is more likely to be found by developing know-why.

The short answer question data from project 1 was coded so that ‘knowing how’ was seen as ‘acts’ – individuals know how something is done and, based on received ideas, actions can be performed well. Everything is explicit and can be observed. This was seen as equating to basic training, behavioural outputs, being based upon information so that direct short-term measurement is possible.  Where answers focused on processes for training, NVQ measurement etc., this was seen to reflect the explicit observable nature of the knowledge. ‘Knowing-that’ was where there appeared to be a recognition that something needs to be done in a certain way. There is a basic skills level of understanding and some tacit issues are now perceived. The perception that there needs to be some cognitive underpinning is key here. Lastly, ‘knowing why’ was seen to be when individuals understood why something had to be done, what the options were, when concepts were understood and when there was recognition not only that this is the reason for something  to be done but also why something else was not.

Table 1: Know That/How/Why

Type of Knowledge

Hits for LO (%)

Hits for non LO (%)

Total (%)

That

24 (18%)

27 (20%)

51 (38%)

How

31 (23%)

30 (22%)

61 (45%)

Why

14 (10%)

10 (7%)

24 (17%)

Total Entries Coded

70 (51%)

67 (49%)

136 (100%)

TABLE 1:  Results Of Types Of Knowledge

What is immediately obvious once more is the similarity of the responses between the types of organisations. It is also clear that this analysis confirms the low incidence of linking knowledge with understanding (24% of ‘know why’ overall). The focus is upon skills and information acquisition rather than cognitive and understanding abilities. The implications of this are that there will be more effort put into gaining more information but not necessarily in developing people to be able to use the knowledge. The focus upon learning will be on acquiring information, not upon cognitive development. When companies are looking to see if they have acquired knowledge they will be looking for evidence of more facts present but not necessarily new ways of thinking. Measurement systems will be looking for explicit examples of knowledge rather than more ephemeral concepts. This data from the short answer questions was corroborated by the interviews in both projects which, as indicated above, focused upon competencies and skills rather than challenge and change. The implication is that the measurement will encourage lower levels of knowledge, thereby actively reducing the potential for new knowledge to lead to transformational change.

7.         Targeting And Rewarding Knowledge Development

The learning organisations were targeting knowledge in order to be able to recognise and measure it. The idea of targeting is recommended in some models (see for example: Nevis et al, 1995; Tosey and Smith, 1999) but, by its very nature, targeted knowledge has to be clearly defined before it has been acquired and thus is more likely to be simple in order to be easily measured.

The problem with implementing reward processes successfully is that there has to be some form of measurement if performance is to be changed – this is so that the desired behaviour can be rewarded and encouraged (Thorpe and Homan, 2000). The measurements needed, if there are to be visible incentives, will drive behaviour more than the desire to achieve the outputs. An example of this was seen in company 3, where many were encouraged to attend training/conferences and other forms of development. However, there was no system for feeding this new knowledge into the organisation. The money was in place and, therefore, the behaviour of discovering new ideas was actively followed, but, in fact, the organisation did not benefit from this. Moreover, in company 4 the measurement of learning was equated to qualifications and so everyone received a bonus upon completing a qualification. Again, there was no measure as to what was really being fed into the company and the debate must be whether, having everyone achieving the same qualifications as each other is helping, or is it in fact merely closing down the company still further as the process of acquiring the same qualification will develop strongly shared models. In all four companies, learning targets (the measurements) were set by appraisal, thus the shared mental models drove the new learning.  This was then reflected and emphasised by the rewards processes and thus the reward structure added to the propensity for closure.

There is also cause for concern here as to how this response to measurement affects, and is affected by, the dominant coalition within organisations. Kofman and Senge state that a learning organisation can aid creativity and development as it focuses on the organisation itself and reduces the ‘race’ against the competition (1993; pp. 9). Yet, if the desired output is seen to be new ideas and change, this is what will probably occur, because management will assume that the measurement of success will be change - whether it is the right one may not be seen as the issue. The competition is now amongst the managers to ‘learn’ more than their colleagues so that they can change things faster. Competition is part of the learned behaviour already in place which will take a great deal of eradication, owing to the constant focus on short-term gain. Thus, it becomes possible that the key factor is to achieve learning, via measurement techniques such as monitoring qualifications, not necessarily to achieve new knowledge. One could argue that this is the focus of figure 1, but it is clear within the model that the learning needs to cultivate knowledge, but there is no evidence that these were the outputs developing.

Learning is being predetermined by the use of learning targets, measurement and appraisals. It can be seen that the current mental models held within an organisation, especially those held by senior management, drive this pre-determination. The shared or dominant mental models construct the learning measurements and drive the learning processes and knowledge development. Consequently, the idea of an open system acquiring ideas from the outside, itself becomes seriously challenged. Without this openness, the usefulness of learning becomes more problematic, because only if there are divergent mental models can the differences and tensions emerge which will generate learning.

If figure 1 is redrawn it can be seen that, because of the fact that knowledge is being pre-determined by the managers, by the systems and by what is already known, the role of mental models in knowledge creation (via the choices of measurement) will be quite different from that implied in figure 1. In figure3 the inputs are re-drawn to reflect what is really happening because of the mental models and their impacts upon measurement and learning process development.

FIGURE 3: The Learning Organisation Model Reflecting The Impacts Of Shared Mental Models

 
(Blackman And Henderson, 2005)

 

 

 

 

 

 

 

 

 

 

 

Culture is no longer shown as this will be determined by the mental models, the stronger the shared models become the more they will determine the culture and its impacts. Elements are being predetermined by what is already known and understood. The more measurement is used as a way of ascertaining the state of learning, the more likely it is that it will reduce the levels of novelty and innovation that are emerging from the learning processes.

8.         Alternative Perspectives On Measurement

The concern is shown that setting up measured and monitored learning opportunities in order to set up a new learning culture can be seen to be flawed as, although the inputs being measured may appear to be likely to develop the desired outputs, in the cases we have studied, in both projects 1 and 2 the actual knowledge outputs do not seem to be measured in a meaningful way. This concern is not new; much training literature vindicates the view that evaluation systems of development and training are sadly lacking (Potter et al, 2003; Davis et al, 2003). The last section of the paper will develop some theoretically formed alternatives for measurement, based upon research currently in progress (project 2). It will be shown that, rather than measuring learning targets and knowledge gained, there needs to be a change to measure the organisational propensity to learn and the range and types of knowledge.

 
There needs to be a different way of considering how to measure learning in a way that will not prevent the creation and acquisition of new knowledge. Building upon what has been said before, it can be seen that the measurement system would need to be designed to develop and reward alternative behaviours. These behaviours would need to encourage: a recognition of the knowledge being created by the learning (Blackman, 2002); the maintenance of more open systems (Blackman and Henderson, 2005); higher levels of ‘know why’ knowledge (Blackman and Henderson, 2006); the transfer of knowledge to others within the organisation and ongoing recognition about what the learning is actually desired to achieve in terms of the outputs of the processes being undertaken. It could be argued that what is needed is a change in the conceptualization of what a learning culture is for: instead of expecting it to encourage more learning, it needs to encourage more knowledge creation in order to enable transformation (Blackman, 2002). If this is so, then a reconceptualisation of the role of culture in learning organisation models can be proposed to be one of challenge and not of learning, then a reconceptualisation of figure 1 will also need to be considered – see figure 4.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Figure 4: The Learning Organisation – An Alternative Input/Output Model (Blackman, 2002)

The differences between figure 1 and figure 4 can be seen to be an overt linking of feedback loops, encouraging a focus upon the desired knowledge outputs at all times. What is meant here is that the desire is for knowledge which needs to be the centre of process development, not necessarily learning. This may, in itself, seem paradoxical but it is hoped that this focus upon knowledge outputs, instead of the acquisition processes, will ensure that organisations concentrate on the reality of the knowledge issues they need to address rather than developing measurement systems for learning processes. Moreover, the inclusion of on-going challenge is a part of the need to recognise that closure can emerge if there is not enough managed reflection and scepticism (Preskill and Torres, 1999; Blackman et al, 2004).

9.         Enable Continuous Learning Opportunities

It is clear that there will still need to be learning opportunities provided and supported within the organisations. However, they will need to be broader based and be focused on the creation of new ideas as well as purely skill enhancement within the same thought frames. The opportunity must be to learn to be innovative and challenging and not be purely about incremental development. The difference will need to be that the learning opportunities need to be focused upon the knowledge that they are designed to develop, rather than the learning itself.

10.       New Culture Of Challenge

In cases where innovation has been clearly identified within the companies in projects 1 or 2 it was where, as a result of the types of learning opportunity outlined above, radically new ideas were developed within the organisations. In one case, a decision was recently taken that any new development strategy must encourage employees to go to something only loosely related to what they do. They then had to come back and explain to a peer group (within 2 weeks) in what ways they had seen anything about their work differently as a result of the development they had undertaken. Results were interesting but “it’s hard because they [peer group] were arguing that it wasn’t different because of what they already knew. I don’t know if I’ll be able to change anything because they are not open minded. I’ve suggested they do something similar but who knows if they will” [project 2]. It can be seen that even where an attempt was being made to do something differently, the current mental models were already undermining it. One way forward may be formal processes of challenge.

Firestone and McElroy (2003) and Blackman et al (2004) have both argued that there needs to be managed falsification of ideas in such a way as to challenge and to encourage differences of opinions and alternative perspectives. If this process was a part of the managed organisational systems, it would fulfil the need to measure and might actively encourage openness. The processes should encourage the debate about all aspects of the organisational knowledge base and current understandings and beliefs.

Developing System Self-Awareness

To be self-aware an entity must be able to describe its thoughts, emotions and even its self-awareness. The objective will be to learn from its own and others’ experience and to share the knowledge gained from this in order to develop even further. To do this successfully, the organisation must actively seek to reflect and challenge who it is and what it knows. This will be encouraged via the use of paradox and challenge (figure 4). However, it is an important category of its own when the issues of measurement are considered. It has already been noted that management, in particular, is prone to impose its mental models upon the organisation as a whole. For this to change, the development of the managers needs to be focused upon being self-aware and being more aligned with the long term knowledge goals.

Developing Paradox

Proponents of paradox within organisations argue that, where there are differing opinions and ideas, instead of seeking coherence in order to reduce cognitive dissonance and, therefore, generating more supportive, safe cultures, the inconsistencies should become the focus of ideas and dialogue so that they may become part of the new theory-in-use (Poole and Van de Ven, 1989; Morgan, 1986). Although some writers on learning organisations intimate that there will need to be conflict in order to develop dialogue and create the tension needed for learning (Buckler, 1998; Eisenhardt et al, 1997; Altman and Iles, 1998), in fact the overall focus upon shared mental models and ‘safe’ cultures tends to lead to an avoidance of dissonance and, therefore, a move away from this tension.

This need to encourage the holding of different beliefs and alternative worldviews, leads to a wish to build a learning organisation model that holds paradox as one of its core elements. Whereas a sceptical view could be seen as negative, this would be embracing difference and encouraging the organisation to focus upon the desired outputs rather than upon the processes developed to achieve it. The use of paradox, already accepted as a solution by many within the knowledge management and organisational learning fields (Poole and Van de Ven, 1989; Morgan, 1986; Snowden ,1999/2000;  Handy, 1994;  Pascale, 1993; Eisenhardt, et al, 1997), should be brought more clearly into the learning organisation literature in order to create multiple mental models. What should be recognised is that the need to measure will always come to the fore within organisations. It is, therefore, important not to argue for concepts such as paradox but to consider how such an idea can be measured and rewarded.

11.       Epistemological Understandings

This means that organisations actually need to consider the type of knowledge that they are creating. They will need to map different frameworks of knowledge and ask themselves what knowledge is and why they want it. There is evidence that, despite there being wide spread systems of knowledge management, many of them do not focus upon the actual epistemological foundations of the knowledge being created (Blackman and Henderson, 2004). This weakness leads to an assumption being made that learning processes or knowledge management systems, merely by existing, will be enough to develop learning. Moreover, in both projects being considered there was a tendency to confuse information and knowledge (Blackman, 2001). This has the result of leading to learning being about accessing information rather than developing new understandings and knowledge

These are fallacious assumptions so there needs to be a measurement of levels of knowledge present. How this can be done will need research, but tracking the impacts that knowledge has upon the ideas outlined above will be a beginning. Organisations can seek to interview their own employees, gaining insights as to their perspectives of knowledge and what implications these have for knowledge development. The measurement will need to be about the levels of change and not the levels of learning and/or information access.

13.       Conclusions

A problem of self-referentiality emerged within the cases studied, because the learning targets were being set within the context of what was already considered important, so that the measurements were input focused, being based upon what the system and the learner were putting into the processes, rather than considering what was emerging out of the system. The more passionate an organisation was about supporting learning the more likely it was to actually prevent effective development as nothing unexpected was occurring and so, instead of monitoring learning encouraging new knowledge and a transformational output, prior experience was presetting the output, leading to a reversal of the knowledge creation process. The targets were being set to confirm what was already known and the common ethos was ensuring that all staff were learning the same ideas. As there was a focus upon developing a strongly shared culture at the same time, this was strengthening the pre-emptive nature of the measurements. The reasons for wanting to measure learning can be clearly understood, but far from encouraging new learning and innovative ideas, unless the measurement systems and monitoring systems are very different from those currently in common usage, the result will be entirely counterproductive, leading to less new knowledge rather than more.

It is necessary, therefore, to develop ways of learning that will encourage different learning behaviours, thereby changing the types of outputs that are likely to emerge. Moreover, the focus of measurement must be upon the outputs of the system and not the inputs. The new figure of the learning organisation indicates this and highlights potential areas for research development.

14.       References

Altman, Y. and Iles, P. , 1998,  “Learning, Leadership, Teams: Corporate Learning And Organisational Change”, Journal of Management Development,  17.1; pp.  44-55.

Arkin, A. , 1993,  “Formula For A Learning Organisation”, Personnel Management, July; pp. 48-49.

Attwood, M. and Beer, N., 1988, “Development Of A Learning Organisation”. Management Education and Development,   19.3; pp. 201-214.

Bell, E., Taylor, S. and Thorpe, R., 2002, “A Step In The Right Direction? Investors in People and the Learning Organisation”, British Journal of Management, 13; pp 161-171.

Bierly, P.E., Kessler, E.H. and Christensen, E.W., 2000, “Organizational Learning, Knowledge And Wisdom”, Journal of Organizational Change Management, 13.6; pp. 595-618.

Blackman, D.A., 2001, “Does A Learning Organisation Facilitate Knowledge Acquisition And Transfer?” Electronic Journal of Radical Organization Theory, February, 7.2.

Blackman, D.A., 2002, “How Learning Organisation Practices Close Knowledge Creation”, Unpublished PhD Thesis, Nottingham Trent University.

Blackman, D.A and Henderson, S., 2006, “Why Managing Organisation Learning Obstructs Transformation”, Knowledge, Action and the Public Concern, Academy of Management Meeting, August 11-16, Atlanta, Georgia.

Blackman, D.A. and Henderson, S., 2005, “Why Learning Organisations Do Not Transform”, The Learning Organization Journal, 12.1; pp.42-45.

Blackman, D.A. and Henderson, S., 2004, “Know Ways In Knowledge Management”, The Learning Organization Journal, 12.2; pp. 152-168.

Blackman, D.A., Connelly, J. and Henderson, S., 2004, “Is Double Loop Learning A Reliable Knowledge Creation Process?” The Learning Organization Journal, 11.1; pp. 11-27.

Buckler, B., 1998, “Practical Steps Towards A Learning Organisation: Applying Academic Knowledge To Improvement And Innovation In Business Practices”, The Learning Organization, 5.1; pp. 15-23.

Caldwell, C., Bischoff, S. J. and Karri, R., 2002, “The Four Umpires: A Paradigm For Ethical Leadership”, Journal of Business Ethics, March, 36.1/2; pp. 153-163.

Carr, A., 1997, “The Learning Organization: New Lessons/Thinking For The Management Of Change And Management Development?” Journal of Management Development, April, 16. 4; pp. 224-232.

Clegg, S., 1999, “Globalizing The Intelligent Organization: Learning Organizations, Smart Workers, (Not So) Clever Countries And The Sociological Imagination”, Management Learning, September, 30.3; pp. 259-280.

Clute, P.W., 1999, “Change At An Oil Refinery: Toward The Creation Of A Learning Organization”, Human Resource Planning, June, 22.2; pp. 24-34.

Coopey, J., 1995, “The Learning Organization, Power, Politics and Ideology”, Management Learning,  26.2; pp. 193-213.

Coopey, J., 1996, “Crucial Gaps in ‘the Learning Organization’: Power, Politics And Ideology”, in Starky, K. (Ed.) How Organizations Learn, London, International Thomson Business Press.

Creswell, J.W., 1994, Research Design, Qualitative And Quantitative Approaches, London, Routledge.

Davis, M.A., Curtis, M.B. and Tschetter, J.D., 2003,  Evaluating Cognitive Training Outcomes: Validity And Utility Of Structural Knowledge Assessment”, Journal of Business and Psychology, Winter, 18.2; pp. 191-206.

De Geus, A.., 1997, “The Living Company”, Harvard Business Review, March/April, 75.2; pp. 51-59.

Dixon, N. M., 1997, “The Hallways of Learning”, Organizational Dynamics Spring, 25.4; pp. 23-35.

Dobson, P.C. and Tosh, M. , 1998,  “Creating A Learning Organization: Training And Development In British Steel's Universal Beam Mill”, Total Quality Management, July, 9.4-5; pp. S66-70.

Dovey, K., 1997, “The Learning Organization And The Organization Of Learning: Power, Transformation And The Search For Form In Learning Organizations”, Management Learning, 28.3; pp. 331-349.

Dowd, S.B., 2000, “Organizational Learning And The Learning Organization In Health Care”, Hospital Materials Management February, 1.3.

Doyle Corner, P., Kinicki, A.J. and Keats, B.W., 1994, “Integrating Organizational And Individual Information Processing Perspectives On Choice”, Organizational Science, 5.3; pp. 294-308.

Easterby-Smith, M. and Araujo, L., 1999, “Organizational Learning: Current Debates and Opportunities”, in Easterby-Smith, M., Burgoyne, J. and Araujo, L. (Eds.) Organizational Learning and the Learning Organization, London, Sage.

Easterby-Smith, M., Snell, R. and Gheradi, S., 1998, “Organizational Learning: Diverging Communities Of Practice”, Management Learning, 29.3; pp. 259-272.

Edmondson, A. and Moingeon, B.J., 1996, “When To Learn How And When To Learn Why: Appropriate Organizational Learning Processes As A Source Of Competitive Advantage”, in Moingeon, B. and Edmondson, A. (Eds),  Organizational Learning and Competitive Advantage, London,  Sage.

Eisenhardt, K.M., Kahwajy, J.L. and Bourgeois, L.J., 1997, “Conflict And Strategic Choice: How Top Management Disagree”, California Management Review, Winter, 39.2; pp. 42-62.

Firestone, J. and McElroy, M., 2003, “The Open Enterprise: Building Business Architectures For Openness And Sustainable Innovation”, Executive Information Systems, Inc.; Accessed August 2006:  http://www.dkms.com/oemarketing.htm.

Flood, R.L., 1999,  Rethinking The Fifth Discipline, London,  Routledge.

Garvin, D.A.., 1993, “Building A Learning Organisation”, Harvard Business Review, July-August; pp. 78-91.

Goh, S.C., 1998, “Towards A Learning Organization: The Strategic Building Blocks”, SAM Advanced Management Journal, Spring, 63.2; pp. 15-20.

Griego, O.V., Geroy, G.D. and Wright, P.C., 2000, “Predictors Of Learning Organizations: A Human Resource Development Practitioner's Perspective”, The Learning Organization, 7.1; pp. 5-12.

Grieves, J., 2000, “Navigating Change Into The New Millennium: Themes And Issues For The Learning Organization”, The Learning Organization, 7.2; pp. 54-74.

Handy, C., 1994, The Age of Paradox, Boston, Harvard Business School.

Hayes, J. and Allison, C.W., 1998, “Cognitive Style And The Theory And Practice Of Individual And Collective Learning In Organizations” Human Relations, July, 51.7; pp. 847-872.

Hill, R.C. and Levenhagen, M., 1995, “Metaphors And Mental Models: Sensemaking And Sensegiving In Innovative And Entrepreneurial Activities”, Journal of Management, November-December, 21.6; pp. 1057-1075.

Holt, G.G, Love, P.E.D. and Li, H., 2000, “The Learning Organisation: Toward A Paradigm For Mutually Beneficial Strategic Construction Alliances”, International Journal of Project Management, 18.1; pp. 415-421.

Iles, P., 1994, “Developing Learning Environments: Challenges For Theory, Research And Practice”, Journal of European Industrial Training, 18.3; pp. 3-9.

Kim, D.H., 1993, “The Link Between Individual And Organizational Learning”, Sloan Management Review, Fall;  pp. 37-49.

Kirkpatrick, D., 1998,  (Ed.) Another look at evaluating training programs, Alexandria, VA, American Society for Training and Development.

Kock, N.F., McQueen, R.J. and Baker, M.., 1996, “Learning And Process Improvements In Knowledge Organisations: A Critical Analysis Of Four Contemporary Myths”, The Learning Organization, 3.1; pp. 31-41.

Kofman, F. and Senge, P., 1993, “Communities Of Commitment: The Heart of Learning Organizations”, Organization Dynamics, 22.3; pp. 5-23.

Le Boeuf, M., 1986, How To Motivate People: Reward: The Greatest Management Principle In The World, New York, Pan Macmillan.

McGill, M., Slocum, J. and Lei, D., 1992, “Management Practices In Learning Organisations,  Management Dynamics, 21.1; pp. 4-17.

Morgan, G., 1986, Images Of Organization, London, Sage.

Nevis, E.C., Dibella, A.J. and Gould, J.M., 1995,   “Understanding Organizations As Learning Systems”, Sloan Management Review, Winter; pp. 73-84.

Pascale, R.T., 1993, “The Benefit Of A Clash Of Opinions”, Personnel Management, October; pp. 38-41.

Pearn, M., Roderick, C. and Mulrooney, C., 1995, Learning Organisations In Practice, Maidenhead, Berkshire, McGraw-Hill .

Pedler, M. Boydell, T. and Burgoyne, J., 1989, “Towards The Learning Company”, Management Education and Development Spring, 20.1; pp. 1-8.

Pemberton, J. and Stonehouse, G.H., 2000, “Organisational Learning And Knowledge Assets - An Essential Partnership”, The Learning Organization, 1.4; pp.184-193.

Pfeffer, J. and Sutton, R.I., 1999, “Knowing ‘What’ To Do Is Not Enough: Turning Knowledge Into Action”, California Management Review, Fall, 42.1; pp. 83-99.

Phillips, J.J., 1999, (Ed.) Handbook Of Training Evaluation And Measurement Methods (Improving Human Performance Series), 3rd edition, London, Kogan Page.

Poole M.S. and Van De Ven, A.H., 1989, “Using Paradox to Build Management and Organization Theories”, Academy of Management Review, 14.4; pp. 562-578.

Popper, M. and Lipshitz, R., 2000, “Organizational Learning: Mechanisms, Culture And Feasibility”, Management Learning June, 31.2; pp. 181-196.

Potter, M.A., Ley, C.E., Fertman, C.I., Eggleston, M.M. and Duman, S., 2003, “Evaluating Workforce Development: Perspectives, Processes And Lessons Learned”, Journal of Public Health Management and Practice, November/December, 9.6; pp. 489-495.

Preskill, H. and Torres, R.T., 1999, “The Role Of Evaluative Enquiry In Creating Learning Organizations”, in Easterby-Smith, M., Burgoyne, J. and Araujo, L. (Eds.) Organizational Learning And The Learning Organization, London, Sage, pp.92-114.

Ryle, G., 1949, The Concept of Mind, UK, Penguin.

Senge P., 1990, The Fifth Discipline, New York, USA. Doubleday Dell.

Simon, H.A., 1991, “Bounded Rationality And Organizational Learning”, Organization Science, February, 2.1; pp. 125-134.

Snell, R.S., 2001, “Moral Foundations Of A Learning Organization”, Human Relations, 54.3; pp. 319-342.

Snowden, D., 1999/2000, “The Paradox Of Story”, Journal of Scenario and Strategy Planning, 1.5, Ark Publications.

Swaab, R.I., Postems, T., Neijens, P., Kiers, M.H. and Dumay, A.C.M., 2002, “Multiparty Negotiation Support: The Role Of Visualization’s Influence On The Development Of Shared Mental Models”, Journal of Management Information Systems, Summer, 19.1; pp. 129-150.

Thorpe R. and Homan, G., 2000, (Eds.) Strategic Reward Systems, Harlow, UK, Financial Times/Prentice Hall.

Tosey, P. and Smith, P., 1999, “Assessing The Learning Organization: Part 2 - Exploring Practical Assessment Approached”, The Learning Organization, 6.3; pp. 107-115.

Watkins, K.E. and Marsick, V.J., 1993, Sculpting the Learning Organisation, San Francisco, Jossey-Bass.

Wetzel, D.K. and Buch, K., 2000, “Using A Structural Model To Diagnose Organizations And Develop Congruent Interventions”. Organization Development Journal, Winter, 18.4; pp. 9-19.

Wishart, N.A., Elam, J.J. and Robey, D., 1996,   “Redrawing The Portrait Of A Learning Organisation: Inside Knight-Ridder”, The Academy of Management Executive, February, 10.1; pp. 7-21.


Contact the Author:

Dr. Deborah Blackman, University of Canberra, School of Business and Governance, Canberra, ACT, 2601 Australia; Tel: 61 (2) 6201 5076; Email: deborah.blackman@canberra.edu.au