http://bmj.com/cgi/content/full/324/7330/156

BMJ 2002;324:156-159 ( 19 January )

Learning in practice

Learning needs assessment: assessing the need

Editorial by Goldbeck-Wood and Peile

Janet Grant, professor of education in medicine. 

Open University Centre for Education in Medicine, Milton Keynes MK7 6AA

j.r.grant@open.ac.uk

Learning needs assessment has a fundamental role in education and training, but care is needed to prevent it becoming a straitjacket

It might seem self evident that the need to learn should underpin any educational system. Indeed, the literature suggests that, at least in relation to continuing professional development, learning is more likely to lead to change in practice when needs assessment has been conducted, the education is linked to practice, personal incentive drives the educational effort, and there is some reinforcement of the learning.¹ Learning needs assessment is thus crucial in the educational process, but perhaps more of this already occurs in medical education than we suspect. The key lesson might be for those who design new systems of education and training: for example, the postgraduate education allowance system in general practice was felt to fail the profession because it did not include needs assessment and so led to ad hoc education to fulfil the time requirements of the system rather than the needs of individual doctors or the profession as a whole. On the other hand, basing learning in a profession entirely on the assessment of needs is a dangerous and limiting tactic. So a balance must be struck.

In 1998 both individual and organisational needs assessment became part of government policy in relation to the continuing professional development and personal development plans of all healthcare professionals.² Thus, it has a role in the clinical governance of the service³ and is therefore much more than an educational undertaking. This integration of needs assessment, education, and quality assurance of the service was first made explicit in 1989 in relation to clinical audit, which would identify practices in need of improvement and ensure that educational and organisational interventions were made to address these needs.⁴ Accordingly, audit was described as "essentially educational" and the educational process surrounding it described.⁵

Long before these recent developments, needs assessment outside medicine was presented as an important part of managed education and learning contracts, which are the predecessors of the personal development plans to be developed for all NHS healthcare professionals.⁶ In his descriptions of adult learning Knowles assumed (he did not claim to have research evidence) that learners needed to feel a necessity to learn and that identifying one's own learning needs was an essential part of self directed learning.⁷ In medicine a doctor's motivation to learn would therefore derive from needs identified during his or her experience of clinical practice. So the pedigree and practice of learning needs assessment, if not the evidence, are well established.

As in most areas of education, for many years there has been intense debate about the definition, purpose, validity, and methods of learning needs assessment.⁸ It might be to help curriculum planning, diagnose individual problems, assess student learning, demonstrate accountability, improve practice and safety, or offer individual feedback and educational intervention. Published classifications include felt needs (what people say they need), expressed needs (expressed in action) normative needs (defined by experts), and comparative needs (group comparison).⁹ Other distinctions include individual versus organisational or group needs, clinical versus administrative needs, and subjective versus objectively measured needs.¹⁰ The defined purpose of the needs assessment should determine the method used and the use made of findings.

Furthermore, even though the concept of educational needs assessment is enshrined in practice, policy, and the educational canon, several factors indicate the need for careful planning and research in this subject (see boxes 1 and 2). Exclusive reliance on formal needs assessment in educational planning could render education an instrumental and narrow process rather than a creative, professional one. This is especially so in a profession where there is inherent unpredictability and uncertainty. Members of any profession require wide knowledge and depth of experiencethe relevance of some of which might not have been obvious at the time of learning. Certainly, learning needs can and should be identified on the basis of what has been experienced and of what more experienced members of the profession know to be relevant, but this must not deter other, more general or even speculative, learning that, at the time, seems to answer no specific need. Possibly no specific learning needs assessment would ever send a person to a large international conference on a generic subject (such as endocrinology, medical education, or management). It is, nevertheless, important that doctors attend such meetings and return with the unexpected and expected benefits that they accrue.

Although the literature generally reports only on the more formal methods of needs assessment, doctors use a wide range of informal ways of identifying learning needs as part of their ordinary practice. These should not be undervalued simply because they do not resemble research. Questionnaires and structured interviews seem to be the most commonly reported methods of needs assessment, but such methods are also used for evaluation, assessment, management, education, and now appraisal and revalidation.¹¹ Together, these formal and informal methods might make an effective battery where there is clarity of purpose. The Good CPD Guide details 46 formal and informal methods of self assessment (see box 3).¹²

The methods listed are both formal and informal, planned and opportunistic, showing that day to day work and encounters have the potential to generate needs as much as do formal methods. Formal needs assessment methods include critical incident techniques, gap analysis, objective knowledge and skills tests, observation, revalidation, self assessment, video assessment, and peer review. Such methods are often used to identify group needs. ^{14 15} Formal identification of needs can also arise from audit, morbidity patterns, adverse events, patient satisfaction surveys, and risk assessment. Most of these tools use quantitative methods that can generate computerised data and cover wider population ranges, but these are often unable to probe into the personal agendas and opinions of individuals.

Types of needs assessment
Methods of needs assessment can be classified into seven main types, each of which can take many different forms in practice.

Gap or discrepancy analysis
This formal method involves comparing performance with stated intended competenciesby self assessment, peer assessment, or objective testingand planning education accordingly. ^{9 16 17}

Reflection on action and reflection in action
Reflection on action is an aspect of experiential learning and involves thinking back to some performance, with or without triggers (such as videotape or audiotape), and identifying what was done well and what could have been done better. ^{18 19} The latter category indicates learning needs.

Reflection in action involves thinking about actual performance at the time that it occurs and requires some means of recording identified strengths and weaknesses at the time. The Canadian MOCOMP programme uses formalised reflection as its basic process.²⁰ Similarly, PUNs and DENs (see box 3) are well known in British general practice.

Self assessment by diaries, journals, log books, weekly reviews
This is an extension of reflection that involves keeping a diary or other account of experiences.²¹ However, practice might show that such documents tend to be written nearer the time of their review than the time of the activity being recorded.

Peer review
This is rapidly becoming a favourite method. It involves doctors assessing each other's practice and giving feedback and perhaps advice about possible education, training, or organisational strategies to improve performance. The Good CPD Guide describes five types of peer reviewinternal, external, informal, multidisciplinary, and physician assessment.¹¹ The last of these is the most formal, involving rating forms completed by nominated colleagues, and shows encouraging levels of validity, reliability, and acceptability. ^{22 23}

Observation
In more formal settings doctors can be observed performing specific tasks that can be rated by an observer, either according to known criteria or more informally. The results are discussed, and learning needs are identified. The observer can be a peer, a senior, or a disinterested person if the ratings are sufficiently objective or overlap with the observer's area of expertise (such as communication skills or management).

Critical incident review and significant event auditing
Although this technique is usually used to identify the competencies of a profession or for quality assurance, it can also be used on an individual basis to identify learning needs.²⁴ The method involves individuals identifying and recording, say, one incident each week in which they feel they should have performed better, analysing the incident by its setting, exactly what occurred, and the outcome and why it was ineffective.

Practice review
A routine review of notes, charts, prescribing, letters, requests, etc, can identify learning needs, especially if the format of looking at what is satisfactory and what leaves room for improvement is followed.

Needs assessment is not the same as assessment in the sense of examination of learning. Assessment systems that lead to academic or professional awards should show certain minimum characteristics, including measurement of performance against external criteria and standards, a decision on adequacy by an assessor, and standardised data gathering.²⁵ Needs assessment might sometimes have these characteristics, but it also might be based on practice, reflection, professional judgment, discussion, and informal data. Needs assessment methods that are limited by the standards of assessment will fall into the trap of assessing only a narrow range of needs.

The main purpose of needs assessment must be to help educational planning, but this must not lead to too narrow a vision of learning. Learning in a profession is unlike any other kind of learning. Doctors live in a rich learning environment, constantly involved in and surrounded by professional interaction and conversation, educational events, information, and feedback. The search for the one best or "right" way of learning is a hopeless task,¹ especially if this is combined with attempting to "measure" observable learning. Research papers show, at best, the complexity of the process.

Multiple interventions targeted at specific behaviour result in positive change in that behaviour.²⁶ Exactly what those interventions are is less important than their multiplicity and targeted nature. On the other hand, different doctors use different learning methods to meet their individual needs. For example, in a study of 366 primary care doctors who identified recent clinical problems for which they needed more knowledge or skill to solve, 55 different learning methods were selected.²⁷ The type of problem turned out to be the major determinant of the learning method chosen, so there may not be one educational solution to identified needs.

Much of doctors' learning is integrated with their practice and arises from it. The style of integrated practice and learning ("situated learning") develops during the successive stages of medical education.²⁸ The components of apprenticeship learning in postgraduate training are made up of many activities that may be regarded as part of practice (see box 4).²⁹ Senior doctors might also recognise much of their learning in some of these elements and could certainly add moresuch as conversations with colleagues.

Thus, educational planning on the basis of identified needs faces real challenges in making learning appropriate to and integrated with professional style and practice. The first step in all of this is to recognise the needs assessment and learning that are a part of daily professional life in medicine and to formalise, highlight, and use these as the basis of future recorded needs assessment and subsequent planning and action, as well as integrating them with more formal methods of needs assessment to form a routine part of training, learning, and improving practice.

   Competing interests: None declared.

(Accepted 17 December 2001)

© BMJ 2002

Related editorials in BMJ:

Learning in practice: a new section in the BMJ.

Sandra Goldbeck-Wood and Ed Peile
BMJ 2002 324: 125-126. [Full text]  

Other related articles in BMJ:

EDITOR'S CHOICE
Health systems: where doctors and patients meet.

BMJ 2002 324: 0. [Full text]  

ALL INFORMATION, DATA, AND MATERIAL CONTAINED, PRESENTED, OR PROVIDED HERE IS FOR GENERAL INFORMATION PURPOSES ONLY AND IS NOT TO BE CONSTRUED AS REFLECTING THE KNOWLEDGE OR OPINIONS OF THE PUBLISHER, AND IS NOT TO BE CONSTRUED OR INTENDED AS PROVIDING MEDICAL OR LEGAL ADVICE.  THE DECISION WHETHER OR NOT TO VACCINATE IS AN IMPORTANT AND COMPLEX ISSUE AND SHOULD BE MADE BY YOU, AND YOU ALONE, IN CONSULTATION WITH YOUR HEALTH CARE PROVIDER.