Managerial assessment of proficiency

Dr. Scott Parry, founder of Training House, Inc.,creator of the tool and 1999 inductee into the ASTD (ATD) and, Academy of Resource Development, Halls of Fame, developed the MAP competency framework by first analysing the results of a series of large competency studies conducted by a number of major organisations. Dr. Parry, sold Training House in 1999 to HRD Press, Inc. who authorized Development Processes Group, plc as the exclusive representative and developer of the MAP assessment within the United Kingdom. Cliff Lansley leads the further development of MAP with fellow Director Cameron Robertson has used the tool in studies across hundreds of major public and private sector organisations and he says' "we now have powerful benchmarking data of leadership competencies across most organisational sectors. What is unique here, is that MAP is the only objective diagnostic tool for managerial and leadership compenece measurement that exists in the world. All other tools are based merely on self-report, 360 degree opinion or subjective judgement by an assessor".

The studies looked at a number of leading corporations to identify the competencies and attributes that were important to the performance of managers. IBM, AT&T, Henley Management College, Ford and Kodak came up with very similar managerial competencies and these were analysed. The developers of MAP selected 12 of the most frequently mentioned competencies, those common to most frameworks, and arranged them in 4 clusters and 2 broad categories: task-handling (left column) and people-handling (right column).

Dr. Scott Parry, developer of the MAP Assessment competency framework and tool, defines competencies as "a group of related skills, knowledge, and attitudes that correlate with success in one’s job and that can be improved through training”.

The assessment involves studying a series of real-life management situations covering a week in the life of a typical manager and his team and answering a series of questions about your judgement of what you've seen.

The situations covered include:

• team meetings
• time management
• delegating
• discipline and empathy
• appraising staff
• performance management
• problem solving
• listening.

Following each short scene the delegates answer a series of questions based on what they have been watching. They are then asked if, in their opinion, the management behaviour they have seen is good or bad practice.

The scenarios are very effective in getting managers to be a part of the assessment process because they are based on the real situations that they regularly have to deal with on a day-to-day basis, resulting in high face validity scores.

There are also two paper-based questionnaires that help determine their preferred styles of managing and communicating and the impact that can have in the workplace.

At the end of the day the assessment is scored online and a personal profile produced within minutes. This is in readiness for Day 2 of the MAP Assessment process.

The second day of the process is the interpretation of the assessment scores providing delegates with a development ‘road map’ for their ongoing development.

If a team is being assessed then a group profile is also produced to take into account the department or team's training and development needs.

Before making the assessment available to clients, Cliff Lansley, one of the developers of MAP validated the methodology in 11 organisations. Managers were selected to cover the full range of proficiency at work from ‘excellent’ to ‘below average’. Working independently, three senior managers assigned ratings on a 5-point scale to each manager being assessed, thereby establishing a rank order.

A total of 253 managers from 11 organisations went through the video-based assessment. Their overall proficiency percentiles (average of the 12 competency scores) were compared with their senior managers’ ratings of their performance at work, using the Spearman's rank order correlation analysis.

Correlations were positive, ranging from .71 to .92. This supported the following assumptions:-

1. the competencies that were assessed enabled the developers to discriminate between high performing managers and their less effective counterparts,
2. the assessment instrument could therefore be used as a predictive index of one's performance on the job,
3. the process constituted the basis for a ‘needs analysis’ for identifying training and development opportunities.

Cliff Lansley, Director and owner of Development Processes Group plc, the lead organisation for MAP across the UK, is undertaking an extended research and validation exercise and the results will be referenced on completion.

The construct validity can be measured by the degree to which the developers’ assessment data agrees with the actual performance of managers at work. But participants are also concerned with face validity, the degree to which they can identify with the video episodes and accept their proficiency profile as accurate. On the post-assessment evaluation sheet, 92% said they had no difficulty relating to the episodes, and 86% said that the scores they received were probably accurate.

A further examination of the list of 12 competencies will support the assumption that they can all be improved via training. This is in contrast to the characteristics, qualities, and attributes that appear in some organisations’ lists of competencies¼ self-confidence, initiative, flexibility, ambition and so on. These are best defined as personality traits, not competencies, according to Dr. Scott Parry. They are typically formed early in life and, short of clinical intervention, are not subject to significant change through participation in a training programme. The developers of MAP restricted the assessment to competencies that can be developed through training.

Famous chef, Raymond Blanc, experienced the tool himself and he outlines in the link below how he used MAP within his organisation and found it to be a good predictor of managerial performance.

MAP has also been used as a central part of qualifications accredited by the Institute of Leadership and Management and the Chartered Management Institute.

MAP has been deployed across the UK by a range of Licensed Delivery Centres by Development Processes Group PLC. One of its centres, Metricate Ltd, have supported its work in assessing and researching competencies in the construction, engineering and logistics industry sectors. The goal was to identify the implications of any deficiencies for the businesses concerned and assess how and where they impacted upon the organisation. (‘Understanding the Real Skills Gap: Management and Leadership’ 2008, Metricate Ltd).

The specific management competencies in deficit on this study, according to a MAP analysis, were identified as:-

1. Training, coaching and delegating
2. Thinking clearly and analytically
3. Listening and organising

From follow-up interviews with managers at Company Board level, there were a number of strategic and operational problem areas for companies stemming from the deficit in the management competencies described:-

1. problems with succession planning for the business and key staff
2. low profitability: chasing turnover at the sacrifice of margin
3. poor productivity: unproductive use of staff at all levels
4. response to globalisation: failure to grow and compete with larger rivals
5. new environmental, technology, techniques, legislation: struggling to respond and develop commercial advantage
6. aging workforce: inability to recruit and retain sufficient new staff
7. skills gaps: staff lacked the management and technical skills to compete successfully

The work of Metricate Ltd went on to confirm in their study that development programs were most effective in resolving the deficiencies in competence where they met the following program design specifications:-

1. A thorough needs analysis undertaken at the outset, to support an agreed understanding of the problems and desired outcomes
2. MAP measurement before and after the intervention, and against relevant industry and national benchmarks
3. The development program should be led ‘from the top down’, with active endorsement by the directors
4. Training program targeted at three levels simultaneously: the individual, teams and the organisation
5. Customized to organizational challenges and personalized to individuals’ needs: this is achieved through interviews, following MAP assessment
6. Modes of delivery matched to all learning styles: such as workshops, coaching and self-study
7. Development of a cross-functional approach involving different teams/ departments, to ensure an integrated understanding of the issues for the organization
8. Establish self and organizational awareness, to embed a culture of autonomous development
9. Programme over an extended period – ‘little and often’ – up to 12 months, to ensure assimilation and minimise the disruption of training
10. Post development assessment to measure progression and ROI (Return on Investment)

• Human resources

• Assessment Information
• Linked Training
• studies and product details about MAP
• a round table discussion on MAP with Cliff Lansley
• the national training and development organisation responsible for MAP, based in the UK
• the MAP tool explained in detail
• Raymond Blanc's experience with the diagnostic tool
• HRD, Academy of Resource Development, Hall of Fame – the Award is presented to scholars in human resource development and related disciplines http://www.ahrd.org/mc/page.do?sitePageId=56783
• Metricate, one of the DPG plc Licensed Delivery Centres providing assessment and development programmes for the construction, engineering and logistics sectors
• Article: Performance management masterclass: the fifth TJ roundtable discussion hosted by DPG plc and Dr Peter Honey examined the impact performance assessment has had on the ever-changing role of the manager

1. HRD Manager's Route to Competence: Cliff Lansley; Developing a Strategic Approach to HRD v. 3 (Paperback); ISBN 1-86093-006-9

1. Design Learning Programmes: Cliff Lansley; (Paperback0; ISBN 1-86093-007-7

1. Training for Results; Scott Parry; ASTD March 7, 2000; ISBN 1-56286-132-8