Guest posting to voced-coord email list
Item 7 of 8: New Standards for Assessors and Trainers

Date: Tue, 27 Jan 1998 11:24:09 +1100
To: voced-coord@owl.qut.edu.au
From: Patrick Griffin (p.griffin@edfac.unimelb.edu.au)
Subject: VocEd:Patrick Griffin guest - (new standards for Assessors and trainers)

A belated Happy New Year to all the onliners. It has been a while since I last sent a discussion to the online group. A mixture of network improvement works and Christmas and New Year intervened. However, it seems that most are starting to come on line again and judging by the cries for communication recently, the chat groups serve an important interpersonal as well as professional purpose.

I enjoyed the coffee debate at the end of the year and the discussion about levels of competence that it focussed on. Perhaps there is room for assessing competence, performance and product. In the case of the coffee this would be the task to make coffee, the process used in making a coffee the quality of the coffee. In the end everyone seemed to agree that it was the quality of the coffee that counted.

This focus on evidence is not too far removed from the work we have been doing here at the University of Melbourne on competency assessment. We are currently finalising the development of the National Training Package for Assessment and Training. I thought I'd outline what we have done and ask for your comments particularly how it might impact on the role of the teacher in VET/industry partnership programs.

A training package consists of three components. The three are the standards, the assessment guidelines and the qualifications of assessors. All endorsed national training packages contain these three sections. Our package sets out what an assessor must know and do to conduct assessments against any set of workplace competency standards. In this way, the new training package has relevance for every VET teacher involved in Traineeships and working in any industrial setting.

What is the Training and Assessment and National Training Package?

The Training Package consists of a combination of endorsed and non-endorsed components. An 'endorsed' component becomes the mandatory materials for assessing against national competency standards for the purpose of issuing any vocational credential. A non-endorsed component contains the materials that may vary from training provider to training provider. These include advice but no mandatory procedures or materials. At the risk of explaining the obvious, I thought it might be worthwhile to review the Training and Assessment Package background.

Endorsed Components
1. Assessor and Trainer Competency standards complete with required evidence of competence;
2. Assessment guidelines, which provide advice and basic knowledge required; and
3. Qualifications of assessors and trainers at different levels of the Australian Qualifications Framework (AQF).

The non Endorsed Components
1. Possible learning strategies
2. Professional development materials for Assessors and Trainers; and
3. Exemplar Assessment materials for use in assessing against the competencies for assessors and trainers

Our task has been to develop the endorsed component of the training package, that is to define the standards, provide the guidelines and establish the qualifications for those who wish to work as trainers and or assessors in workplace settings or in VET where national vocational credentials are issued.

Competency Standards.
National assessment and training competency standards provide a national benchmark for conducting assessments and delivering training. Anyone conducting assessments against any industry competency standard must be registered as an assessor and be proven competent against the both the industry standards for a specific industry and the assessment competency standards. A teacher qualification does not necessarily qualify a person to conduct assessments against the competencies. If the person conducting the assessment is not competent against the industry standards, they have to work in conjunction with someone who is. Every assessor however, regardless of technical expertise, must have demonstrated competence against the assessment standards before they can conduct an assessment for national recognition.

The Assessment Guidelines.
The guidelines provide the overall framework for assessment against the competency standards for assessors and trainers. They are intended to ensure that the competency standards are used as the basis for assessment and for awarding assessor or trainer qualifications under the Australian Qualifications Framework (AQF).

The Guidelines set out the mechanisms and processes to be used so that the chances of producing valid, reliable, fair and flexible assessments of competency against the Assessment and Training Competency Standards are increased. They apply to assessment and/or training whether on-the-job or off the job.

Qualifications
The national qualifications created by packaging of groups of competencies from the Assessment and Training Competency Standards forms the third component of the training package. For the first time, it is possible to gain a qualification in assessment as well as in training.

Importance of Training Package

The information and advice provided by the Training Package is particularly timely in light of a number of recent developments in the vocational education and training sector. Assessment against the competencies is now the major currency of credentials. Once anyone demonstrates competency against a sufficient number of competencies, they are entitled to be issued with a credential regardless of how or where they acquired the competencies.

The assessor's role increases in importance in this context. It is no surprise therefore that government policy now requires any individual conducting assessments for a national qualification whether in the workplace or in a training institution (including schools) should be competent against the Competency Standards for Assessment and Training. All industry training packages are expected to include assessment guidelines, which address assessor qualifications and training. All assessors, including teachers wishing to conduct VET assessments against national competency standards in any industry must be competent against:

• Unit AST 1.1 Conduct Assessment in Accordance with an Established Assessment Procedure
• Unit AST 2.1 Plan and Establish Assessment Procedures
• Unit AST 2.2 Review Assessment

Assessors are expected to use these competencies within an established industry or enterprise based assessment and/or training system. Attainment of these units is a minimum requirement for registered training organisations to conduct assessments for purposes of national qualifications.

People working in any assessment system with specialist functions, such as developing assessment tools and procedures, designing, establishing and managing an assessment system, will be expected to be competent in the following additional units:

• Unit AST 3.1 Analyse Training/Assessment needs
  
• Unit AST 3.2 Develop Assessment Tools
  
• Unit AST 3.3 Design the Assessment/training System
  
• Unit AST 3.4 Establish the Assessment/training System
  
• Unit AST 3.5 Manage the Assessment/training System
  
• Unit AST 3.6 Evaluate the Assessment/Training System

Why?
Assessment is now the basis of all vocational credentials. Workplace assessment can be conducted by anyone competent in the industry standards and who can demonstrate that they have an understanding of assessment procedures. The importance of fair and reliable assessments given the life chances involved for those who are assessed cannot be overstated. The units provide minimal training for people who wish to act as assessors in any workplace or VET setting. For those who will train, assess or manage workplace assessors the demands are higher and the specialist units are designed to cater for these circumstances. Given the importance of this, we have included a code of practice for assessors. Perhaps teachers would like to comment on how these will affect their work in VET.

Who is it for?
The Training Package is intended for persons who are involved in the implementation of the assessment of anyone against Competency Standards in any type of industry or size of organisation. Also a code of practice has been proposed.

An assessor's code of practice.

Assessment specialists have developed a code of ethics and practice (The National Council for Measurement in Education (NCME), 1995. These sets of standards have been modified to provide this the code of practice for competency-based assessment and training.

• The differing needs and requirements of the person(s) being assessed, the local enterprise(s) and/or industry are identified and handled with sensitivity.
• Potential forms of conflict of interest in the assessment process and/or outcomes are identified and appropriate referrals are made, if necessary.
• All forms of harassment are avoided throughout the planning, conduct, reviewing and reporting of the assessment outcomes.
• The rights of the candidate(s)are protected during and after the assessment.
• Personal or interpersonal factors that are not relevant to the assessment of competence must not to influence the assessment outcomes.
• The candidate(s)is made aware of rights and processes of appeal.
• Evidence that is gathered during the assessment is verified for validity, reliability, authenticity sufficiency and currency.
• Assessment decisions are based on available evidence that can be produced and verified by another assessor.
• Assessments are conducted within the boundaries of the assessment system policies and procedures.
• Formal agreement is obtained from both the candidate(s)and the assessor that the assessment was carried out in accordance with agreed procedures.
• Assessment tools, systems, and procedures are consistent with equal opportunity legislation.
• The candidate(s)is informed of all assessment reporting processes prior to the assessment.
• The candidate(s)is informed of all known potential consequences of decisions arising from an assessment, prior to the assessment.
• Confidentiality is maintained regarding assessment results.
• Results are only released with the written permission of the candidate(s).
• The assessment results are used consistently with the purposes explained to the candidate.
• Self - assessments are periodically conducted to ensure current competencies against the Assessment and Training Competency Standards.
• Professional development opportunities are identified and sought
• Opportunities for networking amongst assessors are created and maintained.
• Opportunities are created for technical assistance in planning, conducting and reviewing assessment procedures and outcomes.

A large range of assessment practices has been recommended for use in competency assessment against these standards. I wonder how many of you use these approaches in the workplace or against VET programs already. Here they are:

1. Adaptive Testing

• Sequence of items or tasks depends on responses to previous items or performance on previous tasks
• Performance is assessed via difficulty of tasks completed satisfactorily as opposed to number of tasks completed.

2. Anecdotal Records

• Short narrative report/summary of an event or activity
• Used to support generalisations about the performance of an individual/group.

3. Behavioural Checklist

• Record or form
• Assessor indicates the presence or absence of a specific behaviour
• Assessors need to memorise the recording code, practice making recordings, receive feedback regarding the relative accuracy of their recording.

4. Behaviourally Anchored Rating Scale (BARS)

• Judgements about performance are empirically linked to specific examples of incumbent performance at each level of effectiveness on the scale
• Typically rating recordings are made after the observation.

5. Computerised Assessment

• Uses computers to measure performance on some attribute not necessarily related to computer skills.

6. Essay

• Oral or written narrative description, analysis, explanation, interpretation, opinion and/or summary that demonstrates an individual's use of information and language to generate a coherent document about a particular topic or in response to a question or prompt
• A scoring process needs to be included.

7. In-Basket (in-tray) Test

• Individual receives a collection of documents, letters, memoranda and other materials each of which requires a decision or action
• Within specified time, the individual being assessed must review the materials, determine what actions need to be taken, and establish priorities for completing them.

8. Log

• Journal or diary maintained by individual or group
• May include plans, activities, decisions, work undertaken and competed, time spent, products produced, results, changes and reflections.

9. Observation

• Direct observation of performance in any suitable context
• Records may be made on checklists for later referral and interpretation.
• Use of a recording process that specifies criteria or indicators of merit
• Assessor marks the presence or absence of the attribute being assessed
• Criteria may or may not refer to specific behaviours.

10. Paper Pencil Tests

• Consists of items, questions or problems to be answered by the individual in writing or by marking an answer document
• Usually consist of multiple choice items e.g. matching, true/false, may also include other items e.g. fill in the blank, underline the error, label a diagram or map.

What do you think???

This is my last "guest appearance". I have enjoyed the discussions and the comments. Unlike the other guests I have not had the opportunity to get involved in specific discussions with each of the respondents. The end of the year proved to be far too hectic with commitments here (apart from Christmas parties). I have now just returned from overseas and Janine's reminder to wrap it up.

I trust that you all have a great year and I hope to meet many of you this year as my colleagues and I help the ASTF to develop some assessment materials for Traineeships. If you think you can help in this field (and want to make your mark) let me know.

Patrick Griffin