|
|
|
Indian Pediatr 2015;52:
231-235 |
 |
Portfolio-based Learning and Assessment
|
|
*Mohit Kumar Joshi, #Piyush
Gupta and Tejinder Singh
From *Department of Surgery, AIIMS, New Delhi; #Department of
Pediatrics, University College of Medical Sciences, New Delhi; and
CMCL-FAIMER Regional Institute, Christian Medical College, Ludhiana,
Punjab; India.
Correspondence to: Dr Tejinder Singh, Program Director, CMCL-FAIMER
Regional Institute, Christian Medical College, Ludhiana, Punjab, India.
Email: [email protected]
|
Assessment using portfolios has recently gained wider
acceptance, and is being considered relevant to several educational
aspects in medicine, including formative and summative assessment during
resident training, revalidation and continuing professional development.
In this article, we provide an overview of the use of portfolio as an
assessment and learning tool. We have discussed the rationale of using
portfolio in medical education, its advantages and criticisms, and some
of the associated challenges and dilemmas.
Keywords: Assessment, Feedback, Portfolio, Reflective
learning.
|
|
A
ssessment is a crucial component of educational
system. It is intended to serve the twin purpose of prove and improve.
However, the procedure of assessment in our medical education system is
archaic and needs improvement. The existing assessment system largely
focuses on the outcome of summative assessment. The performance of the
student in the summative assessment is considered as a proof of his
competence, on the basis of which he is provided the license to
practice. However, this snapshot of the student’s performance in a high
stake examination may not be a true reflection of his knowledge and
competence as this fails to take into account his learning and attitude
over the period of his medical training. The summative assessment also
lacks in the content validity and there always remains a possibility of
examiner bias. The fate of the student in such summative assessment also
largely depends on the ‘chance’ or ‘luck’ of getting an easy case, a
lenient examiner, and ‘help’ from seniors or peers.
It is also accepted that mere acquisition of
knowledge and skills is not a proof that the student is capable of
applying this effectively to a real-life scenario. The understanding
that the student ‘knows’ occupies the lowest strata in the Miller’s
pyramid of assessment [1].
More important would be to assess whether the student knows how to apply
this knowledge and how he actually performs it to achieve the higher
levels in Miller’s pyramid of assessment.
There is no attempt, presently, to assess
communication skills, professionalism and empathy which are so vital and
desirable qualities of a doctor. The assessment largely focuses on the
presentation rather than on clinical skills. Another issue with
conventional assessment is lack of opportunity to provide feedback to
the student and not providing opportunity to the student to reflect on
his learning.
A solution to these would be a system where the
‘doctor in making’ is assessed frequently on a day-to-day basis by
various assessors on a range of aspects of his medical training,
including academic as well as non-academic attributes. Numerous tools
have been proposed to achieve this and use of portfolio is one of them.
In literature, portfolio has variously been referred
to as ‘collection of papers and other forms of evidence to document
learning, learning journal or learning diary’ [2-4].
The same may apply to logbooks too as essentially
logbook is also a document to suggest that learning has taken place.
Portfolios, however, should not be confused with a logbook. While a
logbook is maintained by a trainee to record the learning experiences,
portfolio in addition includes critical reflection on such learning
experiences in learner’s own words. This self-reflection component forms
the heart of portfolio and differentiates it from a logbook. This can be
further understood by the definitions provided by some of the stalwarts
of education. Hall defined portfolio as "a collection of
material, made by a professional that records, and reflects on the key
events and processes in that professional’s career’ [5].
Paulson in his article defined portfolio as: ‘a
purposeful collection of student work that exhibits the student’s
efforts…the collection must include…evidence of self-reflection" [6].
The reflective process is crucial to the portfolio
based learning as it stimulates the thought process of the learner,
which helps in deeper understanding of the subject.
Introduced in early 1990s, portfolios are
increasingly being used in undergraduate and postgraduate training both
for formative and summative assessments and for continuing medical
education (CME) of practicing doctors [4,7].
In addition, their role has also been suggested in
licensing examinations and promotion of faculty [2,8].
Rationale for Using Portfolio in Medical Education
Adult learning, which is self-directed, improves with
reflection on one’s accomplishments, strengths and weaknesses.
Literature suggests that the use of portfolios improves the awareness of
the student of his learning, and thereby provides avenues for
self-improvement. In addition, by looking at the records of a trainee,
the teacher or mentor can also assess the needs of the trainee and can
provide him the required help [9].
A recent systematic review demonstrated that
portfolios effectively enhance personal and professional learning in
postgraduate training. It suggested that portfolios enhance
self-learning and support learning processes and outcomes [10].
In addition, various studies have demonstrated
other benefits of portfolio-use in medical education such as improved
tutor awareness of student’s need, emotional support for students,
preparation for postgraduate training, improved clinical learning,
improvement in organization skills, development of collaborative
learning, and improvement in assessment system [11,12] (Box 1).
However, some studies have found limited evidence
to support their educational impact in undergraduate setting [4,9].
|
Box 1:
Why use Portfolio in Learning and Assessment? |
|
• Traditional system of assessment has many inherent
deficiencies.
• Use of portfolio is one way of avoiding
some of these deficiencies.
• Use of portfolios
• Enhances self-learning
• Improves tutor
awareness of student’s need and support required for
students.
• Helps in preparation for PG training.
• Improves clinical learning.
• Improves the IT and organization
skills.
• Develops collaborative learning.
• Improves the assessment system.
|
Types of Portfolios
On the basis of the type of content and their
intended use, the portfolios can be of following three types:
1. Learning portfolio: This portfolio is
used for recording the reflections on learning experiences by the
learner. The learner controls the content and decides the things to
be included. Such portfolios are for the help of the learner only,
and may not be appropriate for evaluation in training programs. Such
portfolios are used during a residency/ training program with
primary aim of self-reflection and formative assessment.
2. Showcase portfolio: This portfolio is
used to demonstrate one’s best work only. Like learning portfolio,
the content is controlled by the person who creates it, but the
content expresses pre-eminent work only. Such portfolios are used
for summative assessment.
3. Assessment portfolio: The aim of this
portfolio is to document what the student has learned. The content
therefore is controlled by the designer and must include the
reflection on the specific learning outcomes, including knowledge,
skills and attitude. The designing of these portfolios should be
diligent enough to generate evidence to evaluate the extent of
required learning.
On the basis of the medium used to record the
portfolios, they can be divided into:
1. Paper-based portfolios: The contents of
portfolio are recorded on papers.
2. Electronic portfolios (e-portfolios):
The content can be recorded electronically in computer -based tools.
Reflection: An Integral Component of Portfolio
Reflection is the essence of portfolio. Without
reflection, portfolio is reduced to mere collection of papers or
e-material similar to a logbook. Reflection has been defined as
"integration of knowledge with action through thought" [13]..
Al Shehri, et al. [14]
defined it as "a cycle of paying deliberate,
systematic and analytical attention to one’s own thoughts and actions,
feelings, thinking and in relation to particular experiences for the
purpose of enhancing perceptions of and responses to current and future
experiences". Malinen
in her book has reviewed the work of five leading experimental learning
theorists, and have observed that reflection is the key component in
Kolb’s, Knowles and Schon’s paradigm [15]. In simple words, Reflection
refers to scrutinizing one’s own learning experience in order to take a
closer look at what has been experienced to explore it in depth.
Reflection can often be performed simultaneously during a learning
exercise or soon after its completion. The idea of reflection is to
inspect the learning experience rather than just living it.
Thinking and writing what has been experienced helps
one to clarify his thoughts and feelings. It digs out the intricacies of
the subject which may not spring up when one is just undergoing an
exercise without looking at the process. Reflection exposes the learner
to the possibilities of meaningful learning which is derived not from
books but from one’s own understanding of his learning experience.
Criticisms Against the use of Portfolio
Despite evidence about their utility, portfolios have
still not gained universal acceptance [7,16]. While one of the reasons
could be the common misconcepts of equating the portfolio with logbook,
other reasons that have prevented their widespread use include:
1. Time consuming: Residents report lack
of time to maintain the records, especially when it includes
reflection. This is understandable as maintaining portfolio add to
the resident’s already busy schedule [6,9].
2. False or incomplete information: When
the candidate is aware that the information supplied in portfolio
can be used for assessment purposes, he may be reluctant in
recording his shortcomings or may supply false information [17].
3. Concern over originality: It may be
difficult to verify whether the contents of the portfolio,
especially the reflection, are the individual’s own work.
4. Concern over structure: Issues have
been raised that at times the content required to be included in the
learning portfolios may not be relevant [5,10]. This can be offset
by careful planning while designing the portfolios. The learning
outcomes needs to be defined and the portfolio should be structured
so that enough evidence can be collected to assess whether required
learning has occurred or not.
5. Non uniformity in development and
standardization of portfolio [3].
6. High inter-rater reliability due to lack of
non-uniformity in the structure of the portfolio and training of the
assessor.
Although the available literature supports the
positive educational impact of portfolio-based learning, this has been
met with mixed success [5].
The major reasons cited are poor understanding of the purpose of
portfolio among the students and teachers, and rigid structure of
portfolio. These perceived shortcomings can be overcome by: training of
teachers and students in portfolio use and sensitizing them towards
their benefit, keeping the portfolio slim, and by keeping the portfolio
structure flexible, learner-centered and user friendly.
Assessment Using Portfolios
Assessment using portfolio can be obtained in two
ways, either self-assessment by the learner or external assessment by
the teacher.
The self-assessment integrated with the reflective
practice gives a clear and meaningful picture of ‘what has been learnt
and what remains to be learned’. This helps the learner to understand
his needs and helps him achieve the learning goals. External assessment
is done by the teacher to assess the extent of learning. The role of
teacher is therefore crucial in external assessment. It is essential for
the teacher to be familiar with portfolio and its purpose. External
assessment using portfolio can be utilized both for formative and
summative assessment. In formative assessment, the assessor has to
provide feedback on the learner’s progress and help him identify the new
learning goals. In summative assessment, it is crucial for the assessor
to identify whether the desired learning goals and competencies have
been achieved.
For the purpose of assessment, the assessment
portfolio should be carefully structured so that it reflects the desired
information which can help assess the extent of learning. It is also
desirable to have a standardized format of portfolio for assessment
purposes.
Various authors have suggested ways to improve the
assessment using portfolios. Bird suggested that the evidence in the
portfolio should be arranged according to the competencies the learner
wants to demonstrate [18]. This may make portfolio assessment fast and
more accurate. He suggested that if the portfolio has clear instructions
like ‘Show how you….’ will help the learner to include the relevant
evidence that demonstrates his performance. Collins suggested
highlighting the most informative part of portfolio by attaching a
little sheet called caption. This sheet should contain the information
as to what the document is and why it is important evidence [19].
Integrating Portfolio in the Present System
With the present understanding that portfolios are
beneficial for learning and assessment, there is a need of their more
widespread use. However, it may be difficult to replace the traditional
way of learning and assessment by portfolio-based learning. The change
has to be gradual so that both the learner and the mentor understand the
potential benefits of the tool, and have time to modify the content to
align with the learning goals. The most important initial step would be
the sensitization and training of mentors. As there is no uniformly
agreed design of portfolio, brainstorming is required to design
portfolio that suit the need of the learner and at the same time achieve
the goals of learning. Maintaining the portfolio should be as easy for
the student and as efficient for the assessor, as possible. This can be
accomplished by guiding the students to include few but important
evidences in portfolio which reflect their learning and comprehension.
Wherever there is a provision of self-reflection, there should be word
limit so as to keep the content relevant and the portfolio slim.
Organizational support is vital during the initial implementation of
this new tool. The use of portfolio should not be an additional but
integral part of a training program. As logbooks are being widely used
in undergraduate and postgraduate training programs, we recommend that
the students be asked to incorporate their reflections in the logbooks’
in addition to recording their learning experiences.
One of the authors (TS) has been using reflective
learning as part of a faculty development program. This program involves
monthly discussions on e-mail about various educational topics, the last
week of the month being devoted to writing reflections on learning [20].
The reflections are written using "what happened- so what- what next"
framework and invokes good responses.
The Challenges
Although literature suggests the use of portfolio for
a varied setting, there are still concerns over the structure, type,
uniformity, acceptance, utilization, time-constraint and educational
benefits of the portfolio [9,10]. The biggest challenge at present is to
generate more evidence on the use of portfolio in medical education and
training.
Another challenge is to change the widespread belief
that portfolio are subjective and non-standardized, and cannot be used
for assessment purposes, especially in high stake examinations [21].
However, it is now well accepted that the qualitative data gathered by
portfolio can be well analyzed using holistic assessment techniques.
When subjected to good qualitative analysis, the portfolio achieves
acceptable inter-rater reliability, and this resolves the issue of
portfolio being subjective [8]. The assessment using portfolios can be
further improved by using some uniform ways of scoring portfolios,
assessment by small groups of trained assessors rather than individual
assessment, and training for assessors.
Even with the current evidence, the use of portfolios
in learning and assessment remains underutilized. This may partly be due
to lack of awareness of the existence or perceived educational benefits
of this tool. Medical Education Units have an important role to play
regarding sensitization of this tool to the teachers and students. The
next challenge now would be to obtain the support of organization and
administration to implement and sustain it. A summary of importance of
portfolio in medical education is provided in Box 2.
|
Box 2 : Portfolio
Essentials
• Portfolio includes recording of a learning
experience as well as description of this learning experience in
learner’s own words.
• This description on learning experience is
known as ‘reflection’.
• ‘Reflection’ is the heart of portfolio.
• Without reflection, portfolio will become
merely a logbook.
• Various types of portfolio are: learning
portfolio, showcase portfolio and assessment portfolio.
• When recorded on paper, they are called
paper based portfolio and when recorded in computer based
devices they are known as e-portfolio.
|
The Future
The evidence regarding the outcomes of the use of
portfolio in medical education needs to be strengthened further. This
can be accomplished by incorporating portfolios in medical training on a
larger basis. Given all the problems with the present system of medical
training and assessment and the understanding that portfolios can offset
a number of these deficiencies, the future of portfolios seems bright.
Portfolios have been utilized across a variety of
health care professionals, although not explored at present [10], they
also hold promise for inter-disciplinary learning. Portfolio use holds
the promise to have a learning program that would have a positive
academic effect on the learner’s knowledge, skills and behavior, and a
better, unbiased system of assessment achieving the highest level of
Miller’s pyramid. This would thereby positively influence the
educational system, the learners, and the society.
Contributors: TS: conceptualized the draft; MJ:
wrote the manuscript; PG: provided critical inputs.
Funding: None; Competing interests: None
stated.
References
1. Norcini JJ, Grant J, Wooding K, Mairead M.
Assessing professional competence. In: Singh T, Anshu, editors.
Principles of Assessment in Medical Education. New Delhi: Jaypee; 2012.
p.49-69.
2. Kadgad P, Kotrashetti SM. Portfolio: A
comprehensive method of assessment for postgraduates in oral and
maxillofacial surgery. J Maxillofac Oral Surg. 2013;12:80-4.
3. Donato AA, George DL. A blueprint for
implementation of a structured portfolio in an internal medicine
residency. Acad Med. 2012;87:185-91.
4. Buckley S, Coleman J, Khan K. Best evidence on the
educational effects of undergraduate portfolios. Clin Teach.
2010;7:187-91.
5. Hall D. Professional development portfolios for
teachers and lecturers. Br J Service Educ. 1992;18:81-6.
6. Paulson FL, Paulson PP, Meyer CA. What makes a
portfolio a portfolio? Educ Leadership. 1991;48:60-3.
7. Driessen E, Tartwjik V, van der vleuten C, Wass V.
Portfolios in medical education: Why do they meet with mixed success?
Med Educ. 2007;41:1224-33.
8. Colbert CY, Ownby AR, Butler PM.
A review of portfolio use in residency programs and considerations
before implementation. Teach Learn Med. 2008;20:340-5.
9. Zubizarreta J. The Learning Portfolio: Reflective
Practice for Improving Student Learning. Bolton, MA: Anker; 2004.
10. Tochel C, Haig A, Hesketh A, Cadzow A, Beggs K, Colthart
I, et al. The effectiveness of portfolios for post-graduate
assessment and education: BEME Guide No 12. Med Teach. 2009;31:299-318.
11. Nairn S, O’Brien E, Traynor V, Williams G,
Chapple M, Johnson S. Student nurses’ knowledge, skills and attitudes
towards the use of portfolios in a school of nursing. J Clin Nurs.
2006;15:1509-20.
12. Tiwari A, Tang C. From process to outcome: The
effect of portfolio assessment on student learning. Nurse Education
Today. 2003;23:269-77.
13. Hart M. Liberation through consciousness raising.
In: Mezirow J, editor. Fostering Critical Reflections in
Adulthood. San Francisco: Jossey-Bass;1990 . P.47-73.
14. Al-Shehri A. Learning by reflection in general
practice: a study report. Educ Gen Pract. 1995;7:237-48.
15. Malinen A. Towards the Essence of Adult
Experiential Learning. SoPhie Academic Press. Jyavskyälä; 2000.
16. Pearson DJ and Heywood P. Portfolio use in
general practice vocational training: A survey of GP registrars. Med
Educ. 2004;38:87-95.
17. Davis MH, Ponnamperuma GG. Portfolio assessment.
J Vet Med Educ. 2005;32:279-84.
18. Bird T. The schoolteacher’s portfolio: An essay
on possibilities. In: Millman J, Darling-Hammond L, editors. The
New Handbook of Teacher Evaluation: Assessing Elementary and Secondary
School Teachers. Newbury Park, CA: Corwin Press; 1990.p.241-56.
19. Collins A. Portfolios for biology teacher
assessment. J Pers Eval Educ. 1991;5:147-67.
20. Anshu, Bansal PK, Mennin S, Burdick W, Singh T.
Online faculty development for medical educators: Experiences of a South
Asian program. Education for Health. 2008; 21:175-83.
21. Roberts C. Portfolio-based assessments in medical
education: Are they valid and reliable for summative purposes? Med Educ.
2002;36:899-900.
|
|
|
 |
|
