Home            Past Issues            About IP            About IAP           Author Information            Subscription            Advertisement              Search  

   
medical education

Indian Pediatr 2015;52: 231-235

Portfolio-based Learning and Assessment

*Mohit Kumar Joshi, #Piyush Gupta and Tejinder Singh

From *Department of Surgery, AIIMS, New Delhi; #Department of Pediatrics, University College of Medical Sciences, New Delhi; and CMCL-FAIMER Regional Institute, Christian Medical College, Ludhiana, Punjab; India.

Correspondence to: Dr Tejinder Singh, Program Director, CMCL-FAIMER Regional Institute, Christian Medical College, Ludhiana, Punjab, India.
Email: cmcl.faimer@gmail.com


Assessment using portfolios has recently gained wider acceptance, and is being considered relevant to several educational aspects in medicine, including formative and summative assessment during resident training, revalidation and continuing professional development. In this article, we provide an overview of the use of portfolio as an assessment and learning tool. We have discussed the rationale of using portfolio in medical education, its advantages and criticisms, and some of the associated challenges and dilemmas.

Keywords: Assessment, Feedback, Portfolio, Reflective learning.


A
ssessment is a crucial component of educational system. It is intended to serve the twin purpose of prove and improve. However, the procedure of assessment in our medical education system is archaic and needs improvement. The existing assessment system largely focuses on the outcome of summative assessment. The performance of the student in the summative assessment is considered as a proof of his competence, on the basis of which he is provided the license to practice. However, this snapshot of the student’s performance in a high stake examination may not be a true reflection of his knowledge and competence as this fails to take into account his learning and attitude over the period of his medical training. The summative assessment also lacks in the content validity and there always remains a possibility of examiner bias. The fate of the student in such summative assessment also largely depends on the ‘chance’ or ‘luck’ of getting an easy case, a lenient examiner, and ‘help’ from seniors or peers.

It is also accepted that mere acquisition of knowledge and skills is not a proof that the student is capable of applying this effectively to a real-life scenario. The understanding that the student ‘knows’ occupies the lowest strata in the Miller’s pyramid of assessment [1]. More important would be to assess whether the student knows how to apply this knowledge and how he actually performs it to achieve the higher levels in Miller’s pyramid of assessment.

There is no attempt, presently, to assess communication skills, professionalism and empathy which are so vital and desirable qualities of a doctor. The assessment largely focuses on the presentation rather than on clinical skills. Another issue with conventional assessment is lack of opportunity to provide feedback to the student and not providing opportunity to the student to reflect on his learning.

A solution to these would be a system where the ‘doctor in making’ is assessed frequently on a day-to-day basis by various assessors on a range of aspects of his medical training, including academic as well as non-academic attributes. Numerous tools have been proposed to achieve this and use of portfolio is one of them.

In literature, portfolio has variously been referred to as ‘collection of papers and other forms of evidence to document learning, learning journal or learning diary’ [2-4]. The same may apply to logbooks too as essentially logbook is also a document to suggest that learning has taken place. Portfolios, however, should not be confused with a logbook. While a logbook is maintained by a trainee to record the learning experiences, portfolio in addition includes critical reflection on such learning experiences in learner’s own words. This self-reflection component forms the heart of portfolio and differentiates it from a logbook. This can be further understood by the definitions provided by some of the stalwarts of education. Hall defined portfolio as "a collection of material, made by a professional that records, and reflects on the key events and processes in that professional’s career’ [5]. Paulson in his article defined portfolio as: ‘a purposeful collection of student work that exhibits the student’s efforts…the collection must include…evidence of self-reflection" [6]. The reflective process is crucial to the portfolio based learning as it stimulates the thought process of the learner, which helps in deeper understanding of the subject.

Introduced in early 1990s, portfolios are increasingly being used in undergraduate and postgraduate training both for formative and summative assessments and for continuing medical education (CME) of practicing doctors [4,7]. In addition, their role has also been suggested in licensing examinations and promotion of faculty [2,8].

Rationale for Using Portfolio in Medical Education

Adult learning, which is self-directed, improves with reflection on one’s accomplishments, strengths and weaknesses. Literature suggests that the use of portfolios improves the awareness of the student of his learning, and thereby provides avenues for self-improvement. In addition, by looking at the records of a trainee, the teacher or mentor can also assess the needs of the trainee and can provide him the required help [9].

A recent systematic review demonstrated that portfolios effectively enhance personal and professional learning in postgraduate training. It suggested that portfolios enhance self-learning and support learning processes and outcomes [10]. In addition, various studies have demonstrated other benefits of portfolio-use in medical education such as improved tutor awareness of student’s need, emotional support for students, preparation for postgraduate training, improved clinical learning, improvement in organization skills, development of collaborative learning, and improvement in assessment system [11,12] (Box 1). However, some studies have found limited evidence to support their educational impact in undergraduate setting [4,9].

Box 1: Why use Portfolio in Learning and Assessment?


• Traditional system of assessment has many inherent deficiencies.

• Use of portfolio is one way of avoiding some of these deficiencies.

• Use of portfolios

• Enhances self-learning

• Improves tutor awareness of student’s need and support required for students.

• Helps in preparation for PG training.

• Improves clinical learning.

• Improves the IT and organization skills.

• Develops collaborative learning.

• Improves the assessment system.

Types of Portfolios

On the basis of the type of content and their intended use, the portfolios can be of following three types:

1. Learning portfolio: This portfolio is used for recording the reflections on learning experiences by the learner. The learner controls the content and decides the things to be included. Such portfolios are for the help of the learner only, and may not be appropriate for evaluation in training programs. Such portfolios are used during a residency/ training program with primary aim of self-reflection and formative assessment.

2. Showcase portfolio: This portfolio is used to demonstrate one’s best work only. Like learning portfolio, the content is controlled by the person who creates it, but the content expresses pre-eminent work only. Such portfolios are used for summative assessment.

3. Assessment portfolio: The aim of this portfolio is to document what the student has learned. The content therefore is controlled by the designer and must include the reflection on the specific learning outcomes, including knowledge, skills and attitude. The designing of these portfolios should be diligent enough to generate evidence to evaluate the extent of required learning.

On the basis of the medium used to record the portfolios, they can be divided into:

1. Paper-based portfolios: The contents of portfolio are recorded on papers.

2. Electronic portfolios (e-portfolios): The content can be recorded electronically in computer -based tools.

Reflection: An Integral Component of Portfolio

Reflection is the essence of portfolio. Without reflection, portfolio is reduced to mere collection of papers or e-material similar to a logbook. Reflection has been defined as "integration of knowledge with action through thought" [13].. Al Shehri, et al. [14] defined it as "a cycle of paying deliberate, systematic and analytical attention to one’s own thoughts and actions, feelings, thinking and in relation to particular experiences for the purpose of enhancing perceptions of and responses to current and future experiences". Malinen in her book has reviewed the work of five leading experimental learning theorists, and have observed that reflection is the key component in Kolb’s, Knowles and Schon’s paradigm [15]. In simple words, Reflection refers to scrutinizing one’s own learning experience in order to take a closer look at what has been experienced to explore it in depth. Reflection can often be performed simultaneously during a learning exercise or soon after its completion. The idea of reflection is to inspect the learning experience rather than just living it.

Thinking and writing what has been experienced helps one to clarify his thoughts and feelings. It digs out the intricacies of the subject which may not spring up when one is just undergoing an exercise without looking at the process. Reflection exposes the learner to the possibilities of meaningful learning which is derived not from books but from one’s own understanding of his learning experience.

Criticisms Against the use of Portfolio

Despite evidence about their utility, portfolios have still not gained universal acceptance [7,16]. While one of the reasons could be the common misconcepts of equating the portfolio with logbook, other reasons that have prevented their widespread use include:

1. Time consuming: Residents report lack of time to maintain the records, especially when it includes reflection. This is understandable as maintaining portfolio add to the resident’s already busy schedule [6,9].

2. False or incomplete information: When the candidate is aware that the information supplied in portfolio can be used for assessment purposes, he may be reluctant in recording his shortcomings or may supply false information [17].

3. Concern over originality: It may be difficult to verify whether the contents of the portfolio, especially the reflection, are the individual’s own work.

4. Concern over structure: Issues have been raised that at times the content required to be included in the learning portfolios may not be relevant [5,10]. This can be offset by careful planning while designing the portfolios. The learning outcomes needs to be defined and the portfolio should be structured so that enough evidence can be collected to assess whether required learning has occurred or not.

5. Non uniformity in development and standardization of portfolio [3].

6. High inter-rater reliability due to lack of non-uniformity in the structure of the portfolio and training of the assessor.

Although the available literature supports the positive educational impact of portfolio-based learning, this has been met with mixed success [5]. The major reasons cited are poor understanding of the purpose of portfolio among the students and teachers, and rigid structure of portfolio. These perceived shortcomings can be overcome by: training of teachers and students in portfolio use and sensitizing them towards their benefit, keeping the portfolio slim, and by keeping the portfolio structure flexible, learner-centered and user friendly.

Assessment Using Portfolios

Assessment using portfolio can be obtained in two ways, either self-assessment by the learner or external assessment by the teacher.

The self-assessment integrated with the reflective practice gives a clear and meaningful picture of ‘what has been learnt and what remains to be learned’. This helps the learner to understand his needs and helps him achieve the learning goals. External assessment is done by the teacher to assess the extent of learning. The role of teacher is therefore crucial in external assessment. It is essential for the teacher to be familiar with portfolio and its purpose. External assessment using portfolio can be utilized both for formative and summative assessment. In formative assessment, the assessor has to provide feedback on the learner’s progress and help him identify the new learning goals. In summative assessment, it is crucial for the assessor to identify whether the desired learning goals and competencies have been achieved.

For the purpose of assessment, the assessment portfolio should be carefully structured so that it reflects the desired information which can help assess the extent of learning. It is also desirable to have a standardized format of portfolio for assessment purposes.

Various authors have suggested ways to improve the assessment using portfolios. Bird suggested that the evidence in the portfolio should be arranged according to the competencies the learner wants to demonstrate [18]. This may make portfolio assessment fast and more accurate. He suggested that if the portfolio has clear instructions like ‘Show how you….’ will help the learner to include the relevant evidence that demonstrates his performance. Collins suggested highlighting the most informative part of portfolio by attaching a little sheet called caption. This sheet should contain the information as to what the document is and why it is important evidence [19].

Integrating Portfolio in the Present System

With the present understanding that portfolios are beneficial for learning and assessment, there is a need of their more widespread use. However, it may be difficult to replace the traditional way of learning and assessment by portfolio-based learning. The change has to be gradual so that both the learner and the mentor understand the potential benefits of the tool, and have time to modify the content to align with the learning goals. The most important initial step would be the sensitization and training of mentors. As there is no uniformly agreed design of portfolio, brainstorming is required to design portfolio that suit the need of the learner and at the same time achieve the goals of learning. Maintaining the portfolio should be as easy for the student and as efficient for the assessor, as possible. This can be accomplished by guiding the students to include few but important evidences in portfolio which reflect their learning and comprehension. Wherever there is a provision of self-reflection, there should be word limit so as to keep the content relevant and the portfolio slim. Organizational support is vital during the initial implementation of this new tool. The use of portfolio should not be an additional but integral part of a training program. As logbooks are being widely used in undergraduate and postgraduate training programs, we recommend that the students be asked to incorporate their reflections in the logbooks’ in addition to recording their learning experiences.

One of the authors (TS) has been using reflective learning as part of a faculty development program. This program involves monthly discussions on e-mail about various educational topics, the last week of the month being devoted to writing reflections on learning [20]. The reflections are written using "what happened- so what- what next" framework and invokes good responses.

The Challenges

Although literature suggests the use of portfolio for a varied setting, there are still concerns over the structure, type, uniformity, acceptance, utilization, time-constraint and educational benefits of the portfolio [9,10]. The biggest challenge at present is to generate more evidence on the use of portfolio in medical education and training.

Another challenge is to change the widespread belief that portfolio are subjective and non-standardized, and cannot be used for assessment purposes, especially in high stake examinations [21]. However, it is now well accepted that the qualitative data gathered by portfolio can be well analyzed using holistic assessment techniques. When subjected to good qualitative analysis, the portfolio achieves acceptable inter-rater reliability, and this resolves the issue of portfolio being subjective [8]. The assessment using portfolios can be further improved by using some uniform ways of scoring portfolios, assessment by small groups of trained assessors rather than individual assessment, and training for assessors.

Even with the current evidence, the use of portfolios in learning and assessment remains underutilized. This may partly be due to lack of awareness of the existence or perceived educational benefits of this tool. Medical Education Units have an important role to play regarding sensitization of this tool to the teachers and students. The next challenge now would be to obtain the support of organization and administration to implement and sustain it. A summary of importance of portfolio in medical education is provided in Box 2.

Box 2 : Portfolio Essentials

• Portfolio includes recording of a learning experience as well as description of this learning experience in learner’s own words.

• This description on learning experience is known as ‘reflection’.

• ‘Reflection’ is the heart of portfolio.

• Without reflection, portfolio will become merely a logbook.

• Various types of portfolio are: learning portfolio, showcase portfolio and assessment portfolio.

• When recorded on paper, they are called paper based portfolio and when recorded in computer based devices they are known as e-portfolio.

The Future

The evidence regarding the outcomes of the use of portfolio in medical education needs to be strengthened further. This can be accomplished by incorporating portfolios in medical training on a larger basis. Given all the problems with the present system of medical training and assessment and the understanding that portfolios can offset a number of these deficiencies, the future of portfolios seems bright.

Portfolios have been utilized across a variety of health care professionals, although not explored at present [10], they also hold promise for inter-disciplinary learning. Portfolio use holds the promise to have a learning program that would have a positive academic effect on the learner’s knowledge, skills and behavior, and a better, unbiased system of assessment achieving the highest level of Miller’s pyramid. This would thereby positively influence the educational system, the learners, and the society.

Contributors: TS: conceptualized the draft; MJ: wrote the manuscript; PG: provided critical inputs.

Funding: None; Competing interests: None stated.

References

1. Norcini JJ, Grant J, Wooding K, Mairead M. Assessing professional competence. In: Singh T, Anshu, editors. Principles of Assessment in Medical Education. New Delhi: Jaypee; 2012. p.49-69.

2. Kadgad P, Kotrashetti SM. Portfolio: A comprehensive method of assessment for postgraduates in oral and maxillofacial surgery. J Maxillofac Oral Surg. 2013;12:80-4.

3. Donato AA, George DL. A blueprint for implementation of a structured portfolio in an internal medicine residency. Acad Med. 2012;87:185-91.

4. Buckley S, Coleman J, Khan K. Best evidence on the educational effects of undergraduate portfolios. Clin Teach. 2010;7:187-91.

5. Hall D. Professional development portfolios for teachers and lecturers. Br J Service Educ. 1992;18:81-6.

6. Paulson FL, Paulson PP, Meyer CA. What makes a portfolio a portfolio? Educ Leadership. 1991;48:60-3.

7. Driessen E, Tartwjik V, van der vleuten C, Wass V. Portfolios in medical education: Why do they meet with mixed success? Med Educ. 2007;41:1224-33.

8. Colbert CY, Ownby AR, Butler PM. A review of portfolio use in residency programs and considerations before implementation. Teach Learn Med. 2008;20:340-5.

9. Zubizarreta J. The Learning Portfolio: Reflective Practice for Improving Student Learning. Bolton, MA: Anker; 2004.

10. Tochel C, Haig A, Hesketh A, Cadzow A, Beggs K, Colthart I, et al. The effectiveness of portfolios for post-graduate assessment and education: BEME Guide No 12. Med Teach. 2009;31:299-318.

11. Nairn S, O’Brien E, Traynor V, Williams G, Chapple M, Johnson S. Student nurses’ knowledge, skills and attitudes towards the use of portfolios in a school of nursing. J Clin Nurs. 2006;15:1509-20.

12. Tiwari A, Tang C. From process to outcome: The effect of portfolio assessment on student learning. Nurse Education Today. 2003;23:269-77.

13. Hart M. Liberation through consciousness raising. In: Mezirow J, editor. Fostering Critical Reflections in Adulthood. San Francisco: Jossey-Bass;1990 . P.47-73.

14. Al-Shehri A. Learning by reflection in general practice: a study report. Educ Gen Pract. 1995;7:237-48.

15. Malinen A. Towards the Essence of Adult Experiential Learning. SoPhie Academic Press. Jyavskyälä; 2000.

16. Pearson DJ and Heywood P. Portfolio use in general practice vocational training: A survey of GP registrars. Med Educ. 2004;38:87-95.

17. Davis MH, Ponnamperuma GG. Portfolio assessment. J Vet Med Educ. 2005;32:279-84.

18. Bird T. The schoolteacher’s portfolio: An essay on possibilities. In: Millman J, Darling-Hammond L, editors. The New Handbook of Teacher Evaluation: Assessing Elementary and Secondary School Teachers. Newbury Park, CA: Corwin Press; 1990.p.241-56.

19. Collins A. Portfolios for biology teacher assessment. J Pers Eval Educ. 1991;5:147-67.

20. Anshu, Bansal PK, Mennin S, Burdick W, Singh T. Online faculty development for medical educators: Experiences of a South Asian program. Education for Health. 2008; 21:175-83.

21. Roberts C. Portfolio-based assessments in medical education: Are they valid and reliable for summative purposes? Med Educ. 2002;36:899-900.

 

Copyright © 1999-2015 Indian Pediatrics