From evaluation to reflection-on-action:
Lessons learnt from the impact of a distance
education programme
NADEOSA Conferen...
Introduction
• Quality assurance at various levels in HE, but
a gap
 evaluation dimension (Mizikaci, 2010)
• Evaluation i...
Background
• ACE (EM) - Continuing professional
qualification
• Paper-based DE program
• Elongated impact study (2007 – 20...
Background
 128 principals
 Interviews (2 individual & 10 focus group)
 30 teachers
 20 principals
Kirkpatrick’s & Ba...
Reflection & Reflective Practice
By three methods we may learn wisdom:
first, by reflection, which is noblest; second,
by ...
Cyclic Process of Reflection
Content
Return
Feelings &
Emotion
Re-
evaluation
Possible
explanations
Outcome
Intelligent
ac...
Reflection Questions
• What valuable lessons can be learnt from the
impact of the discontinued ACE: (EM)
programme on the ...
6 Major Lessons
1. Clear institutional policy on quality
assurance
persistent negative perceptions of DE
programmes (with...
Table 1: Non-completion rates of distance education
students at UP (All programmes)
Year Students who discontinued
their s...
2.Practices guided by a policy on
distance education
Traditional Education policy does not
necessarily apply to DE
Impor...
Examples of data: age, technology, work &
geographical profiles, and graduation &
retention rates.
11
Table 2: Technology...
4. Student support structures
all activities beyond the production and delivery
of course materials
Most students from T...
6. Research focused on DE
Dedicated research unit
Operational & academic research
Purpose is to inform practice
unders...
Concluding Thoughts
• Quality and quality management are
organisational obligations that lie vested in
management’s commit...
Bibliography
• Aluko, R. 2009. The impact of an Advanced Certificate in Education (ACE) programme on the professional
prac...
Thank You!
ruth.aluko@up.ac.za
of 16

Nadeosa 2014 aluko

Reflection-on-action is necessary to derive meaning from one’s experiences. This paper revisits research data from an elongated study on the impact of a distance education programme on the professional practice of graduates. The study focused on 300 graduates and 128 principals, selected through multi-stage and purposive sampling. The researcher used a mixed-methods research design with specific focus on Kirkpatrick’s, and Baldwin and Ford’s training evaluation models. The researcher’s curiosity was triggered by the need to understand possible reasons for the participants’ views, as these are contrary to the norm. This account indicates a clear institutional policy on quality assurance, practices guided by the policy, an ongoing monitoring of the distance education students’ profiles, improved programme design, student support structures, programme design and research focused on programmes as possible reasons. The author argues that higher education practitioners, irrespective of delivery mode, could benefit from the valuable lessons learnt from the exercise. Keywords: Distance education, evaluation, impact, professional development, quality, reflection, reflection-on-action
Published on: Mar 3, 2016
Published in: Education      
Source: www.slideshare.net


Transcripts - Nadeosa 2014 aluko

  • 1. From evaluation to reflection-on-action: Lessons learnt from the impact of a distance education programme NADEOSA Conference (24 – 25 June 2014) Ruth Aluko
  • 2. Introduction • Quality assurance at various levels in HE, but a gap  evaluation dimension (Mizikaci, 2010) • Evaluation in HE (Hall & Hall, 2004)  focus – aims of a program & to what extent  tool for decision-making & assessing quality • Purpose  to maintain / improve quality of products & processes • Tool  reflection 2
  • 3. Background • ACE (EM) - Continuing professional qualification • Paper-based DE program • Elongated impact study (2007 – 2012) Mixed-methods enquiry (surveys & interviews) Surveys: 300 graduates (2004-2006 cohorts) 3
  • 4. Background  128 principals  Interviews (2 individual & 10 focus group)  30 teachers  20 principals Kirkpatrick’s & Baldwin & Ford’s Training Evaluation Models Evidence of value added to individual lives & workplace Suggestions were proffered 4
  • 5. Reflection & Reflective Practice By three methods we may learn wisdom: first, by reflection, which is noblest; second, by imitation, which is easiest; and third, by experience, which is the bitterest. (Confucius) • Experience, meaning, learning & application • Reflection-in & on-action • Trigger for reflection 5
  • 6. Cyclic Process of Reflection Content Return Feelings & Emotion Re- evaluation Possible explanations Outcome Intelligent action 6
  • 7. Reflection Questions • What valuable lessons can be learnt from the impact of the discontinued ACE: (EM) programme on the professional development of graduates and why? • How could these lessons apply to other distance education programmes at the University of Pretoria? • What possible value could these lessons add to the practices of other distance education providers? 7
  • 8. 6 Major Lessons 1. Clear institutional policy on quality assurance persistent negative perceptions of DE programmes (within & outside) Quality – a priority & an iterative process Awareness among stakeholders Possible relationship between student attrition rates & quality 8
  • 9. Table 1: Non-completion rates of distance education students at UP (All programmes) Year Students who discontinued their studies Percentage of total enrolment 2006 419 4% 2007 650 4% 2008 584 3% 2009 762 4% 2010 430 2% 2011 650 3% 2012 689 4% 2013 336 2% Source: University of Pretoria (UDE) (2006–2013) 9
  • 10. 2.Practices guided by a policy on distance education Traditional Education policy does not necessarily apply to DE Importance of policy document 3. On-going monitoring of the DE students’ profile Necessity for quality data to make data-driven decisions regarding programme improvement 10
  • 11. Examples of data: age, technology, work & geographical profiles, and graduation & retention rates. 11 Table 2: Technology profile of students who enrolled for the first time between 2004 and 2013 Year 2004 2006 2007 2008 2009 2010 2012 2013 Number of students 3 187 5 087 5 643 8 011 6 102 5 675 3 354 2 599 Cell phone use 98% 99% 99% 99% 99% 99% 100% 100% Internet use 0% 2% 1% 3% 8% 13% 25% 29% Source: University of Pretoria (between 2004 & 2013)
  • 12. 4. Student support structures all activities beyond the production and delivery of course materials Most students from TE background Examples: contact sessions, tutorial letters, assignments, SMS & an academic enquiry service 5. Programme design Continual review In spite of IT, most countries in Africa are still trapped in the first-generation mode of delivery 12
  • 13. 6. Research focused on DE Dedicated research unit Operational & academic research Purpose is to inform practice understanding trends and issues in terms of topics & methods E.g. mixed-methods 13
  • 14. Concluding Thoughts • Quality and quality management are organisational obligations that lie vested in management’s commitment towards an understanding of quality (Aluko, Fraser, and Hendrikz 2008) • improved performance of distance education students attested to by scholars if quality improves • The necessity for continual evaluation of DE programs & reflection-on-action 14
  • 15. Bibliography • Aluko, R. 2009. The impact of an Advanced Certificate in Education (ACE) programme on the professional practice of graduates. The International Review of Research in Open and Distance Learning , IRRODL, 10 (4) www.irrodl.org • Aluko, FR, WJ Fraser and J Hendrikz. 2008. Some interfaces in conventional and distance education programmes in a postmodern context. South African Journal of Higher Education, 22(3): 484–497. • Aluko, FR, WJ Fraser and J Hendrikz. 2011. Transactional Distance Theory and Total Quality Management in contact and distance learning. Africa Education Review, 8(1): 115–132. • Commonwealth of Learning. 2004. Planning and implementing open and distance learning systems: A handbook for decision makers. Vancouver: Commonwealth of Learning. Available at: www.col.org (accessed 7 April 2011). • Hall I. and D. Hall. Evaluation and social research: introducing small-scale practice. United Kingdom: Palgrave Macmillan, 2004. • Lee, Y, MP Driscoll and DW Nelson. 2004. The past, present, and future of research in distance education: Results of a content analysis. American Journal of Distance Education, 18(4): 225– 241. • Lockee, B., Moore, M., & Burton, J. (2002). Measuring success: Evaluation strategies for distance education. Educause Quarterly, 1, 20-26. • Mizikaci, F. Total quality management in higher education: An evaluation model for practitioners. Germany: LAP Lambert Academic Publishing, 2010. • Pulsipher S. 2009. Preparing quality teachers: Tracking a cohort of students from pre-admission to graduation. Available at: http://education.byu.edu/cites/documents/Student%20ProfileSusan%20Pulsipher.pdf (accessed 7 January 2013). • Rubin, F. 1995. A basic guide to evaluation for development workers. Oxford: Oxfam Publications. • Wilson-Strydom, M. 2004. Programme evaluation and its role in quality assurance. Vancouver: Commonwealth of Learning. Available at: http://dspace.col.org/handle/123456789/351 15
  • 16. Thank You! ruth.aluko@up.ac.za