International    Literacy  Assessment    in  Indonesia   Suhend...
Overview  l  During  the  first  decade  of  the  Third  Millennium,  Indonesia  has  taken  part  in  sev...
Reasons  to  participate  in  international  assessments  l  The  main  reasons  to  participate  in  inter...
Managing  the  Results  l  Results  were  analyzed  and  discussed  among  experts  in  several  academic ...
Improving  the  education  system  l  Results  of  international  assessments  have  grown  awareness  and ...
The Challengesl  The  existing  curriculum:   §  Reading  literacy  is  the  bases  for  Math  and  ...
The Challengesl  Teacher  preparation  is  not  yet  sufficient   §  Most  teachers  receive  their  ed...
Reading Curriculum has not beenchanged since 1994
Lowest Instructional Time in Reading Curriculum
No  emphasis  on  reading  engagement  
Not  focused  on  certain  cogni3ve  processes  
Evaluation is not based on research
Teacher preparation is not yet sufficient
Indonesia   ...
Reading  Performance  and  Social-­‐economic  Background     (PISA  2009)   SCORE > OECD ...
Detailed  Social-­‐economic  Factors  Influencing  Reading  Performance    (PISA  2009)   ...
Engagement  in  Reading,  Learning  Strategies  &  Reading  Performance    (PISA  2009)   Diversity  of  ...
Factors  Influencing  School  Background       (PISA  2009)  School  resources,  school  size,  and  teachers	...
The Strengths…l  The  National  Examination  held  by  the  Ministry  of  Education  and  Culture   (20...
National  Exam  (Grade-­‐9)   Subject   2009   2010   ...
International  Benchmark  Test    (IBT)l  IBT  is  designed  by  ACER  (Australian  Council  for  Education...
IBT  Results  No.   Subjects   Test     Schools   Grade   No.  of     Mean   ...
IBT  Results   23
IBT  Results   24
Lesson  Learned…  l  International  assessments  have  been  used  to  monitor  and  compare  the   qu...
Lesson  Learned…  l  The  need  to  reform  the  curriculum:   §  Results  of  international-­‐level	...
Lesson  Learned…  l  Reading  literacy  is  the  basis  of  Math  and  Science  literacy  development.  	...
Lesson  Learned…l  There  is  a  need    to  continuing  teacher  education  and  appropriate   ongoin...
Thank  you  very  much   Terima  kasih     ...
Dr.  Suhendra  Yusuf o  Project  Director  International  Benchmark  Test  (IBT),  Institut   ...
Our  students  are  at  risk..!  
International Literacy Assessment in Indonesia - Suhendra Yusuf
of 32

International Literacy Assessment in Indonesia - Suhendra Yusuf

Presented at Bincang Edukasi 15 event, 19 February 2013, @america - Pacific Place Mall, Jakarta.
Published on: Mar 4, 2016
Published in: Education      
Source: www.slideshare.net


Transcripts - International Literacy Assessment in Indonesia - Suhendra Yusuf

  • 1. International    Literacy  Assessment    in  Indonesia   Suhendra  Yusuf    Bincang  Edukasi  Jakarta,  19  Februari  2013   1
  • 2. Overview  l  During  the  first  decade  of  the  Third  Millennium,  Indonesia  has  taken  part  in  several   international  studies  on  students’  achievement  i.e.,  PIRLS,  PISA,  TIMSS,  and  IBT  as  an  effort  to   map  its  educational  standards  compared  to  the  global  ones.  l  The  results  were  still  not  yet  satisfactory.  The  students’  average  score  in  PIRLS  2006  was  407     and  PIRLS  2011  was  428;  all  was    below  international  average  score  500.  l  PISA  2000-­‐2009  showed  that  the  highest  score  in  Reading  literacy  was  402  (2009),  Math    391   (2006),  and  Science    395  (2003/6).  l  TIMSS  1999-­‐2011  also  portrayed  the  highest  score  in  Math  (Grade  8)  was  411  (2003)  and  Science   435  (1999).  l  However,  if  the  sample  is  focused  on  limited  numbers  of  students  in  international  schools,  IBT     (2009)  showed  the  highest  score  in  Math  (Grade  10)  was  753  and  the  lowest  523,  average  score   607.  l  Results  of  international  assessments  have  grown  issues  of  discussion  among  the  Ministry  of   Education,  the  House  of  Representatives,  and  experts  in  universities.  Results  have  prompted  a   review  of  education  policy.   2
  • 3. Reasons  to  participate  in  international  assessments  l  The  main  reasons  to  participate  in  international  assessments  are  to  monitor  of  quality  of   education  and  to  describe  and  understand  determinant  factors  and  observed  differences  of   the  students  achievement  among  provinces  in  Indonesia.  l  Program  was  proposed  by  the  Center  of  National  Education  Assessment  (CNEA/Puspendik)   but  the  decision  to  participate  was  made  by  the  National  Education  Research  and   Development  Body,  the  Ministry  of  Education  and  Culture  MoEC).    l  The  CNEA  implemented  the  Assessments.  Experts  from  higher  institutions  were  involved  in   the  preparation,  implementation,  monitoring  and  evaluation  of  the  program.  l  Pros  and  cons  on  the  assessment  participation.  The  opponent  arguments:   §  Sample  was  too  small.  It  did  not  reflect  the  whole  population.   §  Test  items  and  questionnaires  were    out  of  context.   §  Translated  materials  were  not  sufficient  or  inappropriate.   §  Assessment  results  only  exacerbate  or  worsen  the  image  of  educational  achievement   3   internationally.
  • 4. Managing  the  Results  l  Results  were  analyzed  and  discussed  among  experts  in  several  academic   meetings  facilitated  by  the  Center  of  National  Education  Assessment.  l  Seminars  were  held  to  discuss  the  results  in  several  occasions  involving  all   educational  stakeholders.  Results  were  recommended  to  policy  makers.  l  Results  were  also  exposed  to  policy  makers  in  national,  provincial,  and  local   levels.  l  The  MoEC  held  a  hearing  with  the  House  of  Representatives  to  discuss  aspects   of  educational  policy  based  on    critical  educational  issues.  l  Exposure  to  policy  makers  in  the  House  of  Representatives,  national,  provincial,   and  local  authorities  in  education  had  the  most  impact.  l  Results  were  also  published  as  a  reference  book.   4
  • 5. Improving  the  education  system  l  Results  of  international  assessments  have  grown  awareness  and  issue  of  discussion   among  the  MoEC  authorities  and  the  Indonesia  House  of  Representatives.   §  Quoting  results  in  formal  and  informal  meetings   §  Becoming  issues  in  mass  media  l  Results  have  prompted  a  review  of  education  policy.   §  Curriculum  2013  was  among  other  influenced  by  the  international  assessment  results  e.g.,   integrated  subjects  and  more  time  to  read  in  elementary  schools.   §  Data  from  previous  international  assessments  were  usually  compared  and  analyzed.  l  The    CNEA  developed  a  program  similar  to  international  assessment  called  Indonesian   National  Assessment  Program  (INAP)  to  improve  the  quality  of  education.   §  Using  Indonesian  and  local  contexts   §  Referring  to  content  standards  in  the  curriculum   5
  • 6. The Challengesl  The  existing  curriculum:   §  Reading  literacy  is  the  bases  for  Math  and  Science  literacy  development   §  Reading  Curriculum  has  not  been  changed  since  1994.  It  is  not  a  separate  curriculum  area.   There  are  only  language  teachers,  not  reading  teachers   §  Instructional  time  devoted  to  language/Reading  Curriculum  was  the  lowest  (15%)  among   countries   §  Curriculum  has  no  emphasis  on  reading  engagement/  reading  for  enjoyment   §  Reading  curriculum  is  not  focused  on  certain  cognitive  processes  and  does  not  include  the   highest  cognitive  process,  i.e.,  examining/evaluating  content,  language  and  textual  elements   §  Methods  used  to  evaluate  reading  curriculum  is  not  based  on  research   §  Curriculum  for    secondary  school  does  not  include  Chemistry.  Only  32%  of  TIMSS  sample  said   they  have  chemistry  subject,  resulting  in  the  lowest  among  science  strand   §  Geography  is  not  part  of  science  strand;  it  is  a  social  science  strand   §  Less  contextual  and  rational  thinking  process  in  math  curriculum   6
  • 7. The Challengesl  Teacher  preparation  is  not  yet  sufficient   §  Most  teachers  receive  their  education  through  a  teacher  college  program   §  They  are  supervised  during  education  and  have  to  pass  qualifying  exam   §  No  completion  of  probationary  teaching  period   §  No  completion  of  mentoring  or  induction  program  l  ESCS  (Economic-­‐Social-­‐Cultural  Status)  Index  is  the  lowest  among  participating   countries:   §  The  increase  1  point  in  the  ESCS  (parent  education/occupation,  home  possession  / number  of  books  at  home  index)  will  improve  17  points  in  students  performance   §  ESCS  influences  students’  reading  engagement:  enjoyment  in  reading  contributes  43%   and  diversity  of  reading  materials  influences  60%  to  students  performance   §  ESCS  influences  also  students  learning  strategy:    memorization  and  elaboration   strategies  contribute  34%  and  25%  respectively  to  students  performance   7
  • 8. Reading Curriculum has not beenchanged since 1994
  • 9. Lowest Instructional Time in Reading Curriculum
  • 10. No  emphasis  on  reading  engagement
  • 11. Not  focused  on  certain  cogni3ve  processes
  • 12. Evaluation is not based on research
  • 13. Teacher preparation is not yet sufficient
  • 14. Indonesia   Thailand   Tunisia   Brazil   Albania   Hong  Kong-­‐China   Indonesia  lowest  index  does  not  result  in  lowest  achievement…   Uruguay   Azerbaijan   Trinidad  and  Tobago   Jordan   Shanghai-­‐China   Romania   Portugal   Poland   Russian  Federation   Croatia  Lowest  ESCS  Index  (PISA  2009)   France   Italy   Slovak  Republic   Lithuania   Greece   Ireland   Serbia   Switzerland   New  Zealand   United  States   Luxembourg   United  Kingdom   Denmark   Australia   Dubai  (UAE)   Canada   Iceland   1.00 0.50 0.00 -0.50 -1.00 -1.50 -2.00
  • 15. Reading  Performance  and  Social-­‐economic  Background     (PISA  2009)   SCORE > OECD 600 SCORE > OECD ESCS < OECD China,  Singapore,      Japan,  Korea   ESCS > OECD 500 § Thailand   § Indonesia   400 Y = 56X ; R2 = 0,32 § Peru   § Qatar   In  spite  of  ESCS  lowest  index,   300 Indonesia  has  better  achievement   compared  to  higher  indexes  e.g.   200 Peru  and  Qatar   SCORE < OECD 100 SCORE < OECD ESCS INDEX < OCD ESCS > OECD 0-2 -1.5 -1 -0.5 0 0.5 1 1.5
  • 16. Detailed  Social-­‐economic  Factors  Influencing  Reading  Performance    (PISA  2009)   The  increase  1  point  in    the   ESCS  index    will  improve  17   points  in  students’    reading   performance
  • 17. Engagement  in  Reading,  Learning  Strategies  &  Reading  Performance    (PISA  2009)   Diversity  of  reading  materials  and  memorizing  strategy  mostly  influence  reading  performance
  • 18. Factors  Influencing  School  Background       (PISA  2009)  School  resources,  school  size,  and  teachers  quality  influence  significantly  school  social-­‐economy  index  which  has  impact  on  the  students  performance.
  • 19. The Strengths…l  The  National  Examination  held  by  the  Ministry  of  Education  and  Culture   (2009-­‐2011)  for  Grade  9  showed  a  significant  improvement  in  Science  and   English.  In  spite  of  a  decrease  in  Math  and  Indonesian,  the  scores  are  above   international  achievement.   o  Test  items  with  local  contexts  get  better  results  than  those  with  international   contexts.  l  The  International  Benchmark  Test  (IBT)  –  using    limited  numbers  of  students  in   international  schools    as  the  sample  of  the  study  –  showed  the  highest  score  in   Math  (Grade  10)  was  753  and  the  lowest  523,  average  score  607.   o  There  are  1300  schools  of  this  category  (out  of  182.538  schools  with  58  million   students)  all  over  the  country.   19
  • 20. National  Exam  (Grade-­‐9)   Subject   2009   2010   2011   score   sd   score   sd   score   sd  Mathematics   7.60   1.57   7.53   1.28   7.3   1.70  Science   7.32   1.28   7.32   1.17   7.41   1.35  Indonesian   7.38   1.19   7.47   1.08   7.12   1.18  English   7.14   1.45   7.14   1.25   7.52   1.50   20
  • 21. International  Benchmark  Test    (IBT)l  IBT  is  designed  by  ACER  (Australian  Council  for  Educational  Research)  to  assess   the  performance  of  students  from  school  against  local,  national  and   international  standards.   o  The  tests  are  in  English,  Math,  and  Science.   o  The  Math  strands  are  chance  and  data,    measurement,    number,  and    space.   o  The  English  strands  are  comprehension,    punctuation,    spelling,    grammar,  and     vocabulary.   o  The  Science  strands  are  physical  science,  earth  science,  and  life  science.  l  Student  results  for  the  IBT  are  compared  with  TIMSS    2007  for:   o   Mathematics  in  Years  4,  5,  8  and  9   o   Science  in  Years  4,  5  and  8   21
  • 22. IBT  Results  No.   Subjects   Test     Schools   Grade   No.  of     Mean     Score  range   Level   students   score  1.   Math   10   SMAN  8  Pekanbaru   10   242   607   523  –  735  2.   Math   9   SMAN  1  Mataram   10   212   614   419  –  749  3.   Math   10   SMAN  1  Mataram   11   61   618   513  –  687  4.   Math   10   SMAN  1  Mataram   12   48   647   559  –  759   Total   621   419  –  759  5.   English   10   SMAN  8  Pekanbaru   10   242   525   422  –  627  6.   English   10   SMAN  1  Mataram   10   211   505   319  –  602  7.   English   10   SMAN  1  Mataram   11   61   562   499  –  643  8.   English   10   SMAN  1  Mataram   12   46   557   490  –  661   Total     537   319  –  661   22
  • 23. IBT  Results   23
  • 24. IBT  Results   24
  • 25. Lesson  Learned…  l  International  assessments  have  been  used  to  monitor  and  compare  the   quality  of  education  locally  and  internationally  and  to  understand   factors  influencing  the  students  achievement.  l  The  program  has  become  issues  of  discussion  among  the  MoEC   authorities,  the  House  of  Representatives,  and  academician.  Results   have  been  used  to  review  the  education  policy.  l  The  opponents  to  program  participation  particularly  argue  on  the   sample  representativeness,  the  quality  of  test  materials,  and  the   unexpected  results  to  worsen  the  image  of  quality  of  education   achievement  internationally.
  • 26. Lesson  Learned…  l  The  need  to  reform  the  curriculum:   §  Results  of  international-­‐level  studies  might  be  accounted  for  by  differences   in  curriculum  rather  than  intellectual  differences  among  students   (Gustafsson  and  Undheim,  1996)   §  Reading  has  to  be  a  separate  curriculum  area.  There  should  be  more  reading   teachers   §  Instructional  time  devoted  to  reading  has  to  be  increased  at  least  twice   from  current  curriculum   §  Curriculum  has  more  emphasis  on  reading  for  enjoyment   §  Reading  curriculum  has  to  focus  on  certain  cognitive  processes,  to  include   process  of  examining/evaluating  content,  language  and  textual  elements   §  More  research  in  implementation  of  reading  curriculum  has  to  be  conducted
  • 27. Lesson  Learned…  l  Reading  literacy  is  the  basis  of  Math  and  Science  literacy  development.     o  There  is  a  need  to  a  more  comprehensive  approach  to  reading  instruction  to   include  teaching  for  motivation.   o  The  need  of  professional  development  for  subject  teachers,  including   training  in  the  use  of  research-­‐based  reading  instruction  .   o  The  need  of  classroom-­‐based  strategies  for  improving  student  reading  and   comprehension  strategies  across  the  curriculum.   o  More  professional  initial  and  continuing  teacher  education.  The   International  Reading  Association  (1998)  recommended  that  primary   teachers  have  280  hours  of  instruction  in  reading  and  how  to  teach  it.
  • 28. Lesson  Learned…l  There  is  a  need    to  continuing  teacher  education  and  appropriate   ongoing  professional  development  in  reading,  math,  and  science.  l  There  should  be  an  effective  intervention  for  children  experiencing   difficulties.  l  As  of  the  sample  of  the  next  study:  better  to  focus  purposively  on    a   certain  type  of  schools  as  a  benchmark  or  a  model  of  school   development.
  • 29. Thank  you  very  much   Terima  kasih             Suhendra  Yusuf   suhendrayusuf@gmail.com   29
  • 30. Dr.  Suhendra  Yusuf o  Project  Director  International  Benchmark  Test  (IBT),  Institut   Asesmen  Indonesia  (IAI)  in  collaboration  with  Australia   Council  for  Educational  Research  Australia     o  International  Quality  Control  Monitor  for  the  implementation   of  PIRLS  2006  &  2011   o  Researcher  PISA  &  PIRLS  2000  –  2011   o  Associate  Professor  and  vice  Rector  Nusantara  Islamic   University  Bandung   o  Author  Benchmark  Internasional  Mutu  Pendidikan  (co-­‐ authored  with  Bahrul  Hayat,  Bumi  Aksara,  2010)
  • 31. Our  students  are  at  risk..!