Questions and Comments about National, Externally Graded Assessment Exams: A Response to
Author(s): Janet Wilmoth...
ommendsa multi-methodapproachthat
over time (Astin et al. 2003...
used, performanceexpectations, and num-
ber of exams generated?
In conclusion, Wagenaar'sproposal...
of 4

National exam external grader journal

a journal about national exam graded by external parties
Published on: Mar 3, 2016
Published in: Education      Technology      

Transcripts - National exam external grader journal

  • 1. Questions and Comments about National, Externally Graded Assessment Exams: A Response to Wagenaar Author(s): Janet Wilmoth Source: Teaching Sociology, Vol. 32, No. 2 (Apr., 2004), pp. 241-243 Published by: American Sociological Association Stable URL: . Accessed: 07/04/2014 02:14 Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at . . JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact . American Sociological Association is collaborating with JSTOR to digitize, preserve and extend access to Teaching Sociology. This content downloaded from on Mon, 7 Apr 2014 02:14:01 AM All use subject to JSTOR Terms and Conditions
  • 2. QUESTIONSAND COMMENTSABOUTNATIONAL, EXTERNALLYGRADEDASSESSMENTEXAMS: A RESPONSETOWAGENAAR* JANETWILMOTH SyracuseUniversity IN "ASSESSINGSOCIOLOGICALKNOWL- EDGE"TheodoreWagenaarextendsaninvi- tationto readerstoengageina conversation aboutthe meritsof usinga national,exter- nally gradedessay exam to assess under- graduatestudents.This invitationis appro- priate,giventhattherearefew comprehen- sive assessmentinstrumentsin sociology andnone,to my knowledge,thathasbeen endorsedby theAmericanSociologicalAs- sociation. Perhaps, througha collective dialoguewe can arriveat a reasonableas- sessmentinstrumentthatcan be usedby a varietyof sociologydepartments. Wagenaar'sambitiousproposalis timely given the increasingexternaland internal pressuressociologydepartmentsareexperi- encingregardingthedevelopmentof assess- ment methods.As Wagenaarand others have noted,undergraduateassessmentis a priorityof accreditationagenciesandmany college administrators(Weiss2002; Senter 2001). But beyond this externally imposed pressure to engage in assessment activities, assessment can inform internal discussions of undergraduateprograms. Therefore, as- sessment is a potentially useful tool that departmentscan rely on when revising cur- riculumto improvestudentlearning. Despite the potentialof assessmentfor enhancing undergraduateeducation and demonstratingeducational effectiveness, manydepartmentshave reluctantlyunder- takenthe task of assessment.Articlesin TeachingSociology(see, for example,the October2002 specialissue)andAssessing StudentLearningin Sociology(2001)pre- sent department-specificassessment instru- ments thatprovide insight into the possible rangeof assessmentmethods.Wagenaar's articlemakesan importantcontributionto this literaturebecause it provides specific essay questions that can aid a variety of departmentsin conducting assessment. These questionsprovide a startingpoint from which departmentscan tailor their own assessment protocols. However, Wagenaar'sproposednational,externally gradedessayexamraisesseveralquestions. First,whataretheimplicationsof provid- ing a nationalassessmentinstrumentto soci- ology departments?Typically, the develop- mentof anassessmentinstrumentis partof a process of discussion and self-reflection that occurs within departments.Assessment protocolstypicallyflowfromthemissionof the university,college, and departmentin which the assessmentis takingplace and shouldbe consistentwiththe department's specific program goals (Hood, Potts, and Johnson2001). Linkingassessmentactivi- ties with programgoals and mission state- mentsincreasesthe likelihoodthatthe as- sessment will provide information that is usefulto thedepartment.Italsoenablesthe departmentto articulateits uniquerole in the universitycommunityto externalstake- holders.A standardset of assessmentques- tions could circumventthis process and po- tentially lead to the implementationof as- sessment items that are not consistent with the goalsof a particulardepartmentor the broadermissionof theuniversityor college in which thatdepartmentis located. Second, what are the components of an effective assessment protocol? The Ameri- can Association for Higher Educationrec- "*Pleaseaddressall correspondenceto the authorat MaxwellSchool of Citizenshipand PublicAffairs,SyracuseUniversity,426Eggers Hall,Syracuse,NY 13244; TeachingSociology,Vol.32, 2004(April:241-243) 241 This content downloaded from on Mon, 7 Apr 2014 02:14:01 AM All use subject to JSTOR Terms and Conditions
  • 3. 242 TEACHINGSOCIOLOGY ommendsa multi-methodapproachthat tracksstudentexperienceandperformance over time (Astin et al. 2003). However, Wagenaar'sproposedinstrumentonly in- cludesessayquestionsthataregivento stu- dentsat the endof theirundergraduateca- reer.Wagenaareffectivelyarguesthatessay tests are particularlywell suitedto deter- minea student'sabilityto mastercomplex- ity, think critically,and work independ- ently.Dependingon a department'smission and goals, an assessmentcommitteemay alsowantindicatorsof students'knowledge of specific subjects,abilityto implement sociologicalmethodsin anoriginalresearch project,oralcommunicationskills,andun- dergraduateor postgraduateexperiences. Collectinginformationon thesedimensions of studentperformancemayrequiretheuse of multiple-choiceexams, grades from a capstonecourse,evaluationsof studentthe- sis projects,studentself-assessment,focus groups,or alumnisurveys.Itmightalsobe desirableto collect this informationfrom studentsas theyprogressthroughtheirun- dergraduatecareers.This would allow a departmentto tracethe impactits curricu- lum is havingon students'intellectualde- velopment.The use of Wagenaar'sinstru- mentdoes notprecludethe implementation of additionalassessmentmethods.But,pro- vidinga nationalbankof assessmentitems andestablishinga systemof externalgrad- ingforthoseitemscouldpotentiallynarrow the field of vision of departmentalassess- mentcommitteesas they constructassess- mentinstruments. Third,whattypesof knowledgeshouldbe demonstratedby sociologymajors?Further- more,whatarethe mostreliableandvalid methods for assessing this knowledge? Thereis someagreementwithintheAmeri- can SociologicalAssociationregardingthe broadgoals for undergraduateeducationin sociology. LiberalLearningand the Sociol- ogy Major(1991) lists 12 learninggoals for sociology majors. However, it is not clear how Wagenaar's 20 assessment items are relatedto these learninggoals. It would be helpful if the connectionbetween the learn- inggoalsandtheproposedassessmentitems wereexplicated.Forexample,constructing a grid(withthelearninggoalsalongthetop and the assessmentitems down the side) thatidentifieswhichlearninggoalsareas- sessed by which specific items would be informative.It wouldallowdepartmentsto comparethe 12 nationalgoals with their own departmentalgoals, considerwhether the department'scurriculumreflectsthese national learning goals, and determine whetherWagenaar'sproposedassessment itemsadequatelymeasurestudentprogress towarddepartmentalgoals. The challenge for departmentsis to adequatelymeasure theirstudents'understandingof globaldisci- plinaryknowledgeand the knowledgeof specificfoci representedwithinthe depart- ment. Wagenaar'sinstrumentcan poten- tiallybe helpfulon bothaccounts,butit is difficultto determineits utilitygiven that the specificassessmentitemsare not cur- rentlylinkedtoparticulargoals.Inaddition, the reliabilityand validityof these items shouldbe empiricallydemonstratedbefore departmentsinvestsubstantialresourcesin theimplementationof theinstrument. Finally,is establishinganexternalsystem forgradingassessmentexamsnecessary?Is it feasible?Wagenaardoesnotmakea con- vincingargumentforthebenefitsof sucha system.He notesthat"Weiss(2002)found that 'externalreviewers'were rated the highestin termsof perceivedvalueas an assessmentmechanism."But,theseexternal reviewerswerenotgradingstudentexams. Insteadthey were "typicallychargedwith evaluationof severalaspectsof the depart- ment"(Weiss 2002; p. 396). Therefore, althoughexternaldepartmentreviews are perceivedas helpful,thereappearsto be no evidencethat externalgradingof student exams is more beneficialthan internally gradingthe exams. Furthermore,it is not clear how a system of external grading would operate. If therewas a clearinghouse, who would pay for personneland web sup- port? What grading standards would be used? How would differencesbe reconciled across departments regarding the items This content downloaded from on Mon, 7 Apr 2014 02:14:01 AM All use subject to JSTOR Terms and Conditions
  • 4. A RESPONSETOWAGENAAR 243 used, performanceexpectations, and num- ber of exams generated? In conclusion, Wagenaar'sproposal for a national,externallygradedessay assessment exam raises many questions. In principle, the idea of a national test bank of assess- ment items that are clearly linked to the Liberal Learning and the Sociology Major (1991) goals for undergraduateeducationin sociology has promise. However, establish- ing a system for externally grading these exams is not warrantedgiven the lack of evidence regarding the benefits of such a system. REFERENCES AmericanSociologicalAssociation.1991. Lib- eralLearningandtheSociologyMajor.Wash- ington,DC: AmericanSociologicalAssocia- tionTeachingResourcesCenter. Astin, AlexanderW., Trudy W. Banta, K. PatriciaCross, Elaine El-Khawas,PeterT. Ewell,PatHutchings,TheodoreJ. Marchese, Kay M. McClenney,Marcia Mentkowski, MargaretA. Miller,E. ThomasMoran,and BarbaraD. Wright.2003. "NinePrinciplesof Good Practicefor AssessingStudentLearn- ing."AmericanAssociationforHigherEduca- tion.RetrievedOctober8, 2003 (http://www. Hood, DeniceWard,ShellyKeimigPotts,and WilliamS. Johnson.2001. "GeneralGuide- lines for Designingand Implementinga De- partmentalLearningOutcomesAssessment." Pp. 73-93 in AssessingStudentLearningin Sociology,2d ed., editedby CharlesF. Holm and WilliamS. Johnson.Washington,DC: AmericanSociologicalAssociationRecourse MaterialsforTeaching. Senter,MaryScheuer.2001. "AcademicOut- comesAssessment:Fads,Fallacies,andFoot- steps."Pp. 14-25in AssessingStudentLearn- ing in Sociology,2nded., editedby Charles F. HolmandWilliamS. Johnson.Washing- ton, DC: AmericanSociologicalAssociation RecourseMaterialsforTeaching. Weiss, GregoryL. 2002. "TheCurrentStatus of Assessmentin Sociology Departments." TeachingSociology30:391-402. Janet Wilmoth is an associate professor of sociol- ogy, directorof the sociology undergraduateprogram, and senior researchassociate in the Center for Policy Research at Syracuse University. Her researchexam- ines older adult migration, living arrangements,social support,and healthstatus. Inaddition,she is interested in pedagogical issues related to social statistics, re- search methods,and social gerontologycourses. This content downloaded from on Mon, 7 Apr 2014 02:14:01 AM All use subject to JSTOR Terms and Conditions