Assessment: comments archive

The following comments are placed in chronological order, with the most recent at the top of the list.

Date: Sat, 22 May 2004 12:04:14 +0100 (BST)
From: Jocelyn Hart <jocelyn.hart@southkent.ac.uk>

A big thank you to Deliberations for this site.  I have been stumbling over a 3,000 word essay about assessment and am totally bored by the reading list supplied for my course.  It is so refreshing to read comments from people in 'proper' language who do not have to waffle to make a book bigger than it really need be. I am completely re-energised and have got loads of ideas.  Thank you

Date: Tue, 23 Mar 2004 15:23:37 GMT
From: Samantha Cook <Csam501@hotmail.com>
Subject: Should assessment be part of mentoring

I am a mentor myself, and find myself steering away from assessment or discipline.  I feel that pushing this area contradicts the relationship I may have struggled to create.  It jeopardises the rapport thats been built up and mentees then question trust. 

I find that I'm far better positioned to go to the mentee after and discuss with them any issues and in this position I can address the reasons why discipline has taken place or issues involving assessment.  This I find builds on the rapport and relationship and also allows for rationalization.

Currently I am studying for a degree in social work and have been put in a position of having my own mentor whilst on placement.  I have been interviewed by my mentor and they will also be involved in the assessment of my practice and portfolio on completion.  This for me goes against what I know and feel about mentoring, for a start with being interviewed, creates a power imbalance which is a big no no!  How can true rapport be built with major power imbalances like this.  I won't forget who is in the position of power which means I will not let my guard down with regards to the mentor.  I also struggle with the idea of confidentiality in this situation.

Anyway I hope this makes me better at my role of mentor for the sake of my mentees.

Date: Mon, 6 Jan 2003
From: AspireHWT@AOL.com (Harvey Ward Turner)
Subject: Maintaining rigor whilst assessing through a portfolio

Having just stumbled on your site, I would like to reply to Mark Broom (7 Apr 98) on his concern about maintaining rigor whilst assessing through a portfolio. This is an area of concern for me - teaching in particular fine art where there is an extraordinary degree of subjectivity. Learners may have to demonstrate their ability to understand small objective outcomes: (yellow added to blue does makes green) but the far larger and more important wider aesthetic learning asks the learner, yes, but what type of green and why? Of course the answer is predicated on the learners personal choices which truly are the handmaidens of the 'Art'! Assessing the portfolio, especially at pre-degree level, seems too often to rely on looking for and assessing all those little objective outcomes at the expense of the bigger picture. So it seems it may be a question of determining and striking a balance, that recognises that sometimes a learner may have intentionally ignored or not understood fully these methodological niceties, but what they have made nevertheless works!

Date: Tue, 13 Mar 2001 16:36:28 GMT
From: Michael Zeilik (zeilik@la.unm.edu)
Subject: Multiple choice tests and thinking.

Yes, multiple-choice questions can probe "thinking" but it is a lot of work to develop and validate them. It has taken me almost 10 years to write such items for astronomy. I use a simple scheme: (1) concept recognition, (2) near-transfer concept application, and (3) far-transfer concept application. I teach large classes (100 to 300 students) with essentially no help. A large fraction of students really dislike these questions. They call them "tricky". I call them "thought provoking".

Date: Thu, 20 Apr 2000 13:34:28 +0100 (BST)
From: David C Nicoll (david.nicoll@iie.org.uk)
Subject: Teaching Portfolios.

How about using a framework similar to that employed by City & Guilds 7307, where everyone does a basic 'part 1' and the more serious go on to complete a 'part 2'. It works in FE very well.

Date: Fri, 26 Nov 1999 01:00:41 GMT
From: Mara (schweitz@foxvalley.Tec.wi.us)
Subject: Multiple choice tests and higher order thinking.

Yes I do think multiple choice questions can test higher order thinking. It depends on how you structure the question and how the multiple choices are stated. Writing a multiple choice test to include testing of thinking requires a great deal more effort on the writers part. But, the result is well worth it.

Date: Mon, 19 Oct 1998
From: g.isaacs@mailbox.uq.edu.au (Geoff Isaacs)
Subject: Summative & Formative Assessment

Alice Whish (A.Whish@unsw.edu.au) wrote Sun, 23 Aug 1998

I am particulary interested in peer assessment and the issue of summative assessment by students and its contribution to the formative grade.
Any suggestions for further reading or a case study or 2 would be great. I am working with craft studies (jewellery) and Design Studies students.

I can't offer any relevant case studies, although a look at Assessment and Evaluation in Higher Education for the last 10 years or so will provide a number of examples. However I'd like to comment on the first paragraph because it looks a little confused to me.

As I understand it, summative assessment is assessment that contributes to a student's grade and formative assessment is assessment solely for the purpose of feedback (to help inform and form the student's learning). Increasingly this distinction is becoming less useful as all or almost all assessment serves a summative role.

Thus one issue is the extent to which summative assessmnt exercises can also be used to help students to learn (ie used for formative purposes). Several issues come to mind.

First, if the assessment event is "fatal" (as a result the student fails the subject) or very threatening then the student is unlikely to take much notice of the feedback. Ideally no single assessment exercise ought to be capable of failing a student; at worst only a final exam should be able to do so (but see the next point...).

Secondly, if the feedback comes too late for the student to put it to good use (eg if it is from a final exam at the end of a subject, even if that subject is the precursor of another) then the student is unlikely to take much notice of it.

Incidentally, the literature on peer assessment, while by no means unanimous on the topic, does seem to indicate that, with adequate training, students' assessments are in substantial agreement with those of their teachers.

Hope this helps.

Subject: Peer Assessment
From: A.Whish@unsw.edu.au (Alice Whish)
Date: Sun, 23 Aug 1998 06:39:59 GMT

I am particulary interested in peer assessment and the issue of summative assessment by students and its contribution to the formative grade.

Any suggestions for further reading or a case study or 2 would be great. I am working with craft studies (jewellery) and Design Studies students.

Subject: Portfolio Assessment
From: mbroom@glam.ac.uk (Mark Broom)
Date: Tue, 7 Apr 1998

Having been in higher education for just over three years, I am constantly amazed at the number of assessments we put our students through. This process appears to reward the student with a good recall of facts rather than developing a critical reflective thinker.

I am currently developing a BSc (Hons) in Health Care Management and I am engaged in the long process of validation. I hope to adopt portfolio assessment as a vehicle to create this thinking practitioner. This is particularly important to me because the overall aim of this degree is to ensure that post registration nurses can compete on a level playing field with the advent of the government's ten year White Paper entitled "Putting Patients Fist".

I am currently asking myself several questions that I hope will generate debate and in the process solve a few of my problems:

1. Is portfolio assessment an appropriate form of assessmment?
2. Are there any Universities using this assessment tool?
3. How can we maintain academic rigor through this process?

I eagerly await any reply.

Subject: Assessing use of language
From: felicia.zhang@jcu.edu.au (Felicia Zhang)
Date: Wed, 10 Dec 1997

I am a Chinese lecturer at James Cook University, Australia. As the only person teaching all the courses we offer in Chinese, resource is limited. When I assess students' essays in Chinese at an advanced level, I most of the time get the feeling that most errors students make are no longer the kind that involves grammar or other things. The errors instead point to not knowing how to organise an essay on a discourse level. Consequently, no matter how extensive my comments might be, students still cannot gain much from my comments.

That is why now I conduct group feedback sessions to work on things like how to organise an essay etc. Fortunately with some success. Of course, such feedback is only possible when students have been productive in creating and writing thoughts of their own.

In my experience, especially in the learning of some finer points of grammar in a language, the motivation to feel the need to use a particular grammatical usage has to be generated by the learners in order for the learning to make sense.

Subject: Rubrics
From: g.isaacs@mailbox.uq.edu.au (Geoff Isaacs)
Date: Wed, 9 Jul 1997

I keep seeing on the Web references to "rubrics". The definitions given seem somewhat contradictory. Most of them are to uses in US high schools and primary schools.

I would appreciate it if someone were to point me to a good explanation of their use (they look useful for criterion referenced and portfolio assessment) and, especially, to examples of their use in university education.

Examples on the Web would be particularly appreciated!!!

Subject: Assessing Essays
From: smenon@global.co.za (Kirti Menon)
Date: Mon, 7 Jul 1997

I have been researching assessment of students' essays and examining students' responses to feedback provided by tutors. My results are quite dismal as most students do not find the comments useful and are still unable to understand where they have gone both right and wrong.

I teach at a university in South Africa in the English Department. Most of our students are ESL learners. We have no opportunity to carry out alternative assessment but rely on the essay to assess students. Our staff student ratio at the moment is 175:1. This is of sourse an untenable situation.

I would appreciate any comments.

Subject: A National Body for Assessment?
From: d.r.bird@iti.salford.ac.uk (David Bird)
Date: Thu, 28 Nov 1996

My twopenneth - There are a number of issues which we don't really address in our discussions and, unfortunately, some of them have political overtones which colour the whole debate.

I think that the majority of assessors have to confess that the academic quality of the assessing process has had to decline (and probably seriously) due to the increase in numbers of those being assessed. A true assessment exercise, in my mind certainly, would take much longer than I can currently allow it to. The pressure of numbers (double the number of recruits in this institution) means that there is a genuine "halving" of both assessment quality and quantity - I think that many academics would be unhappy to confess to this.

In considering how the use of project teams may enhance the learning experience, we should look at the roles that team members play when engaging on an assignment. Far too often, we find it easy to give the assessment to teams, and "let them get on with it". A structured assessment which has well considered learning objectives and outcomes should provide a reasonable learning experience for all involved. We shouldn't use assessment as a rod - although I'm not sure about how we deal with those who do not want to work - that might be an issue for recruitment, rather than assessment.

Perhaps we can take a leaf from other organisations where learning and assessment are involved. The fact that we link the two together might be where some of our problems lie. Is it time to (re)consider a national examining body for higher education, which provides our syllabus and sets the assessment? The teaching mechanism and the assessment mechanism might well do with parting company, if only perhaps to develop the two issues as separate professions - why not?

Subject: The value of thoughtful assessment
From: j.holzl@qut.edu.au (Janice Holzl)
Date: Mon, 1 Jul 1996

Thank you for a most interesting discussion on assessment. I would like to add a few thoughts on this very important topic. Having been involved in teaching and assessing students (undergraduate and postgraduate) for the past six years, I have learned the value of thoughtful assessment. There are a number of issues which require careful consideration.

Firstly, assessment is inevitably affected by the type of course (theory or clinical based), entry level of the student (first year-first semester graduate),entry experiences of the students (ESL, high school graduates etc), and class numbers (700 - 1000+ students makes for a lot of essays).

Secondly, the goals of the course, expectations of the teachers, expectations of attributes of graduates, and expectations of future employers and other stakeholders all impact on the assessment process (overtly or covertly).

Given these components, it should not surprise us if there are problems with assessment, particularly in light of short time frames and perhaps lack of adequate discussion on the best way to achieve learning outcomes. In my view, assessment should be integrated into the curriculum and given as much thought as all other curriculum components.

Assessment drives student learning. If we foster understanding and invite risk taking, students will adopt a deeper approach to learning. Concentration on tasks and the ability to memorise will result in a surface approach. It may be, however, that given the nature of the curriculum a combination of these approaches is required. Careful consideration of learning goals and outcomes will assist assessment decisions.

Marking Scales
From: sbzml@nottingham.ac.uk (Martin Luck)
Date: Mon, 20 May 1996

My University uses a marking scale in which:
40% is the pass mark,
40-49% = 3rd,
50-59% = 2.2,
60-69% = 2.1 and
70-100% = First;
I expect many other universities do the same.

This system works reasonbly well for subjectively marked coursework and essay-type exam questions - in most cases the mark awarded is determined by ones view of the class into which the work should be placed.

However, things are not quite so straightforward for objectively marked assessments such as MCQ and short answer assessments. With these, I frequently find that the long First Class range and the low pass mark distort the categorisation of the work. This can have a profound but variable effect on the overall assessment of the student, depending on the ratio of subjectively:objectively marked assesments in the module.

At worst it would devalue the meanings of "First Class" and "Fail" in quite a serious way. To get round the problem, I sometimes realign objectively marked work to fit the subjective scale, but this really defeats the object of objective marking.

How do other people cope with this problem? Is it, in fact, a problem? Should I be awarding more Firsts and fewer Fails!?

Any suggestions?

Subject: Fairness in Assessment
From: pm4@soton.ac.uk (Peter J Melhuish)
Date: Wed, 22 May 1996

Assessment must be seen to be fair to both the student and the institution. It should allow base competencies to be demonstrated, but should also allow excellence to be shown. The process must be scrupulously fair, ensuring that all work put forward is that of the student as claimed. It is most of all an opportunity for the student to show the teacher what he or she knows.

Additionally, at undergraduate (especially second and third year levels) assessment should not unreasonably interfere with the main purpose of the entire activity, the process of learning and of development both of skills and of a liking / interest in the subject. We do not work to provide an accredited rating for others to use. We work to educate students in our discipline, as well as to research, etc

Continuous assessment, tests and short examinations seem to me the worst of all worlds. I would personally prefer all undergraduate (after first year perhaps) modules to be formally examined, possibly in open-book mode, and usually for at least three hours. For some practical activities (ie computer systems design) six hours may be preferred.

I see assignment, laboratory, group project and related work as admirable means of learning, but poor means of confirming knowledge and understanding.

We fail many of our students (especially the more able) by not asking them to demonstrate their understanding in a formal and rigorous manner.

We also confuse demonstrating base competencies with demonstration of real academic ability and conceptual understanding.

Universities are not NVQ institutions! We must be able to state what our students know and understand, but there are better ways of doing this than constantly demanding that all students jump through small, uninteresting hoops!

Am I alone in preferring a teaching period, unsullied by assessment, followed by an examination period, where students sit a set of formal examinations? Is this really too simple?"