Barry Jackson

Assessment practices in art and design: a contribution to student learning?

Barry Jackson, (Middlesex University)

Reproduced with permission from Gibbs, G. (ed.) Improving Student Learning - Through Assessment and Evaluation. Oxford: Oxford Centre for Staff Development (1995)

Introduction

This chapter reports the background, intentions and methodology of a three year research project recently launched by the author at Middlesex University. Using theoretical frameworks derived from the work of Saljo, Biggs, Marton Entwhistle and others it explores the problematic issues surrounding approaches to learning in art and design. Although the focus of the research project is in a design context initially, there are many areas of commonality with fine art. Later work will seek to explore these more fully.

 

Figure 1

What are we Doing?

What is the research intention?

The principal aim of the research is to help to improve the quality of student learning in design. It is intended to be practically helpful to teachers and course designers, and the research itself will be largely carried out by teachers as action researchers. It is hoped that the understanding gained from the project will be of more than local value, and will be transferable to other sites and subjects.

The central research question can be summarized as:

'Are we currently using the best assessment methods to improve the quality of students' learning on design courses?'

In answering this we will throw light on several questions:

• How are we currently assessing learning in design?
• To what extent do the existing methods encourage a deep approach to learning?
• Can better methods be found?
• How will we know they are better?

Each of these questions will itself require other questions to be investigated. For example in order to answer the question. 'How are we currently assessing', we will have to find out what is happening according to course documentation and staff, and what students say is happening.

In order to clarify further the objectives of the project, it is necessary to consider why we are doing the research. And to understand this we need to know:

• something about the research and theoretical frameworks which inform the project,

and

• something of the character of learning and assessment in design as it currently exists.

Why are we Doing it?

Relevant theoretical frameworks about student learning

There exists a good research and theory base regarding the nature of learning in higher education. This research suggests that it cannot be taken for granted that existing teaching and assessment practices are working as well as practitioners believe.

The research has collected data about learning from a broad range of academic subjects in a number of different countries. A broadly consistent picture has emerged and it has been assumed that the concepts and structures which have been constructed on this research basis are therefore universally transferable. However, the implications for art and design subjects remain relatively unexplored.

While there is no reason to doubt that the frameworks are also relevant for art and design, there are some interesting problematics in translating some of the concepts into an art and design context in a way which can be easily accessed by teachers in those subject areas.

Of particular interest are theoretical structures and research instruments associated ~ with the concepts of levels of learning outcome, and approaches to learning, which offer ~. potentially valuable insights. Associated with these, and linking them are conceptions of learning. These three interrelated concepts form the theoretical framework in which the i. learning and assessment of students will be evaluated (see Figure 2).

 

Figure 2

These have begun to be applied to art and design in a case study undertaken by myself and a colleague at another institution. The outcomes were promising (Davies, 1993). However, the studies raised more questions than they answered, and left a good starting point for a more focused research programme.

Levels of learning outcome

Students show qualitatively different levels of understanding of what they have learned. They are described as showing different levels of learning outcome.

The work of Marton, Saljo, Dahlgren, Svenson and others on learning outcomes provides a framework for thinking about the level of students' learning as demonstrated by the outcomes, and relates those levels to the approach that students take.

There are several ways of categorizing and analysing learning outcomes. Work has been done to investigate content specific outcomes in a number of subjects (e.g. Dahlgren, 1978), but work is just beginning on content-specific analyses of art and design subjects.

Biggs and Collis (1982) offer an empirically derived general taxonomy of levels of learning outcome, called SOLO (Structure of the Observed Learning Outcome). The categories Biggs and Collis describe are intended to be generally valid, independent of the kind of questions asked, or the subject matter contained in them. Subsequent use of the taxonomy in research seems to confirm the transferability of the taxonomy across a number of disciplines (see for example some of the case studies in Gibbs, 1993).

In the SOLO taxonomy five levels of learning outcome can be distinguished, of increasing complexity:

1. Prestructuralno evidence of anything learned.
2. Unistructural one correct and relevant element is present.
3. Multistructural several relevant elements are present but in an unrelated  way, often in list form.
4. Relational the relevant elements are integrated into a generalised structure; there is evidence of induction.
5. Extended-abstract  the structure of elements is related to other relevant domains of knowledge; answers are not bounded by the question.

Although the SOLO taxonomy is intended to be a generally applicable categorization, and was used in the design case study reported in Gibbs (1993), it is not formulated in a  helpful way for analysing outcomes of learning in design. The taxonomy has been constructed using written or verbal evidence for analysis, and this is clearly not a usual outcome of design learning. Current use of the taxonomy in a design learning context therefore requires that interviews, reports or other written evidence be produced specifically for analysis. The identification of what, in design terms, constitute the various levels of outcome in the SOLO taxonomy remains to be done and is one of the objectives of the current research project.

John Biggs has also suggested a system of qualitative grading for assessment, based on . the five conventional letter grades (A-F) linked to the levels of the SOLO taxonomy (Biggs, 1992 quoted in Brown, Rust and Gibbs, 1994). A similar attempt to use the SOLO levels as the basis for qualitative grading was an important feature of the earlier project in design carried out by the author and colleagues at Falmouth School of Art and Design (see Davies, 1994). The potential of this system has not yet been fully explored and the current research project seeks to develop that work in a new context.

Approaches to learning

Approaches to learning are the strategies which learners adopt in order to succeed at learning. The term 'approach' is used to signify both the learner's intention and the way in which s/he processes information.

The quality of outcome of students' learning has been shown to be closely correlated to the level of approach they take to learning.

The concepts of approaches to learning have been developed in some detail and described elsewhere (Marton and Saljo in Marton, Hounsell and Entwhistle, 1984). Broadly speaking a student may take a 'surface' approach, in which material is learned superficially without evidence of understanding, or they make take a 'deep' approach in which the learning changes the way in which the student understands or perceives the subject and its context.

A deep approach is one which would involve the learner in constructing a personally meaningful and well-formed knowledge base. It will be a significant feature in the development of the designer's tacit 'theory of design'.

The approaches are highly context-sensitive. An individual student may adopt quite different approaches in different classes. In seeking the reasons for this, research has postulated and tested a number of features of course design and delivery which . encourage a deep approach, and a number of factors which are associated with students taking a surface approach.

Features encouraging a deep approach

Four features associated with encouraging a deep approach were identified originally by Biggs (1989):

MOTIVATlONAL CONTEXT
Students are more likely to take a deep approach to learning when the motivation to learn is internal, coming from the student's own needs and desires. A positive climate, in which students 'own' their learning, can be established by involving students in the selection and planning of what is learned and how learning takes place. A good motivational context may be a necessary condition for deep learning. Motivational context is profoundly sensitive to the assessment strategies used.

LEARNER ACTIVITY
Deep learning is associated with activity. Students are more likely to make connections between what is being learned and past learning if they are active rather than passive.

INTERACTION WITH OTHERS
Talking and discussing ideas and concepts with others is a powerful way of reflecting on and testing learning. It provides a means of negotiating and structuring meaning which is more effective than solitary reflection alone.

WELL-STRUCTURED KNOWLEDGE BASE
New learning can only be approached deeply if the student can relate it to their existing knowledge and experience. The growing sense of a visible structure, which the subject matter acquires if it is presented in an integrated way, provides the student with a model of the knowledge they are acquiring, which can then be more easily manipulated and related to other knowledge. Interdisciplinary approaches can encourage a well-structured knowledge base.

Features associated with a surface approach

Research has similarly identified a number of features of course design and delivery which are associated with students taking a surface approach.

HEAVY WORKLOAD
A heavy workload allows no time for reflection and discussion: there is limited opportunity to consolidate the structure of knowledge or to relate it to existing knowledge. The actual workload which is imposed on students may often be heavier than lecturers think. The expectations of various lecturers associated with a course or programme may mean that students have to commit substantial extra time to work on assignments, reading set texts and preparing presentations.

RELATIVELY HIGH LEVELS OF CONTACT TIME
Students need to be independent and active in their learning and to have opportunities to discuss their learning with peers. A high level of contact time may not leave sufficient time for these important activities. It may also encourage the view that learning is passive, which is likely to encourage a surface approach.

EXCESSIVE AMOUNTS OF COURSE MATERIAL
In addition to contributing to a heavy workload, excessive amounts of course material reduce the student's chances of being able to relate the various components of their learning meaningfully, and to build a well-structured knowledge base.

LACK OF OPPORTUNITY TO STUDY SUBJECTS IN DEPTH
If students have no opportunity to develop their learning in depth in any particular area, their 'ownership' and their ability to make personally meaningful sense of it are likely to be diminished.

LACK OF CHOICE IN WHAT IS STUDIED, AND HOW
Lack of choice may reduce the positive emotional climate associated with good . motivation.

ANXIETY-PROVOKING ASSESSMENT METHODS
A sense of anxiety or threat may be conditions which are in themselves sufficient for surface learning. Assessment is the main source of anxiety in any educational experience.

To this list can be added one further feature:

ASSESSMENT METHODS WHICH REWARD SURFACE APPROACHES
The power of assessment to determine students approach to learning has been recognized in several research projects. Assessment methods which permit or even encourage surface approaches to be rewarded can be a strong influence on how students learn.

Work in other disciplines has found a clear correlation between a deep approach and the higher levels of learning outcome. There is some evidence that the same correlation exists in design learning.

Evidence suggests that many, if not most, learners are able to take a surface or deep approach in different circumstances. That is to say that approaches, unlike 'learning styles', described for example in the work of Kola or Honey and Mumford, are not personal unchanging characteristics. They are, by contrast, very dependent on context.

However, there does seem to be evidence of a small proportion of learners who find difficulty in employing a deep approach, even in favourable contexts.

Work by Entwhistle and Ramsden has demonstrated the significance of looking at students' approaches to learning in relation to their orientation. The Approaches to Study Inventory (ASI) identifies three possible orientations:

1. meaning orientation, which correlates with a deep approach;
2. reproducing orientation, which correlates with a surface approach;
3. achievement or strategic orientation, in which a student will be oriented primarily towards being successful, and will therefore readily adopt whichever learning strategy is most likely to gain success.

There is some evidence that a relatively high proportion of design students may enter higher education with an achievement orientation. This will be of interest to the current research project, since it would have significance for the kinds of strategies which might be successful in improving student learning in the subject.

The nature of the connection between the two approaches and the level of outcomes may be illustrated in the Figure 3

 

Figure 3: The relationship between approaches, context and outcomes

Two possible avenues of improvement thus suggest themselves:

1. to design and provide learning contexts which encourage 'achievement orientated' students to take deep approaches, and which reward their appropriate outcomes in assessment situations;
2. to find ways to encourage 'chronic' surface learners to understand and apply deep approaches to their learning.

Conceptions of learning

The approach to learning that a student takes is very sensitive to the context in which learning is done. The features outlined above are examples of this. For some students, however, the ability to take a deep approach appears to be limited by the conception they hold of learning.

This conception will to a large extent determine the student's expectation of what the learning process entails: and of what teaching entails.

From research Van Rossum and Schenk (1984) developed a set of categories of conceptions of learning held by students which is a helpful framework for further work. Learning can be conceived of as:

1. a passive increase in knowledge;
2. memorization of all apparently relevant facts;
3. acquisition of usable facts;
4. abstraction of meaning;
5. understanding reality.

Of these categories, levels 4 and 5 are qualitatively different from the first three, since they involve an active construction of personal meaning on the part of the learner.

The same authors demonstrated a correlation between these more advanced conceptions of learning and a deep approach to learning.

The relationship is illustrated in Table 1. Preliminary work suggests that the descriptions of the various levels of conception are meaningful to students of design, even though they have to translate the descriptions into statements more related to conceptions of learning in a design context.

The three related concepts of conceptions of learning, approaches to learning and levels of learning outcomes provide the structures within which the research project is framed. In order to consider the particular ways in which design learning relates to these, it will be necessary to explore the current character of design education.

The existing character of learning and assessment in design

Design practice requires the exercise of a complex interrelationship of skills, knowledge and understanding. In acquiring the competencies that mark the path from novice towards expert, the developing designer also learns a body of knowledge. Some of this knowledge is explicit and factual; it is usually acquired, in the way of most factual knowledge, from sources of information such as lectures, reading and observation.

(from van Rossum and Schenk, 1984 quoted in Marton, Hounsell and Entwhistle, 1984)

Table 1: Relationship of conception of learning to approach

A more significant part of the designer's knowledge and understanding has more problematic origins: the understanding of what design itself is, and the ability to choose and apply appropriate methodologies, for example. These are learned through the practice itself, in which the reflective designer constructs a personal knowledge-base. This knowledge base, although individually formulated, shares common conceptions with others in the design community and amounts to a tacit theory of designing and design values.

Unlike other academic subjects, design has only a small body of factual knowledge which a student has to acquire and which can be assessed. There are an infinite number of right answers to any problem, but not all answers are right. The theoretical understanding has to be gained by practice and reflection.

The learning process in which designers acquire, formulate and construct theories of designing during the progress of their formal education is not well understood. The teaching of the subject has evolved a set of practices based on assumptions and shared values. These values have certainly contributed to the successful education of many designers - the British design education system is well regarded, particularly by its own practitioners, and, with some reservations, by the industry it mostly serves.

However, these values are not necessarily helpful in encouraging students' . development of learning, and while this may not ultimately be harmful to some students, there are many students who do not perform as well as they might. This fact in itself is associated with one of the concepts underlying design education, that of intrinsic talent. Students who do not perform well, it is argued, are either lazy, or simply lack the necessary talent.

The research project aims to question the validity of such assumptions and to test the hypothesis that the differential between students' performance can be at least partly attributed to the differences in their understanding, and that that in turn can be helped or hindered by the teaching and assessment which the students experience.

There are other features of design education which are relevant. Some of these might be expected to encourage a deep approach to learning, others not.

• Learning is focused on skills, and therefore students are involved in active learning for most of their time.
• Within the culture of design education there is a resistance to the use of verbal language.
• There is a celebration of the individual achievement within the culture which discourages collaborative work.
• There is a high level of intrinsic motivation in students studying design.
• Accomplished professional performers are valued highly as 'masters'.
• Students are expected, and expect, to personally invest a great deal of their own time in their learning, outside of class hours.

The assessment practices in art and design education also have a weight of tradition behind them, but there is little evidence that they are appropriate to encouraging a deep approach. Assessment usually centres on the quality of the design artefact, and this is exactly the area in which connoisseurship plays such an important part. There is an increasing emphasis on using clear assessment criteria and this has helped overcome problems of subjectivity. However there remains an overriding concern with the quality of 'creativity', a notoriously slippery concept, and one subject to considerable variations in personal interpretations.

In the context of this chapter it may be useful to draw attention to some features of assessment in design which seem to me to be important.

For simplicity's sake the description of art and design assessment will be generalized, not to say caricatured, in a way which I am aware does not do justice to the variety of practice which exists now, and has existed to a lesser degree in the past; it does, however, give insight into the cultural context in which innovations are taking place.

When does assessment happen?

Assessment in art and design higher education begins before the students start the course. It starts at admission, when students, typically having spent a year on a Foundation course, are interviewed and assessed in order to predict the probability of their success on the course they are about to enter.

Assessment continues throughout the course, culminating in a grand summative assessment at the end, on the basis of which an appropriate level of award is recommended. This practice continues even on some modular courses.

Many, if not all, projects which students undertake as part of their studies may be assessed, with a formative and/or summative intention. The results of these assessments may contribute to a summative assessment of the student's ability to progress to another level or another module, or to the final award.

There are often stage or interim assessments, points at which students must achieve a specified level before being allowed to progress into the next stage of the course. On modular schemes these are formal end-of-semester assessments.

How does it happen?

Often summative assessment and formative assessment are combined: students are given feedback intended to be helpful to them, in addition to being marked or graded.

Formative assessment often takes place informally in the studio or as part of a tutorial; this practice was until recently almost always one-to-one, and is now more often undertaken in groups. The crit has been a site for critical comment and formative assessment of a kind, although the quality of crits is very variable and the practices vary considerably, both within and between courses.

Summative assessment occurs at the end of projects, at the end of semesters or at stage assessment points, and of course at the exit point. The dominant form of assessment in all of these has been the assessment of the appropriate visual work, usually in exhibition, and usually led and dominated by tutors.

Assumptions and values underlying 'traditional' assessment in art and design

There are several assumptions to be noted underlying 'traditional' assessment practices.

Underlying the 'traditional' practice of the end of course, or end of semester, assessment by exhibition there are several assumptions:

• that students' achievement of course objectives can be judged adequately from looking at the physical artwork products;
•  that students develop progressively towards their best work, and that their 'exit velocity' is the best and fairest measure of their ability;
• that students are novice designers, who, on graduation become qualified, if somewhat junior, experts;
• that the judgement of student' readiness to join the fraternity of designers rests most properly in the hands of proven experts - that is, designers working in education;
• that the concept of 'final assessment' is a useful and valid one.

Although student involvement in project assessment has grown substantially it is still not uncommon to encounter situations in which students participate in critical comment about each others' work, but the work is then taken away by tutors to be marked.

Behind this practice are assumptions that only tutors are adequately informed, experienced and 'reliable' to make these important judgements. Of course this is not the case only in design; in many other disciplines the marking is wholly in the hands of lecturers. But for art and design it is both less understandable and at the same time more difficult to break out of.

It is less understandable because the individual nature of much of the project work being assessed makes it almost certain that the person with the best understanding of the objectives is likely to be the student herself.

It is more difficult to break with because art and design has a hero culture which privileges the judgements and opinions of individual practitioners/teachers. There are few other disciplines which pride themselves so much on the eminence or on the level of professional practice (by which I mean practice as artist or designer) of their staff teaching at undergraduate level.

Specific questions about learning and assessment in design

Attempts to think about learning and assessment in design programmes, in the context of the theoretical frameworks outlined above, very quickly encounter difficulties.

The concepts of 'deep' and 'surface' approach were originally defined in terms of texts - whether a reader seeks for meaning below the surface of a text, or instead tries to memorize the information in the form in which it appears in the text. The concepts were quickly extended to other learning situations where their validity was confirmed. They are now taken to be valid in any area of learning.

However, translating these concepts into a form which can usefully be applied to design is not unproblematic. There are teachers within design education who would assert that the study of design is synonymous with a deep approach, arguing that the intensely personal motivation, the active learning by project and the group critiques of work which characterize design education necessitate a deep approach. They also argue that in design learning there is no place for an equivalent of the simple factual learning which is a feature of a surface approach. There are similar problematics about describing learning outcomes.

We are led to a series of questions:

• what does a deep or surface approach look like in design?
• can the SOLO taxonomy be applied to the normal products of design learning?
• if so, how?
• if not, what evidence or framework may be used to distinguish levels of outcome?
• do current assessment methods permit low-level learning outcomes?
• do current assessment methods encourage a deep approach? how might assessment methods be improved?

These are the questions the research project attempts to address.

How are we Doing it?

An appropriate research framework

The research takes the form of an investigation and case study in the first stage, alongside proposals for innovations and action research into their effectiveness.

The action research paradigm is very appropriate for this kind of study. The research is intended to have practical and helpful outcomes, and to be carried out by staff who are closely concerned with putting the outcomes into practice. It is this lack of division between practitioner and researcher which characterizes action research.

It is in the nature of action research that the specific research aims, the hypotheses underlying the research, emerge more clearly as a part of the research process itself. In this it reflects a characteristic of the phenomenographic research process used by Marton and others. In this research method the categories of data are not prejudged but emerge directly as the result of rigorous qualitative analysis of the data. Both methodologies will be used in the current study.

The action research elements of the project will be grounded in the theoretical frameworks outlined earlier in the chapter. This body of work also provides a useful methodology and valuable research tools.

By using the methods of data collection familiar to readers of the research literature, that is the questionnaire inventory, and analysis of student interviews, it is hoped that insights into the impact of assessment on learning in design can be gained. At the same time the appropriateness, in this context, of the research tools and theoretical frameworks themselves will be tested, and this may lead to the development of more sophisticated instruments.

Action research is based on a cyclical process of action-observation -. reflection-planning - action - etc. The project plans, as reported here, are the outcome of reflection on observations of previous practice. The detailed planning of the first stage is now underway and will lead to activities to take place in the new academic year, 1994-95. The outcomes of the project will be reported when appropriate.

References

Biggs, J.B. and Collis, K.F. (1982). Evaluating the Quality of Learning. New York and Sydney: Academic Press.

Biggs, J.B. (1992). A qualitative approach to grading students, HERDSA News, 14(3).

Brown, S., Rust C. and Gibbs, G. (1994). Strategies for Diversifying Assessment in Higher Education. Oxford: Oxford Centre for Staff Development.

Davies, A. (1993). Improving student learning in design: a case study. In Gibbs (1994).

Davies, A. (1994). Identifying and working with student expectations of learning. In Gibbs (1994).

Davies, A. (1994). Self and peer assessment in design. In Gibbs (1994).

Gibbs, G. (1993). Improving the Quality of Student Learning. Bristol:TES.

Gibbs, G. (1994). Improving Student Learning - Theory and Practice. Oxford: Oxford Centre for Staff Development.

Marton, F., Hounsell, D. and Entwhistle, N. (1984). The Experience of Learning. Edinburgh: Scottish Academic Press.

Marton, F. and Saljo, R. (1984). Approaches to Learning in Marton, Hounsell and Entwhistle (1984).

Dahlgren, L.O. and Marton, F. Students' conceptions of subject matter: an aspect of learning and teaching in higher education. Studies in Higher Education, (1978) 3, 25-35.

Van Rossum, E.J. and Schenk, S.M. (1984). The relationship between learning conception, study strategy and learning outcome. British Journal of Educational Psychology, 54, 73-83.