The paper will investigate the use of the SOLO (Structure of the Observed Learning Outcome) Taxonomy as a possible tool for assessing students. The taxonomy was applied to assess the quality of Pre-Technician Physics students’ practical reports.

The SOLO taxonomy describes the growth of competence of a task across a number of subject areas in order to determine the quality of the learning outcomes. The course of development of a concept or principle is charted by establishing a hierarchy of conceptions of understanding on a topic by topic basis or by using a general taxonomy such as the SOLO taxonomy.

The paper will report on the application of the taxonomy in assessing the Pre-Technician students’ practical reports. Issues such as describing the different levels of competence, establishing assessment criteria and the indication of competence development will be addressed.

Assessment is an essential part of the teaching and learning process because it has such a powerful effect on learning. In the past the technology of assessment has been such that this effect was not always a beneficial one. However, assessment, when correctly applied, promotes learning and improves student performance. In addition, it determines how well a student has achieved certain objectives and provides appropriate feedback to both student and lecturer.

Modern assessment procedures and techniques seem to be moving away from the traditional pen and paper test, to correspond with the new theories of teaching and learning. Although no theory of assessment can be regarded as perfect, much needed reform in assessment procedures has become an important issue in education.

The purpose of this assignment is to investigate the use of the Structure of Observed Learning Outcome (SOLO) taxonomy as a possible tool for qualitatively assessing students in higher education.

The Philosophy

Sociologists and philosophers today regard contemporary society as being post-modern. "Postmodernism is difficult to explain but generally is regarded not as a philosophy but a reflection of the conditions of present day society" (Gericke, 1996, p.1). One of the most important characteristics associated with postmodernism is social constructivism. Social constructivism implies that man lives in a symbolic world:

· Reality is constructed by more than one person.

· The world reflects realities due to the different experiences of different people.

· Meaning is not objective because it is constructed by society.

There are two broad theories concerning the nature of knowledge. The first is called the objectivist tradition and locates knowledge outside the learner. In this paradigm, understanding is getting to know what already exists. Assessments in this tradition are mostly concerned with quantitative measurements. The second tradition rejects the objectivist theories and can be divided into two streams of which constructivism is one. The constructivist tradition regards the learner as the focus in the creation of meaning, rather than the teacher as the transmitter of knowledge. "Learning is context sensitive: the learner brings to any learning situation an accumulation of assumptions, motivations, conceptions and previous knowledge, which will largely determine the nature and the quality of learning" (Jackson, n.d., p.23). Learning and, consequently assessment, is seen in qualitative not quantitative terms. Constructivism is rapidly becoming the dominant espoused theory in education but it remains to be seen if it becomes the theory in use.

Biggs and Collis (1982) suggested that their SOLO taxonomy could be used as a framework for translating some features of constructivism into assessments.

The SOLO taxonomy describes the growth of competence of a task across a number of subject areas in order to determine the quality of the learning outcomes.

As the emphasis of the SOLO taxonomy is on qualitative assessment of student responses I feel that it is necessary to briefly distinguish between the quantitative and qualitative approach to assessments.

Assessment can be both quantitative (how much) and qualitative (how well and in what way).

"In the quantitative outlook, learning is conceived as the aggregation of content; to be a successful learner is to learn more" (Biggs, 1994, p.2). The contents of learning refer to either declarative knowledge or procedural knowledge, that is bits of knowledge, unrelated facts and skills. The process of learning is to collect more and more of these bits, internalizing them and being able to reproduce them accurately. Assessment is seen as involving test situations that reliably indicate whether or not a student can produce the taught content correctly and speedily. The content is regarded as knowledge and is learned as binary units, correct or incorrect. The correct units are summed up to give an aggregate or total score.

"In the qualitative outlook it is assumed that students learn cumulatively, interpreting and combining new material with what they already know thereby changing their understanding progressively as they learn" (Biggs, 1994, p.3). The constructions the learners make become the outcomes of learning. Thus, the aim of qualitative assessment is to determine at what point the students are currently in their understanding of the concept or skill that is questioned. In qualitative assessment it is necessary to chart the course of development of a concept or principle in order to define the development. "This could be done by establishing a hierarchy of conceptions of understanding on a topic by topic basis or by using a general taxonomy that can be applied over a whole range of subjects" (Biggs 1994, p. 3). Research has shown that the SOLO taxonomy could serve as a useful tool for categorising students’ responses in a qualitative way.

Many different interpretations have been given of the SOLO taxonomy and some examples are:

· Helping students to go Solo: teaching critical numeracy in the biological sciences,

· Assessing the structure of learning outcomes from counselling using the SOLO taxonomy

· Evaluating LOGO, the SOLO taxonomy.

My own understanding of the development of the SOLO taxonomy is as follows.

The development of the SOLO taxonomy by Biggs & Collis (1982,1989) was a direct result of investigating the structure of learners’ knowledge. Building upon the developmental learning theories of Piaget and Bruner, Biggs and Collis believe that there are natural stages in the growth of learning any complex material, information or skill and that in certain aspects these stages are similar but not identical to the developmental thinking stages described by Piaget.

Biggs and Collis (1982) suggest that the quality of learning not only depends on the quality of instruction but also on the student’s motivation, developmental stage, and prior knowledge. If students haves no prior knowledge of the content, perhaps due to their lack of interest, they could respond to questions in an egocentric way in an attempt to hide their ignorance. However, Piaget (cited by Collis and Biggs, 1982) linking developmental growth to responses, found this response typical of young children around the age of 4-6. This led to Biggs and Collis’s theory that even though students’ developmental stage might be in the upper levels, their motivation and previous knowledge, in other words, their intrinsic features, will determine whether they always function at that level.

It is common knowledge that most people function well below their potential maximum. This observation has led to a lot of confusion amongst teachers who follow Piaget’s theory, as they might find that their students do not always operate at the same level as their developmental age suggests they should, nor do they perform consistently. It was because of this confusion between the stages of development and certain test results that the SOLO terminology was first introduced. To sum up it seems that when looking at outcomes of a particular learning episode it is not the quality of student that is the issue as Piaget’s theory suggests but the quality of the learning achieved at that particular time. The SOLO level describes a particular performance at a particular time and is not meant as a label to tag students.

The SOLO taxonomy proposes five structural levels through which a learner passes.

These levels are arranged, on the basis of the structural organisation of the knowledge in question, from incompetence to expertise. The levels are as follows: Prestructural, Unistructural, Multistructural, Relational and Extended Abstract. Using this framework it now becomes possible to identify a student’s current level of operation of a particular task through verbal and/or written responses. It provides educators with a powerful tool for the assessment of their students’ understanding of concepts and problem solving.

The SOLO taxonomy provides a systematic way of describing how a learner’s performance grows in mastering an academic task. APPENDIX A gives an explanation of the levels as well as the performance of understanding. The verbs are a description of the relevant performance levels and indicate a particular quality of performance.

At the Prestructural response level the student is simply saying: "I don’t know," repeating the question or conveying some irrelevancy. The student who answers a question at this level is merely seizing the first relevant aspect that comes to mind.

At the Unistuctural level all the responses can be equally correct yet inconsistent with each other. The student is able to discuss content meaningfully, as a reasonable amount of content is known. However, the student does not apply or transfer this knowledge easily.

In the Multistructural level response, closure is determined when more aspects are perceived, but these aspects are not interrelated and inconsistency results. Two answers at this level may give the same amount of information but fail to link up and therefore come to different conclusions. A typical response contains sentences with links such as: "and, and so, and also."

At the Relational level responses are still tied to concrete experiences and inconsistency appears across context. Yet the response gives an overall concept that accounts for the various isolated data that the Multistructural response contains but stays with concepts already taught.

The Extended Abstract response takes up all relevant data and their interrelations and lists them under a hypothetical abstract structure that enables deductions to apply to information that was not included originally, thereby allowing the student to give alternative outcomes and not come to a definite closure or conclusion.

It is sometimes found that students’ responses do not fit exactly into one of the five levels. Biggs and Collis refer to these as transitional responses. These transitional responses occur when a student is nearly at the next level but doesn’t quite make it. This is often indicated by a certain amount of confusion and inconsistency. It is as if a student is handling more information than he can cope with and then loses track of the argument. These responses carry more information than the previous level that the student is emerging from, yet the student is unable to reach the complexity expected at the next SOLO level. Because of this a well-formulated transitional response at a lower level can be mistaken for an Extended Abstract response. Transitional responses are referred to as 1A, 2A, 3A, in APPENDIX A.

The commonly stated aims of learning in higher education are closely connected to a constructivist approach. Higher education is seen as something more than just the simple acquisition of skills. Learning at tertiary level is said to be about the development of understanding and the ability to apply critical judgements to presented knowledge.

"Adult learners in higher education are supposed to grow in competence as they learn. It is not sufficient if learners only accumulate declarative knowledge. What should be focused on, is the adult student’s performative aspect of understanding knowledge. Such understanding will cause students to act differently in contexts involving the content understood (Gericke, 1996, p5)."

Constuctivism emphasises that those internally involved in the classroom, that is, teachers and students, should be heavily involved in assessing the learning progress. This will require a radical change in tuition methodologies as well as course material in higher education. The following are a few examples where changes should be implemented.

Assessments should probably be done continuously to establish students’ gradual growth in competence, and should relate to practical life and job related experiences. Students should be assessed in terms of identifying learning outcomes within diaries, portfolios etc. The systems that give marks should be aborted in favour of grading, so that assessment is done in terms of reaching outcomes, for example, to what extent did the student meet the required outcomes and is able to prove that he meets the required learning outcomes.

The curriculum or unit objectives need to state clearly specific levels of understanding in terms of content that implies the desired performances.

It is quite clear that teachers need to specify the things that they need to do for students in order to demonstrate a particular level of understanding. It should also be possible to select teacher learning activities (TLA) for example portfolios at the Extended Abstract level, that specifically address desired performance of understanding. The planning of TLAs should include teachers, peers and individuals so that a higher level of understanding can be achieved. If teaching is to stimulate competence it will be necessary to select a wide range of different learning related activities.

Using one of the subjects that I lecture as an example, I will attempt to draw up my own model of how SOLO taxonomy could be applied in higher education.

The students are Pre-Technician students who are doing a 6 month bridging program to improve their Matriculation symbols in Mathematics and Science in order to reapply for a main stream diploma course. The lecture activity is a physics practical e.g. Hooke’s Law. This practical is based on theory students’ receive during their physic lectures. The practical is presented to the student as a demonstration to familiarise them with the correct procedure, technique and general requirements of the experiment, after which the students (who work in groups of 2-3) repeat the experiment for themselves. On completing the experiment each student is required to write up their results, observations and interpretations according to a set format. The practical write-up is then assessed using the SOLO taxonomy as a tool. First it is necessary to set up a hierarchical list of performances of understanding as my objectives so that a grading category can be established (Refer to Appendix A).

a. Most desirable (extended abstract)

Students are able to evaluate their decisions and reflect on their results.

b. Very satisfactory (relational)

Student is able to apply, recognise and understand content.

c. Moderately satisfactory (multistructural)

Student understands, and discusses meaningfully

d. Barely satisfactory (unistructural)

Evidence of some effort but student misunderstood some of the instructions.

e. Unsatisfactory outcomes (prestructural)

Lack of effort and involvement.

It is now possible to use the above objectives to form categories that I can use for grading, so that (a) to (e) becomes "A", "B","C", "D" and "F".

Now using Appendix B, I am able to assess each student’s write-up accordingly. I do however wish to point out that this is only a provisional guideline, which requires refining in terms of the curriculum and objectives of the practical session before implementation for assessment.

In this paper I have attempted to show how the SOLO taxonomy could contribute towards meeting the problem of evaluating quality in learning in higher education. It must not be seen as a method of teaching but purely as a structure to assist teachers to make judgements about the quality of learning. However it is important to realise that its use in context does require a certain model of instruction. This implies that the facilitator must have certain definite intentions about the amount of quality learning that is to take place. As Biggs states,

Arnold, S. (1996). Challenge and support: The SOLO taxonomy. [online] Available from Internet: URL. http://www.edfac.unimelb.edu.au/ DSME/TAME/Arnold/TAME-pages/CandS4.html

Biggs, J B & Collis, K F (1982). Evaluating the quality of learning THE SOLO TAXONOMY. Australia: Academic press

Biggs, J B & Kirby, J R (1980). Cognition, development and instruction. USA: Academic press

Biggs J B (1993). From theory to practise: A cognitive system. Higher Education Research and Development, vol. 12 no 1, pp. 73-78.

Biggs, J B (1994). Student learning research and theory - where do we currently stand? [online]. Available from Internet: URL

http:/www.lgu.ac.uk/deliberations/ocsd-pubs/isltp-biggs.html

Biggs, J B (1996). Enhancing teaching through constructive alignment Higher Education, vol. 32, pp. 347-362

Biggs, J B (1993). From theory to practise: a cognitive systems approach. Higher Education Research and Development vol. 12, pp. 73-85

Boulton-Lewis, G M (1994). Teachers as adult learners: their knowledge of their own learning and implications for teaching. Higher Education, vol. 32, pp. 89-106

Burnett, P C (1999). Assessing the structure of learning outcomes from counselling using the SOLO taxonomy: an exploratory study. British Journal of Guidance & Counselling November 1999, vol. 27 issue 4, [online][approximately 13 pp] Available from Internet: ehost@epnet.com

David, L (1999). Helping students to go SOLO: Teaching Critical Numeracy in the biological sciences. Journal of Biological Education, Autumn 99, vol. 33 issue 4. [online]. [approximately 13 pp]. Available from Ebscohost, Electronic information for libraries. Database: Academic Search Elite

Gericke, D H (1996). Assessing transformative learning : the SOLO taxonomy. Unpublished paper.

Jackson, B (no date). Evaluation of learning technology implementation. LTDI Evaluation Studies.