Quantity and Quality in University Teaching

Text

some minor notes on quantity & quality

(university, why?)

1

4:00 2:30 5:00

3:00 4:15

8:00 4:00 4:00

I am a technologist. In thinking about why I might have been invited here, and what I wanted to say, I realised it was important to remember why I use technology in my teaching. It puts the tools of production, of making, into the hands of my students. Making matters. I’m a humanities academic. We have a sort of Cartesian schizophrenia where we accept the separation of mind and body. We have ideas. We write words, abstractions. We don’t much care about layout, fonts, though if we’re any good we do care about words and sentences. You don’t eat in class because it is simply too carnal, and never submit an essay on coloured paper if you want to be taken seriously. This separation of the corporeal body and the idea is what I have worked against in all my teaching, and a lot of my research practice. New technologies let students be creators, contributors, peers. Most importantly, they let us all make different things, differently. It is in this differing that learning happens.

exercise caution

Beware of carpet baggers from the south.

What I am going to wonder and wander about is teaching.

Been invited here before, I liked my ivory tower too much.

What’s a university to do, educationally?

2

The things that make a difference for someone in Melbourne are quite different to regional areas in most other parts of Australia. Last time I was at this conference Carol almost certainly wanted me to talk about video blogging, RSS and the like. But I talked about much more experimental things that video could do. Not that practical.

There is little structure. Just some ideas. I am doing and performing one of the ways in which I think things can be done differently. A mix of little and larger ideas, anecdotes and provocations. I tried it in different orders and there isn’t any that works much better than another. Your experience of it should be mixed. Some of it will be a bit like a silent movie.

LIKE A BLOG

like a blog

This is a style of ‘thick description’.

3

like a blog

This is a style of ‘thick description’. What I am talking about comes from my experience as a media academic and educator.

4

Media is one of the key sites undergoing radical transformation right now. It is exciting, I think it is extraordinary to be a part of it. Media is at the pointy end of things and right now I get to watch once solid institutions melt to become archaeological relics.

This is a style of ‘thick description’. What I am talking about comes from my experience as a media academic and educator. I work in a post industrial media ecology.

like a blog

5

Industrial means things needed a lot of capital, and a lot of specialisation. Expensive to make and access to the means of making media, let alone finding an audience, was highly limited because it was so scarce. It was scarce because it was expensive. The post industrial has reversed all the terms of this economy.

This is a style of ‘thick description’. What I am talking about comes from my experience as a media academic and educator. I work in a post industrial media ecology. I think all of us do.

like a blog

This is a style of ‘thick description’. What I am talking about comes from my experience as a media academic and educator. I work in a post industrial media ecology. I think all of us do. It is an ecology more than it is an economy.

like a blog

6

It is an ecology not an economy because it really is a network. There are asymmetries, bits that are bigger, more important, have bigger impact, but it is still interconnected and inter-related and the relations between the parts are more important than the parts themselves. The links to a site are much more important to Google than the site itself. Relations matter, and this is an ecology. A simple premise from this is that relations between students are as important, if not more, than the relation between the teacher and the students.

MY TRIBE

my tribe

I work in media studies, and more recently honours education across media and communication.

7

All tribes have particular ways of going about doing things and of understanding their world. The most powerful of these are those that have become internalised to the extent where the values they express appear normal, natural, and just, well, common sense. Whatever we call them, they are ideologies, and while we can’t escape ideology I think it is imperative to try to be aware of them and to recognise the coercive force they have. There is an ideology of education attached to being an academic. One thing I’m doing here is naming some of it, and wondering about its legitimacy.

I have been instrumental in rebuilding the media studies curriculum at RMIT in Melbourne. I have also led curriculum renewal for a large honours program within a school of media and communication. Aside from disciplinary specifics the heart of this has been to appropriate a range of strategies and methods from design education and practice and bring them into the traditional humanities curriculum.

it is an effort to dissolve that cartesian distinction between thinking and doing .

my tribe

I work in media studies, and more recently honours education across media and communication. It is a luxury, yet this luxury lets us conceal an awful lot.

8

The luxury is that I get students with high academic skills. The sorts who can write a 2000 word essay overnight and do pretty well. But these students are good enough, and well enough acculturated to the learning regimes of measurement as assessment that it lets education at university be lazy. Provide some content, set the reading, talk to them about it, write an essay, mark it. Really it sounds a bit like an autopilot doesn’t it?

For academically inclined students it is easy to game the system, and these are the ones that we hold up as exemplary, to show that we are good teachers (these are our high distinction students). But they’re high distinction students in spite of us, not because. And they’re high distinction students because they mirror the values of us as academics and that is what we privilege. It does not follow that they are the best students, if best might mean they have learnt or achieved the most through your subject, or have changed their understanding in significant ways.

my tribe

I work in media studies, and more recently honours education across media and communication. It is a luxury, yet this luxury lets us conceal an awful lot. I am not an education academic, though I have been to education conferences.

my tribe

I work in media studies, and more recently honours education across media and communication. It is a luxury, yet this luxury lets us conceal an awful lot. I am not an education academic, though I have been to education conferences. Education academics appear to be as socialised into conservative academic norms as any other discipline or professional body.

9

. Conferences run the risk of being the place where tribes reinforce why they matter to themselves, on their own terms, in their own language. Or, as Kevin Kelly more or less put it, “Institutions will try to preserve the problem to which they are the solution.” We are as guilty of this as any other institution. It’s what tribes do.

SCARCITY

scarcity

I studied media in the 1980s. I went to university to find;

I studied media in the 1980s. I went to university to find; a scholarly community, a video camera, an edit suite, a reference library, experts, and somewhere to argue about ideas.

scarcity

I studied media in the 1980s. I went to university to find; a scholarly community, a video camera, an edit suite, a reference library, experts, and somewhere to argue about ideas. I can now find the community online, the camera is in my pocket, the edit suite is on my laptop, the library begins with Wikipedia, and I can find outstanding blogs.

scarcity

10

The university defines itself via an economy of scarcity. if you wanted these things, you had to go to where they were. They were scarce because they were expensive. This is an industrial model. Universities, to be very crude, had intellectual capital and the tools that went with that (library, video cameras, edit suites). This is not the self definition used, but it is the reason why universities could self define in the way that they did.

I studied media in the 1980s. I went to university to find; a scholarly community, a video camera, an edit suite, a reference library, experts, and somewhere to argue about ideas. I can now find the community online, the camera is in my pocket, the edit suite is on my laptop, the library begins with Wikipedia, and I can find outstanding blogs. Most of my students today go for the same reason I did, fetishising the technology.

scarcity

I studied media in the 1980s. I went to university to find; a scholarly community, a video camera, an edit suite, a reference library, experts, and somewhere to argue about ideas. I can now find the community online, the camera is in my pocket, the edit suite is on my laptop, the library begins with Wikipedia, and I can find outstanding blogs. Most of my students today go for the same reason I did, fetishising the technology. Staff do the same.

scarcity

11

In the media program I teach in students still subscribe to this model. This scarcity is partly an effect of competitive entry - it is academically hard to get in to - but more importantly it comes out in their fascination with thinking that they must use the best equipment and this is how what they are doing matters. A technical fetish is used to generate scarcity. Staff largely approach it the same way. This is a quantitative model where high quality equipment is mistakenly thought to be related to high quality learning and outcomes. It is quantitative because it is about tech standards. In other words they approach this as if we are still in an industrial model.

This happens everytime someone uses the "but it is what industry uses" argument. Lets be clear, a good film or book is not really good because of the tools used to make it.

In 1980 it was enough for the university to offer these things. Quantity was sufficient because of scarcity. Now?

scarcity

BEING ACADEMIC

being academic

I got to be an academic because I am smart.

I got to be an academic because I am smart. Yet at primary and secondary school you get taught without being socialised into being teachers.

being academic

12

The sort of smart I am was easily recognised by teachers at university because it was more or less smart in an academic way. I was a mature age student and keen to learn. I seemed to get the importance of the ‘reflexive’ in art and theory and fitted right into this sort of academic environment. I was a high distinction student. When I became an academic people like me were my best students. These were the ones I taught to. They got theory. The ones who enjoyed my classes and had smart questions. They Iet me think I was pretty good at this since look how good they were. But they were like me when I was a student. I was already like that before I came to the university and apart from providing the place, the university didn’t really do that much more. It certainly didn’t teach me how to be like that, how to get theory and use it. It provided access, because access was scarce. They were industrial times.

I got to be an academic because I am smart. Yet at primary and secondary school you get taught without being socialised into being teachers.

At university good teaching too easily becomes socialising our students into becoming petite academics, little ‘me’s’.

being academic

13

For too many university academics this is the model of good teaching. We use the values and standards of our own profession - a humanities academic - as the default pedagogical values for our students. The essay, abstract theory, the conventions of citation and the short road to hell that is plagiarism. As academics these are the socialised norms of my workplace which I self identified with as a student. But it doesn’t follow that my students self identify with them, nor that these are inherently the most valid elements of what university education is. In a nutshell too many of us in university treat the attributes that drew us into the academy as the pedagogical values of learning in the academy. This is a nonsense. The vast majority of my students are not there to learn how to be academics, and most will never write an essay again, ever.

being academic

As someone with this particular sort of ‘smart’ it turns out I’m good at explaining things. I’m an associative thinker (which is one of the reasons I work in hypertext) and can join up stuff for others.

being academic


But notice what has already happened.

being academic


But notice what has already happened.

14

Even my simple explanation privileges my expertise. My ability to explain. No one told me this might not be the way to be a good teacher. This helped me think I was a pretty good teacher. A qualitative change for me was the shift from understanding that it was not about finding clever ways to explain complex ideas to others, but that my teaching could be the practice of learning and discovery and making knowledge in itself. That my ‘black box’ of theory and doing could be opened and named, prodded, and shown. (And that each student had their own ‘black box’ of learning and knowledge which could be opened, named, prodded and grown.)

being academic


But notice what has already happened. I had become habituated to using my academic norms to judge the academic merit of my students.

15

The mistake, like so many other beginning university teachers, is to then think that these values, the ones I self identify with, are those that matter to everyone else there. But for the largest group, our students, they don’t.

QUANTITY & QUALITY

quantity and quality

I have been thinking about quantity and quality.

16

Quality is my response to why go to university. What I have called a quantitative view i think informs most uni teaching that I see and hear.


I have been thinking about quantity and quality. A quantity is something that has number, scale.


I have been thinking about quantity and quality. A quantity is something that has number, scale. It is pretty easy to measure (it is something that is measurable).


I have been thinking about quantity and quality. A quantity is something that has number, scale. It is pretty easy to measure (it is something that is measurable). It is delightfully empirical.

17

Quantity, for better or worse, has underwritten most assessment for most of my peers. Essays are measured by length, either words or pages, not by ideas or problems. Teaching is a series of weeks, not topics (the weeks come first). When was the last time you heard a student ask about comments on their work, rather than what mark they received? Because of this assessment is, first of all, an economic transaction. For us and for them. (“It’s only worth 10%”.) The best students do the maths, the worse don’t.

numbers are how we measure, so when I teach students how to define their own assessment criteria I insist they be empirical, because otherwise they don’t have any way to measure their achievement. Later, with experience, they can do different.


I have been thinking about quantity and quality. A quantity is something that has number, scale. It is pretty easy to measure (it is something that is measurable). It is delightfully empirical. A quality is an attribute that is independent of quantity, of scale.


I have been thinking about quantity and quality. A quantity is something that has number, scale. It is pretty easy to measure (it is something that is measurable). It is delightfully empirical. A quality is an attribute that is independent of quantity, of scale. How many kids I have is a quantity. How I feel about them is a quality.


I have been thinking about quantity and quality. A quantity is something that has number, scale. It is pretty easy to measure (it is something that is measurable). It is delightfully empirical. A quality is an attribute that is independent of quantity, of scale. How many kids I have is a quantity. How I feel about them is a quality. This is independent of quantity. (Though a change in quantity can cause qualitative change.)

18

A quality is a quality independent of how much of it there is. The redness of a red is that red regardless of much of it I have. You measure quality by things like intensity, and to turn quality into a quantity, into something that can be measured, is hard to do.

I think university learning needs to move from a quantitative to a qualitative model, though this is different to the quality assurance models that we have.

For me learning should provide a qualitative change in understanding for those involved. This is not the same thing as knowing more about something. It is knowing something differently. Often it might be something small, but its consequences are large (it is an ecology).

I also think our teaching models, which in spite of everything remain largely content driven, are a consequence of our assessment regimes. Not the other way around. I think if you were given permission to assess qualitative outcomes, real qualitative ones, then what and how we teach would change dramatically, and quickly.


19

I did mention on the previous slide that quantity can lead to qualitative change. This graph shows an example of this. It is something I ask my honours students to do at the end of their honours year. The horizontal line is neutral, above it means something for the better, below something for the worse. In this one the green line is their knowledge about their topic. Purple the course their research has taken in relation to where they thought it would go. The pink their understanding of what research is as a practice. What I look for is a steep line up, somewhere. When that is there then there’s been a qualitative change in their understanding. For this student this happened partly around their project (the purple line, it was an experimental audio drama) but most strongly around their understanding of research as a practice, the pink with two big shifts through the year. This is a case of where quantity leads to qualitative change as the scale of honours, the number of words required, helps a shift in understanding.

“How do you know you are assessing what your students have learnt?” From here it was but a skip and a jump to turnitin (.com). It is an untenable and irrelevant arms race that has nothing to do with learning, but a lot to do with the acculturation of students into the habitus of the university (as if they care).


20

This is an example of what I’m talking about. Here quantity is all dressed up as a qualitative outcome. Bah, hooey.

Here the “quality” learnt was that over 3 years the students learnt how to cite and write more like an academic. Why do we think students are at university to learn how to be academics?. These students were in a discipline related to medical imaging and diagnosis. When they are looking at my MRI scan I don’t give a toss if they can write like mini-me academics, I want to know that they have outstanding diagnostic skills. As an educator I see little connection between one and the other. This makes one of the most common mistakes in university education where we mistake the values that matter for us as academics - that is our professional values - with those that matter educationally. This is academic narcissism. Being able to write, academically, matters to me much more, than my students. And I imagine not to most medical imaging professionals in their daily practice.

“How do you know you are assessing what your students have learnt?” From here it was but a skip and a jump to turnitin (.com). It is an untenable and irrelevant arms race that has nothing to do with learning, but a lot to do with the acculturation of students into the habitus of the university (as if they care). Evidence of copy and paste does not tell me what has been learnt.


21

My question at the end was simply how do you know the students don’t know the copy and paste material? He said he didn’t. His work is celebrated in the university as an exemplary model of technological enabled learning.

I think there are simple category errors happening that confuse quantity with quality. More does not equal better. Whether this is oversight, compliance, auditing, assessment, access, contact. Doing more (quantity), in universities, is not a solution. It is doing differently (quality). My problem is that if a university has defined itself via scarcity, what is my job, now?


22

From my point of view he has built a complex academic compliance machine that rewards behaving like ourselves. Cite often, cite well, cite properly, and we reward that. I think it is a significant effort of time to simply acculturate students to our own values, which we then parade as good teaching.

This is an example of using a quantitative tool (turnitin.com) to achieve what are mistakenly thought to be qualitative outcomes. The measurement was the decline of unattributed citation, this is just about metrics. This is to be socialised into a normative set of standards that matter to academics, but not necessarily to these students. What I am railing against is the way the academy normalises this sort of coercive power as pedagogy and legitimate. It is the stuff of ideology.

I think there are simple category errors happening that confuse quantity with quality. More does not equal better. Whether this is oversight, compliance, auditing, assessment, access, contact. Doing more (quantity), in universities, is not a solution. It is doing differently (quality). My problem is that if a university has defined itself via scarcity, what is my job, now? And what is ‘quality’?


PARADIGM CHANGE?

paradigm change

23

This is the Lytro camera. Will be available next year. Rather than focussing light onto a plane it captures the entire light field. This means after the photo is taken you can vary the focus of the image. Forever, for the photographer and the viewer. This is a paradigm shift, the sort that is possible when the computer is used not only as a channel of delivery but as a fundamental actor in the doing.

I am the first or second person in the world to have started videoblogging.

paradigm change

I am the first or second person in the world to have started videoblogging. However, my idea for videoblogging is quite different to what it has become.

paradigm change

I am the first or second person in the world to have started videoblogging. However, my idea for videoblogging is quite different to what it has become. I still think video for the internet age will happen.

paradigm change

I am the first or second person in the world to have started videoblogging. However, my idea for videoblogging is quite different to what it has become. I still think video for the internet age will happen. Like blogs did for writing.

paradigm change

I am the first or second person in the world to have started videoblogging. However, my idea for videoblogging is quite different to what it has become. I still think video for the internet age will happen. Like blogs did for writing. With blogs we have trackback, blogrolls, pings, blogs and blog posts. It is a network sensible thing made up of whole fragments. It is porous and granular.

paradigm change

24

We now take for granted what a blog is, but it is worth remembering how large a change they represent for writing practice. I remain fascinated with their cinematic qualities, which revolves around how they are whole fragments. A post is always whole, like a shot. But like a film a blog is made up of a serially arranged collection of posts, if you don’t have posts you don’t really have a blog. The key thing that blogs formalised is an architecture that lets these basic units remain addressable, singular and whole after becoming part of something larger. this is a qualitative change for writing.

I often meet staff who tell me blogs don’t work with their students.

paradigm change

I often meet staff who tell me blogs don’t work with their students. Students complain about having to do yet more blogging. They keep multiple blogs for different subjects. They are told what to write and assessed on just these.

paradigm change

These are all bad models of blogging in university.

I often meet staff who tell me blogs don’t work with their students. Students complain about having to do yet more blogging. They keep multiple blogs for different subjects. They are told what to write and assessed on just these. The same can be said about the essay, but no one, students or staff, ever ask why another essay has to be written.

paradigm change

25

The essay is part of the ideology of the institution, hence it is unquestioned. As is the tute paper. The exam. Vary from these and the students are as likely to get ansty as other staff. It is not because they are conservative, it is because they have been so squashed by everything to have simply gotten to university in the first place.

Remember it is about difference, about not repeating the same.

I often meet staff who tell me blogs don’t work with their students. Students complain about having to do yet more blogging. They keep multiple blogs for different subjects. They are told what to write and assessed on just these. The same can be said about the essay, but no one, students or staff, ever ask why another essay has to be written. If you don’t blog how do you imagine you could teach with a blog?

paradigm change

I often meet staff who tell me blogs don’t work with their students. Students complain about having to do yet more blogging. They keep multiple blogs for different subjects. They are told what to write and assessed on just these. The same can be said about the essay, but no one, students or staff, ever ask why. If you don’t blog how do you imagine you could teach with a blog? It is a verb and a noun.

paradigm change

Blogging requires a change in practice. This is a qualitative change. You can get there via quantity, but numbers by themselves won’t do it.

paradigm change

paradigm change

26

With a practice such as blogging there is resistance from students as it is simply different to the forms of academic practice and assessment that they have been conditioned to. It is just much messier than what they have learnt counts as learning. However, much like the graphs from the honours students there is always a tipping point, and once you get over that tipping point blogging more or less takes care of itself. To get over that tipping point requires coercion, or seduction, and the strongest forms of coercion and seduction you have as a teacher is assessment. So you need to spend a lot of your assessment budget on these things at the beginning, and later you more or less get this budget back again to spend on other things. Self assessment works the same way, too. As does the use of other, alternative protocols and technologies.

AFFECTIVE PEDAGOGY

affective pedagogy

If scarcity is no longer the reason to go to university, what is? What is the ecology of the post industrial?What do they learn from me if not my expertise?What is this idea of ‘quality’?

affective pedagogy

University today should provide an experience that changes how I understand what I understand. To learn how to ‘be’ a learner in my discipline. To learn a vocabulary of tacit knowledge that is ready to hand. Quality is not quality assurance (which is an audit culture’s effort to make the quantitative appear qualitative). It is a qualitative change in understanding, a difference that makes a difference in how something is known.

27

This comes out of my interest and appropriate of design research and education where tacit knowledge, knowing how, is recognised as a quality, as hard to teach, hard to learn, hard to assess. But it is also recognised that it is tacit knowledge that provides the schema that let you deal with difference, with wicked problems, with change, with the way things always talk back. To learning how to listen for and have a conversation with this ‘back talk’, the way your material, your stuff and ideas always push back. Learning how to ‘be’ I take from a recent comment by Stephen Downes about the point of getting a university education. This is not about content but a mode of engagement with learning and disciplinary knowledge that helps you become a member of a tribe, with the proviso that you also recognise the problems that this has.

affective pedagogy

This qualitative change is what I think Meyer calls a threshold concept and can include Perkins' notion of troublesome knowledge.

With information so near to hand knowledge, in a university, should always trouble.

affective pedagogy

A threshold concept is:

transformative

probably irreversible

integrative

potentially troublesome

How? We shift towards other, differing, new making — how we make (social, shared, public, personal, aggregative), what we make (mixed media, logical forms outside of the teleological, writing with, not to, media), and who manages the making (students and teachers as collaborators).

A qualitative change in the forms used to express knowledge, in what counts as knowledge, and how it is conducted.

affective pedagogy

28

So this is simply a call for learning studios, kindergartens if you like, with an emphasis on making, but in a tertiary context. A making with a meta level of reflective and critical engagement that is modelled, expected, performed, assessed, and scaffolded all the way through. That measures and rewards the sorts of change in learning that should be rewarded, rather than just knowing a lot.

paradigm change

29

As I finished this I thought there really isn’t much I’ve said. So I took a breath and remembered that in universities most of what I’ve said is radical and met with blank stares of misunderstanding and miscomprehension. Even down right hostility. For instance my students assess their own participation in a subject. They do this extraordinarily well, with rigour and responsibility. I have a protocol about how I do this, which I share with others. Most refuse to believe that it works. It does. Universities no longer can rely on privilege (which is just another form of scarcity) but once you take away all the ways we think we matter, educationally, I think there is not a lot left. The emperor, if you like, is not wearing any clothes. It is for us to step up here because we all know learning does matter. It is not only that governments don’t get it, neither do many of our administrators, our peers, or our students. So think about one thing, one small thing, that could be achieve a qualitative change for your students, and do it. Don’t ask permission, ask forgiveness.

thank you

Adrian Mileshttp://vogmae.net.au

RMIT University

Quantity and Quality in University Teaching

Education

Transcript of Quantity and Quality in University Teaching