Skip to main content
Loading Explore guides…
Classroom Setup & Motivation · Acedra
Home
Explore
Other
Classroom Setup & Motivation
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
172 segments
Copy
Hide video
0:00
The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free. To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu JANET RANKIN: All right, great. I looked over some of the Mud cards we had from last time. And there were some really good questions, some really good points, and one lingering one
0:33
from the previous time that I forgot to address. Someone said, could we please just move the desks into an arrangement where we could see each other. It was a fabulous suggestion. Today we can't do it. But I'm going to try to do it in the future whenever the logistics allow, because I think you're absolutely right. A little bit more of a U, or a circle, would be great for this class. So thank you for that suggestion. Another one was that we didn't get to motivation theory.
1:02
And I think I mentioned this last time, the idea that there's a huge amount of stuff that's been written and published on motivation and theories of motivation. And I'd be more than happy to discuss this as the semester goes on. And it might be a great topic for one of those lunches that somebody was going to organize, one of those informal lunches that someone was going to organize.
1:29
Or maybe we can carve out some space at the end of the semester and pop it in, take something out and pop it in, because it's really fascinating. It's really interesting, but it doesn't quite fit anywhere in particular. So thank you for bringing that up. And keep bugging me about it. The other thing is someone asked if all those learning theories that we discussed were equally valid. And yes, they are equally valid. I mean, there are situations where being behaviorist could help.
2:02
There are situations where being constructivist could help. It depends on what you're trying to have students learn. So you want to make sure that you tailor the way that you're teaching to support what it is you want students to learn. And we'll get into that in a lot more detail today when we talk about developing learning outcomes, and then also when we talk about tailoring your instructional activities to support those learning outcomes. So this class, this particular class today
2:33
is a balance between developing course design, developing intended learning outcomes, which for me is at the heart of it. If you can come up with some good intended learning outcomes, you basically have designed your course. And I'll show you how. I mean, you can't just ignore it, but that's the hard work, and then the act of writing constructing a syllabus. So once you've got those learning outcomes, you have an incredible anchor for the rest of your course.
3:02
And there was just a very timely haiku. I put it on the wiki. But there's this Twitter feed, academic haikus, which if you want to waste any time and read them, it's a way to waste time. But there was one here, "that question you have? I know where the answer is-- on the syllabus. So it was, like I said, it came out yesterday. I thought it was pretty timely. And it's true. If you write a good syllabus, it should really clear up a lot of, not the content, not the understanding level issues
3:34
students have, but the issues with respect to when's the test, do we have to know this, how will this be graded, what's important? All those things should be in your syllabus. So that's why that's there. So here's a little slide, a graph that I saw a few years ago, and I thought it's a really good way to start this discussion. So on the y-axis we have sort of an assessment, a proclamation of students understanding. They got it, they didn't get it. And I think the word in French is [FRENCH].
4:11
So for some reason I find it very cool to say [FRENCH]. Got it. I don't know. So whether they get it, like whether after the tests, you've given a test, and you go, man, that student really got it. That student really gets this material, and whether they passed the exam or failed the exam. So the size of the circle is more or less the number of people that fall into those quadrants.
4:41
So anybody surprised by this plot?
4:51
Yes Rachel. AUDIENCE: [INAUDIBLE] JANET RANKIN: OK. All right. Well, there's the segue. So we want, we don't want the green, but we more or less expect the green, right? If you get it, you pass the exam. If you pass the exam, you get it. That's that, and we like to have lots of people have that happen too. And then we also acknowledge that somebody isn't going to get it, and they're not and they're not going to pass the exam. And that kind of maybe makes us sad, but it's not surprising.
5:24
And then anybody can kind of have a bad day or kind of a fluke. So sometimes, sure, somebody will fail the exam, and actually really understand the material. Generally speaking, that doesn't happen, and then there's often a reason for it. But it's that red circle that Rachel mentioned that is troublesome, at the least, troublesome. So why might that happen?
5:55
Why so many red circles? Yeah Dave. AUDIENCE: Because the test was set up for basic memorization, or you could figure out with a lucky guess with multiple choice. So you could easily pass just with good test-taking skills or a good memory. JANET RANKIN: Excellent. So you pass with sort of good test taking skills. Now I want us to hang on to that. There's an underlying assumption in Dave's comment. So I'd like to hang on to that thought, but let's hear a few other conjectures and see.
6:19
Rachel, why do you think you might have passed it? AUDIENCE: So often I feel like TAs are either really good or really terrible when it comes to the exam reviews, and they're mainly good. They often told us too much of what was going to be on the exam. And so then you kind of just knew what the test was going to be if you could well on it if you [INAUDIBLE]. JANET RANKIN: OK. That's interesting. Yes Gordon. AUDIENCE: Yeah, it's possible that [INAUDIBLE] tests [INAUDIBLE].
6:50
When it comes to true life situations and something that [INAUDIBLE] knowledge that was gained from that material, then [INAUDIBLE].
7:03
JANET RANKIN: OK. So Gordon is sort of kind of flirting with this idea of you as the instructor, this real life knowledge, this is what it means. They have to have this practical working knowledge for us as the instructors to say they got it, right? But the exam isn't measuring it. And that's exactly what, that's what Dave was saying as well. This idea that the exam is measuring one thing, but it's not really measuring whether they got it. When we go home and think what does
7:38
it mean to be a good material scientist, or what does it mean to be a good chemist, we don't think, oh, it means that you can define what a mitochondria is, or you can balance a simple equation.
7:52
And yet, we put that on the exam. It's also a bit of the idea that students are good at taking tests. They're good at gaming systems. They're good at listening to TAs or TAs may have not given out appropriate information. So it comes from both sides. It comes from the students and it comes from the professors. But you have control over this. You shouldn't be writing an exam that people can get a good grade on, but then when you go home and say, oh, they got a good grade on this test,
8:26
but they didn't get it. You need to define what it means to get it in your field, to get this subject, whatever it is that the test is about. And then you need to write a test that measures that. And too many times we don't do it. We're kind of rushed, or we don't really know. We haven't really thought about how to measure whether somebody gets it. So the first step in all this is to define what it is that means that they got it. And that's the first step, and that's the hardest step.
9:03
But once that's done, the rest is easier, and it will save you from this. OK? OK. So that's the topic of today's class. And if you needed a little more motivation, I'd like you to read this paragraph, and I'd like you to tell me what it's talking about. But I'm just going to give you about 2 and 1/2 minutes to read it. And then we'll discuss what it means. What do you think this is talking about?
9:41
Yes David. AUDIENCE: Yeah, I think it stresses about how we can arrange by say, maybe schedules or activities that put them into groups. Then maybe if we form a group, how we plan [INAUDIBLE].
10:04
JANET RANKIN: OK. So arranging schedules or activities. OK.
10:24
OK. Something else? AUDIENCE: When you're planning, how to go about teaching a module, [INAUDIBLE].
10:42
JANET RANKIN: Planning teaching. What else? Anything? AUDIENCE: It's like a mailroom operation or something. JANET RANKIN: Mailroom.
10:58
Something from the back, maybe? You know you're smiling. AUDIENCE: I couldn't think of anything. JANET RANKIN: All right. Well, then don't say anything. AUDIENCE: The first is the course content, the course outline, and what you are going to teach the students, [INAUDIBLE].
11:21
JANET RANKIN: OK. So students, grouping students-- Adam right? Adam. OK. So content, arranging content. Anybody in the back?
11:43
Flyer?
11:47
Yes Gordon. AUDIENCE: I think you talked about sorting materials. JANET RANKIN: Sorting materials. So we'll go with maybe sorting.
11:59
So if I tell you-- so these are all great guesses. They're as good as what I'm going to tell you the answer is. So this was written to describe the process of doing your laundry.
12:16
Like doing the wash. You take the clothes and you put, maybe you put the dark clothes in one bin and the white clothes in another bin. And then you fold them and you put them away. It's impossible. This happens almost every time I show this little piece of writing. People don't generally get it. They get the fact that it has to do with sorting, and some planning, some arranging things, but they don't get that it's about the laundry. If you skim it-- you don't have to totally reread it--
12:48
but if you skim it with the idea that it's about doing the laundry, you probably can understand it better, maybe. So now I'm going to ask why did I show you this now, today, before the third, the beginning of the third class? Yes. AUDIENCE: I think it's sort of about arrangement of course content [INAUDIBLE]. JANET RANKIN: So it could be about how you arrange the content in the syllabus. [? Hina. ?] AUDIENCE: I think it's easy to describe methodology as a person more in a different field.
13:34
But if there's no context for the audience about what this is, it doesn't matter that you've laid out everything. No one knows [INAUDIBLE]. JANET RANKIN: Right. It's very detailed at a certain level, but it's still really hard to understand, right? Any other ideas? Yes. AUDIENCE: I think it's interesting. There's a lot of value judgements in it. Like, it's really easy. And one can never tell with comments like that. Because I'm trained to figure out what they're saying,
14:05
it's actually just frustrating me more, because I don't know what they're talking about. And it's really easy-- JANET RANKIN: All right. So that's a great point. So we're talking about writing learning outcomes, about organizing the course around writing learning outcomes. If you're using phrases, or you're using language that's just, there's just like throwaway words in there or value judgments, it's not doing your students any good, right?
14:33
And also, that if you have this beautiful course that's beautifully organized even, wonderfully organized, but if you don't tell them what it is you're trying to do with it all-- what it is they're going to be able to do with it all, whatever it is you expect them to do-- they're not going to be able to follow you. So imagine a procedure as fairly straightforward as doing the laundry. Now imagine, OK, you're going to be able to select the right, the appropriate method
15:05
to integrate equations, so methods of integration, by separation, by card or substitution or whatever. So if you don't tell them what it is you're expecting them to do, even if you're very good at explaining what it is they're going to do, they won't get it. So this is my sell for why when you write a learning outcome, you have to write a learning outcome that's clear and that's understandable to the students, that doesn't have a lot of extra words and that tells them
15:42
exactly what they're supposed to do.
15:48
So I have some learning outcomes for today and I hope that they're clear. You'll be able to describe the components of the constructive alignment and backward design processes. And those, constructive alignment anyway, was in the pre-class reading. You'll be able to classify the content of a course that you might teach. So what do I expect students to do with this content, to be able to do with this content? You'll be able to create measurable, specific, and
16:22
realistic learning outcomes for a class you will teach, and you'll be able to state the components of a syllabus. So I feel that those are pretty understandable. You may disagree with me, or maybe you want to wait.
16:43
So constructive alignment, it was in the pre-class reading, right? Anybody want to just give a brief summary of it?
16:54
I mean it's kind of up here, but in your own-- AUDIENCE: I think constructive alignment is about aligning assessment, method of assessment, with course content. [INAUDIBLE] assess [INAUDIBLE] type of assessment so that what the teacher has in mind, the course content or the content I want them to get, to test it with the assessment process. Then we would also do the assessment, and then come up with the intended outcome. JANET RANKIN: Right. They'll achieve the learning outcomes.
17:34
And if you do this right, you get rid of that red circle on that graph. You get rid of students who did well on the assessment but don't really know what's going on, because you have aligned the assessment with what it was you wanted them to know or be able to do. So that's important. I just think this is a very, very compelling image to keep in your head, that these three things have to be linked. And as I said before, this, you will see that the top part, writing the learning outcomes
18:08
is the tricky part, it's the hard part. It's the part that requires a lot of thinking and work. When you get a good learning outcome, the other two things almost take care of themselves. OK? Did anybody have any, does anybody have any questions about something from the reading associated with constructive alignment or some other idea from the reading? Yes Gordon. AUDIENCE: I just wanted to make a contribution. I think that this constructive alignment is one of the things
18:42
that the intended outcome should measure. So I think maybe that's confusing [INAUDIBLE] and also how we can test and be sure and confident. JANET RANKIN: Right. Right. I think the measurability part for scientists and engineers is kind of intuitive. Like why would you say something if you couldn't tell whether or not it was true, or not from a scientific point of view. What's the point of saying, making a proclamation about what somebody should be able to do if you can't measure it in the end?
19:31
So it's comforting to say, OK, I'm going to make learning outcomes that are measurable. But it also can be a bit challenging. But we'll do that. We're going to do that today. Some other comment? Nina. AUDIENCE: I guess as far as [INAUDIBLE] Do people write learning outcomes that have no constructive alignment process, like choosing to write learning outcomes, are you going through the prcoess? It seems decoupled a little bit in the reading, but it seems like you can't have one without the other.
20:06
JANET RANKIN: You owe it to yourself-- and I don't know what people do, and I guarantee you people write learning outcomes without thinking about whether they can assess them or measure them, I guarantee that happens in real life-- you owe it to yourself to try very hard not to do that. OK? And I think the important thing to do is, often we say, oh, my god, this is the best assignment. This is the greatest question. It's this project, and they're going to do this, and they're going to whatever.
20:34
They're going to build a race car. And if you sit down and say, well, what do I want them to get out of that? If you can't articulate it in a way that's consistent with your class, consistent with the course as it sits in this series of their courses, or as it sits in the institution, then you either need to rethink the assignment, rethink the project, or think a lot harder about what learning outcomes that activity advances. That's a very good point. It's really kind of iterative.
21:08
And I think it might be easier to see with this diagram. This is another-- so John Biggs is from the UK. And so there's sort of the UK camp of people and they like John Biggs. And then there's Wiggins and McTighe. Grant Wiggins and Jay McTighe are from the US. They wrote primarily for the K through 12 audience, but it's completely applicable. And they're from the US, so they have kind of a US following. It's just how it works. It's essentially the same thing.
21:40
It's identify the results, so the intended learning outcomes. Figure out how you're going to get them there. So the black is Wiggins and McTighe and the red is Biggs. So plan the learning experiences that are going to support them, achieving the learning outcomes, and really help you measure whether they got them or not. And then determine what evidence will be that acceptable evidence that they achieved the learning outcomes and what are the assessments.
22:07
So real often we do have this project that we want. Oh, this is a great problem. Often in thermo-- I taught a lot of thermo-- there's like, oh, they're going to do this double loop, double reactor cycle with the refrigerant and a heating cycle. And it's going to be coupled and they're going to use the output from one for the input in the other and all this crazy stuff. Like, this is a great assignment. And you kind of, I know this is a great assignment. If I can't back up from this assignment--
22:40
whether or not I put it here or here-- if I can't back up and say after a successful completion of this assignment, you will be able to describe the components of the Brayton cycle, you will be able to define the increase in entropy for a system with x boundaries, If I can't articulate those things, then I need to either change the assignment, or, well, then I need to change the assignment. OK? But you can start with the activity, and then back up and say, well, what
23:16
learning outcomes does that support, and then, am I OK with that? Is that relevant to this class? So it's often a very iterative, the arrows kind of go this way, but it's often iterative.
23:31
So I had asked that you come to class with some topics from a class you're likely to teach. That was on the assignment page. If not, start thinking now. And so you want to have, for most classes, most undergraduate classes, there's a bunch of topics. You might have 10 topics, 20 topics. And I don't care right now about the scale of the topics, whether they're really big. You probably don't want to go with like first law, second law. That's probably a little bit too big,
24:07
but on the order of 5 to 15 topics. Jot down the topics. And then I have a chart here. So the chart has two sides, and the columns are Bloom's Taxonomy, which you don't have to worry about that specifically, but you can look at the words that are associated with it. And then the second column is what students should be able to do with each topic. So let's say my topic is first law for closed systems. So I might think about, OK, do I want them to be able to define, to just state
24:51
the first law for closed systems? Do I want them to be able to use the first law for closed systems in order to solve certain kinds of equations? Do I want them to be able to derive the first law? Do I want them to be able, whatever it is. So you're going to look at the columns, the categories on the left side, and you're going to think about your topic. And you're just going to decide which box in the middle column to put the topics into. You don't have to worry about writing anything.
25:20
You're just going to distribute the topics to the level of, it's really cognitive processes that are associated with the things on the left. Remember, understand, apply, analyze, evaluate, and create. So we'll take about five minutes. And you just do that by yourself. And just fill in that middle column of the table with the topics from your course. Do you want more explanation, Rachel?
25:55
It's starting to make you think about this part of the beginning circle. But you don't want to write any sentences yet. But it's starting to help you think about what are they going to do with these topics. What do I want them to do? Is it remember, or is it create something using these things? Where is it? Those are two ends of the spectrum. And in your chart, the things that at the top are the more remember, right? It's more about kind of a recall information.
26:33
And then the things in the back of the chart tend to be more higher order cognitive processes, like synthesize, apply, create.
26:50
Yes. AUDIENCE: [INAUDIBLE] JANET RANKIN: So don't worry about the third column. Don't worry about the last column yet. We're going to do that later. So all you have to do is just say where each topic falls loosely. OK. Does everybody have a chart? OK. AUDIENCE: Do you want us to write as many topics as possible? JANET RANKIN: Write as many topics as possible, but distribute them through the chart. Right? Not everything is going to be remember, right?
27:22
So some things will be remember, some things will be apply, some things will be analyze, some things will be create. OK?
27:31
If you have any questions, just raise your hand. All right. So what we're going to do is I'm going to ask you to just keep those with you. We're going to do another activity later on using those as a starting point, but I want you to just keep that sorting with you. Any comments or questions about the experience, about what you just did?
27:58
Was it easy? No. It can sometimes be a challenge. Well, what the heck you were supposed to do with this anyway? What do I really want them to do with it? This is the beginning of that process whereby you define exactly what it is you want students to do, know or be able to do. And then you can start to think about how you're going to assess whether or not they can do it or not. But it's very different if I were to say I want everybody to be able to state the first law of thermodynamics versus I
28:29
want you to be able to use the first law of thermodynamics to define chemical, mechanical, and thermal equilibrium in a simple composite system. That's different. That's very different.
28:46
So that's what we're starting to do.
28:51
So now we have kind of the seeds. We thought about which topics kind of we want to have, what we want to do with each of the topics. But now we really have to turn them into learning outcomes, into real statements. By the end of this class, you will be able to, by the end of this course, you will be able to. And just as an aside, remember that sometimes you'll notice that for every class, every session we have, I have learning outcomes. But there's also overarching learning outcomes
29:24
for the course. So I will use them interchangeably, but they'll be at a slightly different scale. The ones for the class are a little bit more fine grained. As was mentioned in the reading, and as Gordon mentioned, they have to be specific, measurable, and realistic. If they don't have those properties they're not going to be particularly useful, or as useful. All right. And we'll see some examples of that in just a sec. So here's what they're not. They're not topics. So you came with topics.
30:06
You will leave today with learning outcomes. But topics are not learning outcomes. The first law of thermodynamics is not a learning outcome. You need a verb with it. What do you want me to do, what do you want your students to do with the first law of thermodynamics? What do you want them to know about the first law of thermodynamics? So they're not topics. They're not things that you will do.
30:37
It's easy to say, I could have a list a mile long. I will talk about the first law of thermodynamics. I can check that off pretty easily. I mean I just did. I just talked about the first law of thermodynamics. It says nothing about what you learned about it, what your students learned about it. So it's not about you. Get over it. That was a joke.
31:06
It's not value statements. As Adrian pointed out, some of these think this is easy, this is good, this is useful. Any of those things don't belong in a learning outcome.
31:22
And it's not your hopes and dreams for your students. I mean, ultimately, it is. You hope that they will be able to use the first law of thermodynamics, da, da, da. But it's really about what they're doing, about what they're going to do. So no. Here's a little example.
31:43
This is just from like an earth and planetary course. Students will understand plate tectonics. How's that?
31:54
Is that OK for a learning outcome? Why not? AUDIENCE: It's too broad. JANET RANKIN: It's not very specific. What the heck about plate tectonics? What else?
32:11
AUDIENCE: Measurable. JANET RANKIN: Thank you. OK, the measurable part. This word understand, strike it from your brains. It's a property that we love. We love understand. But you can't flippin' measure it. You can't go in there and measure it. You can measure lots of things that tell you whether students understand, but you can't measure understanding.
32:42
And I think we'll see that with some more examples. I don't think anybody is earth and planetary here.
32:49
So how about this one? Students will be able to interpret unfamiliar tectonic settings based on information on volcanic activity and seismicity. Again, I don't think any of us are geologists. But the idea that I've told, or the person has told us what we're going to use to interpret these situations, and then we're going to make predictions based on those interpretations, I mean, that's an assumption. So it's specific and now it's measurable. Even though we're not geologists,
33:22
I suspect we can think of how we could write a test question that would ask students to do this. And then we could see whether they did it or not. So that's measurable. And probably, by doing this, by having students do this, we get a pretty good measurement about whether they understand plate tectonics. But it's much more specific and it's much more measurable.
33:58
From here, this is from the University of Minnesota, which has a nice website on learning outcomes. They're not from your point of view, as we mentioned. They're not what you're going to do. I'm going to introduce students to the major turning points and processes in North American history. You can say anything you want, but it doesn't mean the students learned anything. So it's not about you. It's what they're going to do. They're going to list and describe
34:21
the turning points and processes in North American history.
34:26
I'm going to create an understanding of the formal constructs of physical design. That one is like, this one is, it's like, I don't even know what it means, right? Create an understanding of the formal constructs of physical design. I don't even know what it means. And so it's like that laundry example. Your students aren't going to know what it means, and it's not going to help them. It's not going to help them access the material. So the ones on the left are all teacher-centered
34:56
and they're not very useful. The ones on the right are student-centered and they are generally useful. But there's some things wrong with the things on the right. AUDIENCE: Understand. JANET RANKIN: Understand, yes. So what would you write instead of understand the formal constructs of physical design, whatever the heck that means? What might you use? AUDIENCE: Identify. JANET RANKIN: Identify. OK. That might be like a list, right, or identify or list, pick it out of the lineup kind of thing.
35:34
This is a nice one. Explain to an intelligent non-expert, an INE. And so explain this to somebody who's not in the field. If you could do that, it probably means that you get it. Somebody I know used to say, explain this to my four-year-old brother. Explain-- what was it-- angular momentum, to my four-year-old brother. I would argue if you can explain it to a four-year-old kid, you probably understand it. So you can use that.
36:13
Use formal constructs to design something. I mean, that's kind of an inversion. How about the one on the bottom. Understand gender, race, ethnicity, socioeconomic class, understand how they have shaped Americans' lives. Completely unmeasurable, right? So what would you write instead?
36:41
AUDIENCE: Describe. JANET RANKIN: Describe how, sure.
36:49
Anything else? AUDIENCE: [INAUDIBLE] it seems like you're trying to get [INAUDIBLE]. JANET RANKIN: Right. So that's a great point, is that we don't know where this faculty member put this on their table. We don't know whether it really just states some ways that, list some ways that these things have affected Americans, North Americans' lives, or Americans' lives-- which is kind of a remember thing, just make a list-- or whether it's do some analysis on some situation.
37:22
We don't know that, but you as the instructors get to decide that for your classes. So that's an important thing. You get to call all the shots. You get to make the decision. We can only guess about what this person meant, and therefore, his or her students can only guess about what he or she means.
37:46
So you have on your table this hierarchy of Bloom's Taxonomy. It's often shown in this triangle, this pyramid. There's a handout which has exactly the same strata on it. And it just has some handy dandy verbs associated with it. But Benjamin Bloom came up with this classification of cognitive processes way back when, in the 1950s. And it is this idea that it is this pyramid, that at the base of the pyramid are these ideas like remember. And those are basic cognitive processes, remember, list.
38:27
And then it moves up the pyramid to things that require a higher level of cognitive processing. Rather than just pulling facts out of storage and stating them, you're going to do something with those facts. So as you go higher and higher up the pyramid, note that he chose-- really unfortunately-- to call the second level understand. However, there are words that can help you. If it really is understand that you want to measure, there's words that describe that a little bit better.
39:03
They're in your table, and they're also in that second table that I handed out. So arrange, list, label. For understand-- describe, relate, recognize, explain, those things. Gordon. AUDIENCE: Just a little [INAUDIBLE]. When it comes to understand, maybe if you put it like [INAUDIBLE]. JANET RANKIN: Understand to what? AUDIENCE: Understand to an [INAUDIBLE]. Maybe that's what we can measure. Understand to this level or to this [INAUDIBLE].
39:33
JANET RANKIN: But how, can you give me a specific example? AUDIENCE: I wish I could go back to the last slide [INAUDIBLE] can describe [INAUDIBLE].
39:59
JANET RANKIN: OK. So I see where you're going. Does someone have a suggestion? So think about that laundry. Think about that description of doing a laundry.
40:10
Can somebody take what Gordon said and get where he wants to go, but get there in perhaps a more direct way.
40:25
Katherine. AUDIENCE: [INAUDIBLE] JANET RANKIN: Exactly. So if you're going to do that, and I see where you're coming from, understand so that you can describe, in order to describe, or to the level of describe. Just forget the understand part, and just say describe, or define, or whatever it is you want. But yes, I mean, that first step is saying, what do I mean by understand? And when you say that, then it generally rewrites itself. And so the more direct you can be, generally, the better.
41:01
And keep that laundry example in your head. Yes. AUDIENCE: You're asking a student [INAUDIBLE]? JANET RANKIN: So I would say, well, let's just open that up. What do you think about that, of asking students? So do you understand this stuff about learning outcomes? AUDIENCE: I think as teachers, when you, in the front of the class, you're teaching students and you say do you understand, they say yes. But you see from psychology you know who doesn't understand.
41:34
JANET RANKIN: So David says as I ask you that, do you understand? Most people are going to say, hey, I understand, sure. Don't think I'm an idiot. I understand. Right? So I completely agree. Students are not likely to say they don't understand. What else? AUDIENCE: That's difficult for the teacher to [INAUDIBLE] JANET RANKIN: Well, so do you want to say something about that? AUDIENCE: Yeah, I agree. I think the part that we do understand as nonspecific is the learning understanding [INAUDIBLE].
42:30
As an instructor you have to know, understand what you're trying to get at, which I think is where the clicker question is beautiful, because you know already what you're trying to evaluate that they're catching in class in real time, if you can see who's getting at the nuance of what you're teaching. JANET RANKIN: Yes. I think that's an excellent, excellent point, that many of these activities that we do, these small group activities, or the clicker questions or other activities that students
43:04
are doing in class are helping them learn. But they're also formative assessments of whether or not they get it or not, whether or not they understand. And you as the instructor can see it. If I walk around, if I listen, if I look at who answered what question on the clickers, or on the raising your hand, multiple choice questions, whatever methods I used here, I get a better, a much better sense of who understands or doesn't understand. If all I'm doing is lecturing I can't tell.
43:38
And I can ask do you understand, but they're not likely to tell me. Which is kind of the other point is that sometimes-- and I think you were alluding to this, Gordon-- that sometimes students don't know whether they understand or not. So we need to give them the opportunities to find out whether they understand or not. And arguably, if you're just lecturing and you're a great lecturer-- you're eloquent, you don't stumble, the writing on the board is awesome, all this stuff--
44:11
students will think they understand maybe when they don't, because they're not having to confront any misunderstanding, and it all seems very nice. So these activities where you actually have students do something, say something, where you ask them a specific question, that's when you can determine whether or not they understand or not.
44:36
And because they are not likely to say they don't understand, either because they just don't want to admit it, or because they really don't know whether they don't understand. So you need to give them measurements to find out whether they get it or not.
44:52
Yes. AUDIENCE: Sometimes, for example lecture and the activity has [INAUDIBLE]. For example, we have lectures and the teacher just [INAUDIBLE] and then we have expectations as we do exercises. So do you recommend that we also do some activities during the lecture or just [INAUDIBLE]? JANET RANKIN: No, do them during lecture. I mean, the model that we're trying to do here is the model that you should do even if you have a hundred or 200 people in your class. You can use clickers.
45:25
You can use pair share discussions. You can do all sorts of things to break it up, and it's effective. There's a paper coming up for next week, a meta-analysis, active learning strategies, how they actually, students learn more in a lecture if you actually ask them to talk to each other, break up, share, et cetera. And I think when we talked about misconceptions, remember that example with the coin? How many students got it, could draw the free body diagram before and after?
45:57
I mean, before and after in a straight lecture, there's hardly any learning gain. But if you incorporate active learning, the learning gains go way up.
46:25
So that's Bloom's Taxonomy. And an interesting take on Bloom's Taxonomy, there's a woman, Kathy Schrock, who-- it's a nice website and the slides we posted so you have the link-- but she decided, she has this theory that it's not really so pyramidal. That everything really is dependent, these cognitive processes are dependent on each other at a certain level. And so she's drawn it as a gear, where everything sort of supports the act of creating, which is up at the top.
46:51
But the other ones are sort of around the side, but they all contribute to the ultimate cognitive process of creating. So that might be a nice way to think about it. And then she's taken the specific words, like analyzing, and said, OK, what's an instantiation of analyzing, outlining, deconstructing, organizing, structuring, surveying whatever? So it's the same idea, it's just a different graphical representation.
47:27
So what you've done is you've taken your topics, put them in those cells in the table. And I just want to point out kind of how as the instructor you have a lot of power about where you're putting those, where you've decided to put those. And it may be different for a first year undergraduate class, or a second or third year graduate course. It could be the same topic, but you're going to put them in a different category. So that's why when I asked you to bring a topic,
47:56
it's for a specific class. It's for a specific group of students. So let's say we take the idea of interstitial sites. So I have a crystalline structure, it's got an order of periodicity of atoms. And then I have spaces between the atoms, those are the interstitial sites. And so I could say you'll be able to identify the interstitial sites. So I could say where's the tetrahedral site in a face-centered cubic or something, whatever? And students just really have to find it.
48:27
I could say calculate the maximum size of an interstitial atom that could occupy that space, an ion that could occupy that space in a particular crystal structure. So those are two very different levels, but it's the same topic. X-ray diffraction, I could say apply Bragg's Law to calculate d-spacing. So that's very much a turn the crank kind of thing. I know the equation for Bragg's Law, I stick things in, I solve for the unknown.
49:00
I could then also ask you to index unknown diffraction patterns, meaning identify an unknown crystal structure based on the diffraction pattern. That's a completely different activity, completely different set of cognitive processes than just using Bragg's Law. But you have to decide what it is. And arguably, if all you've had students do during the class is sort of problems that they applied Bragg's Law and they solved for d-spacing-- so they solved for the incident angle
49:29
or they solved for the wavelength or whatever it is-- you really can't throw this at them on the exam, right? Or conversely, if you spent the whole time, they've indexed, they've been in the lab indexing unknown diffraction patterns, and then this is the question you ask them on the exam. Well, then you have people that perhaps get it without really, perhaps pass the exam without really getting it. So that's just something to keep in mind. The other thing to note is let's say
50:00
this is my intended learning outcome-- calculate the maximum size of the interstitial atoms in a variety of crystal structures. How do I know whether students get that or not?
50:16
What's that? AUDIENCE: It's easy. JANET RANKIN: It's very easy. I give them a crystal structure and I say calculate the maximum size of the ion that can fit in the space. So I have figured out how, the measurement is easy, it's cake. It's totally straightforward. Now I haven't told them, OK, I'm going to give you this crystal structure and I'm going to ask you this spacing, this ion size. I haven't given it away, but I have told them what's expected of them.
50:51
And my colleague wrote these for VSEPR theory. It's the same topic, and it's different. Identify the common geometric shapes found in simple molecules. Explain the assumptions of this theory. Apply the theory to predict 3D structures. Compare and contrast the geometry of a certain molecule as predicted by two different theories. Evaluate the accuracy of each theory for a particular set of compounds, and then create some recommendations. So it's the same topic, but you decide
51:27
what it is that students should be able to do. OK? So I have some, just a little exercise for us. So they should be specific, measurable, realistic. I'm going to put them up here and we're going to say, you're going to tell me what's wrong with them. So number one, t-tests are like a statistical analysis. So is anybody a t-test kind of person? We can skip that one. How about gaining appreciation for the use of linearization techniques? What's wrong with it? AUDIENCE: It's not specific.
52:08
JANET RANKIN: It's not specific. AUDIENCE: It's not measurable. JANET RANKIN: It's totally not measurable, exactly. I mean, how do I know that you appreciate it? Don't do it. All right. Great. Have an intuition for the most effective method of integration for a given problem. AUDIENCE: It's vague. It's not measurable. JANET RANKIN: So I've heard it's not specific, it's not measurable, and it's not vague, I mean, and it's vague. So right, so have an intuition, I cannot measure that.
52:46
I have not figured out a way to measure intuition. I can tell whether you can do it, right, but I can't tell you whether you have an intuition for it. Well, how would you rewrite it? Actually, how would you rewrite 2?
53:06
AUDIENCE: Use [INAUDIBLE] JANET RANKIN: So it could be use the techniques. Another suggestion? AUDIENCE: Make something using the evaluation techniques. JANET RANKIN: Calculate something. Make a first pass estimate using a particular linearization technique. AUDIENCE: Describe when you would use a linearization technique. JANET RANKIN: Describe when you would use it, describe why you would use it, all of those things. AUDIENCE: List those areas where [INAUDIBLE]
53:41
JANET RANKIN: List the areas where it's useful, yes. Great. All right. For number 3, the gain an intuition, we decided it's vague, it's not measurable.
53:54
So how would you rewrite it?
54:03
AUDIENCE: Compare. Compare the areas integration technique [INAUDIBLE] Solve a particular problem using different kind of [INAUDIBLE]. JANET RANKIN: OK. So solve a problem, and then we need a little more. Did you have something? AUDIENCE: Identify the most effective method of--
54:30
JANET RANKIN: Let's hear what Rachel had to say. Rachel. AUDIENCE: Yeah. Learn how to identify the simplest [INAUDIBLE]. JANET RANKIN: Right. So you probably don't want to say learn how. You would say be able to identify. Right? Gordon. AUDIENCE: [INAUDIBLE] JANET RANKIN: Right. I mean, you could just say integrate-- AUDIENCE: [INAUDIBLE] JANET RANKIN: Integrate a problem using whatever. Select a technique to integrate a particular equation.
55:16
And again, those things end up being more measurable. So 4, provide problem solving tools and strategies. What's wrong with that one? AUDIENCE: It's too wide. JANET RANKIN: It's too wide. AUDIENCE: It's instructor centered. JANET RANKIN: It's instructor centered. What else? AUDIENCE: But it's measurable. JANET RANKIN: Well, I can measure that I provided it. Here, I provided these handouts, right? OK. Check. It happened. Whether or not you could do anything with them,
55:47
I don't know at this point, right? That's the problem. AUDIENCE: It doesn't tell us what kind of problem and what kind of tools we're going to use. Just who gets to choose the strategies. JANET RANKIN: Right. Right. It's very, very general, totally not specific. So what could we do? I mean, we don't know what this instructor had in mind, but what could you do if this was your class? What would you say? AUDIENCE: What about develop instead of provide? JANET RANKIN: Develop.
56:13
And again, it would have to be student centered, so the student will be able to develop problem solving tools and strategies. That's for x, exactly. We want to make it a little more specific and a little more focused, because that's pretty broad. I mean, I have developed a problem solving strategy that lets me get out the door without tripping. Does that mean I satisfied your learning outcomes? Probably not. So in order to do something-- Use thermodynamics to solve engineering problems.
56:56
AUDIENCE: That's not specific as well. JANET RANKIN: It's not specific at all. AUDIENCE: Engineering is [INAUDIBLE]. JANET RANKIN: Yeah. I mean, it's crazy broad on a number of fronts, right? I mean, it's like engineering is huge. Thermodynamics is pretty darn big, too. So which part of thermodynamics, which part of engineering?
57:21
Build an SAE race car. AUDIENCE: [INAUDIBLE] JANET RANKIN: What?
57:28
AUDIENCE: It's not realistic. JANET RANKIN: Right. It's pretty specific. Build a functioning SAE race car. It's very specific, and I can totally think of the metric of how I would measure whether students could do it. I would just ask them to build the race car. But they're never going to do it in a class, in one class period, in one course. It's unrealistic, exactly. Learn to use Laplace transforms to solve differential equations. AUDIENCE: [INAUDIBLE] JANET RANKIN: It's funny.
58:02
I mean, you can imagine the instructor saying you will learn to use that. But let's go, Katherine, work your magic with this one.
58:16
AUDIENCE: [INAUDIBLE] JANET RANKIN: Right. You're my go-to person for when you want to get rid of all the extra words. I'm going to call on you. Right. Use Laplace transforms. That you can measure. You can give them an equation and see if they can use Laplace transforms to solve it. And yes, you might want to be a little more specific about the kinds of differential equations they use, and know how to upper diagonalize a matrix.
58:51
Katherine. AUDIENCE: . Upper diagonalize a matrix. JANET RANKIN: Upper diagonalize a matrix. That's all you have to say, right? Forget the extra words. And that is really, I have just written the assessment question for that. I know it's totally measurable. I give you a matrix and I ask you to upper diagonalize it. I haven't given it away. I haven't told you you will upper diagonalize this matrix, right? Memorize these equations that you're
59:22
going to need to write down in order to upper diagonalize it. But I have told you that that's what's expected of you. AUDIENCE: I have a comment on it. It's pretty useful when a student is trying to evaluate the use of the class. For example, the career fair tomorrow [INAUDIBLE]. There's a million [INAUDIBLE] to get jobs. OK, how are they going to evaluate me? And if you could come in and say, this happened and this is what we did, as opposed to having, or have them
59:51
evaluate me on some random knowledge. And then I don't know that I could respond, but I do know stuff. So this is really helpful in kind of a broader sense. JANET RANKIN: Right. Great. No, I'm glad. And I think it's so true. And people often think, well, you're giving it away. But you're not giving anything away. You're just telling them the expectations. You're not giving them the problems, you're not teaching to the test, but you're just telling them
1:00:20
what they're going to be able to do at the kind of high level, or medium level. You're telling what they're going to be able to do. I will say-- so ABET, the Accreditation Board for Engineering Teaching-- every accredited engineering program has to list, they have A through K outcomes. And if you look it up, ABET--
1:00:44
and I'm sure you see us as accredited-- if you look up, they have to, they used to do a lot more bean counting. But what they would do is the outcomes would be critical thinking and problem solving skills, and they were a little bit broad. But every department that got accredited had to show how various courses contributed to the outcomes. So somewhere in your department should be a pretty detailed list of how 6 double 00 whatever, did this. Or 6 yada, yada--
1:01:23
I know your grad students-- but did this. The problem is they don't have to do it for grad student courses. But it might be helpful if you looked at that, because I would imagine that sometimes the recruiters would be looking at that information, so that might be a commonality. But every program in the US that's accredited has to have, show at least how they've addressed those outcomes. So you might want to take a look at that. But yeah, if it was a little more explicit
1:01:50
it would certainly be better. And I'm glad it's helpful.
1:01:55
So I think we can see, you can start to see how there are better and worse learning outcomes, that if they're not measurable they're not so useful. If they are measurable, they're quite useful. So now what I'd like to do is revisit the worksheet. And what we can do is we're going to take about five minutes where you just sit quietly and write out specific intended learning outcomes. So you'll be able to upper diagonalize a matrix, you will be able to use Laplace transforms
1:02:33
to solve this type of differential equation, whatever you've written in this column. And note that if you put it in this column, let's say you put it in the apply column, then when you write the learning outcome it's going to be you will be able to interpret data using blah, blah, blah. You will be able to model the vibration, model a car suspension system using linearization techniques. They're going to be whatever, wherever you put it in the box. These are some of the verbs, these
1:03:09
are the verbs you're going to use in your learning outcome. Does that make sense?
1:03:17
And I have even more verbs for you if that's helpful. Did I pass those out? You got those, right? AUDIENCE: Yeah. JANET RANKIN: So there's more verbs. It's the same thing. I mean, it's consistent. So take five minutes by yourself and do that. And then you're going to get into pairs and you're going to discuss. You're going to trade worksheets and give each other feedback. All right? So five minutes of quiet and then we'll pair up.
1:04:03
So if you can pair up, you don't necessarily have to pair up with people that are sitting next to you. You might want to get a different perspective, so feel free to get up and move around. You can talk to somebody that wasn't in your group. So maybe if you go back and talk with Alex, maybe. We need one group of three. Oh, are you a group of three? So now just trade sheets of paper with your partner, or if you're a threesome you can figure out how to do it.
1:04:33
And then review the learning outcomes critically. Guys, just one sec. So just review your partner's learning outcomes critically. Make sure that they're measurable, realistic, and specific. If your partner has used the word understand, help him or her work through that to get a better word. So you're going to be a critical friend here. And then you guys can discuss how to improve it. And then I'm going to pass around some flip chart paper and each one of you will write, maybe
1:05:05
just write your name at the top, and write two or three learning outcomes on the flip chart paper. And then we can stick them up around the room, and then everybody can see everybody else's, all learning outcomes. [INTERPOSING VOICES]
1:05:22
JANET RANKIN: Was there a question? AUDIENCE: Oh, no. I just think-- AUDIENCE: I think [INAUDIBLE].
1:05:32
JANET RANKIN: How are you guys doing? Doing OK? [INTERPOSING VOICES]
1:05:58
JANET RANKIN: So you can use this zone if you need it.
1:06:05
As you're looking at other people's learning outcomes, think about how clear, how easy it would be, or not easy, to come up with an exam question, or a project, or an assessment or measurement of whether or not the student reached, attained the learning outcome. You can put them up here, Ina and Shau.
1:06:34
OK. And as I said, make sure that you've read everybody else's. I'm going to say a few words about them.
1:06:45
I think we may have a record. This is like the first time ever that no one has used the word understand, which I think you're all to be commended for.
1:07:00
So David, make sure you don't write understand. You build me up just to knock me down.
1:07:12
AUDIENCE: [INAUDIBLE]. JANET RANKIN: You can, or you don't have to. It doesn't matter.
1:07:18
Does anyone have any comments, either about the exercise itself, or about somebody's learning outcomes, that you see around?
1:07:33
Again, the measurement, the thing you want to look at when you look at your own learning outcomes and those of others, is are they specific, measurable, and realistic? And would you know if a student achieved those learning outcomes? So I would say about 90% of these are very specific, which is great. I mean, there's some incredibly specific ones here. The more specific, the easier it is to measure. So that's one thing. That's a useful thing. You want to make sure you don't get so specific
1:08:08
that there's only like one question you could ask. So that's the balance. Sometimes you can't tell that unless you're in the discipline. Do we have other comments about any of these learning outcomes, something you don't get, something that looks particularly interesting?
1:08:33
Yes Gordon. Perhaps you can sit down if you're done reading. AUDIENCE: I'm looking around and I'm seeing [INAUDIBLE].
1:08:52
I don't know how it's going to be. How are we going to test that? JANET RANKIN: Well, I guess you have to come up with a reaction that you're confident they haven't seen before. So if you can do that, then you can ask them to propose a mechanism for it, I suppose. If the person that wrote it wants to-- I know who wrote it-- but if the person that wrote it wants to talk a little bit about it, that would be great. If not-- AUDIENCE: OK, So a lot of people are
1:09:22
going to just try to memorize each specific outcome. But it's easier to just identify your activity group and find patterns within them. So if you can try [INAUDIBLE] They can't just [INAUDIBLE].
1:09:50
JANET RANKIN: Right. And this is a great point. I mentioned it to one of the groups. But sometimes we'll say explain how blah, blah, blah, works. And that's a great thought, that, oh, they're going to be able to explain it, right? But many times they can fall back on a memorized explanation or sort of kind of a canned procedure. So if you really want them to be at the level of explain, you will have to kind of make some things up to get them out of, to get them away
1:10:21
from the ability to just pull something completely from memory. So this is an example of how you might do that.
1:10:34
Other observations or questions? AUDIENCE: I also [INAUDIBLE] something that all [INAUDIBLE].
1:10:52
JANET RANKIN: So perhaps the person that wrote it wants to comment on it. AUDIENCE: OK. So [INAUDIBLE].
1:11:19
I think what we have, just [INAUDIBLE].
1:11:26
So we write and compile.
1:11:31
So you write a code and then you compile it. JANET RANKIN: Right.
1:11:40
And this is a great example of how having people that aren't in the field read them can help clarify things for your students, who are by default not experts. So perhaps to an expert, it totally makes sense, of course, code. But to somebody that's not an expert, maybe that's confusing. Maybe the use of code twice was confusing. To an expert, it's totally readable. It's totally understandable. Perhaps to a novice it isn't. You're laughing. AUDIENCE: It's funny, because we're both [INAUDIBLE].
1:12:14
JANET RANKIN: Yeah, that was kind of random. Well, that's a perfect example, right? It's a perfect example of how getting maybe other people that aren't in your field to look at them can help make them more accessible to your students. OK. This is great. Oh Hina. AUDIENCE: I had, it's on a different topic. But really, I'm talking about buzz words in learning outcomes. I think with engineering, you're really excited about telling people about real world problems. I guess that's the reason why.
1:12:46
I guess it's more of an opinion question. As you write your learning outcomes I think in some ways it's better to be specific about the real world problems that they're going to attack. Because I've seen that on syllabuses all the time. I'm very excited about it. I think it's a way to teach people in your class. It's going to help them know what tools they're getting. JANET RANKIN: Right. I mean, real world problems in particular really do kind of encapsulate a whole pile of hopes and dreams.
1:13:19
So it may help to be a little more specific. You can use it in your learning outcomes, but then maybe describe it in a little more detail in your syllabus, the kinds of real world problems, or why we care about real world problems. They're messy. They're ill-defined. What about them is so important? What about them is so important? Try to minimize the buzz words. But at some level, they may not know about something until they're completely through with the course,
1:13:49
and you may have to use that word. So there's a bit of a balance. What I'm going to do before, I'll probably bring all these back to my office, take pictures of them, and put them up on the wiki so you'll have a record of them. I wanted to just as we go out here, my point is really that the hard part is these learning outcomes. And that once you write the learning outcomes, as we said before, you kind of know how you're going to test whether students got them,
1:14:23
and you know what you're going to do to help them get them. The syllabus itself is just an articulation of that. It's a description of the course, an articulation of the learning outcomes stated, like the laundry example, with the title. This is what you're going to do. By the end of the class this is what you'll be able to do. It's kind of a promise that that's what you're going to do. It motivates students to take the class, to stay in the class, to engage with the class.
1:14:56
It keeps you on track. So at the beginning of the semester maybe I'll be very clear, but four weeks in, and you're a little bit fuzzy, because you've been dealing with all sorts of stuff. And you kind of go, what the heck. What's this in here for? Well, if you've lined everything up at the beginning using your learning outcomes as the anchor, then you don't have to worry about that. You just follow the map. So it really, really helps you and the students
1:15:25
get through the semester in a logical way. And it tells the students what they can expect of you, what you can expect of them. So it's motivational, structural, and it's evidence, hopefully of what they did, but certainly of what you hope to do with them.
1:15:50
So for the post-session assignment, you'll have to write up more learning outcomes from your course. You can use these as the basis for sure. Hopefully, that table will help you as you write more of them. And I will go in and give you feedback. I want to make sure that everybody knows that you should read each other's postings. So a lot of you had great examples from the first class about how you would use these learning theories in your courses and how you teach. They were fabulous examples.
1:16:21
Make sure you read those postings of others. That's one reason why it's up on a wiki and not just you handing it in to me. So you have the Mud cards. Fill them out if you have any more information that you want. And otherwise, we'll see next week.
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat