learning about learning

The other big experience that shaped how I view educational systems was being a TA at MIT. On the whole it was a really disheartening experience. I believe that is much harder and more worthwhile to construct a healthy system — be it educational or otherwise — than critique an existing, failing one, and so I haven’t thought much about TAing at MIT in the past few years. However, I’m currently living at an MIT house (I’m the Resident Advisor at pika!). Watching people struggle with a system they’re unaware of has made me think that these observations are still of use.

After coming back to MIT from Germany, I felt completely unwilling to participate in the educational system. My overall feeling was that the whole system could be run so much better and had so little thought put into it, and yet it didn’t particularly bother anyone. I had an amazing resistance to taking classes where I felt like the instructor wasn’t trying or caring much and consequently dropped nearly all of the classes I signed up for my first term back.

I took 4 classes my last two terms at MIT, and dropped my former plans of becoming a mathematician mid-way through applying for a Ph.D. My interest in math fell apart as I realized that there were many more important things to work on, especially in the worlds of learning and education. Of the four classes I took, was being a teaching assistant (TA) for 18.02 — multivariable calculus. It satisfied my last graduation requirement and with my rising interest in education it would be a neat thing to do.

For background, 18.02 is a required course at MIT — all students must take it to graduate. It’s the second math class in the introductory one-year calculus sequence. At many universities, this is a two-year sequence. The class itself was about 200-300 students, and I taught a 1-hour recitation twice a week with 20 students and also graded their homework.

1. My own growing perspective on education

I’d started reading John Holt’s How Children Fail and How Children Learn that semester, which were really shaping how I thought about teachers, learners, and educational systems. Those books began to give me an articulation for the intuitions I’d developed in Germany, and made me look at grades as unnecessary and something that simply got in the way of people’s learning processes. Instead of giving students the message “You haven’t fully mastered this concept yet,” grades gave students the message “You haven’t fully mastered this concept yet and now you’re being punished for it.” I began to become aware of how grading cuts into someone’s natural feedback process when learning — changing an intuition like “Man! I wish I understood lift better, it’s so mysterious and interesting” to “Man, there’s no way I can get an A in this class now.” Thinking about Holt in terms of my experiences in Germany made me understand how toxic grades and similar judgements are and how one can thrive when rid of them.

I also saw how liberating it was to be free of notions like “I need to work harder” and “I’m not working hard enough” in Germany. I began to recognize that kind of thinking as symptomatic of a poor educational system — putting the student in a place where despite their natural curiousity, they felt both overwhelmed (I can’t get everything done) and unsatisfied (I need to be doing something differently…) Since students fundamentally can’t change their learning environment — they don’t have control over their own learning process — the only place they have to turn is inward. This leads to the very common spiral of plaguing, nagging thoughts of self doubt: “Maybe if I worked harder, I wouldn’t be behind,” or “Maybe if I slept enough, I wouldn’t be so tired,” etc. My experience in Germany, suddenly free of my own self-doubt once I was happily learning all day, has led me to view claims like these, or similar ones like “I’m just not good at this” or “I just haven’t tried enough” as signs that the system isn’t working, not the student.

2. The first exam

The first few weeks of TAing were pretty ordinary. Myself and my students got used to the rhythm of the class, I got used to balancing presenting material and taking questions in recitation, and so on. The first exam we had was the first event that began making me suspicious of the class.

The test grades were what you’d expect — some students did well, some did ok, and a few failed. TAs were asked to email any student who failed, letting them know how they could make up a test. I remember looking at one student’s failed exam and his related homework and seeing clearly that of the three weeks of material covered, he’d understood the first two weeks fine and not the third week. As far as using the test as objective feedback goes, this meant he had a week’s more of learning to do for the course.

As far as the metric used by the class though, this meant he’d failed his first test: a deeply demoralizing event. On top of that, he’d have the spectre of that failure for the next two and a half months: there was no way from him to makeup that failed grade and get a high mark in the class now — his average would be too weighed down. So while the message ought to have been “Ah! You haven’t understood cross products yet — you have these ideas to catch up on!”, it becomes a stigmatizing failure that lasts for the whole term: “You could pass this class if you work extra hard.”

I emailed my student asking him to meet. I was eager to show him that there were just 2 big ideas he was missing and then he’d be on par with the rest of the course: the looming feeling that not only did you fail a test, but most everyone else in your class didn’t makes the event that much more defeating and confusing: why can’t I do this but most everyone else can? It took a bit of pushing to get him to meet with me — I think he politely declined the email, and then after recitation one day I got his attention in the hallway and asked him if he had time to go over the material right then. He did, and a half hour later he was really relieved I’d pulled him aside. As I suspected, he’d viewed the failed test as a sign that he was really far behind, and by the end of our session he was much more relaxed.

The rest of the semester went fine for him and he remained very grateful for that intervention. It’s a nice story because it wraps up cleanly, but of course it’s not the norm. It got me thinking: why was it such a big deal for him to learn how to handle cross products right here, right now? Beyond the test — the artificial environment — he had no need for them. When we’d met, he explained that he’d been busy with his other classes, having had several exams that week, and just hadn’t gotten to calculus yet. This seemed reasonable enough to me: he was doing his work as best as he could, and the idea that he hadn’t learned how to do cross products by an arbitrary date causing so much stress just seemed preposterous to me. What was the point?

3. Copying

About midway through the term, I began to notice that a lot of the problem sets handed in were duplicates of other problem sets. From my point of a view as a grader, it would be amusing to see the changes people would put in their problem set — like substituting a “y” instead of an “x” throughout a problem to attempt to mask an otherwise 6 pages of calculus that was line-for-line identical. On one of the weeks almost half of the work was copied. Some of those may have been the originals, but it was still a lot! Another undergrad TA had noticed the same thing; graduate TAs didn’t grade their own section’s homework, so it was only a few staff that noticed this trend.

I set out to look at copying objectively — past the usual moral claim that it’s a wrong thing to do and to discern what it meant about the class. It first occurred to me that copying was a literal waste of time: instead of spending an hour or two trying
to understand the material at hand, one spends it transcribing equations line for line for pages. Beyond that, there’s the mental space that someone is in when they’re copying. By copying, someone is acknowledging that they can not or do the requested homework or that they don’t want to, yet they still feel obliged to appear to have done so. It’s like saying “I can’t or don’t want to do this, but I have to.”

The point of homework is to give the student practice with the skills covered in a course, and to give the instructors feedback as to what the students are understanding. What’s happening here is that a student who is copying feels that the practice offered is either not doable (the student does not understand, feels too exhausted, or both) or not useful (the practice does not seem worthwhile or the material does not seem worthwhile.) If the material isn’t doable, that feedback is of critical importance to the instructor. Whether it’s because the student is too exhausted to learn, as is often the case at MIT, or is just lost in the class, the student would benefit from the class slowing down and addressing this.

It’s also valuable feedback if the student sees no point in the exercises. Learning German was effortless because there was a natural context for the skill — communicating in German. Likewise, when there is no natural context for a skill, learning is a struggle. If you don’t see a reason to learn a skill, why would you learn it? Again, this is huge feedback for an instructor to have.

In either of these cases, instead of this feedback going to the instructor through a simple conversation or email, the student feels there is no use in being honest about how they feel, so much so that they spend an hour or two doing something pointless in order to appear as if they did indeed do the homework. This feedback gets masked because to give that feedback honestly — to have a conversation instead of handing in a copied problem set — would likely be poorly received, and worse, one would be graded harshly for it if the conversation didn’t go well.

4. Bibles

Taking this view and applying it to 18.02, where at its peak half or so of the problem sets were copied shows a pretty bleak picture — almost half of the students are pretending to do the homework and feel bound to do so. And yet this is no quirk particular to this class. I remembering touring fraternities as a freshmen, and one of their perks was their collection of “bibles”: collection of all the notes, homework, and tests from previous years. This idea of copying problem sets is such a normal one that it’s well established in MIT’s residential culture. The idea that a class isn’t working for you in some way and yet there’s no way to change that is an accepted one.

I came to realize that part of why students’ bibles work is because professors also have bibles. The course my professor was teaching was handed to him by the previous professor. It was the complement to the student’s bible: the lectures, problem sets, and exams were all in the course package. It felt on one hand ridiculous (the professor hands out problems he hasn’t thought about, and the students hand in answers pretending to have thought about them) and on the other offensive. The class was on a rigid track: there were so many set lectures, homeworks, and exams. There was no room for deviating if the class was stuck on one idea, understood another quicker than expected, or if there was an insight as to where the class should go instead. The class was like a train — steady and immovable. Knowing how much effort students put into accomodating a class’ assignments, it bothered me to realize that it was already pre-determined that the class wouldn’t respond to the student’s needs.

This begins to make sense of common student questions and concerns like “Why do we have to do this problem when we haven’t covered it in class yet?” or “I’m not sure what’s going to be on the exam since on the last exam there was material we had only just started…” If a class’ homework/test structure is predetermined, it’s easy for a lecturer to be out of sync with the questions he’s assigning because he simply hasn’t read them over or thought them through. While I can’t tell you how common this in courses at MIT, I would expect it to be the case in the majority of introductory classes.

Another post-doc in the math dept. — one of my favorite teachers at MIT, Emma Carberry — told me as she was applying for professorships that she felt quite frustrated with the high-pressure academic system. She was applying to liberal arts colleges to teach at because she wanted to be in an environment where she was rewarded, or at least acknowledged, for putting a lot of time into her teaching. She said that at MIT and in this tier of academia, the only metric was how well your research was going, and so putting time into teaching well was something that you were implicitly punished for professionally. This anecdote still amazes me; I often wonder why MIT bothers having classes when they don’t value them.

Students often have a few great classes at MIT. With an instructor who carefully thinks about the interaction between all of the course’s components: homework, tests, lectures, and so on,
and integrates feedback as it comes, a lecturer can create a good learning environment for a student at MIT. These professors are unfortunately rare because MIT’s professional system selects against this, as was the case with Emma Carberry. The professor I was working for told me that he wanted to be teaching grad students in his field of research, but was assigned the introductory class and could do nothing about it. I thought the professor was doing a decent job too (he did lots of things well — taking feedback from TAs, worked to create good materials for recitations) and yet at the end of the day, it was clear that this was not a project the professor was interested in. The lack of choice the professor had (“I don’t want to teach this class, but I have to”) led to the automatic production of the class from the bible, which in turn led to most of the students having the same reaction (“I don’t want to take this class, but I have to.”)

5. Tests

Midway through the semester, as I began to notice all of the copying, I began to really see the class for more of a charade: both the professor and majority of students were there because it was a required class, not because they thought it was worthwhile. Exam grading only furthered this along.

The professors and the TAs graded the exams, and each staff member got a question to grade for 3 hours or so with a partner. It was a pretty mind-numbing experience, grading 150 or so of the same exam question again and again. On this exam, the question that I graded was one where 2/3rds or so of the students made the same mistake. I remember thinking that I wished I had a stamp for how many times I wrote down “can’t used Green’s theorem when the line integral isn’t closed!” It began to dawn on me as I did this over and over again, that this was a clear sign the class did not understand the concept in any intuitive way. This was great feedback (the course pace should slow down and go over this again, ideally approaching it in a way that develops more intuition), but feedback that had no place in a predetermined class. The results of the tests got turned into numbers, the numbers into a distribution, and that was the feedback that the course received. The details of which concepts had been mastered by the group and which concepts hadn’t been understood at all — the feedback that mattered — was left behind in the wake of lots and lots of exam scores.

This exam just added on to the feeling that the class was a big, stressful game of pretend. The students didn’t know how to do the problem, and the mistake stemmed from trying to match patterns: there are 4 big ideas being tested, 6 questions on the test, and if you match them up right the test will turn ou
t fine. The feedback from the test that students didn’t understand Green’s Theorem didn’t fit into the course’s structure and so was ignored. The staff gave tests because they had to, the students took tests because they had to, and the course went on, staying on track.

6. Context

There were about three weeks left of the class at this point, which covered material like Stokes’ theorem. Stokes’ theorem is something that everyone at MIT recognizes — having had to take 18.02 — and hardly anyone knows, including math majors. I myself never had a strong intuition for why Stokes theorem was important, and tried to find a good context to present the theorem in.

I came across Schey’s Div, Grad, Curl and all that , which explained vector calculus and the material we were covering oin the context of electricity and magnetism. The book was great — it was very simple and answered my own personal questions about why this material was valuable.

It didn’t help for presenting it well though. I tried once, and realized that I was trying to elucidate one abstract concept — vector calculus — by putting in context of another abstract concept — electricity and magnetism (E&M). This was particularly weak because many of these students were taking E&M at the same time as 18.02 — there was no guarantee that E&M was something these students had any familiarity with, and furthermore, any intuition for.

I then asked the simple question — how many of these would actually use this material, based on their declared major? I surmised that of the 20-some majors at MIT, the ones that would use 18.02 extensively (something more than just adding a week’s worth of a material to a course to explain a needed mathematical tool) were mathematicians, physicists, and anyone who studied flow (so mechanical and civil engineers,) I was willing to bet another two majors used the material in ways I couldn’t think of, but that for the rest of them, beyond those six, their disciplines were not reliant on this material in any way. That quick estimate puts the number of students who would use 18.02 later on to be somewhere between 1/4 and 1/3 of the student body. Yet everyone was taking it!

I also thought about my friends who were physicists — who in E&M used vector calculus all the time. Most of them told me that the way they’d learned vector calculus was by learning E&M. This made sense to me too: you learn something by using the knowledge, not by preparing to use it. It made perfect sense to me: needing to use vector calculus and would create a much more powerful context for understanding and remembering the material than the artificial one of 18.02. Only 1/4 or 1/3 of the students who actually learned this material, and it seemed clear that they were better off learning it in the context of their discipline anyway. What was the point?

6. Feedback

One of the things that was so powerful in Germany was having complete control over my learning environment: being able to fold in the feedback from each thing I did into my daily process. One of the reasons that 18.02 felt like such an ineffective class to me was that the feedback loop was too large to change quickly and too fragmented to understand what it should change.

I already discussed the example of homework: where students hide their honest feedback in bowing to the system. And the example of testing: where the staff, prioritizing grading, ignores the real feedback generated. Beyond this was just the hierarchical mess of having 1 person responsible for 300 people’s learning.

I remember one example where a problem set contained an unusually difficult problem. The TAs had trouble doing it and couldn’t figure out how to do it. Some TAs unwittingly gave out false solutions in office hours, not realizing they didn’t know how to do it correctly. There were tons of questions that week about the problem, and their was nothing illuminating about solving it. It was supposed to be practice for a calculus idea, but it was really an exasperatingly long geometry problem.

This problem easily added on at least 2 hours of work to each student’s problem set or furthered the “I can’t do this so I better copy it” mindset. It would’ve taken the professor or the course admin maybe two hours to do the problem set, find that problem, and throw it out. But, since they were just handing out problem sets handed to them, this problem was kept in and a group of students wastes a good 300 – 600 hours.

Letting one person control 300 people’s time is the core structure of this and other lecture-based course. It’s pretty tough to give a good hour-long presentation, let alone three on a week on something you don’t particularly care about, as was this professor’s lot. There’s virtually no room for feedback: most students, when confused or stuck, hold in their questions because it’s quite difficult to have a conversation in a 300-on-1 environment, proceed to forget their questions, and struggle to follow the rest of the lecture.

There are good lecturers at MIT. However, it seems to be akin to being a good performer: lecturing, or performing, well is a rare skill and one that takes work. Giving a good lecture means having a good command of your voice, your blackboard, and having an intuition for how to present your material in a way that is engaging and not confusing. Looking at feedback from the lecturer’s point of view, it is harder to get feedback the larger your class is. A lecture to a 300-person audience has to be one where the instructor is already familiar with most of the pitfalls and confusions. It takes a lot of work to become experienced and knowledgable enough to be a good lecturer. It can be done, but given MIT’s stance of prioritizing research in an academic’s career, a good lecturer is going to be the outlier and not the norm.

Despite this, the lecture is regarded as the most important part of the course. In my term in 18.02, students attendance was highest in lecture, second highest in recitation (a 1 on 20 environment, where at least one or two questions per student could be fielded), and lowest in office hours (a 1 on 3 conversational environment.) I was fascinated by this; my students knew they could come to my office hours for anything, even a recap of the lectures that they’d decided not to go to. Very few did, despite it being the most active us of their time, the environment where they could ask the most questions and address their own confusions. The worst opportunity in this sense — lecture, where they had no way to engage but to passively listen and hope they didn’t get lost too quickly — was the best attended. It seemed to me that by putting students, in an environment that prioritizes the lecturer above all, they will unwittingly waste their time trying to make use of the lecture, absorbing 15 – 20 minutes of material for the hour they spend there.

7. Demoralizing students

Midway through the class, one or two female students told me, independently “I used to think I was good at math until I came to MIT.” It made me crazy to hear — I wanted to explain to them, as succinctly as they told me their self-doubts, all of the systemic things I’d been noticing and explain that they should by no means take this class and its grades personally. The impact of struggling in a class is huge — the take home message is not “I have not understood as much vector calculus as some other people in this class.” It’s “I’m not good at math.” I always wondered if it was just coincedence that the students who told me this were female, or if it was the result of carrying the weight of the stereotype “woman aren’t as good at math as men” around, and finally they had an experience which confirmed it.

I remember wondering one weekend why we (the 18.02 staff) were stressing people out so much about things like finding the
volumes of arbitrary shapes. I got to a point where the whole class seemed preposterous, and eventually even repulsive. For the final, one student told me that his plan was to get no sleep the night before the test because if he studied, and then slept, he’d forget all the equations he’d just learned at wouldn’t be able to use them on the test. Another final came back stained in Pepto-Bismol because the student had brought it with him to the exam, trying to calm his stomach down from pre-test anxiety and vomiting. What was the point of all this? To help people learn how to calculate flows and volumes?

I came to see the course as something that the students didn’t need and not designed in their interests. It was taught and it was taken because it was required of both parties. It’s main result wasn’t to get people excited by these ideas, or even to understand them, but mostly to pretend that they knew what was going on and wait for the class to stop. In the meanwhile, it was a thoroughly demoralizing experience for them, one with repercussions — making math seem impossible — that go exactly against the point of having the class in the first place.

As I finish writing this, my great hope is that this analysis will help someone understand, a little more closely, what is happening to them in their own struggles in college. I think that this kind of system is incredibly difficult to see when you are inside of it, and I hope that my different perspective from being a TA, and from thinking about education non-traditionally, are of use. I am really happy to talk more about this if it strikes anyone — just leave a comment or send me an email.