Monthly Archives: March 2020

Getting dialogue online

Bank in the nineties, I facilitated a meeting with Frank Elter at a Telenor video meeting room in Oslo. There were about 8 participants, and an invited presenter: Tom Malone from MIT.

The way it was set up, we first saw a one hour long video Tom had created, where he gave a talk and showed some videos about new ways of organizing work (one of the more memorable sequences was (a shortened version of) the four-hour house video.) After seeing Tom’s video, we spent about one hour discussing some of the questions Tom had raised in the video. Then Tom came on from a video conferencing studio in Cambridge, Massachusetts, to discuss with the participants.

The interesting thing, to me, was that the participants experienced this meeting as “three hours with Tom Malone”. Tom experienced it as a one hour discussion with very interested and extremely well prepared participants.

A win-win, in other words.

I was trying for something similar yesterday, guest lecturing in Lene Pettersen‘s course at the University of Oslo, using Zoom with early entry, chat, polling and all video/audio enabled for all participants. This was the first videoconference lecture for the students and for three of my colleagues, who joined in. In preparation, the students had read some book chapters and articles and watched my video on technology evolution and disruptive innovations.

For the two hour session, I had set up this driving plan (starting at 2 pm, or 14:00 as we say over here in Europe…):

Image may contain: Espen Andersen, eyeglasses

Leading the discussion. Zoom allows you to show a virtual background, so I chose a picture of the office I would have liked to have…

14:00 – 14:15 Checking in, fiddling with the equipment and making sure everything worked. (First time for many of the users, so have the show up early so technical issues don’t eat into the teaching time.)
14:15 – 14:25 Lene introduces the class, talks about the rest of the course and turns over to Espen (we also encouraged the students to enter questions they wanted addressed in the chat during this piece)
14:25 – 14:35 Espen talking about disruption and technology-driven strategies.
14:35 – 14:55 Students into breakout rooms – discussing whether video what it would take for video and digital delivery to be a disruptive innovation for universities. (Breaking students up into 8 rooms of four participants, asking them to nominate a spokesperson to take notes and paste them into the chat when they return, and to discuss the specific question: What needs to happen for COVID-19 to cause a disruption of universities, and how would such a disruption play out?
14:55 – 15:15 Return to main room, Espen sums up a little bit, and calls on spokesperson from each group (3 out of 8 groups) based on the notes posted in the chat (which everyone can see). Espen talks about the Finn.no case and raises the next discussion question.
15:15 – 15:35 Breakout rooms, students discuss the next question: What needs to happen for DNB (Norway’s largest bank) to become a data-driven, experiment-oriented organization like Finn.no? What are the most important obstacles and how should they be dealt with?
15:35 – 15:55 Espen sums up the discussion, calling on some students based on the posts in the chat, sums up.
15:55 – 16:00 Espen hand back to Lene, who sums up. After 16:00, we stayed on with colleagues and some of the students to discuss the experience.

zoom dashboard

The dashboard as I saw it. Student names obscured.

Some reflections (some of these are rather technical, but they are notes to myself):

  • Not using Powerpoint or a shared screen is important. Running Zoom in Gallery view (I had set it up so you could see up to 49 at the same time) and having the students log in to Zoom and upload a picture gave a feeling of community. Screen and/or presentation sharing breaks the flow for everyone – When you do it in Zoom, the screen reconfigures (as it does when you come back from a breakout room) and you have to reestablish the participant panel and the chat floater. Instead, using polls and discussion questions and results communicated through the chat was easier for everyone (and way less complicated).
  • No photo description available.

    Satisfactory results, I would say.

    I used polls on three occasions: Before each discussion breakout, and in the end to ask the students what the experience was like. They were very happy about it and had good pointers on how to make it better

  • We had no performance issues and rock-steady connection the whole way through.
  • It should be noted that the program is one of the most selective in Norway and the students are highly motivated and very good. During the breakout sessions I jumped into each room to listen in on the discussion (learned that it was best to pause recording to avoid a voice saying “This session is being recorded” as I entered. The students were actively discussing in every group, with my colleagues (Bendik, Lene, and Katja) also participating. I had kept the groups to four participants, based on feedback from a session last week, where the students had been 6-7 and had issues with people speaking over each other.
  • Having a carefully written driving plan was important, but still, it was a very intense experience, I was quite exhausted afterwards. My advice on not teaching alone stands – in this case, I was the only one with experience, but that will change very fast. But I kept feeling rushed and would have liked more time, especially in the summary sections, would have liked to bring more students in to talk.
  • I did have a few breaks myself – during the breakout sessions – to go to the bathroom and replenish my coffee – but failed to allow for breaks for the students. I assume they managed to sneak out when necessary (hiding behind a still picture), but next time, I will explicitly have breaks, perhaps suggest a five minute break in the transition from main room to breakout rooms.

Conclusion: This can work very well, but I think it is important to set up each video session based on what you want to use it for: To present something, to run an exercise, to facilitate interaction. With a small student group like this, I think interaction worked very well, but it requires a lot of presentation. You have to be extremely conscious of time – I seriously think that any two-hour classroom session needs to be rescheduled to a three hour session just because the interaction is slower, and you need to have breaks.

As Winston Churchill almost said (he said a lot, didn’t he): We make our tools, and then our tools make us. We now have the tools, it will be interesting to see how the second part of this transition plays out.

Dealing with cheating

At BI Norwegian Business School, we are (naturally and way overdue, but a virus crisis helps) moving all exams to digital. This means a lot of changes for people who have not done that before. One particular anxiety is cheating – normally not a problem in the subjects I teach (case- and problem oriented, master/executive, small classes) but certainly is an issue in large classes at the bachelor level, where many answers are easily found online, the students are many, and the subjects introductory in nature.

Here are some strategies to deal with this:

  • Have an academic honesty policy and have the students sign it as part of the exam. This to make them aware of they risk if they cheat.
  • Keep the exam time short – three hours at the max – and deliberately ask more questions than usual. This makes for less time for cheating (by collaborating) because collaboration takes time. It also means introducing more differentiation between the students – if just a few students manage to answer all questions, those are the A candidates. Obviously, you need to adjust the grade scale somewhat (you can’t expect all to answer everything) and there is an issue of awarding students that are good at taking exams at the expense of deep learning, but that is the way of all exams.
  • Don’t ask the obvious questions, especially not those asked on previous exams. Sorry, no reuse. Or perhaps a little bit (it is a tiring time.)
  • Tell the students that all answers will be subjected to an automated plagiarism check. Whether this is true or not, does not matter – plagiarism checkers are somewhat unreliable, have many false positives, and require a lot of afterwork – but just the threat will eliminate much cheating. (Personally, I look for cleverly crafted answers and Google them, amazing what shows up…).
  • Tell the students that after the written exam, they can be called in for an oral exam where they will need to show how they got their answers (if it is a single-answer, mathematically oriented course) or answer more detailed questions (if it is a more analysis- or literature oriented course). Who gets called in (via videoconference) will be partially random and partially based on suspicion. Failing the orals results in failing the course.
  • When you write the questions: If applicable, Google them, look at the most common results, and deliberately reshape the questions so that the answer is not one of those.
  • Use an example for the students to discuss/calculate, preferably one that is fresh from a news source or from a deliberately obscure academic article they have not seen before.
  • Consider giving sub-groups of students different numbers to work from – either automatically (different questions allocated through the exam system) or by having questions like “If your student ID ends in an even number (0,2,4,6,8) answer question 2a, otherwise answer question 2b” (use the student ID, not “birthday in January, February, March…” as this will be the only marker you have.) The questions may have the same problem, but with small, unimportant differences such as names, coefficients or others. This makes it much harder to collaborate for the students. (If you do multiple questions in an electronic context, I assume a number of the tools will have functionality for changing the order of the questions – it would, frankly, astonish me if they did not – but I don’t use multiple choice myself, so I don’t know.
  • Consider telling the students they will all get different problems (as discussed above) but not doing it. It still will prevent a lot of cheating simply because the students believe they all have different problems and act accordingly.
  • If you have essay questions, ask the students to pick a portion of them and answer them. I do this on all my exams anyway – give the students 6 questions with short (150 words) answers and ask them to pick 4 and answer only those, and give them 2 or 3 longer questions (400 words or so) and ask them to answer only one. (Make it clear that answering them all will result in only the first answered will be considered.) Again, this makes cheating harder.

Lastly: You can’t eliminate cheating in regular, physical exams, so don’t think you can do it in online exams. But you certainly can increase the disincentives to do so, and that is the most you can hope for.

Department for future ideas
I have always wanted to use machine learning for grading exams. At BI, we have some exams with 6000 candidates writing textual answers. Grading this surely must constitute cruel and unusual punishment. With my eminent colleague Chandler Johnson I tried to start a project where we would have graders grade 1000 of these exams, then use text recognition and other tools, build an ML model and use that to grade the rest. Worth an experiment, surely. The project (like many other ideas) never took off, largely because of difficulties of getting the data, but perhaps this situation will make it possible.

And that would be a good thing…

A teaching video – with some reflections

Last Thursday, I was supposed to teach a class on technology strategy for a bachelor program at the University of Oslo. That class has been delayed for a week and (obviously) moved online. I thought about doing it video conference, but why not make a video, ask the students to see it before class? Then I can run the class interactively, discussing the readings and the video rather than spending my time talking into a screen. Recording a video is more work, but the result is reusable in other contexts, which is why I did it in English, not Norwegian. The result is here:

To my teaching colleagues: The stuff in the middle is probably not interesting – see the first two and the last five minutes for pointers to teaching and video editing.

For the rest, here is a short table of contents (with approximate time stamps):

  • 0:00 – 2:00 Intro, some details about recording the video etc.
  • 2:00 – 27:30 Why technology evolution is important, and an overview of technology innovation/evolution processes
    • 6:00 – 9:45 Standard engineering
    • 9:45 – 12:50 Invention
    • 12:50 – 15:50 Structural deepening
    • 15:50 – 17:00  Emerging (general) technology
      • 17:00 – 19:45 Substitution
      • 19:45 – 25:00 Expansion, including dominant design
      • 25:00 – 27:30 Structuration
  • 27:30 – 31:30 Architectural innovation (technology phases)
  • 31:30 –  31:45 BREAK! (Stop the video and get some coffee…)
  • 31:45 – 49:40 Disruption
    • 31:45 – 38:05 Introduction and theory
    • 38:05 – 44:00 Excavator example
    • 44:00 – 46:00 Hairdresser example
    • 47:00 – 47:35 Characteristics of disruptive innovations
    • 47:35 – 49:40 Defensive strategies
  • 49:40 – 53:00 Things take time – production and teaching…
  • 53:00 – 54:30 Fun stuff

This is not the first time I have recorded videos, by any means, but it is the first time I have created one for “serious” use, where I try to edit it to be reasonably professional. Some reflections on the process:

  • This is a talk I have given many times, so I did not need to prepare the content much – mainly select some slides. for a normal course, I would use two-three hours to go through the first 30 minutes of this video – I use much deeper examples and interact with the students, have them come up with other examples and so on. The disruption part typically takes 1-2 hours, plus at least one hour on a specific case (such as the steel production). Now the format forces me into straight presentation, as well as a lot of simplification – perhaps too much. I aim to focus on some specifics in the discussion with the students.
  • I find that I say lots of things wrong, skip some important points, forget to put emphasis on other points. That is irritating, but this is straight recording, not a documentary, where I would storyboard things, film everything in short snippets, use videos more, and think about every second. I wanted to do this quickly, and then I just have to learn not to be irritated at small details.
  • That being said, this is a major time sink. The video is about 55 minutes long. Recording took about two hours (including a lot of fiddling with equipment and a couple of breaks). Editing the first 30 minutes of the  video took two hours, another hour and a half for the disruption part (mainly because by then I was tired, said a number of unnecessary things that I had to remove.)
  • Using the iPad to be able to draw turned out not to be very helpful in this case, it complicated things quite a bit. Apple’s SideCar is still a bit unpredictable, and for changing the slides or the little drawing on the slides I did, a mouse would have been enough.
  • Having my daughter as audience helps, until I have trained myself to look constantly into the camera. Taping a picture of her or another family member to the camera would probably work almost as well, with practice. (She has heard all my stories before…)
  • When recording with a smartphone, put it in flight mode so you don’t get phone calls while recording (as I did.) Incidentally, there are apps out there that allow you to use the iPhone as a camera connected to the PC with a cable, but I have not tested them. It is easy to transfer the video with AirPlay, anyway.
  • The sound is recorded in two microphones (the iPhone and a Røde wireless mic.) I found that it got “fatter” if I used both the tracks, so I did that, but it does sometime screw up the preview function in Camtasia (though not the finished product). That would also have captured both my voice and my daughter’s (though she did not ask any questions during the recording, except on the outtakes.)
  • One great aspect of recording a video is that you can fix errors – just pause and repeat whatever you were going to say, and the cut it in editing. I also used video overlays to correct errors in some slides, and annotations to correct when I said anything wrong (such as repeatedly saying “functional deepening” instead of “structural deepening”.) It does take, time, however…

My excellent colleague Ragnvald Sannes pointed out that this is indicative of how teaching will work in the future, from a work (and remuneration) perspective. We will spend much more time making the content, and less time giving it. This, at the very least, means that teachers can no longer be paid based on the number of hours spent teaching – or that we need to redefine what teaching means…

Moving your course online: Five things to consider

Another video on moving to video-based teaching, this time about some things to consider to make the transition as easy for yourself as possible (as well as increasing the experience for the students):

From the Youtube posting:

Many teachers now have to move their courses online, and are worried about it. Teaching online is different from teaching in a classroom, but not so different: The main thing is still that you know your material and care about the the people at the other end. There are some things to consider, however, so here are five tips to think about when you move your course online:

  1. Talk to one student, not many.
  2. Structure, structure, structure (much more important in online teaching).
  3. Interaction is possible, but needs to be planned.
  4. Bring a friend: Teach with a colleague, for mutual help and a better experience.
  5. Use the recording as a tool for making your teaching better, by reviewing it and editing it yourself.

Five tips for better video teaching

In these viral times, a lot of universities will need to switch to video teaching, and for many teachers, this is a new experience. Here is a short (and fast) video I made with five – non-technical – tips for better video conferencing and teaching.

To sum it up:

1. Sound is more important than picture.
2. Look into the camera!
3. Don’t make the obvious mistakes: Background, lighting, and clothing.
4. Be lively! The medium consumes energy, you need to compensate.
5. Get to know the tools.

Good luck!