This video will tell you all you want to know about the rigging of historic sailboats (tall ships excluded) with just enough detail to make it a learning experience rather than an overview. And if you want to see more of Leo and his amazing project rebuilding the historic gaff cutter (yep, it will be explained) Tally Ho, go here. If you want to support him, go here.
I have always loved the USA, but it has been hard to do that lately. But this poetry reading by Amanda Gorman well, saved my day. I am two months shy of my 60th birthday and very aware of it. Praise to this 22 year old woman for raising my spirits!
This, truly, is the USA that I love and cherish.
What the heck, I am suffering from low productivity today anyway. So: I can heartily recommend Rick Beato‘s channel Everything Music if you are in need of distraction. He is. a music theorist and producer, first because Youtube famous with a video of his son having perfect pitch, and discusses all kinds of music theory. Most will like his lists of greatest guitar solos and so on, but I think his best video so far is this one, which was recorded, I see, the day before Eddie van Halen died:
Now, back to work, you hear?
Last year (with Chandler Johnson and Alessandra Luzzi) and this year (with Chandler, Jadwiga Supryn and Prakash Raj Paudel), I teach a course called Analytics for Strategic Management. In this course executive students work on real projects for real companies, applying various forms of machine learning (big data, analytics, whatever you want to call it) to business problems. Here is a list (mostly anonymised, except for public organizations) list from this year:
- One group wants to use machine learning to predict fraud in public security contracts in a developing country
- A credit agency wants to predict which of their customers will pay their bills by the end of the month
- An engineering company wants to predict the number of hours needed to meet demand for each month in each department
- One group wants to predict housing prices within Oslo, to help house sellers get a realistic estimate of what their property is worth
- A higher education provider wants to predict which students are likely to fail or not qualify for an exam, to be able to intervene early
- A couple of municipalities want to predict who will accept a kindergarten allocation or not
- A telecommunications company wants to predict which customers will churn
- An Internet product company wants to predict necessary capacity for picking and shipping work every day
- One group wants to predict the likelihood of a road closing due to bad weather, in order to warn truck drivers so they can detour
- One group wants to predict the future financial health of companies based on employee engagement numbers
- One group wants to predict efficiency of production in a wind power park
And last year we had these projects:
- An investment company wanted to predict bankruptcies from media events
- Ruter, Oslo’s public transportation authority wanted to predict the number of passengers (for each station, to great precision) for one line on the metro
- A telecommunications company wanted to predict customer feedback scores from analyzing customer interactions (so the customer does not have to answer a survey afterwards)
- The Norwegian Health directorate wanted to predict general physician “fastlege” churn
- A commercial TV station wanted to predict subscriber churn
- An insurance company wants to identify customers likely to buy a group insurance package
- An online gaming company wanted to predict customer churn
- A large political party wanted to predict membership churn
- One group wanted to start a company based on using machine learning to diagnose hearing problems
- A large retail chain wanted to predict churn based on customer purchase patterns
Yesterday began with a message from a business executive who was concerned with the security of Zoom, the video conferencing platform that many companies (and universities) have landed on. The reason was a newspaper article regurgitating several internet articles, partly about functionality that has been adequately documented by Zoom, partly about security holes that have been fixed a long time ago.
My answer is “probably not” – at least not for the security holes discussed here, and for ordinary users (and that includes most small- to medium sized companies I know about).
It is true that video conferencing introduces some security and privacy issues, but if we look at it realistically, the biggest problem is not the technology, but the people using it (Something we nerds refer to as PEBKAC – Problem Exists Between Keyboard and Chair.)
When a naked man sneaks into an elementary school class via Whereby, as happened a few days ago here in Norway, it is not due to technology problems, but because the teacher had left the door wide open, i.e., had not turned on the function that makes it necessary to “knock” and ask for permission to enter.
When anyone can record (and have the dialogue automatically transcribed) from Zoom, it is because the host has not turned off the recording feature. By the way, anyone can record a video conference with screen capture software (such as Camtasia), a sound recorder or for that matter a cell phone, and no (realistic) security system in the world can do anything about it.
When the boss can monitor that people are not using other software while sitting in a meeting (a feature that can be completely legitimate in a classroom, it is equivalent to the teacher looking beyond the class to see if the students are awake), well, I don’t think the system is to blame for that either. Any leader who holds such irrelevant meetings that people do not bother to pay attention should rethink their communications strategy. Any executive I know would have neither time nor interest in activating this feature – because if you need technology to force people to wake up, you don’t have a problem technology can solve.
The risk of a new tool should not be measured against some perfect solution, but against what the alternative is if you don’t have it. Right now, video conferencing is the easiest and best tool for many – so that is why we use it. But we have to take the trouble to learn how it works. The best security system in the world is helpless against people writing their password on a Post-It, visible when they are in videoconference.
If that’s hard, maybe you should cancel the whole thing and send an email instead.
The way it was set up, we first saw a one hour long video Tom had created, where he gave a talk and showed some videos about new ways of organizing work (one of the more memorable sequences was (a shortened version of) the four-hour house video.) After seeing Tom’s video, we spent about one hour discussing some of the questions Tom had raised in the video. Then Tom came on from a video conferencing studio in Cambridge, Massachusetts, to discuss with the participants.
The interesting thing, to me, was that the participants experienced this meeting as “three hours with Tom Malone”. Tom experienced it as a one hour discussion with very interested and extremely well prepared participants.
A win-win, in other words.
I was trying for something similar yesterday, guest lecturing in Lene Pettersen‘s course at the University of Oslo, using Zoom with early entry, chat, polling and all video/audio enabled for all participants. This was the first videoconference lecture for the students and for three of my colleagues, who joined in. In preparation, the students had read some book chapters and articles and watched my video on technology evolution and disruptive innovations.
For the two hour session, I had set up this driving plan (starting at 2 pm, or 14:00 as we say over here in Europe…):
14:00 – 14:15 Checking in, fiddling with the equipment and making sure everything worked. (First time for many of the users, so have the show up early so technical issues don’t eat into the teaching time.)
14:15 – 14:25 Lene introduces the class, talks about the rest of the course and turns over to Espen (we also encouraged the students to enter questions they wanted addressed in the chat during this piece)
14:25 – 14:35 Espen talking about disruption and technology-driven strategies.
14:35 – 14:55 Students into breakout rooms – discussing whether video what it would take for video and digital delivery to be a disruptive innovation for universities. (Breaking students up into 8 rooms of four participants, asking them to nominate a spokesperson to take notes and paste them into the chat when they return, and to discuss the specific question: What needs to happen for COVID-19 to cause a disruption of universities, and how would such a disruption play out?
14:55 – 15:15 Return to main room, Espen sums up a little bit, and calls on spokesperson from each group (3 out of 8 groups) based on the notes posted in the chat (which everyone can see). Espen talks about the Finn.no case and raises the next discussion question.
15:15 – 15:35 Breakout rooms, students discuss the next question: What needs to happen for DNB (Norway’s largest bank) to become a data-driven, experiment-oriented organization like Finn.no? What are the most important obstacles and how should they be dealt with?
15:35 – 15:55 Espen sums up the discussion, calling on some students based on the posts in the chat, sums up.
15:55 – 16:00 Espen hand back to Lene, who sums up. After 16:00, we stayed on with colleagues and some of the students to discuss the experience.
Some reflections (some of these are rather technical, but they are notes to myself):
- Not using Powerpoint or a shared screen is important. Running Zoom in Gallery view (I had set it up so you could see up to 49 at the same time) and having the students log in to Zoom and upload a picture gave a feeling of community. Screen and/or presentation sharing breaks the flow for everyone – When you do it in Zoom, the screen reconfigures (as it does when you come back from a breakout room) and you have to reestablish the participant panel and the chat floater. Instead, using polls and discussion questions and results communicated through the chat was easier for everyone (and way less complicated).
I used polls on three occasions: Before each discussion breakout, and in the end to ask the students what the experience was like. They were very happy about it and had good pointers on how to make it better
- We had no performance issues and rock-steady connection the whole way through.
- It should be noted that the program is one of the most selective in Norway and the students are highly motivated and very good. During the breakout sessions I jumped into each room to listen in on the discussion (learned that it was best to pause recording to avoid a voice saying “This session is being recorded” as I entered. The students were actively discussing in every group, with my colleagues (Bendik, Lene, and Katja) also participating. I had kept the groups to four participants, based on feedback from a session last week, where the students had been 6-7 and had issues with people speaking over each other.
- Having a carefully written driving plan was important, but still, it was a very intense experience, I was quite exhausted afterwards. My advice on not teaching alone stands – in this case, I was the only one with experience, but that will change very fast. But I kept feeling rushed and would have liked more time, especially in the summary sections, would have liked to bring more students in to talk.
- I did have a few breaks myself – during the breakout sessions – to go to the bathroom and replenish my coffee – but failed to allow for breaks for the students. I assume they managed to sneak out when necessary (hiding behind a still picture), but next time, I will explicitly have breaks, perhaps suggest a five minute break in the transition from main room to breakout rooms.
Conclusion: This can work very well, but I think it is important to set up each video session based on what you want to use it for: To present something, to run an exercise, to facilitate interaction. With a small student group like this, I think interaction worked very well, but it requires a lot of presentation. You have to be extremely conscious of time – I seriously think that any two-hour classroom session needs to be rescheduled to a three hour session just because the interaction is slower, and you need to have breaks.
As Winston Churchill almost said (he said a lot, didn’t he): We make our tools, and then our tools make us. We now have the tools, it will be interesting to see how the second part of this transition plays out.
At BI Norwegian Business School, we are (naturally and way overdue, but a virus crisis helps) moving all exams to digital. This means a lot of changes for people who have not done that before. One particular anxiety is cheating – normally not a problem in the subjects I teach (case- and problem oriented, master/executive, small classes) but certainly is an issue in large classes at the bachelor level, where many answers are easily found online, the students are many, and the subjects introductory in nature.
Here are some strategies to deal with this:
- Have an academic honesty policy and have the students sign it as part of the exam. This to make them aware of they risk if they cheat.
- Keep the exam time short – three hours at the max – and deliberately ask more questions than usual. This makes for less time for cheating (by collaborating) because collaboration takes time. It also means introducing more differentiation between the students – if just a few students manage to answer all questions, those are the A candidates. Obviously, you need to adjust the grade scale somewhat (you can’t expect all to answer everything) and there is an issue of awarding students that are good at taking exams at the expense of deep learning, but that is the way of all exams.
- Don’t ask the obvious questions, especially not those asked on previous exams. Sorry, no reuse. Or perhaps a little bit (it is a tiring time.)
- Tell the students that all answers will be subjected to an automated plagiarism check. Whether this is true or not, does not matter – plagiarism checkers are somewhat unreliable, have many false positives, and require a lot of afterwork – but just the threat will eliminate much cheating. (Personally, I look for cleverly crafted answers and Google them, amazing what shows up…).
- Tell the students that after the written exam, they can be called in for an oral exam where they will need to show how they got their answers (if it is a single-answer, mathematically oriented course) or answer more detailed questions (if it is a more analysis- or literature oriented course). Who gets called in (via videoconference) will be partially random and partially based on suspicion. Failing the orals results in failing the course.
- When you write the questions: If applicable, Google them, look at the most common results, and deliberately reshape the questions so that the answer is not one of those.
- Use an example for the students to discuss/calculate, preferably one that is fresh from a news source or from a deliberately obscure academic article they have not seen before.
- Consider giving sub-groups of students different numbers to work from – either automatically (different questions allocated through the exam system) or by having questions like “If your student ID ends in an even number (0,2,4,6,8) answer question 2a, otherwise answer question 2b” (use the student ID, not “birthday in January, February, March…” as this will be the only marker you have.) The questions may have the same problem, but with small, unimportant differences such as names, coefficients or others. This makes it much harder to collaborate for the students. (If you do multiple questions in an electronic context, I assume a number of the tools will have functionality for changing the order of the questions – it would, frankly, astonish me if they did not – but I don’t use multiple choice myself, so I don’t know.
- Consider telling the students they will all get different problems (as discussed above) but not doing it. It still will prevent a lot of cheating simply because the students believe they all have different problems and act accordingly.
- If you have essay questions, ask the students to pick a portion of them and answer them. I do this on all my exams anyway – give the students 6 questions with short (150 words) answers and ask them to pick 4 and answer only those, and give them 2 or 3 longer questions (400 words or so) and ask them to answer only one. (Make it clear that answering them all will result in only the first answered will be considered.) Again, this makes cheating harder.
Lastly: You can’t eliminate cheating in regular, physical exams, so don’t think you can do it in online exams. But you certainly can increase the disincentives to do so, and that is the most you can hope for.
Department for future ideas
I have always wanted to use machine learning for grading exams. At BI, we have some exams with 6000 candidates writing textual answers. Grading this surely must constitute cruel and unusual punishment. With my eminent colleague Chandler Johnson I tried to start a project where we would have graders grade 1000 of these exams, then use text recognition and other tools, build an ML model and use that to grade the rest. Worth an experiment, surely. The project (like many other ideas) never took off, largely because of difficulties of getting the data, but perhaps this situation will make it possible.
And that would be a good thing…
Last Thursday, I was supposed to teach a class on technology strategy for a bachelor program at the University of Oslo. That class has been delayed for a week and (obviously) moved online. I thought about doing it video conference, but why not make a video, ask the students to see it before class? Then I can run the class interactively, discussing the readings and the video rather than spending my time talking into a screen. Recording a video is more work, but the result is reusable in other contexts, which is why I did it in English, not Norwegian. The result is here:
To my teaching colleagues: The stuff in the middle is probably not interesting – see the first two and the last five minutes for pointers to teaching and video editing.
For the rest, here is a short table of contents (with approximate time stamps):
- 0:00 – 2:00 Intro, some details about recording the video etc.
- 2:00 – 27:30 Why technology evolution is important, and an overview of technology innovation/evolution processes
- 6:00 – 9:45 Standard engineering
- 9:45 – 12:50 Invention
- 12:50 – 15:50 Structural deepening
- 15:50 – 17:00 Emerging (general) technology
- 17:00 – 19:45 Substitution
- 19:45 – 25:00 Expansion, including dominant design
- 25:00 – 27:30 Structuration
- 27:30 – 31:30 Architectural innovation (technology phases)
- 31:30 – 31:45 BREAK! (Stop the video and get some coffee…)
- 31:45 – 49:40 Disruption
- 31:45 – 38:05 Introduction and theory
- 38:05 – 44:00 Excavator example
- 44:00 – 46:00 Hairdresser example
- 47:00 – 47:35 Characteristics of disruptive innovations
- 47:35 – 49:40 Defensive strategies
- 49:40 – 53:00 Things take time – production and teaching…
- 53:00 – 54:30 Fun stuff
This is not the first time I have recorded videos, by any means, but it is the first time I have created one for “serious” use, where I try to edit it to be reasonably professional. Some reflections on the process:
- This is a talk I have given many times, so I did not need to prepare the content much – mainly select some slides. for a normal course, I would use two-three hours to go through the first 30 minutes of this video – I use much deeper examples and interact with the students, have them come up with other examples and so on. The disruption part typically takes 1-2 hours, plus at least one hour on a specific case (such as the steel production). Now the format forces me into straight presentation, as well as a lot of simplification – perhaps too much. I aim to focus on some specifics in the discussion with the students.
- I find that I say lots of things wrong, skip some important points, forget to put emphasis on other points. That is irritating, but this is straight recording, not a documentary, where I would storyboard things, film everything in short snippets, use videos more, and think about every second. I wanted to do this quickly, and then I just have to learn not to be irritated at small details.
- That being said, this is a major time sink. The video is about 55 minutes long. Recording took about two hours (including a lot of fiddling with equipment and a couple of breaks). Editing the first 30 minutes of the video took two hours, another hour and a half for the disruption part (mainly because by then I was tired, said a number of unnecessary things that I had to remove.)
- Using the iPad to be able to draw turned out not to be very helpful in this case, it complicated things quite a bit. Apple’s SideCar is still a bit unpredictable, and for changing the slides or the little drawing on the slides I did, a mouse would have been enough.
- Having my daughter as audience helps, until I have trained myself to look constantly into the camera. Taping a picture of her or another family member to the camera would probably work almost as well, with practice. (She has heard all my stories before…)
- When recording with a smartphone, put it in flight mode so you don’t get phone calls while recording (as I did.) Incidentally, there are apps out there that allow you to use the iPhone as a camera connected to the PC with a cable, but I have not tested them. It is easy to transfer the video with AirPlay, anyway.
- The sound is recorded in two microphones (the iPhone and a Røde wireless mic.) I found that it got “fatter” if I used both the tracks, so I did that, but it does sometime screw up the preview function in Camtasia (though not the finished product). That would also have captured both my voice and my daughter’s (though she did not ask any questions during the recording, except on the outtakes.)
- One great aspect of recording a video is that you can fix errors – just pause and repeat whatever you were going to say, and the cut it in editing. I also used video overlays to correct errors in some slides, and annotations to correct when I said anything wrong (such as repeatedly saying “functional deepening” instead of “structural deepening”.) It does take, time, however…
My excellent colleague Ragnvald Sannes pointed out that this is indicative of how teaching will work in the future, from a work (and remuneration) perspective. We will spend much more time making the content, and less time giving it. This, at the very least, means that teachers can no longer be paid based on the number of hours spent teaching – or that we need to redefine what teaching means…
Another video on moving to video-based teaching, this time about some things to consider to make the transition as easy for yourself as possible (as well as increasing the experience for the students):
From the Youtube posting:
Many teachers now have to move their courses online, and are worried about it. Teaching online is different from teaching in a classroom, but not so different: The main thing is still that you know your material and care about the the people at the other end. There are some things to consider, however, so here are five tips to think about when you move your course online:
- Talk to one student, not many.
- Structure, structure, structure (much more important in online teaching).
- Interaction is possible, but needs to be planned.
- Bring a friend: Teach with a colleague, for mutual help and a better experience.
- Use the recording as a tool for making your teaching better, by reviewing it and editing it yourself.
In these viral times, a lot of universities will need to switch to video teaching, and for many teachers, this is a new experience. Here is a short (and fast) video I made with five – non-technical – tips for better video conferencing and teaching.
To sum it up:
1. Sound is more important than picture.
2. Look into the camera!
3. Don’t make the obvious mistakes: Background, lighting, and clothing.
4. Be lively! The medium consumes energy, you need to compensate.
5. Get to know the tools.
You think of many things when a friend dies.
When I was about 16, I went into the forest with some friends to watch the 50-km cross-country race at the Holmenkollen ski festival. One of the stars that year was Juha Mieto, an enormous (more than two meter) Finn who had to be careful with his strength, as he tended to break his ski poles. One of my friends decided to try to keep up with Mieto, just to see how long he could do it. My friend was a reasonably good skier, and managed to keep up for about 300 meters. Mieto, of course, kept going for 50 kilometers.
Sometimes I had the same feeling when interacting with Clay. Not because he was an imposing giant in the physical sense, but because of his incredible work capacity and ability to follow things through. I felt I could keep up with him for a while, enjoy it – and he would then go on, endlessly doing so much more than I was capable of. Clay was so many things: A famous scholar, a life-changing teacher, an adviser, a church leader, husband, father of five – and seemed to do it, if not effortlessly, at least conclusively, with a degree of self-discipline hard to imagine. For me, he was a friend.
We met as students at Harvard Business School in 1990 (he started one year before me), in an Organizational Psychology course with 140 papers on the reading list. Doing that alone just wasn’t possible, so we formed study groups of 5-6 people, writing summaries of papers for each other and occasionally meeting to discuss them before class. I still have the notes. Clay was different in that he added vry observations to his summaries, showing an ability to reflect and a degree of irreverence that wasn’t much visible elsewhere.
We became friends of a sort, spending much time studying in the cramped basement of the doctoral student house at HBS. Like me, Clay had a family and biked to work, but he would be in much earlier than me. After lunch, he would take a nap in his carrel, lying on the floor with his feet on the office chair. I remember him coming to school one day, shaking his head: His son had dunked a basketball – and he was twelve years old.
Clay’s research was on the evolution of technologies, specifically on generations of hard disks, a project that eventually became the The Innovator’s Dilemma. I got to see how his theory developed through seminars, papers and discussions, including some blind alleys. I was in a different field, but was more interested in technology than most of my peers and think I was one of the people outside his department who early on thought his work interesting and understood the implications, though I do not think I contributed in any meaningful fashion aside from encouragement.
Clay graduated in record time and became a professor, and I needed a friendly face on my thesis committee – so I asked him. Eventually I graduated and moved back to Norway, but kept my consulting job in Boston and travelled there quite often. Clay became famous, and, cashing in a favor, I invited him over to Oslo to speak at BI Norwegian Business School. He came in January, on his way home from the World Economic Forum in Davos. It was cold and dark and he gave a lecture the audience referred to as “life-changing”. I asked him if there was anything he wanted to do in Norway. There was one town – Drammen – he had always wanted to visit, since his great grandfather had been repeatedly arrested there for being a Mormon missionary. (To put this in context: Coming to Oslo and wanting to see Drammen is equal to landing in New York and asking to see New Jersey.) So we went there, in my colleague Øystein Fjeldstad’s car. It was foggy and bitterly cold and Drammen was every bit as dreary as you can imagine. We went back to Oslo, dropped off Clay at the luxurious hotel we had booked for him and urged him to try the gourmet restaurant. When his expenses came in, it turned out he had gone to McDonald’s. Our CFO solemnly informed me that I was free to invite this guy anytime I wanted.
In 2007 I thought I had come up with a way to redeem myself and my country and arranged a “Disruptive Cruise” – a weeklong trip on Hurtigruten with Clay’s family. The idea was to create a nice experience for execs from interesting companies and for Clay to have a great vacation with his family and some good discussions. Economically it just did not work – a combination of an economic downturn, Norwegian executives’ unwillingness to spend a week away during what for them was summer vacation, and my listless performance as a salesperson meant that the whole thing became a highly personal, rather low-budget thing – but Clay and his large family liked it. Clay spent much of the time typing on his computer, but found time to see the midnight sun from the ship’s hot tub, and experience both the bridge and the machine room (where passengers are not normally allowed), in addition, of course, to the coasts and mountains of western and northern Norway.
My partner in crime for this trip was Trond Østgaard, who was chairman of the Hurtigruten Appreciation Society as well as a prominent citizen of Drammen. Hence, Clay and family could visit Drammen in style this time, being shown around the old City Hall (where his great-grandfather had been imprisoned) by the Vice-Mayor and have his photograph taken next to the portrait of the sheriff who had done the arrests.
We stayed friends, though we did not spend much time together. I would occasionally pop into his office at HBS, where we swapped anecdotes and talked about technology. I tended to come up with examples, he would think about how they would play out. We would discuss processor modularity, telecommunications competition, hospital management and (a lot) the coming disruption of business schools. Again, I don’t think I contributed much to his research, aside from coming up with a few examples and suggesting ways to communicate things (not that Clay needed any help there). Perhaps my main contribution was to introduce him to Øystein Fjeldstad (the third guy in the picture on top here), whose “value configurations” made it into a couple of Clay’s articles and books. The way to talk to Clay was not to explain things, but to state examples and wait (not long) for him to work out the consequences himself.
I would occasionally see him when he swung by Oslo for a talk, once or twice being the MC myself. But Clay was incredibly busy with audiences constantly demanding his disruption stories (“I am becoming my own theory here,” he would comment, ruefully), I stopped travelling so much to the US, so we saw less of each other. In one sense we were very different: Clay was deeply religious, I am an ateist, and I could never reconcile his scientific mind with his religious views. We talked about it on a few occasions, agreeing to disagree. Mostly, we would talk about our families and our job experiences, stepping back and seeing what it all meant. For all his fame, Clay was invariably down to earth, a great support for me when two of my children became seriously ill – and I believe I played at least a small part like that for him, too. Clay’s health was not good – diabetes, cancer, heart problems and a stroke that made it difficult for him to form words, but he never complained and kept working – a bit much if you ask me (and his family).
I learned a lot from Clay, and he (politely) said he had learned from me. I learned about how to think critically and clearly, and how to be principled and persistent when you believe in your analysis. In my career this has helped me understand that I should build my career on what I am good at and what I can and want to do, not what the organisations I work for see as the correct career path. His way of thinking has enabled me on a number of occasions to listen to what job the customer want done – not a very intellectual concept, but one that is surprisingly effective – and apply that both to offerings I have developed myself and to my analyses of various companies and industries. In my teaching, he has influenced me enormously – when I finally felt secure enough to teach technology in a business school through cases, and cases only, it was his course I started with.
I keep judging my ideas up against what he would have thought. Just a few days ago, I discussed an idea for a paper with Chandler Johnson, a colleague at BI: Is machine learning disruptive to traditional management research (or traditional research in general)? We swapped some ideas back and forth, and I suggested we type it up as a short outline and send it off to Clay to hear what he thought about it.
And now he is no more.
In addition to his management books, Clay wrote and spoke about how to evaluate your life – and said that in posterity, he would not be judged for being a famous business school professor, but for how he had helped other people.
I don’t believe in an afterlife, but I do think that as a person, you exist as long as somebody remembers you, how you were, and what you did for them. For me, at least, Clay will exist for the rest of my life, and, I am sure, for many of my students.
And my thoughts go to Christine, and to her and Clay’s children and grandchildren, who have lost infinitely more than the rest of us have and for whom that form of remembrance is a small consolation – but hopefully, a consolation nevertheless.
James May has bought a new car – a blue Tesla Model S.
Well, I never thought the day would come when I would be ahead of Top Gear, but I am actually on my third blue Tesla (fourth if you count the toy car) as this picture will testify:
(Incidentally, the Model S is for sale. Interested? You would have the same car as James May…)
Not sure I qualify as a petrolhead, since I have never owned an Alfa Romeo, but at least I can congratulate James May on a car purchase I fully agree with.
UPS – United Parcel Service – is a paragon of operational excellence in the United States. For all I know, they are excellent in other countries as well. In Norway, they are the worst transportation company you can think of – at least when it comes to administration and customer service.
I order quite a few things from abroad, and whenever the company I order from tell me they have sent it with UPS I groan – because I know the package will be late, the paperwork will be cumbersome, and the customer service slow.
Here is the latest example: I order an electric bicycle from Sweden, from Kringla.com. The price is given including Norwegian customs and taxes. I pay the net sum to the bicycle company, and UPS is supposed to send me a link with payment information, since they cannot send the package out before customs has been paid.
Except they did not send it until I called them after 10 days (“because someone forgot to send it” according to customer service.) I paid, the packaged was delivered (the driver was great, incidentally, carried the heavy package behind the house since we were away.)
Then, two weeks later, I get an invoice for the customs and taxes. I call in, and they say (after a long wait on the phone) that it is paid, but they send out an invoice not marked as paid in case I need it for documentation.
I have interviewed top management in UPS in the States, the company is known for excellent delivery services and innovation in logistics. In Norway – check out this page, with 33 reviews, all of them one star…
Can someone in top management in the US please contact the Norwegian organization and tell them to get a grip?
Warning: These are my notes from an ACM webcast. Misunderstandings, skips, jumps and errors (probably) abound. Caveat emptor.
The Power and Limits of Deep Learning,” presented on Thursday, July 11 at 1 PM ET/10 AM PT by Yann LeCun, VP & Chief AI Scientist at Facebook, Silver Professor at NYU, and 2018 ACM A.M Turing Award Laureate.
Deep Learning (DL) has enabled significant progress in computer perception, natural language understanding, and control. Almost all these successes rely on supervised learning, where the machine is required to predict human-provided annotations, or model-free reinforcement learning, where the machine learns policies that maximize rewards. Supervised learning paradigms have been extremely successful for an increasingly large number of practical applications such as medical image analysis, autonomous driving, virtual assistants, information filtering, ranking, search and retrieval, language translation, and many more. Today, DL systems are at the core of search engines and social networks. DL is also used increasingly widely in the physical and social sciences to analyze data in astrophysics, particle physics, and biology, or to build phenomenological models of complex systems. An interesting example is the use of convolutional networks as computational models of human and animal perception. But while supervised DL excels at perceptual tasks, there are two major challenges to the next quantum leap in AI: (1) getting DL systems to learn tasks without requiring large amounts of human-labeled data; (2) getting them to learn to reason and to act. These challenges motivate some the most interesting research directions in AI.
- supervised learning works, but requires too many samples
- convolutional networks: using layers to tease out compositional hierarchy
- other approaches: reinforcement learning,
- use convolutional networks and a few other architectural concepts, requires huge number of interactions with clearly defined universe – takes 80 hours to reach performance a human uses 15 minutes to reach. In the end, it does better than the human, but it takes a long time
- impractical for non-electronic settings (self-driving car would need to crash thousands of times
- better approach: (deep) multi-layer neural nets
- alternates linear/non-linear layers
- supervised machine learning, such as stochastic gradient descent
- figure out tweaking by computing gradients by back-propagation (automatic differentiation)
- architecture of neural networks – figure out sparse networks, not using all connections, based on research on visual cortex
- first using simple cells, then combining them
- convolutional neural network builds on this idea, but introduces back propagation
- turn on/off each neuron based on the portion it sees, then combine
- shows examples through the nineties, such as recognising numbers (for checks)
- neural networks out of fashion with AI researchers, realized that they could recognize multiple objects
- research on moving robots, did not need training data
- moving on to autonomous driving by classifying pixels
- 2010: Deep learning revolution, driven by speech recognition community
- largely responsible for lowering of errors in SR
- 2012: (Alexnet) Krizhevsy et al, NIPS 2012, other nets, large networks
- better and better performance, dramatic increase in number of layers
- current record: 84% image recognition
- trying to find the minimal architecture that gives performance
- Facebook: billions of pictures, each goes through 6 convnets
- Mask R-CNN: instance segmentation, two stage detection system, identifies areas of interest and send them to new networks
- RetinaNet: One-pass object recognition
- other works, recognizing background,
- image recognition, such as finding femurs (for hip ops) by taking in the whole 3D picture rather than using layers
- autonomous driving
- everyone uses convnets
- good for perception, not for reasoning
- for this: introducing working memory (differentiable associative memory), need to maintain a number of facts, “memory network”, a neural net with an attached network for memory, essentially soft RAM
- transformer networks, every unit is itself a neural network, works with translation (dynamic convolution)
- Facebook; dynamic neural nets: networks that put out networks
- Challenge: How can humans and animals learn so quickly?
- children learn largely by observation
- learn about gravity between 6 and 9 months, just by observation
- solution(?) self-supervised networks
- not task-directed, comprises most of our own learning (cake example)
- very large networks (see slide on process)
- works for speech recognition and text, filling in 15-20% of blanks in text
- does not work for filling in missing parts of images (yet)
- works partly for speech recognition
- summary: works with discrete data (text, partly speech), much more difficult with continuous data, because we do not have good ways of parameterization
- predicts the average of all possible futures, results in blurry images…
- Adversarial training: prediction under uncertainty:
- generator that makes prediction, discriminator that determines whether it is good or not
- works well for generating images of people that don’t exist, clothes that has not been designed yet
- important with video prediction for self-driving cars, that is where the demand is
- Self-supervised forward models: training self-driving cars to predict it environment by adding latent variables, randomly sampled
- Final slide: Theory follows invention, will deep learning result in a theory of intelligence?
- children learn largely by observation
(did not take notes during question session, should have don (might add them later), talk available at learning.acm.com)
For the past two days I have run a seminar on case teaching for faculty at the University of Stavanger Business School, by initiative and invitation from Ken Wathne. (The picture above was taken at 11pm, which tells you something about summer in Norway and the gardening at UiS.)
I love case teaching (obviously) and think it is not only a very useful tool to use and skill to have – but also a way for business schools to future-proof their business model. In a time of Youtube and learning-on-demand, case teaching offers a way to analyse and learn real leadership skills in complex decision situation – something that cannot easily be automated, like standard business courses.
Here are some links to things I have used in the seminar:
- A useful article on what case teaching is (because many people get this wrong). [My comments on this one is in Norwegian, but by all means read the original]: Garvin, David (2003): “Making the case“, Harvard Magazine.
- A useful article: Christensen, Clayton C. and Michael Raynor (2003) “Why hard-nosed executives should care about management theory”, Harvard Business Review. Well-written article on how to create theory – and why it is important to understand what works in some contexts does not work in others. I use this article as a rationale for using case teaching to test theory (simulated) practice – case teaching is simulation of decision situations, and is great for finding out if theories are theories or mere beliefs and buzzwords.
- Case sources: I primarily use the Harvard Business Press case collection. Anyone teaching at a degree-granting institutions has access (you will have to register with an email address belonging to an institution and be able to point to an affiliation, such as a personal web page at the institution.) There are other case sources too – here is a list.
- Material for students on how to prepare. I like Bill Ellet’s Case Study Handbook, but it can be a bit much for smaller courses or if you use cases only sparingly. HBSP allows you to buy just some its chapters, so that is an alternative. With my colleague Hanno Roberts, I have made a video series for students, originally for BIs MBA program with Fudan University in Shanghai. The videos are a bit long and rambling, but you can always make your own. For complicated cases I have a note on how to analyze complex case using a structured time line.
- A blog post on how to do effective student feedback.
- Course design: You can find many examples on the Interwebs, so I steal with pride. Here are some of my own: GRA6834 Business Development and Innovation Management (M.Sc., one semester, weekly sessions), GRA68175 Technology Management and Disruptive Technology (two day module, EMBA), and my memo to students for my course Strategic Technology and Information Management in Shanghai (four day module, more structured and instrumental than a usual case course.)
- Writing cases: A blog post on how to write a teaching case. You can, of course, use anything – articles, blog posts, videos – as a basis for discussion. Most important is that there is room for real discussion – that there is enough detail, that the material illustrates what you want to get across in a way that is not obvious and allows for interpretations and complexity.
I have come to learn that there are no boring industries – one always finds something interesting in what at first may looks fairly mundane. And that is something I am trying to teach my students, as well.
Andrew Camarata is a young man who works for himself with excavators, bulldozers, gravel, stone, earthworks and so on. He lives and runs his business in the Hudson Valley just south of Albany, New York and in the winter he does, among a lot of other things, snow plowing.
In this video, he will tell you almost everything there is to know about how to plow snow commercially in rural United States and make money from it.
The interesting point about this video (and a lot of other videos he has made, he has a great following on Youtube) is that he provides a very thorough understanding of business design: In the video, he talks about acquiring and maintaining resources, understanding customers (some are easy, others difficult, you need to deal with both), administration and budgeting, ethics (when to plow, when not to), and risk reduction (add the most complicated jobs with the greatest risk of destroying equipment last in the job queue, to reduce the consequences of breakdowns).
For a business student, this is not a bad introduction to business, and Camarata is certainly a competent businessman. In fact, I see nothing here that is not applicable in any industry.
When it also comes in a pedagogically and visually excellent package, what’s not to like?
Case teaching is not just for the classroom – increasingly, you can (and some schools do) offer discussion-based (or, at least, interaction-courses online.) My buddy Bill Schiano and I wrote a long note on how to do this for Harvard Business Publishing back in 2017 – and I then completely forgot about it.
Recently, I rediscovered it online, published by HBP – so, here goes: Schiano, Bill and Espen Andersen (2017): Teaching with Cases Online, Harvard Business Publishing. Enjoy. (More resources available at the HBP Teaching Center page.)
Incidentally – should you (as a school) consider doing courses this way, beware that this is not a cost reduction strategy. You probably will need to pay the online teachers more than those doing regular classrooms, simply because it is more work and quite a bit more design, at least in the beginning. But it may be an excellent way of reaching student groups you otherwise could not reach, for geographical or timing reasons.
I teach a M.Sc. class called “business development and innovation management”, and challenge students to write Harvard-style cases about real companies experiencing issues within these areas. The results are always fun and provide learning opportunities for the students: You may not provide the answer for the company, but you get a chance to really learn about one company or one industry and dive into the complexities and intricacies of their situation. That knowledge, in itself is valuable when you are hitting the job market.
Here is a (fairly anonymized) list of this year’s papers:
- disruption in the analytics industry: One group is studying SAS Institute and how their closed software and architecture model is being challenged by open-source developments
- disruption in the consulting industry: One group wants to study a small consulting company and how they should market some newly developed software that allows for automated, low-cost analysis
- establishing a crypto-currency exchange: One group wants to study strategies for establishing a payment and exchange service for crypto-currencies
- marketing RPA through a law firm: One group wants to study how a large law firm can market their internal capabilities for RPA (robotics process automation) in an external context
- fast access to emergency services: One group wants to write a case on Smarthelp and how that service can be spread and marketed in a wider context
- using technology to manage sports club sponsorship: One group wants to study how to develop strategy for a startup company that helps participation sports clubs with gain corporate sponsorships
- electronic commerce and innovation in the agricultural equipment sector: One group wants to study how a vendor of farm equipment and supplies can extend their market and increase their innovative capability through ecommerce and other digital initiatives
- machine learning in Indian banking: One group wants to study how machine learning could be used to detect money laundering in a large Indian bank
- social media analysis in consumer lending: One group wants to study an Indian startup company that uses digital indicators from users’ online behavior to facilitate consumer financing for online purchases
Al in all, a fairly diverse set of papers – I am looking forward to reading them.
Together with Chandler Johnson and Alessandra Luzzi, I currently teach a course called Analytics for Strategic Management. In this course (now in its third iteration), executive students work on real projects for real companies, applying various forms of machine learning (big data, analytics, whatever you want to call it) to business problems. We have just finished the second of five modules, and the projects are now defined.
Here is a (mostly anonymised, except for publicly owned companies) list:
An IT service company that provides data and analytics wants to predict customer use of their online products, in order to provide better products and tailor them more to the most active customers
A gas station chain company wants to predict churn in their business customers, to find ways to keep them (or, if necessary, scale down some of their offerings)
A electricity distribution network company wants to identify which of their (recently installed) smart meters are not working properly, to reduce the cost of inspection and increase the quality of
A hairdressing chain wants to predict which customers will book a new appointment when they have had their hair done, in order to increase repeat business and build a group of loyal customers
A large financial institution wants to identify employees that misuse company information (such as looking at celebrities’ information), in order to increase privacy and data confidentiality
NAV IT wants to predict which employees are likely to leave the company, in order to better plan for recruitment and retraining
OSL Gardermoen want to find out which airline passengers are more likely to use the taxfree shop, in order to increase sales (and not bother those who will not use the taxfree shop too much)
- a bank wants to find out which of their younger customers will need a house loan soon, to increase their market share
- a TV media company wants to find out which customers are likely to cancel their subscription within a certain time frame, to better tailor their program offering and their marketing
- a provider of managed data centers wants to predict their customers’ energy needs, to increase the precision of their own and their customers’ energy budgets
- Ruter (the public transportation umbrella company for the Oslo area) wants to build a model to better predict crowding on buses, to, well, avoid overcrowding
- Barnevernet wants to build a model to better predict which families are most likely to be approved as foster parents, in order to speed up the qualification process
- an electrical energy production company wants to build a model to better predict electricity usage in their market, in order to plan their production process better
All in all, a fairly typical set of examples of the use of machine learning and analytics in business – and I certainly like to work with practical examples with very clearly defined benefits. Over the next three modules (to be finished in the Spring) we will take these projects closer to fruition, some to a stage of a completed proposal, some probably all the way to a finished model and perhaps even an implementation.
Since I sometimes give classes on how to do case teaching and have written a book on the subject with Bill Schiano, I am sometimes asked how to write a teaching case. The following blog post is a quick tips & tricks collection (essentially an update of this post.)
Why are you writing this case?
First and foremost: Cases are written for a teaching purpose – and to write a teaching case, you need to have a teaching objective in mind. It is not enough to have an interesting company. Even the best company story needs to have a pedagogical point, a theory or dilemma to illustrate. So don’t write a teaching case just because you happen to know someone in a really interesting company – it does need to be a good story, but it also need to have a purpose.
The standard outline
Cases – particularly the standard HBS case – follow an outline that can seem rather trite, but which is very effective. It is something like this:
- 0.5 page: Intro: The protagonist is introduced, typically pondering a question of some importance. The idea is to tell the students from which perspective the case is written, to set the scene – and that is all there is to it.
- 1 – 1.5 pages: Description of the company – not the whole history, but the relevant details, explaining what the company is doing, how they make their money. Most companies are to a very large degree formed by their history, so the relevant parts need to be told.
- 1 page: Industry. Companies exist within a context, and you need to set it. Explain the industry, its evolution, and the company’s position within it. Do it succinctly, but leave more detail in than what is strictly necessary.
- 1 – 5 pages: Specific issue. This is the meat of the case, the issue at hand, the story to ponder. Make sure you tell it logically and cooly, not leaving anything out, but also conveying the complexity of the situation.
- 0.5 page: Conclusion, typically with the protagonist wondering what to do, often with some sort of event (board meeting, etc.) where he or she has to present a solution to the problem.
Most cases are just that – one case. You can have a B case and even a C case, but keep them short, since they have to be handed out and read in class. The B case should explain what the company did and perhaps introduce a new problem, the C case, if necessary, should bring some sort of closure, explaining what eventually happened. In my experience, it is very hard to get discussion after a C case – the students become exhausted. As a novice case writer, especially if you are writing about a company with a long history, it can be tempting to create a long string of small cases, but in practice this seldom works well – for one thing, it forces the discussion into a very predictable path.
No theory. A good case should be a description of an interesting situation, frequently a decision point – and nothing else. This means that there should be no theory and no discussion of the case in the case itself. Save that for the teaching note, or write a separate academic article about it. Not only does this make the case more realistic, it also means it can be used for more purposes than the one initially envisioned. This can be quite challenging for the traditional academic writer – but it is actually good practice to only present the facts (though, of course, which facts you choose to present constitute a discussion of sorts).
No hero, no villain. When teaching students how to analyze a case, I always start by saying that for most business situations, if is useful to begin the analysis with the assumption that people are not stupid and not evil. Consequently, when you write a case, make sure it has no heroes and no villains. If a case has a clear-cut hero or villain, it is a sign that you have not done enough research. Write things so that the students can see the issue from many perspectives.
No judgement. I frequently find, when reading prospective teaching cases, especially if written with the help of the communications department of a company, that judgements tend to be embedded in the text itself, though it is supposed to be as neutral and purely descriptive as possible. Do not write that a company is “leading” – instead, describe what they do, perhaps adding a bit of industry numbers for comparison, and let the decision about how leading the company is be up to the student. Do not write “Smith was a highly accomplished manager, leading a successful technology implementation.” Instead, write “Smith, at 29, was promoted to be the youngest vice president in company history after conceiving and overseeing the introduction of the first machine learning platform in the industry.”
No consultingese. Be very careful of weasel terms – is the company embarking on a strategic alliance or merely collaborating with another company for a specific purpose? (As a friend of mine said: “I buy all my socks at Marks and Spencer. That does not mean I have a strategic alliance with them.”) Avoid terms that are not well defined, and be precise in your language. Remember, a teaching case has the longest legs when it describes a human situation (since humans change slowly) and to not tie the narrative to specifically to a technology or a situation. (See the example of Fabritek below.) I quite often use old technology cases, and when the students complain, ask them to take out a pen and change the dates mentioned to current times, update the technology capabilities accordingly – and see if anything else changes.
A really well written case has dramatic structure – there is a beginning, a middle that builds up the story, and a really compelling issue at the end. The best cases are almost like a detective story, where you have to dig deep into the analysis to find surprising and sometimes counter-intuitive conclusions. One example of a “detective story” case is Fabritek 1992*, a very old (first published 1969, rewritten by Jan Hammond) case about a quality control issue in a small mechanical workshop. (Hat tip to Robert D. Austin, eminent case teacher, for making me aware of this case and showing me how to teach it.) The case is excellent because it starts with the company (strategic level), proceeds to describe a new situation and a new process (organizational or business logic level) and then introduces the problem (operational level.) Analyzing the operational details leads to one conclusion, which then can be discussed in terms of the organization and its business logic, which can then be placed into a strategic context. The case is excellent because it allows links between these levels – and also teaches the students that the devil indeed resides in the details, and that you as a manager better be very close to how the business you are leading works and makes money.
A second case which shows quality and innovation is iPremier, written by Robert D. Austin and Jeremy C. Short, the first and only graphic novel (cartoon) case I am aware of. The story is about a small online gift company being attacked by hackers, exposing glaring gaps in their security procedures and forcing managers at various levels to make some really hard decisions. The graphic format is excellent in making the various characters real (though they, on average, tend to be way too good-looking for a normal business situation), illustrates technical issues in a way that is very understandable even by non-technology students, and has a cracking good storyline with a B and a C case. I like to introduce a few technical cases in my courses because, well, I don’t think there is enough technology in business schools, and this cases answers very well because it illustrates that certain technical decisions very much require top management attention – ignore (or mindlessly delegate) technology understanding and responsibility at your peril. The graphic format also provides a welcome break from the standard case verbiage, which can be a trifle dour on occasion.
Details, details, details!
Research cases – the kind that is published in refereed journals – tend to be written from a very specific viewpoint, and only facts pertaining to that perspective is included, often in a very abstract format. A teaching case is the direct opposite: It needs lots of details, frequently made available as exhibits (graphs, pictures, documents, tables, etc.) placed at the end, after the main text. A teaching case writer, when visiting a company to write about it, needs to notice the small details, much like a really good journalist does. I tell my students that they should prepare each case so well that they feel like they have worked in the case company – and to allow them to do that, you need to provide the operational details necessary. (Incidentally, having more details than strictly necessary has the added benefit of making the case realistic – in the real world, you have to decide what is important and what is not.)
Doing it – and reading about it.
I am not aware of many books about how to write a good teaching case, with one exception: Grandon Gill (pictured), professor at University of South Florida and an excellent case teacher, has written a book called Informing with the case method, which is available for free download in PDF, MOBI and EPUB format from his web site. It has lots of details, tips and tricks, not just about case writing, but also about case teaching and course planning. (For the latter, of course, I am duty bound to recommend Bill Schiano’s and my book Teaching with Cases: A Practical Guide.)
Last but definitely not least: Don’t underestimate how much work writing a proper business case is. Getting the details right, describing the dramatis personae, and making the storyline compelling is quite a challenge, in many dimensions different from the traditional academic article. On the other hand, should you get it right, you will have a very effective teaching tool for many years to come.