Category Archives: Business development

Analytics IV and V: Projects

asm_topLast year (with Chandler Johnson and Alessandra Luzzi) and this year (with Chandler, Jadwiga Supryn and Prakash Raj Paudel), I teach a course called Analytics for Strategic Management. In this course executive students work on real projects for real companies, applying various forms of machine learning (big data, analytics, whatever you want to call it) to business problems. Here is a list (mostly anonymised, except for public organizations) list from this year:

  • One group wants to use machine learning to predict fraud in public security contracts in a developing country
  • A credit agency wants to predict which of their customers will pay their bills by the end of the month
  • An engineering company wants to predict the number of hours needed to meet demand for each month in each department
  • One group wants to predict housing prices within Oslo, to help house sellers get a realistic estimate of what their property is worth
  • A higher education provider wants to predict which students are likely to fail or not qualify for an exam, to be able to intervene early
  • A couple of municipalities want to predict who will accept a kindergarten allocation or not
  • A telecommunications company wants to predict which customers will churn
  • An Internet product company wants to predict necessary capacity for picking and shipping work every day
  • One group wants to predict the likelihood of a road closing due to bad weather, in order to warn truck drivers so they can detour
  • One group wants to predict the future financial health of companies based on employee engagement numbers
  • One group wants to predict efficiency of production in a wind power park

And last year we had these projects:

  • An investment company wanted to predict bankruptcies from media events
  • Ruter, Oslo’s public transportation authority wanted to predict the number of passengers (for each station, to great precision) for one line on the metro
  • A telecommunications company wanted to predict customer feedback scores from analyzing customer interactions (so the customer does not have to answer a survey afterwards)
  • The Norwegian Health directorate wanted to predict general physician “fastlege” churn
  • A commercial TV station wanted to predict subscriber churn
  • An insurance company wants to identify customers likely to buy a group insurance package
  • An online gaming company wanted to predict customer churn
  • A large political party wanted to predict membership churn
  • One group wanted to start a company based on using machine learning to diagnose hearing problems
  • A large retail chain wanted to predict churn based on customer purchase patterns

Dealing with cheating

At BI Norwegian Business School, we are (naturally and way overdue, but a virus crisis helps) moving all exams to digital. This means a lot of changes for people who have not done that before. One particular anxiety is cheating – normally not a problem in the subjects I teach (case- and problem oriented, master/executive, small classes) but certainly is an issue in large classes at the bachelor level, where many answers are easily found online, the students are many, and the subjects introductory in nature.

Here are some strategies to deal with this:

  • Have an academic honesty policy and have the students sign it as part of the exam. This to make them aware of they risk if they cheat.
  • Keep the exam time short – three hours at the max – and deliberately ask more questions than usual. This makes for less time for cheating (by collaborating) because collaboration takes time. It also means introducing more differentiation between the students – if just a few students manage to answer all questions, those are the A candidates. Obviously, you need to adjust the grade scale somewhat (you can’t expect all to answer everything) and there is an issue of awarding students that are good at taking exams at the expense of deep learning, but that is the way of all exams.
  • Don’t ask the obvious questions, especially not those asked on previous exams. Sorry, no reuse. Or perhaps a little bit (it is a tiring time.)
  • Tell the students that all answers will be subjected to an automated plagiarism check. Whether this is true or not, does not matter – plagiarism checkers are somewhat unreliable, have many false positives, and require a lot of afterwork – but just the threat will eliminate much cheating. (Personally, I look for cleverly crafted answers and Google them, amazing what shows up…).
  • Tell the students that after the written exam, they can be called in for an oral exam where they will need to show how they got their answers (if it is a single-answer, mathematically oriented course) or answer more detailed questions (if it is a more analysis- or literature oriented course). Who gets called in (via videoconference) will be partially random and partially based on suspicion. Failing the orals results in failing the course.
  • When you write the questions: If applicable, Google them, look at the most common results, and deliberately reshape the questions so that the answer is not one of those.
  • Use an example for the students to discuss/calculate, preferably one that is fresh from a news source or from a deliberately obscure academic article they have not seen before.
  • Consider giving sub-groups of students different numbers to work from – either automatically (different questions allocated through the exam system) or by having questions like “If your student ID ends in an even number (0,2,4,6,8) answer question 2a, otherwise answer question 2b” (use the student ID, not “birthday in January, February, March…” as this will be the only marker you have.) The questions may have the same problem, but with small, unimportant differences such as names, coefficients or others. This makes it much harder to collaborate for the students. (If you do multiple questions in an electronic context, I assume a number of the tools will have functionality for changing the order of the questions – it would, frankly, astonish me if they did not – but I don’t use multiple choice myself, so I don’t know.
  • Consider telling the students they will all get different problems (as discussed above) but not doing it. It still will prevent a lot of cheating simply because the students believe they all have different problems and act accordingly.
  • If you have essay questions, ask the students to pick a portion of them and answer them. I do this on all my exams anyway – give the students 6 questions with short (150 words) answers and ask them to pick 4 and answer only those, and give them 2 or 3 longer questions (400 words or so) and ask them to answer only one. (Make it clear that answering them all will result in only the first answered will be considered.) Again, this makes cheating harder.

Lastly: You can’t eliminate cheating in regular, physical exams, so don’t think you can do it in online exams. But you certainly can increase the disincentives to do so, and that is the most you can hope for.

Department for future ideas
I have always wanted to use machine learning for grading exams. At BI, we have some exams with 6000 candidates writing textual answers. Grading this surely must constitute cruel and unusual punishment. With my eminent colleague Chandler Johnson I tried to start a project where we would have graders grade 1000 of these exams, then use text recognition and other tools, build an ML model and use that to grade the rest. Worth an experiment, surely. The project (like many other ideas) never took off, largely because of difficulties of getting the data, but perhaps this situation will make it possible.

And that would be a good thing…

A teaching video – with some reflections

Last Thursday, I was supposed to teach a class on technology strategy for a bachelor program at the University of Oslo. That class has been delayed for a week and (obviously) moved online. I thought about doing it video conference, but why not make a video, ask the students to see it before class? Then I can run the class interactively, discussing the readings and the video rather than spending my time talking into a screen. Recording a video is more work, but the result is reusable in other contexts, which is why I did it in English, not Norwegian. The result is here:

To my teaching colleagues: The stuff in the middle is probably not interesting – see the first two and the last five minutes for pointers to teaching and video editing.

For the rest, here is a short table of contents (with approximate time stamps):

  • 0:00 – 2:00 Intro, some details about recording the video etc.
  • 2:00 – 27:30 Why technology evolution is important, and an overview of technology innovation/evolution processes
    • 6:00 – 9:45 Standard engineering
    • 9:45 – 12:50 Invention
    • 12:50 – 15:50 Structural deepening
    • 15:50 – 17:00  Emerging (general) technology
      • 17:00 – 19:45 Substitution
      • 19:45 – 25:00 Expansion, including dominant design
      • 25:00 – 27:30 Structuration
  • 27:30 – 31:30 Architectural innovation (technology phases)
  • 31:30 –  31:45 BREAK! (Stop the video and get some coffee…)
  • 31:45 – 49:40 Disruption
    • 31:45 – 38:05 Introduction and theory
    • 38:05 – 44:00 Excavator example
    • 44:00 – 46:00 Hairdresser example
    • 47:00 – 47:35 Characteristics of disruptive innovations
    • 47:35 – 49:40 Defensive strategies
  • 49:40 – 53:00 Things take time – production and teaching…
  • 53:00 – 54:30 Fun stuff

This is not the first time I have recorded videos, by any means, but it is the first time I have created one for “serious” use, where I try to edit it to be reasonably professional. Some reflections on the process:

  • This is a talk I have given many times, so I did not need to prepare the content much – mainly select some slides. for a normal course, I would use two-three hours to go through the first 30 minutes of this video – I use much deeper examples and interact with the students, have them come up with other examples and so on. The disruption part typically takes 1-2 hours, plus at least one hour on a specific case (such as the steel production). Now the format forces me into straight presentation, as well as a lot of simplification – perhaps too much. I aim to focus on some specifics in the discussion with the students.
  • I find that I say lots of things wrong, skip some important points, forget to put emphasis on other points. That is irritating, but this is straight recording, not a documentary, where I would storyboard things, film everything in short snippets, use videos more, and think about every second. I wanted to do this quickly, and then I just have to learn not to be irritated at small details.
  • That being said, this is a major time sink. The video is about 55 minutes long. Recording took about two hours (including a lot of fiddling with equipment and a couple of breaks). Editing the first 30 minutes of the  video took two hours, another hour and a half for the disruption part (mainly because by then I was tired, said a number of unnecessary things that I had to remove.)
  • Using the iPad to be able to draw turned out not to be very helpful in this case, it complicated things quite a bit. Apple’s SideCar is still a bit unpredictable, and for changing the slides or the little drawing on the slides I did, a mouse would have been enough.
  • Having my daughter as audience helps, until I have trained myself to look constantly into the camera. Taping a picture of her or another family member to the camera would probably work almost as well, with practice. (She has heard all my stories before…)
  • When recording with a smartphone, put it in flight mode so you don’t get phone calls while recording (as I did.) Incidentally, there are apps out there that allow you to use the iPhone as a camera connected to the PC with a cable, but I have not tested them. It is easy to transfer the video with AirPlay, anyway.
  • The sound is recorded in two microphones (the iPhone and a Røde wireless mic.) I found that it got “fatter” if I used both the tracks, so I did that, but it does sometime screw up the preview function in Camtasia (though not the finished product). That would also have captured both my voice and my daughter’s (though she did not ask any questions during the recording, except on the outtakes.)
  • One great aspect of recording a video is that you can fix errors – just pause and repeat whatever you were going to say, and the cut it in editing. I also used video overlays to correct errors in some slides, and annotations to correct when I said anything wrong (such as repeatedly saying “functional deepening” instead of “structural deepening”.) It does take, time, however…

My excellent colleague Ragnvald Sannes pointed out that this is indicative of how teaching will work in the future, from a work (and remuneration) perspective. We will spend much more time making the content, and less time giving it. This, at the very least, means that teachers can no longer be paid based on the number of hours spent teaching – or that we need to redefine what teaching means…

Practical business development

I have come to learn that there are no boring industries – one always finds something interesting in what at first may looks fairly mundane. And that is something I am trying to teach my students, as well.

Andrew Camarata is a young man who works for himself with excavators, bulldozers, gravel, stone, earthworks and so on. He lives and runs his business in the Hudson Valley just south of Albany, New York and in the winter he does, among a lot of other things, snow plowing.

In this video, he will tell you almost everything there is to know about how to plow snow commercially in rural United States and make money from it.

The interesting point about this video (and a lot of other videos he has made, he has a great following on Youtube) is that he provides a very thorough understanding of business design: In the video, he talks about acquiring and maintaining resources, understanding customers (some are easy, others difficult, you need to deal with both), administration and budgeting, ethics (when to plow, when not to), and risk reduction (add the most complicated jobs with the greatest risk of destroying equipment last in the job queue, to reduce the consequences of breakdowns).

For a business student, this is not a bad introduction to business, and Camarata is certainly a competent businessman. In fact, I see nothing here that is not applicable in any industry.

When it also comes in a pedagogically and visually excellent package, what’s not to like?

Beyond Default

71gkby-vpilDavid Trafford and Peter Boggis are those kinds of under-the-radar strategy consultants that ever so discreetly (and dare I say, in their inimitable British way) travel the world, advising enormous companies most civilians have never heard of about such issues as how to organise your internal departments so that they are capable of responding to technical change. (I should know, because I worked with them, first in CSC and then in the Concours Group, between 1994 and 2009.)

Now David and Peter have begot a book, Beyond Default, that provides a perspective on strategy and organisational change less built on fashionable frameworks than on solid experience. Their focus is on how organisations fail to see changes in their environment and develop strategies – real strategies – to adapt to them. The reasons are many, but most important is the fact that organisations have developed processes and measures to do what they currently do, and the focus on those particulars does not permit stepping back and seeing the bigger picture. Instead, companies carry on towards a “default” future – and, crucially, that future may be declining. Companies need to know what they don’t know and what they do not have the capabilities to do – and to acquire those capabilities when necessary. To do that, the authors advocate experiential learning – seeing for yourself what the future looks like by seeking it out, preferably as a group of managers from the same organisation experiencing and reflecting together.

The authors have a background as IT consultants, and it shows: They very much think of organisations as designed systems, with operating practices and (ideally) articulated operating principles. While eminently logical, this way of organising is hard to do – among other things, it requires thinking about organisations as tools for a purpose, and that purpose has to be articulated in a way that gives direction to its members. Thinking about your principles can make you articulate purpose, but it is very hard not to make the whole process a bit self-referential. Perhaps the key, like for Newton’s second law of thermodynamics, is to keep adding external energy, constantly identifying and understanding ramifications of technical and other change – a process that requires energy, if nothing else.

Both authors care about language and explaining and discussing what happens in a way that can be understood by the organisations they are trying to help. This means that they primarily use examples and stories, rather than frameworks (beyond simple illustrations), to convey their points. They end each chapter with a set of questions the reader can has him- or herself about the organisations they manage – and do not, in any way, try to offer simple solutions. As such, the book works best when it talks about how to explain strategic necessities and start on a strategic journey – through collective leadership, not “great man” charisma. It works less well when trying to explain strategic analysis, perhaps because the authors have too much experience to settle on a simple, all-encompassing method.

Well worth the read, not least for the senior executive trying to understand a new world and wanting an explanation held in a language that fosters understanding rather than just excitement.

Accenture on Cognitive Computing

(Notes from an Executive Short Program called Digitalization for Growth and Innovation, hosted by Ragnvald Sannes and yours truly, in Sophia Antipolis right now.Disclaimer: These are my notes, I am writing fast and might get something wrong, so nothing official by Accenture or anyone else.)

Cyrille Bataller is a managing director and the domain lead for Intelligent Computing, with the Emerging Technology group at Accenture. He leads Accenture’s exploration of the Cognitive Computing field

Cognitive Computing

Skilled work increasingly done by machines – doctors, lawyers, traders, professors etc. Large potential impact, also societal, but exciting new technology. Cognitive computing is IT systems that can sense, comprehend and act – and are perhaps the most disruptive technology on the horizon. They can interact with their environment, learn by training instead of coding, and can analyse and use enormous amounts of data. The quest for cognitive computing has a long history – and has had a varied history, with periods of advances and periods of setbacks, or at least less interest. We differentiate between “strong” AI – achieving consciousness – and “weak AI” – having the machine mimic certain capabilities of humans.

Accenture’s ambition has been to create a toolbox of cognitive computing capabilities under a single architecture, such as text analytics, research assistants, image analysis, multimedia search, cognitive robots, virtual agents, expert systems, video analytics, identity analytics, speech analytics, data visualisation, domain-specific calculations, recommendation systems and self-adjusting IT systems. These are all examples of human traits and capabilities as well…

Some detail – image analysis is about deep learning, using neural networks to mimic how the brain works. Use layers of increasingly complex rules to categorize complex shapes such as faces or animals. In one case, they used neural networks to recognize and classify images of cars as undamaged, some damage or totalled to an accuracy of 90%. By applying text analysis as well, this could be used by an insurance company to analyze insurance claims. Google has acquired a company called DeepMind to look at images on the web, eventually recognizing cats.

Another example is use of robotics in business operations – from minibots, companies such as XL and AutoHotKey to automate application software, to standard robots to using virtual assistants to do user support, for instance. You can have a team of robots in the cloud to work alongside your real back office. These robots can observe and learn and gradually take over or optimize these activities, such as updating documents (powerpoints, for instance). Could be used to observe how an accountant works, for instance, and issue certification based on following proper procedures. You could have 20-30 people in the back office and 100s of robots learning from them and doing the regular stuff.

Virtual agents: Amelia, handling a missing invoice situation. Amelia can understand, learn and solve problems. Generates a flowchart which can then be optimized and analyzed.

Video analytics is another application that is receiving lots of attention. One problem with CCTV videos is that they are almost only used forensically – checking a certain date and time – because nobody has time to watch the videos themselves. Can be used to count cars and recognize events, measure queuing times at customs and immigration, measure service times, monitor for lost objects on a train, analyze the age distribution on public transport, detect emotions, understand the hot and cold zones in the shopping center, find leaks in an industrial setting, etc.

The following framework (from the perspective document referenced below) shows a four-quadrant framework that can help guide companies in choosing the approach towards cognitive computing:


See for more information and a point of view document, written by Cyrille Bataller and Jeanne Harris.