Benno’s appointment as VP Engineering, for all its good sides, was also part of Steve’s divide-and-rule approach – in this case designed to side-line me. It meant that Steve established a direct link with Benno bypassing me. I could also feel a distance developing between Benno and myself, which particularly saddened me, given that he was my star student and like a son to me. Whether this was the result of Benno thinking he had to actively compete with me, or it was Steve actively bad-mouthing me, I can’t tell (but I’ve seen a lot of bad-mouthing of third parties to consider the latter entirely possible).

Of course, there was also a (to a degree inevitable) clash between purity and engineering realities. And I felt that I had to give Benno the freedom to fulfil his role and earn the respect of the team, so I was very careful to avoid seeming at odds with him or undermining him. My confidence in Benno’s technical insight made this easier.

But one of the effects was that I was increasingly excluded from technical design discussions and decisions.

Circumstances made this relatively easy. Part of OK’s deal with NICTA was that the outcome of the still progressing research program (the seL4 microkernel and its formal verification) would go to OK, and I was to oversee the completion of that program. So I was 50% OK Labs CTO and 50% group leader in NICTA. As a part-timer I inherently couldn’t be running too much on a day-by-day basis.

In hindsight, this arrangement was a grave mistake. By the time seL4 was ready, OK had had to build its market and products on the existing technology, and wasn’t the right vehicle for marketing seL4. (That’s not the full story, but more about that later). We should have left seL4 out of the deal, and I should have either had a stronger day-to-day involvement or just be a consultant. But that’s with the benefit of hindsight…

So, my part-time status helped Steve to remove me from most decision-making. But I was excluded from too many other things too.

Many of the personal highlights of my OK experience were talking to customers, be it engineers, VPs or CTOs. I learned a lot about real-world trade-offs, market drivers and technical insights. And my presence helped the company’s reputation. One memorable example was a technical meeting at Nokia, with engineers and CTO-office staff, including a known virtualisation sceptic. As Abi told me afterwards, that sceptic after the meeting commented “he’s the smartest CTO I’ve met, and totally devoid of bullshit.”

But bullshit-freedom isn’t an asset in Steve World. In fact, he thought he could do anything I could do (just better, of course). For example he seemed to truly believe that having sat through one of my presentations, he could do it himself just as well. The few times I observed him doing this, I felt like hiding in a corner for embarrassment, and did my best to prevent damage (typically involving kicking him under the table). But it was surely embarrassing, and any engineers around couldn’t help noticing that. Steve simply can’t keep his mouth shut when he thinks he knows something, even if there are real experts around. I can only imagine what went on when he was on his own.

And I was around less and less. Once Steve hired a VP Sales (who sang to Steve’s song book without missing a beat) I was generally not taken to customer meetings any more. This was partially compensated by some of our top engineers moving into sales, such as Abi, another one of my star students. At least they knew what they were talking about, but generally weren’t in a position to take a stand against bullshit. The rare exceptions included field-application engineer Josh (also a former student) and sales guy Tony. The two of them brought in the biggest non-Qualcomm contract during my time at the company (from Motorola). In fact the two were the only successful sales team we ever had (despite Steve heaping praise on the VP Sales at the Board whenever they managed not to completely mess up a sale). But they didn’t take any bullshit, so they were pushed out and massively bad-mouthed. This became a pattern.

There were instances crying for my involvement, but mostly I only learned about them afterwards (or not at all). At one stage, several opportunities started to develop in China. One of our engineers, native Chinese, who was involved in the technical evaluations told me later “it’s because of you, you’re well-known in China” (something I was quite oblivious to, but had later independently confirmed). The natural reaction should have been to fly me over and talk about the technology and vision. But no, Steve and his VP-Sales clone knew better. In the end, none of the China opportunities went anywhere.

Another case was DoCoMo, the main Japanese mobile operator. They had, in a 2006 white paper co-authored by Intel proposed the concept of the dual-persona phone (two logical phones, work and private, on a single physical handset, separated by virtualisation). In 2009 they were looking at developing one. There was a competitor in play, whose only advantage over us was a security evaluation (similar to Common Criteria but extremely low-grade – glorified tyre-kicking). DoCoMo was going to go with us if we could tick that box. However, for us it would have been insane to go through the expense of an essentially worthless security “evaluation”. By the time I heard about it, they had been talking in circles for 6 months! I joined the next phone conference, and within an hour we had a solution. Too late, as it turned out, things had changed internally in DoCoMo and the project never happened.

There were plenty of other opportunities where I never got a chance to contribute. Besides Steve’s systematic side-lining of myself, a contributing factor was the cultural ignorance of Chicago (i.e. OK headquarters). Steve in particular prides himself of “understanding” other cultures, but, like is technical “insights”, this is very superficial stuff he’s read in mags and blogs. He utterly fails to understand the appeal of technical authority in continental European as well as Asian cultures. People there have high respect for professors and value their opinions, especially where the professor is strongly identified with a particular technology (as is the case with myself and the microkernel technology that formed the basis of OK products).

As such, the standard approach should have been to get me in front of senior engineers/CTOs of any prospective customer. Steve (and his VP Sales) never understood this, despite my attempts to tell them. But what do you expect from someone who seems to believe that bullshit is an adequate substitute for substance? One telling instance of complete lack of appreciation of culture was when he asked a junior marketing staff to approach leaders of big companies across a range of industry sectors. That way, the CEO of BMW got an email “Dear Norbert, I’d like to tell you about our virtualisation products”. The mail probably never reached Reithofer, but if it did, it could only get us filed in the category of dubious operators, rather than a respectable technology provider, and would certainly have done far more harm than good.

As a consequence I grew increasingly frustrated with my lack of ability to contribute in a meaningful way. This was compounded by my observation that the company strategy was wrong, based on wishful thinking rather than analysis of market needs and technical facts. (More on this later.) I was asked to write white papers and technical blogs (something I enjoyed greatly) but that’s a pretty minor role for the CTO and founder. And I was supposed to screen invention disclosures and work with the inventors and the patent attorneys on getting patens filed. That stopped after I assessed two of Steve’s “inventions”, which he got Josh to write up. After finding that the inventive step was non-existent, I wasn’t troubled with any further ones…

I also wasn’t effective as a director. I was caught between trying to avoid being seen as undermining the CEO while trying to voice my concerns. I failed miserably at the latter. Basically I was very naive (probably hard to believe by people who know me) but I was also quite aware of Steve’s bad-mouthing machine, which I had seen at work more than enough. I should have taken a stronger, independent stand at the Board, but that’s water under the bridge. It’s a reflection of my inexperience, which Steve exploited well.

More about the board later.

Basically, my position became more frustrating as time went on. Sometime during 2009 I had given up any hope that OK would be a major success. I stayed on partially because I thought I owed it to the many engineers who were there primarily because of me, and partially because, having been at the conception and birth, I wanted to stay around for the funeral. It couldn’t be far off.

We had a world-class engineering team, by any definition. Except for one or two we hired from outside, the complete team that spun out of NICTA on 1 January 2007 consisted of my former students (and Carl, who never was a student at UNSW but had worked with us as an engineer since the early days of NICTA). Most had years of experience hacking microkernel code, and all had an excellent understanding of what makes a microkernel tick, and how you build systems on top of L4. Each could have doubled or tripled their salary by moving to Silicon Valley. They also had grown up working in a tight-knit team, so personnel-wise we were in good shape.

Before we could actually spin out, a (to me) unexpected hurdle arose: the contract negotiations. Steve took a very hard-line approach with the employment contracts. While most high-tech startups tend to provide fairly liberal conditions, what Steve offered was generally the legal minimum. In addition there were very broad and severe non-compete clauses (extending for a full year after leaving the company). While our legal advice was that this was almost certainly unenforceable in NSW, Steve insisted on them anyway.

The result was a staff revolt. About half of the staff refused to sign the contracts in the offered form, some making extensive changes. What followed was a lengthy (4 weeks or so) period of to-ing and fro-ing until we had a contract people were willing to sign. The end effect was that the company started on a sour note. I have to accept some of the blame here, as the original contract didn’t feel right to me, not the way I wanted to treat my students. But, again, I was inexperienced, didn’t trust my instincts in this unfamiliar territory, and trusted Steve to know what he was doing. Clearly a misjudgment on my side, and one I regret to this day.

In the end, Chuck, one of the A Team, decided not to join OK Labs, but instead moved to the Valley to join the Core OS Team at Apple, where he still works happily. At least he didn’t bear any grudge against me, we’re still good friends and meet up for a beer or three whenever I’ve got spare time in SF.

Not so for two others, Alex and Hal, who did sign on but never recovered from the original confrontation. They left about a year later, triggered by another poor handling of a situation by Steve: he had forgotten to perform a stock split before assigning stock options to staff, with the result that they got 2.5 times the percentage they were meant to get. Instead of saying “sorry, guys, we stuffed up” he tried to sell the correction as an improvement to staff, with a mail starting “good news! the price of your stock options has been reduced”. Not a smart way to treat grown-up engineers.

This created more bad blood. And, while I left these management issues in Steve’s hand (bad call, I know!) staff saw both of us as “management”. And it’s safe to say most staff were there primarily there because of me, and saw this as a betrayal of the trust they had in me. This was particularly the case with Alex and Hal, the two who left after a year. Hal hasn’t spoken a word to me since. It was my advanced OS course which had brought him to UNSW (as a postgraduate coursework student), so this hurts and saddens me.

My takeaway: If something seems wrong, do something, familiar territory or not. I should have known that, of course. One of the challenges was that I’m used to be able to trust my instincts, but this was an unfamiliar environment where I didn’t feel I could trust them, and Steve did a convincing show of knowing what he was doing. He also turned any doubt immediately into a personal trust issue, a great way to prevent any frank and open discussion. As I learned over the years, this was symptomatic: he would never admit to any mistake, and the mere suggestion that his approach wasn’t good would be seen as betrayal. The exact opposite to how I’m used to running things: frank and open discussions and looking for consensus.

After the bad start things improved quickly, and morale in the engineering team was very good once most people didn’t have to deal with Steve directly. Steve appointed Benno as VP Engineering, which helped to create that distance. Benno had been my star student, one of the most experienced in the team, despite his young age. He had proven himself in the early Qualcomm project, and is generally one of the sharpest people I know. Making him VP Engineering (rather than getting an experienced manager from outside) was a good move: Benno had more or less grown into the role already and did an excellent job on it. He is a real leader, respected by his peers, absorbed a lot about software processes, and also was a visionary Chief Architect.

We had excellent engineers, and a VP Engineering in charge who, despite his youth, provided strong leadership. The team achieved many amazing feats, generally delivering high-quality software very quickly. One of the most satisfying comment came from the Motorola engineer responsible for the virtualised phone project. He said “there were no bugs!” with the look on his face of someone who for the first time enters a Tardis and sees that it’s bigger inside. And we got consistently glowing feedback from customers after some of our engineers visited them, they never failed to impress through their insight and professionalism.

So in engineering we clearly had what it takes to be successful. All we needed was a plan and leadership to execute it, the engineers would build whatever was required.