How SMS set back the mobile internet by ten years

Mari Matsunaga, the yin to Steve Jobs' yangStop me if you’ve heard this one before: an obsessively perfectionist corporate maverick defies conventional wisdom and launches a radical new mobile “i”-phone that offers built-in email integration, an app store with simple payments and, finally, a decent way to surf the web, all tied together with revolutionary ease of use.  Sales go gangbusters, the company mints billions, mobile Internet usage skyrockets and competitors are left scrambling to catch up.

I am, of course, taking about Japanese operator NTT DoCoMo’s launch of i-mode in 1999, spearheaded by Mari Matsunaga (pictured), who clawed her way up in the intensely male world of Japanese business and whose insistence of putting user experience first almost scuttled the project several times.  i-mode phones were equipped with an “i” button that opened up a browser in one click, allowing users to surf in a curated portal of i-mode sites, some for subscribers only, most free but with in-site purchases charged directly to the user’s mobile bill, or to access the wider internet by entering URLs manually.  Downloadable “i-appli” Java apps were launched shortly thereafter.  Each phone also came with the “i-mail” email service that was completely interoperable with the general Internet (and other Japanese phones), delta some fairly drastic limitations on message size since we’re talking 320×200 screens and download speeds measured in kilobits.  And since this was all running on packet-switched data, the billing scheme couldn’t have been simpler: pay per byte sent or received.

“So”, I hear you ask, “if this was so awesome, why have I never heard of this?”  There are many theories, but I’m going to lay out the case for an unconventional one today: it’s all SMS’s fault.

Why SMS was awesome

SMS is the GSM mobile standard’s Short Message Service, and by telco standards it has a venerable pedigree, being first dreamed up in 1984 (yes, that’s thirty years ago).  Fundamentally, SMS is a hack; a glorious hack, to be sure, but a hack nonetheless.  SMS is built on the Signalling System 7 (SS7) protocol, designed in the 1970s for controlling landline calls.  Unlike most previous telco signalling systems, which controlled calls by inserting tones directly into the voice stream and let phone phreaks run wild with blue boxes, SS7 has a separate control channel used primarily to signal when calls start and end, means it transmits blank frames the rest of the time.  SMS squeezes text content into those empty gaps, which explains both why telcos were so fast to adopt it — it needed precisely zero additional hardware to roll out — and where that famous size limitation comes from: the frame size is exactly 140 octets, which is enough for 160 7-bit characters.

The other reason telcos love SMS is the money.  Remember, if you’ve already got a GSM network, SMS costs nothing at all to transmit.  (Yes, there is some cost involved in setting up SMS store-and-forward centers and interconnects and whatnot, and I made my living for close to a decade working in the field, but compared to actually building a network, this stuff is a rounding error.)  However, people in the 1990s were already well-trained to pay for each call, letter or fax, so charging (say) a flat dollar a piece for each SMS still sounded like a bargain, and the telcos, buying SMS for $0 and reselling them at $6500/MB, weren’t about to disillusion them.  In 2013, US operators alone minted 21 billion dollars from SMS, and you can still today get charged that $1 a piece if you happen to send an SMS while roaming in the wrong place.

Crazy Frog Crazy HitsSo far, so awesome (if you’re a telco, at least), and Worse is Better when it comes to victory in the marketplace.  But SMS’s sheer hackiness made it an evolutionary dead end, despite increasingly Sisyphean efforts to build more and more complicated infrastructure on top.  For example, while SMS was originally meant purely for transmitting text, some clever noggins came up with the idea of using six of those precious 140 bytes for a User Data Header that, bit by bit, could say things like “I am a logo!”, so you could squeeze in the 1008 bits needed for a 72×14 monochrome bitmap representing the Vodafone logo.  Or if you were so greedy that 134 bytes wasn’t enough, you could use a few more of those bits to say “I’m piece 3 of 7!”, and get the phone to concatenate those SMSes together into a chain that might contain as much as a kilobyte of info, just enough for a Crazy Frog ringtone (that’ll be $4.99, thank you).  Unfortunately, SMS is a UDP-style best-effort-only service with no guarantees of delivery, meaning that the longer the message, the more likely it was that some pieces would fall by the wayside.  And while there is a “delivery notification” bit that’s supposed to confirm a message was received, and whose absence can trigger redelivery, those notifications are also SMSes and are themselves often dropped along the way.  Remember when, in the olden days, you’d occasionally get 3 or 5 copies of a message?  Somebody was dropping your DNs.

Meanwhile, in Japan, they didn’t have SMS, because they didn’t have GSM.  The Japanese telcos naturally noticed that this SMS thing was taking off in Europe and built their equivalents for the Japanese 2G standard PDC, but the operators never managed to sort out talking to each other, and by 1996 or so they just said “fuck it” and deployed good old e-mail instead.  This was easy enough technically, since PDC offered packet-switched data and WAP 1.0 didn’t, and easy enough financially, since they didn’t have to worry about cannibalizing non-existent SMS revenue.  The standard undiscounted data rate in Japan circa 1997 was around 0.3 yen per kilobyte, and because a short e-mail is about 3 kB (headers and all), it cost around one US cent to send/receive one to/from any device in the world that groks SMTP.

Why MMS is so bad

In 2000, NTT’s arch-competitor J-Phone launched Sha-Mail, the world’s first mobile photo messaging service, which (then as now) was an instant killer app quickly duplicated by NTT as “i-shot”.  This was implemented using standard e-mail attachments and charged for at packet data rates, meaning around 10 yen ($0.10) for the 30 kB images taken by camera phones circa 2001, and phones and pictures sold like hotcakes.

Back in Europe, the WAP hype wave had just crested and crashed when it turned out people weren’t willing to pay one euro per minute to surf an unusably sluggish crude approximation of the few Internet sites that had bothered to completely rewrite themselves with WML.  Yet the first GSM packet data scheme, GPRS, was starting to gain traction and it started to dawn on operators that SMS just wasn’t going to cut it for images.

The MMS network architectureSo they came up with the replacement: MMS, the Multimedia Messaging Service, and you can already sense what a kludge it would be from that very first buzzword, which has killed all it touches.   To grossly oversimplify, MMS works by sending magical SMSes called “WAP pushes” that tell the receiving phone to open a WAP connection to an MMS server, from which they then proceed to download what are essentially e-mails crammed full of bizarre custom MIME parts, which are then rejiggled into an approximation of the original message, defined with a laughably overcomplicated custom slideshow language not too distant from Powerpoint.   For additional hilarity, the protocol provides no way of checking what the receiving device supports: instead, it’s the operator‘s job to attempt to manually track which phone number corresponds to which phone model.  (Thought experiment: imagine if the Internet worked this way, and instead of User-Agent strings, you had to phone up your ISP every time you changed device, browser or window size.)  This grotesque Rude Goldberg contraption is defined by eleven (11) different specifications, imaginatively named MM1 through MM11, each laying out the edges of one piece of the puzzle, and (unlike SMS) requiring very considerable resources to deploy.  Unsurprisingly, nobody could agree on how to actually implement the damn thing, making MMS hysterically unreliable to the point that, in A.D. 2014, I still consider it a minor miracle to successfully transmit one.

But MMS did have one feature that operators considered very, very important: it let them continue charging per piece, although many operators socked you for data usage as well.  So if you were visiting Mexico and Aunt Tilly sent you a 300 kB picture of her cat, she’d pay a flat $2 or something to send it, and you’d pay $30 to download it at $100/MB.  Or, more likely, she’d pay $2 to send it, but her phone or your phone or your home operator or your roaming network or any of the gazillion boxes talking incompatible variants of MMn along the way would barf and lose it.  And then Aunt Tilly would get pissed that you didn’t reply to her message, and never touch MMS again, which is in fact pretty much what happened.

Why i-mode never took off elsewhere

3G_NormalNow operators around the world did notice that this i-mode thing looked like a license to print money, and 17 of them ended up eventually launching it around the world, offering i-phones (small p) with little i-buttons that opened i-sites that charged i-ncredible amounts of money and otherwise looked a whole lot like the Japanese i-mode i-cosystem.  Yet every single one of them failed.

Partly this was technical: not everybody/everywhere had GPRS, and i-mode over plain WAP, which essentially involves a dialup connection every time you press a button, is atrociously slow.  Partly this was timing: so many fingers were burned in the bonfire of WAP malinvestment that few were willing to go all-in on another whiz-bang mobile invention, making the chicken-and-egg problem of getting sites to create i-sites tagged with i-mode’s “compact HTML” extensions that much harder to hatch.  Partly this was the cutthroat nature of the market: at least one operator I know of launched i-mode more or less entirely as a cynical ploy to extract better terms from Nokia.  But mostly it was just because none of those operators had the guts to go all-in on i-mail, because they were all so wedded to sucking on the SMS teat and could not bear to exchange $1 now in easy SMS profit for $0.01 forever in data revenue.  And the straw that broke the back of the few brave operators that did launch it?  The fallback mechanism for sending e-mail to non-i-mode phones was…  MMS.

Instead, it was BlackBerry who grabbed that market opportunity in the West, letting you read business e-mails on your CrackBerry and later launching BlackBerry Messenger (BBM), giving its own users a first taste of rock-solid, instant, all-you-can-eat messaging.  What’s more, it wouldn’t be too far-fetched to argue that BBM was such a hit in North America only because SMS was so bad there: half the operators in the US and Canada were using CDMA, which doesn’t natively support SMS.  But this was a completely proprietary protocol, not an ecosystem others could use, and BlackBerry did its best to make sure BBM only worked properly on their own devices, also opting for easy money now over a chance of future relevance later.

Jobs delivers us the keys to the new kingdom

So that was the state of the rest of the world until 2007: SMS ubiquitous for lack of interoperable alternatives, MMS strangled in the cot, BBM minting money in its little niche.  Only then came along the iPhone and the Cambrian burst of mobile internet evolution, which saw Android make mobile e-mail a commodity and the rise of alternative messaging systems like Skype, Facebook, Snapchat, WhatsApp, etc.  In telco lingo, equal parts dismissive and terrified, these are to this day called “Over-the-Top” (OTT) because they finally scaled over that charge-per-piece wall put in place by SMS in 1984, and brought the rest of the world to where Japan was in 1999.

Meanwhile, in Japan, Matsunaga was railroaded out of NTT barely a year after launching her creation, and is now a director at Bandai. NTT lost over $10 billion on its i-modal overseas misadventures but grabbed nearly two thirds of the Japanese market, with around 50 million i-mode users at its peak in 2008.  But they refused to sell the iPhone until 2013, by which time it had lost over 20% of that market share to nimbler rivals like SoftBank, which went on to buy Sprint.

And that’s where my story ends.  Now, I’m not saying i-mode had any real chance to rule the world, GSM was not a good fit technically and SMS was already entrenched by the time it came along.  But if SMS hadn’t existed, the world would almost certainly have had an i-mode equivalent much sooner, and you can only wonder how the world would look today if it had.