Dear Tim Cook: please let scientists and students run their research software on iOS

This is the second unanswered email I sent Tim Cook on the subject of the iOS software lockdown. Here are the first and the third.
In short, there is a long tail of software that cannot be distributed through the App Store, for a variety of valid reasons, but would be ideal for the iPad or the iPhone. That is really too bad for researchers and all the users of specialized software who cannot use their iDevices in their work.
At last year’s WWDC you revealed that Apple has 16 million registered developers. That is a truly amazing number.
I am writing about all the other people who write software as part of their job, just not for the App Store.
A large contingent of those programmers is made of scientists. From archeologists to zoologists, they all write experimental software for their own use, for their students and for their colleagues. They often it release as free or open source software for everyone else to build upon. And many would love to run that software on their iPads and iPhones instead of lugging their laptops around.
The breadth and quality of specialized open source packages is every bit as amazing as the number of Apple registered developers. Unfortunately, most of it cannot be used on iOS for two main reasons:
1. Scientists use languages like Matlab, R, Python, Julia, Haskell, OCamL, Visual Basic, etc. because they pick languages they are comfortable with and that are best suited to the job at hand. Because of the restrictions on importing code into iOS apps or on installing software not compiled with Xcode, their programs cannot be run on iOS.
2. Open source packages that compile to native code are distributed as shared libraries. Those libraries (and their dependencies) compile on Linux, macOS and sometimes Windows with little or no change. For iOS, significant work would be required from the volunteer maintainers to support Xcode and static libraries. The result is that only the relatively few packages that are useful in mainstream iOS apps end up being supported.
In order to solve that problem and let scientists (as well as many other categories of programmers) run their software on their iPhones and iPads, would you please consider allowing non-App Store software on iOS, just like on macOS?
Thank you very much for your time and attention.

Dear Tim Cook: please let us use the programming tools of our choice on the iPad

This is a letter I sent to Tim Cook (Apple’s CEO 😉 a couple of months ago askim him to allow language interpreters, compilers and shared libraries on the iPad. I later sent him two more emails later.

I have not received a reply. I did not really expect any, but I hope that Apple top managers who are responsible for the iOS programming lockdown are at least aware of the (high) costs. Having millions of apps and millions of registered developers does not mean that all bases are covered. Far from it!

I am aware that many people have raised this issue before. We just need to keep asking. WWDC is around the corner: this would be the most wonderful surprise for all the iPad and programming enthusiasts!

Hi Tim,
I love my iPad and I use it for hours every day, mostly reading books, technical articles, news, social media, etc.
I also own a 12.9″ iPad Pro. That’s a beautiful machine, but it is a bit frustrating to me that I cannot program on it when programming is such a big part of my professional life.
There is an enormous ecosystem of free and open source software that make developing algorithms and generally tinkering with code and ideas delightful. Any Apple Machine Learning (for example) engineer will confirm this.
The restrictions on importing and compiling code on iOS, or on installing system-wide libraries, unfortunately make those open source packages impossible to use on my iPad.
There is some great open source software for iOS, but is mostly geared towards app development and it is a very small part of what’s out there.
What I would love to have as a scientist and a programmer is the option to install software outside the App Store, and the ability to use compilers and interpreters for any programming language of my choice. Basically the ability to do on our iPads what we do everyday on our Mac and Linux boxes.
My generation grew up with the Apple II, the Macintosh, the Commodore 64, the TRS-80, etc. Those machines were made for programming: there was literally nothing else I could do with my first computer.
If Steve Jobs’ analogy about PCs and trucks is correct, then young people with iPads and iPhones will no longer have the ability to tinker with code outside school or work. I hope you agree that it would be sad. So please open up the iPad to compilers, interpreters and shared libraries!
Thank you very much for your time and attention.

The Apple TV 5 built in LG televisions?

This is pure speculation. My modest entry in the prediction game.

In the early days of the Apple TV, people thought Apple was going to build a real television set, not just a set-top box. One potential problem was that iDevices had a much shorter lifecycle than TVs. It was feared that the computing hardware would become obsolete faster than people would be willing to buy a new TV. One imagined solution was that the Apple TV would be a replaceable module slotted inside the TV (remember PCMCIA cards?).

Well, apparently the Apple TV is not selling well. Neither is the iPad, the other device where one can watch movies and TV series comfortably. So Apple does not have with the movie industry the kind of leverage that the iPod gave it with the music industry.

On the other hand, Apple seems happy to outsource (kind of) the sale of Mac displays to LG. So why not also let LG manufacture Apple (real) TVs, with the Apple TV part as an upgradeable module (or not). LG alone sold 28 million television sets in 2016. That would move the needle!

What about the other major TV manufacturers? Philips seems to have settled on Android as its smart TV platform. Samsung is developing its own system. Anyway, a Samsung TV with Apple inside does not sound right, does it? Maybe Sony?

Games with in-app purchases offer valuable life lessons for kids and developers


gemstl;dr Mobile games with “bags of gold” in-app purchases teach kids and developers that:

  • if you have deep pockets, you don’t need to be smart or work hard to win
  • without money, skill and hard work won’t get you anywhere. And no second chances.
  • building honest games is its own reward. Don’t expect to be making any money on top of it.

Who could argue with that?

As a developer, I enjoy letting my children try out new iPad games. I tell them to avoid the ones that sell “chests of coins” or “bags of gems”, because you need to pay to keep playing and that’s not how honest apps should work.

But recently, after someone told me about a certain game that is raking a million dollar a day, and has the tech press drooling all over it, I decided to perform an experiment. I installed the game for my eldest daughter and asked her to see how far she could get before having to buy anything. Just two hours later, her ragged, penniless and underarmed villagers were facing certain death in a desperate attempt to take over their neighbor.

Maybe it was a matter of skill. After all, my daughter had just discovered the game. Maybe the next attempt would be more successful? No such luck, we could not find the “restart” button. Even deleting and reinstalling the app did not work! The developers clearly took great care to make sure one could not enjoy the free game too much.

Renting digital media instead of buying it is nothing new. We don’t purchase ebooks, TV shows, music or movies. We just rent them. But at least we know in advance how much it is going to cost us. With a pay-to-play mobile game, we know that the more we enjoy it, the more expensive it gets. I don’t know if it is even possible to “win” at those games, and I am not going to find out.

To be clear, I have nothing against in-app-purchases that unlock specialized features, subscriptions, or are substitutes for free trials or even paid upgrades. But $99 for a chest of gems, seriously?

So Apple, you have repeatedly shown that protecting your customers was a high priority. Personally, I find a scam much more offensive than some nudity or political speech, and I don’t think I am alone. If you must censor the latter, can you please do something about those pay-to-play games? After all, the base version of an app is supposed to be fully functional, and the guidelines for In-App purchases prohibit “intermediary currencies”. The recently-added warning “Offers In-App Purchases” is not enough, since all the App Store lists are infested.

I know that many developers (mostly indie) despise that practice. Yet, among the hundreds  of “app discovery” web sites, I could not find a single one devoted to regular purchase-once-and-play-forever games. Maybe it is time for a “honest games alliance” or something?





Why Siri had to start in beta

Bashing Siri, the iPhone 4S virtual assistant, seems to be fashionable these days.  Mat Honan declares it “Apple’s broken promise“. CNN reports on Siri’s alleged anti-abortion bias (via Danny Sullivan). Colbert weighs in. John Gruber remarks how weird it is for Apple’s flagship new product to be “so rough around the edges”, yet notes that it will be easier to improve voice recognition while it’s being widely used.

It’s not just easier, it’s the only way!

I worked on speech recognition with IBM Research for nearly six years. We participated in DARPA-sponsored research projects, field trials, and actual product development for various applications: dictation, call centers, automotive, even a classroom assistant for the hearing-impaired. The basic story was always the same: get us more data! (data being in this case transcribed speech recordings). There is even a saying in the speech community: “there is no data like more data“. Some researchers have argued that most of the recent improvements in speech recognition accuracy can be credited to having more and better data, not to better algorithms.

Transcribed speech recording are used to train acoustic models (how sound waveforms relate to phonemes), pronunciation lexicons (how do people actually mis-pronounce words, specially people and place names), language models (spoken phrases rarely conform to the English grammar), and natural language processors. And that for each supported language! More training data means the recognizer can handle more variations in voices, accents, manners of speech, etc. That’s undoubtedly why Nuance for example offers a free dictation app.

It is tempting to consider Siri as some kind of artificial intelligence, who, once trained properly, can answer all sorts of questions.  The reality is that it is a very complex patchwork of subsystems, many of which handcrafted.

To improve Siri, engineers must painstakingly look at the requests that she could not understand (in all languages!) and come up with new rules to cope with them. There are probably many, many gaps like “abortion clinic” in the current implementation, which will be fixed over time. When Apple states “we find places where we can do better, and we will in the coming weeks”, they are plainly describing how this process works.

It is important to understand that unlike Apple’s hardware and app designs, Siri’s software could not have been fine-tuned and thoroughly tested in the lab prior to a glorious release. It had to be released in its current form, to get exposure to as much variability as possible all the way from the acoustics to the interpretation of natural language. For each of the funny questions that Apple’s engineers had anticipated, poor Siri has to endure a hundred others.

If the rumors of a speech-enabled Apple TV are true, then Siri will soon have other challenges. For example, far-field speech recognition is notoriously more difficult than with close-talking microphones. She had better take a head start with the iPhone 4S.


[UPDATE There has been a lot of interest in the article, I thought I would clarify a few things]

-I have no inside information. Everything I wrote about Siri is an educated guess based on my own experience. I may be totally wrong, and I probably missed some important parts of the story.

-I did not mean to imply that Siri’s system is rule-based. I am convinced that it relies heavily on statistical learning. But someone has to train, fine-tune, test and debug statistical algos with new data and new use cases. Sometimes you just throw in the new data and press the “retrain” button. Sometimes you have to dive in and adapt algorithms. And sometimes, in order to squeeze the last few percentage points, you may write some old-fashioned rules, like for Siri’s quirky replies.

-As a few commenters pointed out, Apple has already gathered a lot of data from the previous Siri app. I think they used it to build the best system they could, which is already quite impressive IMO. They had to release it to be able to go even further. New data brings diminishing returns: at some point, 20% or 50% more data is insignificant, you want 10x or 100x more.

Why I am grateful to Apple and Steve Jobs (a developer’s perspective)

I fell in love with computers at 14.  My dad had taken me to the local community center.  They had a “computer initiation” class on a bulky Commodore CBM system.  Some people proudly brought their own Sinclair ZX-81 along.  I had never imagined that anything like that was possible.

Soon I had my own computer, a TI-99/4A, and I taught myself to program in BASIC, then in Assembler.  I mostly made games and traded them on tapes, by mail, with the TI99 club.  I have been programming and tinkering with computers since then, at work and at home.

There was a lot of excitement about computers, countless user groups and programming magazines.  The first word processor, the first spreadsheet, the first CAD program, the first graphical interface were all invented.  New software and hardware companies would pop up like mushrooms.  There was an incredible sense of opportunity and of endless possibilities.

After that?  Not so exciting stuff.  We got laptops, then netbooks, cheaper and crappier every year.  Bloated and buggy operating systems.  Browser wars.  Viruses. Enterprise applications that, even in 2011, would be better suited to a green 80×24 terminal. Write-once, run-ugly-anywhere software. Crash-prone smartphones so difficult to use that only computer geeks and corporate email addicts could be bothered.

The only bright spot? The Internet and web applications made computers even more useful, and brought some excitement back.

Most innovation happened in the browser, because regular users were wary of installing any third-party software.  With byzantine installation procedures, compatibility issues and rampant malware, I can’t blame them.

Then came the iPhone and the iPad.  Developers could again craft innovative software and put it directly in front of millions of users.  And a networked, pocket computer packed with sensors sure opens the door for plenty of innovation!

Customers now try and buy many apps because it is so simple and because they are reassured that it is safe.  Indie developers do not have to worry about distribution, billing, payment processing or returns, they can focus on what they do best: design and coding.

Apple has led the way and raised the bar for the whole industry.  Google and Microsoft have had to come up with their own platforms.  There is now a healthy competition that was sorely missing during the previous decade(s).

But most of all, Apple has restored a sense of wonder for what computers (in the form of smartphones and tablets) can do.  It has brought back the excitement and the endless possibilities that motivated so many software developers like me thirty years ago.

Thank you Apple, and thank you Steve for that.

[image: Jonathan Mak]