Rainblog

Responsive design, the currently trendy new way of getting your websites to work unaltered across a range of devices, has been getting a lot of attention lately. I’m a big fan of responsive design: I do a certain amount of web-browsing on an iPod Touch, and - for all Apple’s pinch-to-zoom wizardry, reading non-responsive pages can be painful.

Responsive design is pretty new, but its adherents have figured out a good set of basic techniques which work well in most cases. Unfortunately, in the world of the web, no problem stays solved for long. Just when everything looked rosy, Apple came along and messed everything up for everyone.

They did this, not in the way that Internet Explorer used to - by being too lame - but by being too good. The problem is Apple’s new high-resolution Retina display, as shipped in their latest piece of so-sexy-I-want-to-lick-it-all-over consumer electronics, the new iPad. The problem, basically, is this: the Retina display is so good that images sized for a standard resolution display look ugly. Now, the responsive folks want a way to serve up clean, crisp hi-res images to the iPad and its ilk, and boring, stodgy old lo-res images to those poor souls still suck on 72dpi displays.

My sometime colleague Kelly Sutton does a good job of summarizing the issues in a blog post about the future of retina images on the web. He also proposes a solution of his own. He - rightly, in my view - thinks that handling things in Javascript, using a library such as foresight.js, is probably not the way to go. This issue - getting the right content for your client - is basic enough that it shouldn’t be left to ad hoc Javascript. It should be up to the browser to figure out what it wants and how to ask for it it.

Apple, having unleashed their monster on the world, had to tackle the problem themselves on their own site. They did so by loading the standard images and then - using Javascript - probing the server to see if it had any hi-res resources available, and downloading those. This, as Kelly implies, is madness, not Sparta. It causes both the hi- and lo-res versions to be downloaded (wasting bandwidth); it adds in a redundant HEAD request for each image, and it depends on Javascript.

Kelly proposes that instead the browser itself should be capable of sending HEAD requests for the possible variants of each image. If the page includes an ‘image.png’, and the browser is running on a device with a Retina display, it should send a HEAD request for both ‘image.png’ and for ‘image-2x.png’. Depending on what it gets back, it should then decide which image to load into the page. In this way, you avoid wasting bandwidth by downloading a lo-res version that you’re only going to throw away.

There are some problems, however. First, the number of requests is climbing: for each image, you’re sending two - or more - HEAD requests, followed by a GET once you’ve decided which image you’re actually going to use. In these days of Keep-Alive connections, that may not be such a big deal, but it seems inelegant. If nothing else, the server’s error log files are going to explode with 404’s. If the site’s ‘not found’ handling involves any significant processing, the server will also take a noticeable hit.

Then there’s the question of layout. Some browsers will hold off on laying out the page until they know how big an image is going to be. Yes, we should all be specifying our image sizes in the <img> tag or, better, in the CSS. Not everyone does, though. So the browser either has to wait until all the HEAD requests and the final GET request have returned, or it has to start laying things out and then do it all over again (producing an ugly page redraw) once the final results are in. They have to do this now anyway, but with more requests in the pipe, the wait is longer.

How can we avoid this awkward fumbling with HEAD requests, with the user agent essentially feeling the web server up to see what it’s got? My suggestion would be: don’t play a guessing game; if you want something, come straight out and ask for it.

The HTTP-Accept header is one of the lesser-used features of the HTTP standard. It allows a browser to say “I can handle these particular formats, and here’s my order of preference”. In theory, the web server could inspect the list and return the most appropriate content. In practice, almost no servers do anything useful with the HTTP-Accept header. Many do, however, do something very useful with the Accept-Encoding header: if a browser uses Accept-Encoding to say that it can accept compressed content, the server can give back the web page in compressed form, for substantial savings all round.

In principle, HTTP-Accept and Accept-Encoding represent, for me, a better way to approach this problem. The browser says explicitly what it would like; the server, if it can, honors the browser’s preferences. There are no extra HEAD requests, no waiting: the browser’s desires are expressed in the request, and the server comes straight back with the content that it wants.

So a browser could send, for example, a header called Accept-Resolution, with values such as ‘low’, ‘standard’, ‘high’, depending on the resolution of its screen (and the speed of its connection). The server then hands back the appropriate version of the image.

You could probably implement this today, using mod_rewrite rules. You could even - if you don’t mind browser-sniffing - do it on the server side without waiting for someone to implement an Accept-Resolution header. 

There’s one big gotcha, and that’s caching proxies. If Joe Desktop requests a standard image for their PC and then Jane iPad uses the same proxy, Jane is going to see the jaggies on her Retina display. If Jane gets there first, Joe gets served up a gigantic  image he can’t use (and if Joe Desktop is actually Joe Smartphone on a slow 3G connection, that’s going to hurt).

So either proxies will need to be updated to handle the ‘Accept-Resolution’ header, or images will have to be served with no-cache directives. I’d like to rule out the second ‘solution’ for reasons of economy, and suggest that proxies will need to be made resolution-aware.

Are there other options? I can think of just one more. Instead of having the browser say what it would like, the server can tell it what it can have, using a response header. When the agent requests the initial web page, the server returns a header that says, in effect, ‘hi-res spoken here’. That response would be an invitation to the browser to dynamically rewrite each image request to specify a hi-res version instead of the base version (we can use Kelly’s proposed naming conventions). Before the agent even knows which images it’s going to need to download, the server has already told it what it’s allowed to ask for. If there’s no hi-res version available for any particular image, the server just returns a standard-res version. Finally, as a not-insignificant bonus, the different resolution versions have different names, so we get around the whole caching problem.

I agree with Kelly that this is something that belongs in the browser, not in Javascript, but I’m convinced that the long-term solution needs to use headers, not HEAD requests.

Apple’s new amended developer agreement for iPad/iPhone development has been generating lots of talk over the past few days. Everyone has a theory as to why Apple chose to ban third-party development environments, ranging from a grudge match with Adobe, to multi-tasking issues, to a deliberate attempt to make Cory Doctorow’s head explode. OK, I made the last one up. No one’s saying that yet, but we shouldn’t rule it out.

In correspondence with Greg Slepak, Steve Jobs essentially gives his papal blessing to the theory put forward by John Gruber, which hints that while there’s some truth in the official line that cross-platform tools deliver inferior user experiences, it’s really about Apple maintaining a lock on the market. A market where developers can target a meta-platform over and above Apple’s mobile devices is one that Apple can’t control. That’s a threat to Apple’s business model so they’ve seized the chance to try and cripple the meta-platform from the outset.

Even without Jobs’s endorsement, it’s clear that Gruber’s probably right, but he doesn’t spell out all the implications. Apple’s move doesn’t just target the meta-platform; it also targets competing platforms, such as Android.

Consider a world where cross-platform tools such as those offered by Adobe were permitted by the Apple SDK agreement. If you’re developer X, you build your application using those tools, and release it for all the possible mobile platforms simultaneously. Now Apple’s mobile platform is just another platform. Sure, developer X’s shiny new application is available on iPhone, but it’s also available on Android and who knows what else. There’s no compelling reason to choose an iPhone if you want to run developer X’s new app.

The problem gets worse if Apple allows cross-platform tools in principle, but rejects apps submitted if they don’t measure up to their expectations of quality (which would be the case if Apple were serious about rejecting cross-platform tools in order to guarantee the best possible user experience). Developer X submits their new killer app, and Apple rejects it because the scroll bars work the wrong way or it doesn’t multi-task nicely. Developer X shrugs and says “At least I can sell it on the other platforms.” Now the killer app is available on rival platforms … but not on Apple’s. That’s a disaster from Apple’s point of view.

Developers have limited cycles. If they can use a cross-platform tool, then putting in so many man-hours of work will let them target all the platforms supported by the tool. But if Apple won’t let them use a cross-platform tool, then the man-hours needed to bring a product to market on all the platforms just doubled: they need to develop and test on Apple’s environment, and on the cross-platform tool. If the developer can’t afford to make that investment, they need to make a choice: develop for Apple’s platform, or for the other guy’s?

What Apple is counting on is that they currently have the most attractive platform. If developers have to choose to target only one, Apple hopes that it will be the iPad/iPhone. That can also become self-reinforcing: if more and better apps are available for Apple’s platform, the platform becomes more attractive. If developers have more experience working with Apple’s tools, they’ll be less likely to develop for other platforms using other tools. Apple is trying to use its early lead and Section 3.3.1 to starve the competition.

This isn’t about Adobe. They’re just collateral damage. What’s really at stake is dominance of the mobile platform, and Apple is using everything in its toolbox to ensure that it, and no one else, comes out on top.

Apple doesn’t want to be first among equals. Apple wants to be first. Period.

Our office is full of iPads, but it looks as if for one new owner the honeymoon is already over. Our sales director would really like to take a Keynote presentation from his desktop to his iPad (where he has an iPad version of Keynote), but the iPad doesn’t want to make it easy for him. And the language he’s been using is terrifying.

Apple’s suggested methods for getting a file off your desktop and onto the device apparently include “mailing it to yourself”, “putting it on a website”, and using iTunes file-sharing. The first two options smack of desperation. As for the last one, if you’re the kind of person who says “I need to move this file from here to there, so naturally I’ll use my music player software”, your mind must work in a very strange way.

For a company that prides itself on design and usability, Apple has some strange blind spots. I’ve written before about the bizarre issues involved in syncing an iPod, where every application ends up inventing its own method for sharing data with the desktop. The fact that the iPad launched without a good, transparent way of moving data between the desktop and the device is odd enough, but it may even be a deliberate choice. Apple recently went after the makers of the ebook reading software Stanza and GoodReader, forcing them to change their apps so that they could no longer browse the device’s filesystem for content transferred there using a USB sync utility.

It’s still not clear why Apple insisted on the change. The official statement says that Stanza violated the developer agreement by using private APIs. Insisting that developers use only public APIs is fair enough, but Apple could easily have made the filesystem accessible through a public API. If they don’t, is it because unrestricted filesystem access might permit piracy or does Apple feel that users need to be shielded at any cost from the mind-destroying horror that is a modern hierarchical filesystem? Has Apple chosen not to simplify moving content between devices as a matter of policy, or is it because they haven’t yet found a good, user-friendly way to manage the process?

During my time at Sony, I once saw an informal demo by Jun Rekimoto, a man who probably comes up with a dozen brilliant UI solutions before breakfast every day. At one point in the demo, he showed how to transfer a file between two computers by tapping on the screen of one with a lightpen, then walking across the room and tapping on the screen of the second. Of course the file wasn’t really “in” the lightpen; the transfer took place across the network, and the business with the lightpen was largely sleight of hand. But it was also a brilliantly simple metaphor that was instantly understandable. You pick up the file here, you put it down over there.

Apple needs to get over whatever odd scruples are holding them back and make data transfer between devices work like that. You need to be able to walk up to your desktop machine with your iPad or your iPhone in hand, and move files between the two with a single, simple motion. The whole business should just work - like Jun’s demo, and like everything else on the iPad - with a simplicity and obviousness that could easily be mistaken for magic.

But in the meantime, I’m afraid you’re going to need to keep mailing files to yourself.