Annoyed.

On Android, it appears android.graphics.Region (which is used internally for clipping against a Path) is not anti-aliased. This means if you set a clipping path that is not rectangular, such as a rounded rectangle, then the interior clipping path will have jaggies.

My workaround for my rounded rectangle clipping with a border is to draw the border after drawing the interior, and allowing the stroke to be wide enough to cover the jaggies. But IMHO it’s a kludge.

Why I hate custom protocols over HTTP.

One recent trend is to use HTTP in order to send data between a client and server. Between protocols built on top of SOAP and XML/RPC (and yes, I’ve built code on top of XML/RPC, and have a Java XML/RPC library), it’s not all that uncommon to send text commands over HTTP.

And it makes sense: HTTP generally is not blocked by various internet providers while other ports are firewalled, and it is well supported across the ‘net.

As a rule, however, I’m generally opposed to overriding an existing protocol for private use. My instincts are if it is possible for me to open a port from a client to the server that is not in use by an existing protocol, then use that port instead.

With HTTP, there are a number of downsides. HTTP is essentially a polling protocol: ask a question, wait for an answer, get an answer. There is a lot of plumbing that has gone into HTTP in order to work around the performance issues revolving around HTTP–but because it is essentially a polling protocol, there is little you can do to bypass a resource that takes a long time to download besides opening up a second connection. (Protocols like LDAP allow multiple logical connections over the same physical TCP socket.)

HTTP has also become somewhat more complicated over the years, with things like optional keep-alive settings and an array of possible return codes. All of this makes sense if you’re building a web browser (though some of it is a bit over-engineered: I don’t know if 418: “I’m a teapot” is a joke or a sarcastic response to things like 449: “Retry With”), but for a simple RPC protocol, we really don’t need more than “success”/”failure”/”exception.”

And today I learned another thing that just confirms my “don’t override someone else’s protocol; just build your own” instinct.

As designed the client I’m working on initialized a connection by requesting information on static resources that may have changed. So I’d do an “init” call, and wait for a response. As part of the request call, the server team specified that I should send “if-modified-since” with the date of the last response, so I can tell if I should update the cached response. (This was modified from the original idea, which was to simply use an integer version.) This client runs on Android both over WiFi and over the cell network.

You can guess what happened next.

Yes, T-Mobile rolled out a new proxy server to reduce 3G network traffic by automatically detecting and caching server responses, and sending ‘304’ errors on the init call. Well, if you send ‘if-modified-since’, you better process 304 errors, right?

My client didn’t.

And so it means the 130,000 people running our client software–died. Hard.

The first time you run the application it would sync up just fine. But the next time you’d hook up, T-Mobile would detect that your request hadn’t changed, and would send a 304 response–which the client would not understand, and eventually shut down claiming the client could not connect to the server.

And we never tested this. Of course we never tested this. Our server never sent the 304 exception, and so we never had a way to test this. In retrospect, of course “everyone knows” that if you send an if-modified-since, you should process the 304 exception.

The fix was simple, as all such things tend to be once they are discovered and understood.

But it would never happened if we had never overridden an HTTP protocol, where there are layers we don’t fully understand (until they break) running on a network which can insert any ol’ proxy (some with bugs we may never understand) between the client and the server.

Why we shouldn’t unify mobile and desktop UI frameworks.

John Gruber: Google’s Microsoft Moment

It makes no sense to me why Chrome OS isn’t based on Android. Maybe there’s a good answer to this, but Google hasn’t given it.

While I don’t understand why Google has two completely separate operating systems, one based on Davlik (a Java VM clone) and another based on Javascript (sharing more in common with the Palm WebOS platform than with Android), I do know why no-one will ever be able to successfully create a framework that unifies mobile and desktop operating systems. It’s why Apple has a separate MacOS and iPhone OS UI framework.

It’s because of the needs of each platform.

Bottom line is this: on a desktop application you can create a rich window and environment that displays everything at once. The best example of this is the Apple Mail application: in one window I see all of my mailboxes, the status of all of my mail boxes, the list of mail in my selected mail box, and the current selected message.

Mobile applications, on the other hand, have limited real estate to work with. Thus, instead of having a single window, the iPhone version of Apple Mail has a separate windows for showing accounts, showing mailboxes in an account, showing messages in a mailbox, and showing an individual message. Within a mobile version of the software we also have a notion of a “view stack”; a stack of modal views which push and pop onto a stack of views–something that has no counterpart in the desktop world.

The development model also differs: on the desktop I may have a model and controller object (from the MVC design pattern) which drives multiple views simultaneously. But on the phone, I need multiple controllers attached to the same model, each controller contains information about what was selected higher up in the view stack.

Knowing that, we can answer the question that John Gruber links: “If I make a screen two inches smaller, should I use Android instead of Chrome OS? If the keyboard works with my fingers instead of my thumbs, I should use Chrome OS and not Android?”

The answer is simple: is the screen so small that your application must be represented as a stack of views (like the iPhone mail application) or can everything relevant be placed into a single window (like the Apple mail application)? If the first answer, use the mobile version of the operating system. If the latter, use the desktop version.

Touch screen dead zone.

While chasing down a usability bug, I discovered something interesting about the hardware for the Google developer phone, which I also suspect plagues the release G1 and G2 phones. The problem is an issue with the touch screen technology used by HTC.

The bottom line is this: there is a border to the left and right of the screen (and, I suspect, the top and bottom) where the finger’s location is not reported. On a piece of test software, when dragging my finger left to right, I found that when my finger was within 20 pixels of the left border, the position was reported as 0–and on the right border, as 319.0. This, despite being able to see my finger’s location reported with one- and two- pixel increments elsewhere on the screen.

I’ve also found that finger-down events within that 20 pixel border are not reported, unless accompanied with a drag outside of that 20 pixel border. Thus, if you design a UI where the user is expected to tap within 20 pixels of the border, the tap event will not be detected by the current hardware. A tap and drag, however, will be detected, which is why when you tap in the notification area at the top of the Android screen it doesn’t necessarily work–but a tap and drag will reliably open the notification area.

Just something to keep in mind when designing for the Android: tap areas near the left or right of the screen that may be 40×40 pixels in size will have the unfortunate problem that half of the potential touch area will be dead to tap events.

WebView on Android

I ran into a weird crashing bug on our application: when we closed an Activity containing a WebView display, on occasion we’d get a SIGSEGV crash. This reliably happened about one in a half-dozen times.

The fix appears to be to call WebView’s destroy() method on the view inside the Activity during the Activity’s onDestroy:

    protected void onDestroy()
    {
        fWebView.destroy();
        super.onDestroy();
    }

I’m not sure exactly why this is the case. But it appears to resolve my issues.

Thoughts on dealing with multiple Activities and UIViewControllers

In a typical model/view/controller implementation (setting aside, of course, the debate over what MVC really is), we implement a model (which maps to a document, file, or database), a controller (which manages the views), and a collection of views, generally contained within a window.

Now this model maps nicely into the iPhone API and the Android API: the basic controller class for the iPhone is UIViewController; for Android it is android.app.Activity. And of course the view hierarchy represents the UIViews and android.view.View objects, both built-in objects and custom objects. The controller object also makes the ideal target for iPhone delegates and data objects and for the various Android interfaces which do essentially the same thing.

On the desktop, most applications are built with either a single model and a single controller. Rarely do we have multiple controllers opened onto the same model, and when this happens, more often than not the multiple controllers are multiple instances of the same class declaration: multiple text editors opened onto the same text file, for example.

Mobile devices are different. On a mobile device you can have a top level screen with one controller drill down to a separate screen (with a separate controller) which displays an individual item, which drills down to a third screen showing information about that individual item, which drills down into a fourth showing the setting for a single field.

In this example, we have four (increasingly fine grained) views into the same model: in the first screen we have a top level view; in the next, a view of an item, and so forth. The original method I was using in order to code for this particular model was to create multiple model-like objects and pass them down to individual view controllers:

Now this model tends to be somewhat informal: my top-level model is (for example) an ArrayList of Items. When it’s time to edit an item, we pass the Item as the sub-model object into the sub-control display. And when it’s time to edit some piece of the Item, we pull that piece out and pass that to the next sub-control display.

The whole process becomes messy for a variety of reasons. First, it creates a strict dependency in the flow of views; even if we use interfaces and protocols in order to isolate a control (and its respective views), we still have a lot of work to do if we want to rearrange views. Second, it creates the problem that we somehow need to pass information about which sub-item was updated back to the previous view controller. This becomes problematic since the events for UIViewController and Activity don’t map well to the notion of “saving” or “marking dirty”: there is no single event we can easily grab that consistently means “You’re going away and the view above you is becoming active.”

Android presents a few twists on this as well. First, when your Android Activity goes into the background it can be deleted from memory in a low-memory situation. That is, you can find yourself in the situation where your Activity goes away and has to be rebuilt from scratch. This implies that you must strictly separate the sub-model from the controller (as opposed to the iPhone, where you can informally combine the model and view controller into the same UIViewController class). Second, well-factored activities can be set up to be brought up by external applications: it is possible your Item editor activity can be brought up by a separate application if it knows how to launch you with the correct Intent.

It strikes me that the proper way to handle this is to have a single unified model, and instead of passing sub-models, we instead pass a small record indicating what part of the model we are working on:

The idea is that the Δ object represents a small record which indicates to the sub-controller which part of the model it is working on. This has the advantage that updates are communicated to each of the controllers which are working on this model as the object updates. This also works well on Android, as individual control/view components can disappear and re-register themselves; the delta record is passed in the Intent which created that component, so when the Activity is reconstructed the Intent can be used to put the view controls back into the correct state.

I have a simple nutrition journal application I’ve been building for the iPhone which I was going to rewrite from the ground up using this model. In future posts I’ll update the post to indicate if this works well or not.

Thoughts on Netbooks.

One of the things that the company I’m working for has considered is creating a Netbook version of our location-based software. So I bought a Netbook (The MSI Wind 120) in order to understand what all the fuss is about.

My observations:

(1) This is not a new product category.

To me, a new product category is a product which I interact with differently than with existing products. For example, I interact with my iPhone in a completely different way than I do with my 17″ laptop computer: I keep my iPhone in my pocket and pull it out to look thing up quickly, while my laptop gets pulled out of a briefcase and unfolded on a table.

A Netbook is a small laptop. It doesn’t fit in my pocket; I have to take it out and put it down on my lap or a table. I have to boot it up. It’s more convenient: I can use a smaller briefcase. It’ll more easily fit on the airline tray table in coach. But it’s a laptop computer.

(2) The best part about a Netbook computer is not that it is small, but that it is cheap. If you need a second computer or you cannot afford a good desktop computer, and you’re using it primarily for text editing or web browsing, a Netbook computer makes an excellent low-cost choice: I only paid $300 for mine.

(3) Other than this, the keys are cramped (making it a little harder to touch-type on), the screen is small (1024×600 pixels), making it inconvenient to use for text editing, and there is no built-in CD-ROM drive. (The Samsung SE-S084B external USB DVD-writer is around $60 and works great with the MSI Wind.) Thus, while it is an excellent low-cost choice, it’s clear it’s a low-cost choice: you are giving up a lot to save the $500-$1000 span between the 10″ laptop and a well-equipped 12″ laptop.

(4) A cheap laptop will never dethrone the iPhone in the mobile space. On the other hand the eagerness of mobile carriers to come up with something to dethrone the iPhone may force them to consider lowering the price of an all-you-can-eat data plan for laptops, which means a wireless cell card and/or built-in 3G wireless in laptops will undoubtedly be coming down the pike in the near future.

The real question, however, is will it be too little too late: a proliferation of free and cheap Wifi hotspots may make all-you-can-eat 3G wireless for laptops a terrible value proposition unless you need to surf the net out in the boondocks. (On the other hand, if you are in construction or farming, where you routinely work in the boondocks, 3G wireless for laptops will be a god-send.)

(5) A small form-factor touch-screen tablet will be a new product category if it satisfies the following requirements:

(a) Fast boot time. A touch-screen tablet needs to go from off to on in 2 seconds or less.

(b) It should be two to three times the size of the original iPhone. To get an idea of what I mean, here is a pictures showing the relative sizes of my iPhone, an HP 50g calculator, the original Kindle, the MSI Wind, and my 17″ Macbook Pro.

The iPhone is an ideal size to fit in a pocket, but once you get to the size of the HP calculator (one of the larger calculators out there), you need to put it in a backpack or a briefcase or purse. Around the size of the MSI Wind, and you need a dedicated carrying case for the device.

To me, an ideal “new product category” item would be somewhere between the size of the HP 50g calculator and the Kindle, with the Kindle being the top size for such a device.

(c) Battery capacity should be enough to allow the device to surf the ‘net and use the CPU full-boar for a minimum of 4 hours. The iPhone gets its monumental lifetime between charges from very clever power utilization: when surfing the ‘net, once a page is downloaded to your phone, the CPU is turned off. (It’s why Javascript animation execution stops after 5 seconds.) But if you write software that constantly runs and is not event-driven, especially software that uses the ‘net at the same time, the iPhone battery will drain in less than an hour.

I believe for such a small form factor touch screen device to do the trick it needs about 4 times the battery capacity of the iPhone.

Once you reach this size and have something that is “instant-on”, you now have a device that is big enough to work on where you are–and perhaps balance in one hand while you use it in another–but not so big that you need to find a table at Starbucks to pull it out. In fact, such a device would occupy the same product category space (in terms of size, form factor and how a user could interact with it) as a large calculator.

Which means one application which would be ideal for such a device would be a port of Mathematica or some other calculator software which would put the HP 50g to shame. Another application that would be ideal would be web surfing; ideally such a device would devote more disk caching than the iPhone does to web surfing. Also, vertical software for engineers, and e-book readers, would also work.

The idea here is to create a device that straddles the mid point between the iPhone “pull it out, look it up, put it away” 30 second use cycle, and the laptop “gotta find a table at Starbucks so I can pull it out of my briefcase” 1-5 hour use cycle.

And the MSI Wind (and other clamshell shaped cheap laptops) ain’t it.

Update: However, the CrunchPad very well may be the product I’m thinking about, assuming there is a way to install new software on the unit.

The importance of sending a view size changed event on a mobile device.

On Windows Mobile 5 (and I assume the same is true of 6 and 7), the small bar at the bottom which shows the current meaning of the two smart buttons is not a separate chunk of real estate taken away from the rest of the application; instead it is a floating HWND object. It is up to your application to know if that HWND object is present, and to make sure you don’t obscure the HWND.

On the iPhone with the v2.2 and v3.0 OS, the slide-up keyboard is essentially a floating window which overlays your UI. It is your responsibility to know if the keyboard is present, and if so, resize the contents of your view accordingly. That means if you have a table view with an edit text field that is graphically in the area where the keyboard will appear, you have to figure out a way to move the table contents up and down as the keyboard appears and disappears.

I contend that the region of the screen devoted to the keyboard or custom keypad or the like should not be handled as pop-up windows overlaying your user interface. Instead they should be handled as events that resize the main “window” of your user interface. And instead of hooking a half-dozen different events that could change the form factor of the screen, all of these events should be handled exactly the same way they’re handled on a desktop application: with a resize event sent to the “window”, which then percolates down the view chain.

Unfortunately it appears no-one agrees with me. And so we’re stuck doing all sorts of complicated stuff–including Android, which tears down and rebuilds the “Activity” (the thing which manages the views in a UI) rather than simply send a resize event.

List View Context Menus

This was driving me nuts all weekend, because I became misguided.

On Google Android, I want to create a custom list view, and I want to handle context menus. Of course the right way to do this is:

(1) In your activity that contains your list, call setOnCreateContextMenu with a class that will be handling the context menu callbacks. (The activity already implements the interface, so you can just pass it your activity.)

        fListView.setOnCreateContextMenuListener(this);

(2) Implement the onCreateContextMenu and onContextItemSelect items as usual.

Where I ran into problems was with another suggestion from another web site that suggested in my custom list adapter that I call the view’s registerForContextMenu routine. The problem with that is that you don’t get the list view’s yellow highlight and animated fade effect.

So… How do people use their phones?

Way back in June when I started working for the startup I’m working for now, I made an observation about how people will use their smart phones–a fact which was ignored, and which I believe is hurting our app now. And it reflects how we use different computing form-factors in general.

A desktop is not a laptop is not a pocket computer.

A desktop is an immersive environment. Consisting of a very large display screen and keyboard, a desktop computer is something that generally has a fixed location, but can have a lot of attached or non-portable devices that allow you a great deal of power. Desktop computers are perfect for the daily grind of working at a fixed location.

Laptop and portable computers are less immersive simply because they are smaller. In general, you use a laptop computer by sitting down at a given fixed location, setting up your environment (pull out laptop, open it up, turn it on, find a wireless connection, plug it in), and working. Battery life for a laptop is not all that important unless you set up in an area without an outlet–at which point battery life is invaluable. Battery life is also invaluable if you use a laptop to go from meeting to meeting, where setup time needs to be as short as “take out, flip open.”

Phone computers are the least immersive device, and the one that interests me the most because it is the one that is the least understood. The few people I’ve talked to about phone computers have talked about creating compelling immersive environments for the phone–and tackling the problem of figuring out how to create an immersive environment on a tiny form factor. But phone computers are not immersive.

Certainly there are “immersive” games being designed for the iPhone computer. But there are also a lot of people who seem surprised that some of the most successful applications for the iPhone are things like fart jokes. The reality is that these successful tiny applications are not simply successful because the App Store is improperly structured–though this contributes. No, they are successful because people use the iPhone differently than they use laptop computers.

You see, most people use the phone in the following way:

(1) They pull it out of the pocket and press the “unlock” key.
(2) They tap a few keys. Perhaps they’re getting someone’s phone number, or placing a call, or pulling the finger.
(3) They then put it back into their pockets.

In other words, the tiny, non-immersive form factor combined with a non-existent setup time (as compared to laptops or desktops) means that most interaction with the phone is going to be a very quick cycle of “pull it out, ask a question, get an answer, put it back.”

Now there are times when people are using their phones for an extended period of time. But that extended period of time will stem from two sources. First, a user may be stuck on a subway or in an airport and have nothing to do–at which point they’ll use the phone as a source of entertainment. They’ll pull it out and watch a movie or TV show, or browse the web, or play a game. Because they’re in a circumstance where they’re killing time, however, the game or program should be able to save state quickly, and be entertaining immediately. A first person shooter that has no objective but quickly blowing people away makes more sense than a ‘capture the flag’ game or a game which takes some extended period of time to complete the level: you don’t know when the user will be back on the subway or stuck in an airport. And he may only be killing five minutes.

The second reason why a user will continue using an application (IMHO) is when he pulls it out, asks a question–then doesn’t get the answer he’s looking for, so re-asks the question in a different way. But this is not a good model of user interaction: the user is not interacting with your application because they’ve suddenly gone into an immersive interaction. They’re frustrated! because they’re not getting the answer they want.

Because of this, information-providing applications should provide information quickly. All the usual rules of computer-human interaction apply: 1/4th of a second is the longest delay between touching something and having it react. Two to four seconds is the longest delay for doing some computation–and during this time a spinning animation should be shown. If there is something that requires more time (such as getting the GPS location or hitting a server), a status display should be put up if this takes more than 5 seconds, letting the user know exactly what the software is doing. And if the entire interaction takes longer than perhaps 15 seconds, you better have a very good reason why–because chances are the user is going to flip the application off and put it away, unless he really needs the information.

But the bottom line is this: if your application provides information, it needs to work in a very quick cycle of “pull the phone out, ask a question, get an answer, put the phone away.” If you are designing your application to be an immersive environment or to function in any way other than in a “pull it out, ask a question, get an answer, put it away” loop, you’re designing your application for failure.