The Personal Form Factor

The desktop computer used to be the hub of your digital life. Then the smartphone joined the family. The tablet was supposed to fit in between those two, and the smartwatch was your phone companion. In the end, the phone became the new digital hub; your most personal form factor.

The smartwatch seemed poised to take over that role, with a new ultra-portable form factor, and a promise to untether you. And it has indeed turned out to be great, for some, but is completely irrelevant for others, and even at its best, it did not succeed in replacing the smartphone as the single device you take with you. In fact the whole family of device categories seems to have stabilized in recent years: the tablet is great for some, the tiny laptop is great for some, the big laptop is great for some, and the giant high-end PC — the “truck” — is great. For some.

Remaining at the center of it all, though, there’s your smartphone. With increasingly mature operating systems, it can increasingly do more things for you. It’s not too difficult to make an argument that most people could cut out every other form factor, keep only their smartphone, and still get by perfectly fine. Some do already, I’m sure, which is a factor that no doubt will help inform the next category of personal devices.

There’s probably always going to be room for multiple form factors and screen sizes, one for each specialized use case. In between all of these, however, there’s likely to still be a single hub device. Is it a future watch?  Is it a pocket sized rectangular slate with cell capabilities? Let’s speculate, because that’s what this blog does.

A New Category

Most successful hardware categories seem to share a few traits:

  • They solve specific problems
  • They are easy to interact with

From a technology point of view, the only constant has been miniaturisation, resulting in increasing amounts of power being available at increasingly lower costs. The biggest barrier to physical device miniaturisation however, perhaps more-so than battery concerns, has been an interaction model that just hasn’t kept up.

The smartwatch UI is impressive, but despite valiant attempts, a whole slew of actions are still much better done on your phone. Perhaps one day when rock solid voice input mechanisms, digital assistants, gestures, and other magical science fiction interaction methods reach a combined threshold of quality, the watch might replace your phone, but until such a time, the watch is unlikely to replace the well-tested benefit of a big screen. And so for our most personal device, we’re likely stuck with traditionally proven interaction models like tapping buttons to do actions. Fitt’s law suggests the same: the bigger a button is, the easier it is to tap. (That might sound obvious, but the actual math has been done to prove it too.)

Remember when it was claimed that Apple would never make a phone larger than 4″ because touch targets would be out of reach for single handed use? Yet here we are. Given the current interface limitations, the challenge is propping as much screen into as pocketable a form-factor as possible, while still letting you reach the most important actions with a single hand.

Incidentally, if mobile VR ever takes of, which seems likely, the screen will have to be high resolution, and as scratch-resistant as possible. When you’re looking straight at something that’s an inch from your eye, every speck of dust shows up. As such, it would be really nice if the device doesn’t scratch just by being pocketed.

Big screen, one-hand operation, pocketable, doesn’t scratch? Tall order.

Fold It

Remember the Nintendo Game & Watch? For its time, it had a pretty big screen (with room for more), was pocketable, and barely scratched.

game-watch
Donkey Kong Nintendo Game & Watch, 1982 (Wikipedia)

We’re probably not looking at the next iPhone up there. But we may be looking at the inspiration for Microsoft’s next attempt at a phone. With zero marketshare to speak of, Microsoft is desperate to either invent a new category they can own, or just stand out as unique. They might not succeed in doing so, but they might well try. In a way, they already tried it once, with the Microsoft Courier, and arguably got a few things right, way before its time.

The foldable form factor solves a number of problems that are increasingly becoming evident: $800+ pocket devices that go splat when you drop them are arguably fundamentally flawed in their design. Since transparent aluminium hasn’t been invented yet, it’s probably time someone started experimenting with form factors that are just slightly more durable than a sheet of thin glass. In fact, Lenovo has already gone down this path in a tablet form factor.

Maybe the next category of devices isn’t foldable. Maybe it’s bendable. Maybe it slides or transforms. Maybe there are three screens, of which two are hidden when the clamshell is closed. Maybe we’re just looking at a magnetised keyboard case.

Whatever it might be, such a new device is probably going to need new interaction models that can scale to multiple screens. What might one-handed operation look like on a dual-screen folded device — will the interface adapt and place important interaction only on the one screen? Only near the bottom? What happens to the UI when you rotate the device? Would you want a pen?

6″ is too big a phone for most. Surely no-one in their right mind would want a 7″ phone, right? It just so happens that if you put two 5″ phones next to each other, the combined screen would be about 6.4″. It would be close to a square in its aspect ratio, somewhat like a book:

page-1

Would it work? Impossible to tell, and it would depend on a beautiful combination of hardware and software. Like the good old 1982 Donkey Kong Game & Watch.

Apps Are Dead

“There’s an app for that” was a popular phrase thrown around a few years ago. The phrase optimistically noted that in this technological era, whatever task was at hand, you could probably find an app on your pocket-carried supercomputer that would help you right out.

What the phrase did not include was the notion that in order to get said app, you had to open the app store, search, pick the right one, possibly pay 99¢, type in your app store password, wait for it to download, open the app, grant permissions to contacts, location or whatever else the app might want until finally you could use the app. Unless you had to register and/or sign in to use it, possibly add a credit card.

Apps were revolutionary for their time, no doubt. But in a world where cars drive themselves and pizzas are best ordered from your watch, some of the qualities of apps as we know them are starting to seem a bit menial. The numbers seem to suggest this as well, as apparently half of U.S. smartphone users download zero apps per month.

At its most atomic scale, an “app” is a little window on your phone that does things your operating system might not do on its own. When the iPhone was first introduced, the phone dialer was presented as “just another app”. It was no different, it was suggested, than any other apps you’d install, and every app added new features to your phone.

But what about replacing features? What about augmenting features? Sure, WhatsApp can create a dialer app that lets you call using their service instead of the cell network. But they’ll lose out on all kinds of systems integration into the OS: what if someone calls you, can you answer with WhatsApp?

It varies from platform to platform the amount of integration apps are allowed to do. While iOS is mostly closed down, you can still replace the keyboard. Android allows you to replace many aspects such as the browser, and yes, the dialer as well. But even then Android is still very much Google’s platform, and there are key aspects of the operating system that are still off-limits.

Apps of today are also very much hardware specific. Android apps look and behave a specific way and iOS apps look and behave a specific way. Some apps are cross-platform, available on both. This has worked fine for a world where people carry a smartphone in their pockets, but what happens when we stop doing that?

In order to speculate what the app of the future might look like, let’s summarize some of the challenges posed by the current interface:

  1. Finding and installing apps is cumbersome
  2. Having to manage your identity and sign into every app is a pain
  3. Having to remember or save passwords for every service is dangerous
  4. Trusting an app with your credit card information is both cumbersome and risky
  5. Generally, closed platforms are advantageous mostly for the platform vendors
  6. Future hardware categories are likely to demand drastically different or adaptive apps

If you’re Apple or Google, it might sounds like #5 — being the platform vendor — is a good place to be, and so it might dampen any initiatives to make new touch points that’ll fully open the home turf to competing apps. But there’s an argument to be made that they will have to, or be left behind.

Metaplatform

Enter Facebook. Most of the world is on it. They have your name, address, credit card, contacts, and probably photos. Facebook is you; it’s your identity, and you can use it to sign in, pay and communicate. Facebook is a metaplatform. So is Amazon, and so is Microsoft. Neither of these three have a mobile hardware play to speak of, but they have services and ecosystems you wouldn’t want to be without. To a certain extent, so do Google and Apple, but it’s a competitive space and Facebook arguably has the upper hand on the identity aspect, while Amazon has for the payment aspect. And so in five years maybe it doesn’t matter how good Apple Pay is if you can’t get your “Prime discount” when using it, and it doesn’t matter how good Google Duo is if none of your friends are using it.

People use Facebook. People use Amazon. People use them even if they have to use a browser to do so, and their webpages run well. More so, the secret sauce that makes them run well — React, AWS etc. — is available to anyone. To an extent it doesn’t matter which platform these run on — all they need is a browser. In five years time, will you care whether that browser runs on Android or iOS?

Apple and Google obviously care, and the thing they need to do in order for their platforms to be relevant in the metaplatform future, is open up. The platform that opens up the most integration touch points in their operating system will be able to provide the better user experience for your metaplatform of choice. People might choose Android simply because Facebook runs better on it.

In a future where apps run anywhere, the underlying platform becomes a checkbox. We’re already seeing the baby steps towards this with React Native, and progressive web apps.

Future Apps

So what might the app of the future look like? Perhaps a better question to ask is: how would the app of the future work? 

A few problems need to be solved. Instead of installing a separate app for every airline you fly with, and only when you fly, then perhaps the website should be allowed to perform as well as were it actually native. If you fly that airline a lot — pin it — it’s now “installed”. Unpinning it uninstalls it. Identity wise, you are signed into your operating system with whatever cloud account you prefer. This cloud maintains your passwords and your credit cards. Through biometric authentication, apps can tap into this information when you allow it. No signups, no cumbersome passwords to remember.

Increasingly, apps won’t install themselves as icons in a grid, but instead hook into touch points in the operating system and become actions for intents provided. It’s long been the case that the best apps are the ones that focus exclusively on solving very specific problems and tackling specific use cases. The ultimate refinement of this is the complete reduction into taking action based on a user intent.

In comparison, the apps of today are very linear in their flow. You pick an app, pick an intent, complete your task:

Current Apps.png

When the intent of an app becomes available before the app itself is even launched, suddenly the flow to completing tasks can be highly streamlined:

Future Apps.png

Instead of hunting for icons, perhaps your homescreen will simply list intents contextual to location, time of day, and habits.

Intents also enable interoperability between apps. Some intents are present already today — on Android, the “share” intent lets you transport content from one place to another. Imagine every app as an intent-based action — the plug-in platform, where apps add, replace or augment any intent you might have. Call someone, text someone, take a note are all existing intents that apps can handle the actions for on Android. But who’s your digital assistant, and can it place a WhatsApp call when you ask it to?

The process will be gradual, but it’s likely to kill off most apps that aren’t able to transition to being simply actions. There’s going to be room for the occasional “pinned app” for use cases to which there aren’t universal handlers, or for apps that create new intents (“I want to tweet”), but those are likely to be exceptions.

When metaplatforms provide all the infrastructure and apps merely tap into them, will this make for a more closed web? That remains to be seen — could be that it becomes more open, as web-apps increase in what they can do and where they run. But one thing’s for sure — you’ll spend spend less time hunting and pecking between icons in a grid, as menial tasks are being handled by intents and actions. In fact we might finally be able to go to a dinner with friends without everyone putting their smartphones next to their plates. In this golden future, maybe that Swarm check-in is handled by your digital assistant.

The best user interface is invisible, and in the plug-in future there might not be a lot to look at. Apps are dead. Soon enough, there will be an action for that.

The App Icon

The new Pixel phones by Google have started shipping, and one feature of the new phone is a redesigned home-screen which favors round icons:

google-pixel-phone-8297.jpg

Photo from The Verge

Android has previously been known for allowing any arbitrary silhouette to go in their icons. There have been guidelines for how app icons should be sized and how they should look, but these guidelines haven’t been enforced, leading to a metaphorical wild west of competing standards:

now-launcher.png

Incidentally, the new Pixel homescreen applies icon normalization, which resizes icons that do not follow guidelines:

pixel-launcher.png

Icon normalization isn’t perfect either, though, and it also doesn’t prevent app builders from creating wildly different icons. The move to round icons feels like a last ditch attempt at reaching for some kind of consistency in app icons, even if the icons themselves lose a great deal of creativity in the translation:

roundicons.png

Icons from Android Police

The thing is, outside of doing basic icon normalization, the Pixel phones don’t enforce the round icons, so unless a developer actually makes one, you’ll still end up with a mix of styles.

There Is No Single Style

Having consistent and beautiful app icons throughout an operating system is a beautiful dream, one that I’m sure many designers have had. But so long as app developers have to provide this icon themselves, the styles are going to vary as much as all human art does. It’s human nature, it’s an expression of individuality, a difference in taste, and a result of varying degrees of design time invested.

It might even be done intentionally for differentiation. While having an icon that blends in, fits, looks right among the other icons on the platform might make users happy, the app developers might want their icon to stand out (even if like a sore thumb), scream to the sky: look at me, my developers have families to feed, launch me.

Many attempts have been made at producing consistency where none is found, icon normalization is just one of them. Samsung phones put all icons on a squircle badge, Xiaomi phones offer multiple approaches ranging from badging and masking to replacing icons with huge icon packs to replace every icon on your homescreen.

Perhaps the most successful approach has been that of the iPhone, and probably due to the rather strict limitations imposed: no transparency, therefore no custom silhuettes. Every icon gets cropped into a rounded rectangle. Even then, icons on the iPhone differ wildly in style.

The story is the same on Windows and macOS. Windows 8 and 10 courageously tried to retire the app icon in favor of live tiles that could even be resized, so the user had more control of the aesthetics, and recommending flat white, fairly easily designed motifs:

windows10-tiles.png

Spot the odd one out

… but even then,  no matter how specific your guidelines for app icon designs are, designers can choose to not follow them.

Even if the guidelines were enforced, I suppose, there’s no guarantee the results would be consistent.

The Why of App Icons

It’s always prudent to ask the question: what problems are we trying to solve? Why are there app icons in the first place? I suspect Microsoft asked the same question, leading them to try live tiles.

The obvious answer is that app icons exist because apps exist, and apps work in a specific way. Your operating system provides a platform on which an app runs, and it’s then the job of the app to solve problems, give information, make you productive. If you need to write text, you open a writing app. If you need to edit photos, you open a photo editing app. This is how it’s always been. But will it stay this way?

We have to go deeper!

Why did you want to write text? Why did you want to edit a photo? What happens to the text when you’re done writing, are you going to send it somewhere, publish it? What happens to the photo when you’re done editing?

What if the operating system was intent-based instead of just a platform for a wild west of apps? You want to call someone, use the operating system dialer, but use it with WhatsApp. Want to make a reminder? Use voice actions, but save it in Evernote. Want to listen to music? Press the play button, which you mapped to Spotify instead of Apple Music.

We’re getting ahead of ourselves, and I still have that upcoming post about how apps might look in the future — Update: here’s that post. But the point is — perhaps app icons don’t really matter in the future. Perhaps the platforms of the future finally break the shackles of “a grid of icons” and relieve us of the menial tasks of jumping in and out of apps. In such a future, perhaps an app doesn’t even need an icon — perhaps the app is more like a plugin that hooks into touch points of the operating system, replacing or giving alternatives to what’s already there.

Keeping such a future in mind, it’s hard not to look at Google’s round-icon efforts and smile like a loving parent: aaw that’s so cute, bless your heart for trying!

I like round icons. I like silhouetted icons. I like really well-designed icons, and I love that Google and Microsoft are trying their best to foster consistency among app icons. But perhaps the battle won’t be relevant in a few years. There will still be good designs and bad designs, of course, but perhaps that battle won’t be fought on your homescreen or desktop.

coda.png

Home Button

Let’s discuss, for the moment, the Home button. It’s the quintessential interface on smartphones.

feature-iphone.png

As pioneered by the iPhone, its behavior arguably defined the user experience of the rest of the operating system. It suggested apps were a modal experience, and that a press of the button would always exit the app and take you home. In-app navigation would have to happen elsewhere. The home button conveyed a simplicity of interface that immediately resonated. Despite doing only a single thing, it was worth spending an entire physical button on it.

Since then, the button has taken on many new behaviors. First off, the double-click happened as something you could tie to a custom shortcut. Eventually that would open the multi-tasking tray. We got a long-press that would fire up Siri. When the form factor grew beyond 4 inches, we received an optional “reachability” feature that would require a double-tap (not click). Triple click could be mapped to an accessability feature.

Android

Before we go into the feasibility of adding more gestures to the previously single-purpose iPhone home-button, let’s look at what Android did. In the early days, it wasn’t pretty. There would be physical buttons for Back, MenuHome, and Search. Home and Back buttons worked (pretty much) like you’d expect, but Menu and Search were problematic. The former, in particular, meant that apps had no visible UI for extra features. You’d open an app, and some aspects would be buried in an invisible menu you had to click the menu button to see. It was classic mystery meat navigation, and search was almost as bad. Search would open an apps contextual search feature, if it existed. Otherwise it wouldn’t do anything. On the home screen, search would fire up Google. In the browser it would set focus on the addressbar. I may be misremembering bits here, can you blame me?
Oh, and most buttons had longpress. Longpress back, I believe, would open up history in the browser. Longpress home would access voice navigation, I think. Longpress search, for whatever reason, would invoke the multi tasking menu.

There were a lot of qualifiers there, suggesting I might be misremembering. That’s because invisible features are almost always a bad idea. Hiding possibly critical app features in an invisible menu reduced the discoverability to those willing to play whack-a-mole. Same with search, and frankly, longpress.
Before we get to that, though, Android 4.0 Ice Cream Sandwich (technically Android 3.0 Honeycomb, though I’m not sure anyone will recognize that version) improved things a fair bit and the Android systembar has been fairly stable ever since. Here’s the latest iteration:

feature-android

For the purposes of this post, we’ll be discussing the Android “spec” version of the systembar, which features these three onscreen buttons in “back, home, overview” order. Some OEMs, like Samsung or HTC, would make the buttons physical, or flip around their order.

Android has a Back button, a Home button, and an “Overview” (multitasking) button. Initially, there were no longpress actions tied to any of the buttons. Instead, a vertical swipe would invoke Google Now, or Search, whatever your phone came installed with.

Redesigning an interface is like dancing a waltz. It’s usually two steps forward, one step back. In the case of the Android systembar, the swipe up to enter Google Now gesture was crazy making. I would accidentally invoke it every once and again, but my daughter playing with drawing apps would invoke it constantly and lose her place.
Eventually (5.0 Lollipop I believe) the swipe gesture would be replaced by a longpress, and in 6.0 Marshmallow you could even disable the longpress.

The Goldilocks Principle

The elephant in the room is the burning question: which is better? What’s the right amount of system buttons? Should buttons be physical or on-screen?

In the end I think that’s mostly a question of personal taste, preference, who’s using it, and what one is used to. However if the purpose of an interface design is finding a beautiful balance of usability and discoverability, we can probably extrapolate a few general guidelines.

First of all, basic usability favors visibility. Anything you bury under non-standard gestures such as double-click or longpress are going to go unnoticed by most people, and only utilized when such a gesture is learned to be necessary. Ever see someone doubleclicking a hyperlink in a web-browser? That’s arguably a bad user experience design resulting in the wrong lesson: double-click to make sure it works. Online stores are still paying the price for this, having to disable the buy button once pressed a single time, or verbosely spelling out: “only press once”.

The swipe gesture is a powerful gesture when used right. In older Androids it invoked search, which was a terrible decision — as mentioned it was too easy to invoke, and just not useful enough to warrant such an action. When used to swipe between homescreens, or scroll a page, however, there are few better gestures.

Which brings us to the Back action. On Android it’s a permanent systembutton. It mostly does what you expect it to do, except for a few cases where it doesn’t. It’s supposed to always take you back to the previous screen, but what if you just launched an app and accidentally hit back, should you exit out to the homescreen?
On iPhone, there isn’t a system back button. Instead, back-buttons are added by the developer to the top-most toolbar of an app, ensuring they are contextual and pretty predictable. They’re even labelled (most of the time), which is hard to beat from a usability perspective.
On the flipside, they can be far away from your thumb, especially on large screen phones like the Plus models. To alleviate that, there’s the side-swipe, for lack of a better term: you swipe right from the left edge of the screen to go back to the previous screen you were on. Not quite as predictable, perhaps, as swiping left and right inside a photo gallery, but certainly convenient. The main downside of the side-swipe is that it can limit what apps can do with the same gesture. Once a gesture is reserved by the system, you should probably be mindful to not interfere with it.

Incidentally the side-swipe gesture suggests when it might be appropriate to add multiple actions to systembuttons: poweruser features. If a feature is complementary but not required learning in order to use an interface, it can add a lot of value to the experience. The side-swipe seems to do that — you don’t have to learn it in order to use an iPhone, Back buttons are still there. Android 7.0 Nougat has a similar power-user feature that is absolutely not required learning: double-tap the multitasking button to quickly switch to your last used app. It’s like alt-tab for your smartphone.
I’m not finding it easy to defend mapping the multitasking feature to double-click on the iPhone, however. It seems like a feature so valuable you’d want to make it esaily discoverable.

Siri vs. Google Assistant

We come to it at last, the long-press features. On the iPhone, holding the home button opens Siri. Same with Android 5.0+ which would fire up Google Now, and with the upcoming Pixel phones, Google Assistant.

Pixel phones feature a little colored-dot animation as you press and hold, which seems to be an attempt at adding discoverability to the feature. I don’t buy it. The onscreen buttons are small already so you’re likely to cover the animation with your thumb, and nothing beats a label regardless.
The Google Assistant is available through other means, though. The homescreen — the launcher as it’s called — has a big Google logo tab in the top left corner, which also invokes it. You can also use the “OK Google” hotword. So while the home-button longpress isn’t very discoverable per se, the Assistant itself should be. That embraces the poweruser nature of adding longpress features to systembuttons.

Siri is less discoverable. You have to enable it first (otherwise you just get “voice control”). Then it’s there on the longpress. You can also enable a hotword detection, “Hey Siri”, to invoke it. There isn’t a widget you can add, or an app icon you can tap.
Perhaps that’s okay, though. Perhaps the assistants don’t have to be discoverable, yet. They pass the litmustest of can you use and enjoy the phone without it, and in both cases I’ve never used them for much other than setting timers and reminders.

I can’t help but feel like that’s about to change. Assistants, neural networks and AI seem to be evermore encroaching on the smart devices we use. Today it’s smart replies in Google Inbox and on smart watches. But what do the interfaces of tomorrow look like?

It feels like most of the design of the original iPhone sprung from the concept of having that singular home button. What would a phone look like if that button, instead of taking you home, was your assistant?