What’s next for Design and User Experience (UX) in 2017

(updated Jan 2017 with data visualization and a comment on VR.)

Looking for my next big thing means talking to a lot of people. I’m sometimes asked what I see next for design and user experience. I also run into students who ask advice for their careers. The two answers are connected.

I see these trends over the next few years:

No great innovation in visible UI
The strengthening of invisible UIs
- Think voice activated devices like Amazon Echo
Big data
Machine Learning and AI (and bots)
More data visualization

Students should look for companies that are set up to succeed in these areas.

A Recent History of UI Design

First a bit of history.Â We need to understand how we got here to see where we will go next.

The “Aqua” look and feel in Mac OS X

iOS 6x.: The last skeumorphic iOS

Android 4.x “Icecream Sandwich”

Skeuomorphism and “Lickable UIs”

Around 2010, Skeoumorphic UIs driven by Steve Jobs’ push that a UI should be “Lickable” drove the Mac and iPhone to have round, glistening designs (the Aqua Look and feel).

Aqua Style Button: there were hundreds of tutorial on the web showing how these were made

Android (Icecream Sandwich) and Open Solaris (Nimbus L&F that I worked on) did the same. Graphic design had a heyday, and all of us spent many hours learning how to and creating aqua buttons (I know I did), and textures like “Eggshell”.

While I labored diligently creating aqua buttons and other similar UIÂ components, I was not a fan of skeuomorphism. If skeuomorphism was bad in Microsoft’s BOB, it was bad in other places as well. I prefer matching the UI to the device. The UI is a glass screen. It should not look like wood grain.Â (ignoring the factoid that I took many, many pictures of wood grain…)

Enter Touch and Gestures

When Apple introduced the iPhone, they disrupted many businesses. In the UX space, the biggest contribution of iOS was gestures. Touch (and lickable interfaces) existed before, but the capacitive touch screen really pushed the envelope on gestures. Pinch, pinch out, swipe, swipe from an edge, all these came into vogue. In the past few years (approx 2010-2013), the big improvements in UX were in gestures and aesthetics / visual design.

Two standout apps that really leveraged the technologyÂ at the time wereÂ Clear, the todo list app, and Roam BI Analytics, the data visualization app.

Clear Gestures

Some of the gestures in Clear.

Clear is a todo app that has a delightfully consistent set ofÂ gestures to navigate up, down and to add new lists and items to lists. Many of the clear gestures (like swipe to complete / delete) made their way into other apps. E.g. in iOS mail, you can now swipe to delete an email.

RoamBi Analytics took boring old data and made it sexy by using both vibrant visuals as well as the rules of physics. For example, its Pie charts have inertia, and when you spin them you hear a clicking sound. The combination makes it delightful to play with your data, which in turn should make you better at understanding it.

The end of Skeuomorphic, the start of flat

Windows 8 came along and introduced edge gestures. But its greatest contribution to the UI world was the flat look and feel (Metro). Tiles, buttons, icons, everything was flat. And it worked.

iOS 7, the first iOS to move to a flat UI.

List views in Android using the Material Design language

Soon after, both iOS and Android switched to flat designs. Google’s Material design is considered by many designers to be a great design language. I like flat designs because they are true to the material (no pun intended).

The flat design languages focus on typography and color palettes, designers spend a lot less time doing buttons and curves and the like. The focus pushed designers to think about flow and layout and less on visuals.

The current (2016) state

We now have really, really good designs for smartphones, the web, tablets, convertibles and desktops. They use flat designs, touch, gestures and are getting more sophisticated. For a while, we’ll see small changes. Force touch, watches, activity trackers, none of these has made huge changes to UI design.

UX 2020

(Apologies to my old team where we spent a lot of time talking about UX 2020.) The big shift in UX in the next few years will be not at the front end, but at the back end. Our devices are all connected to servers in the cloud (just the way theÂ computer that I’m writing this blog post onÂ is connected to servers hosted by Dreamhost).Â There are gobs of data sitting on those cloud servers. The companies that find innovative ways of using those gobs, and the interfaces that work seamlessly with them are the ones that will create the next big move in UX.

UX designs will be driven by:

Cloud design. Software will assume data in the cloud, which is quite different from UX that assumes that data is on the device. As an example of the difference, when I type a document in Microsoft Word, the data is local. For this blog, the data is in the cloud. Gestures that users make on their devices must be supported by cloud data. For example, when a user types into a field, that data must go back to the cloud and the cloud throw up suggestions. UXs that understand this architecture and leverage it this will be successful. Of course, this transition is well under way, and “cloud first” design is the way to go.
Big data: Closely related to cloud design is big data. Specifically where the companies who run the cloud services figure out ways to leverage aggregate data to help individual users. For example, when the search field above is driven by big data, it will suggest trending topics.
ML and AI: As organizations learn from their globs of data using machine learning and AI, UI designs will need to take this into account. Some of what this means is unclear, but it is safe to assume that the UI will morph based on the result coming back from the cloud.
Data visualization: We are collecting ever more data, about people, devices, interactions, behavior, the list goes on. Human beings need help understanding the data. I expect there will be lots more work on data visualization, making data visualizations more common. The dashboard concept above is an example.
Machine Learning, AI will enable invisible UIs. Specifically, voice recognition (like Siri, Cortana and Google Assistant) and the ability to generate sentences and voice responses will make devices where the only UI is verbal. Amazon Echo is a front runner, as is the voice recognition UI in cars. Sadly, we have this in two of our cars, and they both are hard to use.

In order to make these experiences great (take the car voice command for example), UX designers need to work on hitting close to 95% success. This likely means reducing the voice recognition features and focusing on the most common use cases, and making those successful.

Advice to UX designers

In order to succeed as UX designers, we need to excel atÂ cloud based design, and learn big data, ML and AI. We need to understand what these technologies can do, and understand how to apply them to our designs.

I asked a couple of folks to review this post, and MV asked: “I am curious to know and would like you to expand on how you think UX will be more focused on the backend rather than the frontend. I see your examples but would also like to know how you imagine what they day of UX designer focused on the backend would look like.”

Designers will need to design the front end while “looking through to the back end”. At the front end, users will see the UI and will have goals to meet based on what they have done, and their expectations. The responses will be generated in the back end. Designers will need to tune the front end to make the UI showÂ the intersection (subset)Â of what users expect and what the backend can accomplish. In a lot of cases, this will mean fewer capabilities visible at the front end till the back end’s capabilities improve.

My thoughts on VR

Some folks have suggested that VR will make a breakthrough in 2017 and become commonplace. I disagree. I think that VR relies too much on VR headsets. Until either VR headsets become common place, or VR doesn’t need headsets and can somehow render on phones, tablets and desktop screens, VR will remain a niche.

Advice to students

I mentioned that I’ve recently met students in design and HCI programs looking for advice on what to do and where to look for jobs. My advice is:Â look for companies that are in the cloud, who can leverage big data and apply ML and AI to it. They will innovate more rapidly than the others and set the direction for design. You need all of them, just being in the cloud or just having big data will not be enough.

😀

Thanks to MayaV for her feedback on this post.

The Best Airline Boarding Sign Yet

Alaska Airlines Boarding Display

This is the best airline boarding sign / display I’ve seen to date. Its by Alaska Airlines. The display hangs close to the gate and prior to boarding has things like departure information (or arrival info if a flight is arriving). When the airline is ready to get people on board, the display switches to this image. It quickly lets you (a passenger) figure out if your row is boarding or not.

At some point in the future I hope to see the displays include a diagram of the rows indicating which rows are boarding. Many flights board “from the rear of the aircraft”. A diagram showing the rows would make this easier to understand.

Why Water Saving Flushes are “Wired” Wrong

Snapshot and closeups of the watersaving pressure flush

As a user experience designer, I can’t help constantly evaluating the designs of new user experiences. I’m also very conscious about conservation and believe that reducing consumption is critical. So I was delighted on a recent trip to Oregon to find that one of the visitor centers was fitted with a two setting flush. Push the lever one way and get a small amount of water, push the lever the other way and get a regular flush. Most of the time, the toilets are only expelling liquid and the little flush works fine. This saves water.

The problem is that the flush doesn’t match the UE principle of making the easiest behavior match the default action. For example, in a dialog box, this is like putting focus control on the “OK” button. The most easy “gesture” with the lever is to push the lever down. But pushing down does a regular flush, which most of the time wastes water.

The flush should have been designed with a push down for a small flush, and a pull up for a regular flush. That would match the easiest action (push down) with the default behavior (small flush). This would save the most amount of water. The small percentage of users who pushed the lever down when they should have pulled up will either repeat their action or read the instructions and pull the lever up.

How to Confuse ATM Users

Closeup of the Envelope Insertion slot in an ATM

ATMs are great. You’re no longer restricted by where your bank is and when it is open. But, I wish they’d spend a little more time getting the designs right. The image above is from an ATM I used. The ATM has Braille, which is wonderful. You can see the Braille in the strip above the envelope insertion slot. I can’t read it, but I assume it says that the envelope slot is below. This is a very nice touch, great attention to detail. In addition, all the edges are gently rounded, there are no sharp corners, overall a very well executed design.

Given the attention to detail, what I don’t get is the image showing you how to insert envelopes into the slot. Its bad…

The biggest problem is that the orientation of the envelopes+slot icon and the arrow is wrong. It looks like you’re supposed to pull the envelopes out of the slot.
The second is that the envelopes look like they are going into the slot along their width, rather than the edge.
It also looks like you’re supposed to put in a stack of envelopes.
The arrow may have been intended to indicate where the slot is. However, that is confusing. It is so close to the slot+envelope graphic that it looks like it telling you what action to perform.

ATM With the insertion graphic moved to the slot

A better design would be to move the envelope insertion graphic to the deposit slot (and fix its orientation) like the image above.

Companies are always looking for ways to cut costs. When I managed Sun’s industrial design team, I learned about keeping costs down. The graphics are printed onto the metal using silk screening. Images on two separate bits of metal is twice the cost. The two bits of metal (silver and gray in the picture above) might come from different sources. The silk screen should not bother trying to tell you where the slot is. It is large compared to the graphic that it is hard to miss. Also, trying to show the envelopes / envelope stack is too much detail, and adds to the confusion by suggesting the envelopes go in face down. Simplifying the graphic has a better result.

Simplified Silk Screen icon

I’m guessing the team that designed the ATM had a decent design. Most likely, someone who was responsible for cost cutting made some design changes without consulting the design team. Whenever Sun’s team needed to cost cut, they worked with the ID team who modified their design to fit the constraints.

What phone ads say about design philosophies

Close up of Droid x "Eagle Eye" ad from Google / Verizon

Snapshot from Apple "Haircut" ad for Facetime

The ads for Droid phones and iPhones are like windows into the design souls of the teams behind them. If you watch TV, its hard to miss the ads, but if you haven’t seen them, they are embedded at the end of this post.

The iPhone / Facetime ads are totally about the user’s experience. They are shown to you as if you were the person holding the phone. This makes you part of the ad. Talk about “an immersive experience”… The ads are so good at communicating their message that don’t refer to the product (except for the end credits). The are all about communicating the experience that their customers have. This reflects the design philosophy behind the iPhone, where the experience rules. The ads are also funny and / or tug at your heart strings, which doesn’t do the product any harm.

The Google / Verizon Droid ads are different. They focus on product features, like the “1 GHz Snapdragon processor.” They tell you you can’t handle the speed. They tell you that the technology is unlike anything you’ve ever seen before, that your eyes won’t be able to keep up, though you’d be able to find your human friends if you had any.

My initial reaction was that the Apple ads were brilliant and that the Droid ads were geeky. The Apple ads reflect the very best of user centered design and are completely focused on expressing the user’s experience. The Droid ads refer to arcane details about the phone that are seem relevant to the experience. But there is more to it than that.

The Apple ads are aimed at people who don’t care about the underlying technology, but just want it to work. That probably describes 90% or more of iPhone customers. The ads show them an experience they could have. The “There is an App for that” iPhone ad series was similar: it talked about what users could use their iPhone for. iPad ads are the same as well. All this points to a consistent philosophy of getting things to just work for the user.

If the world was made up only of people who wanted things to just work, all smartphone buyers would buy an iPhone, and the phone wars would stop. But the very healthy sales of Android phones tell a different story. There are a whole set of folks who do care about speed, and processors, and features, and Droid phonesÂ suit them perfectly. I suspect they find the canned experience of an iPhone stifling. Its much easier to tinker with a Droid than with an iPhone, and that appeals to many folks. When I was in Taiwan earlier this year, a kid said to me that his HTC phone was better than my iPhone. I’m sure that in many ways it is.

The point is that the technical features of the Droid phones are part of the experience. Some people like to drive sports cars knowing that their car is capable of speeds they will never use. (And I won’t get started on SUVs…) My wife will tell you that I buy tools which can do many things that I’ll never do. Just like that, some folks will buy Droid phones because of what they are capable of. They like the freedom to customize what is in their phones, and like the possibilities their phone enables.

As designers we need to keep in mind what drives users. Some are very happy with Apple like simplicity, some want the features. The other thing that designers need to be true to is the organization’s design philosophy. The product (and the ads) should reflect the design philosophy, just like the Droid and the iPhone reflect the philosophies of the companies they come from.

So: Your Product Team Wants “Innovation”

If you’ve ever been asked by a product team for an innovative design, you should read Don Norman’s recent article. Its in the March – April 2010 issue of Interactions Magazine. Don has a way of pissing people off with his ideas. This article “Technology First, Needs Last: The Research-Product Gulf” is a no exception. He says it was controversial even before it got published.

I like controversy because it makes me think.

Don says: “Design research is great when it comes to improving existing product categories, but essentially useless when it comes to breakthroughs.” He says that most design techniques lead to small incremental changes in product design. Breakthrough innovations are difficult. There is an interesting parallel in evolution. It is now believed that simple genetic changes based on Darwin’s theories caused only minor changes in species. The wide range of variations in species that exist today were caused by discontinuous changes brought on by mutation. It is as if genetic changes, like design theories and methods cause only small changes: for big leaps we need mutations. Design mutations perhaps?

If Don’s right (and I think he is), this is a big problem for a design team. Every design team gets asked for innovative designs. But, if all the techniques we have only make incremental changes, then we have a problem.

So what can a design team do when it gets told that the next release needs to be innovative? My reaction to this is something along the lines of “GREAT! Now lets talk!” I’m not into complaining that the product team doesn’t know anything about design (they don’t have to); or that the request for “innovative” sets the design team up for failure. I am delighted when I get to work with a product team that wants an innovative product. The opposite is a death march. When a product team says that the next product must be innovative, I hear two things. First I hear that they care about their product, though they may not have the vocabulary to express what their design needs are. Second I hear the product team say “Hey, we need a great product, can we count on you to deliver a great design?”

What we as designers need to do is to change the conversation. The conversation needs to move from creating an innovative product to one about the right design. Along the way the design team needs to build credibility and get the product team to trust the designs. This is not hard.

Interaction designers are great at understanding what users need. Most of us don’t apply that skill to understanding what the product team needs. We need to apply a little of that skill to the product team, to understand their motivations, what they consider a successful design, and what they consider innovative. That is a series of conversations between the design and product teams, and they don’t have to be formal. The conversations must be at two levels. One is the mundane: what are the feature and performance goals of the product. What should a user be able to do? What is the competition? What are the performance metrics? etc. This is all boiler plate design work that we all do.

The other set of conversations is aspirational. The product team needs to love their product. The design team needs to understand what it will take to make that happen. They must understand what the product team considers innovative. They should look at examples from the product team. They should have examples of their own ready to show the product team. By showing the product team what they consider innovative (and why), the design team builds credibility and shows that they know what innovation is. That helps answer the hidden question “Can we count on you to deliver a great design?”

The designers will usually find that the product team’s definition ofÂ innovative can be broken down to a few relatively simple things. Sometimes its color: some teams like dark colors, others light. Some like gradients, some animations. These attributes serve to narrow down the range in which the designers need to explore, and can usually come up with a a set of variations that the product team likes and can choose from.

Next the design team needs to establish that they have the design chops that the product team is looking for. Once the product team has confidence in the design quality, the discussion will move away from “innovative” to “the right design.” For products that are old and crufty establishing design cred is surprisingly easy. The designers can build prototypes that do away with the cruft, and create a clean, simple experience.

Caiman Installer Progress Indicator

The Caiman installer for Solaris / Open Solaris that we worked on a couple of years back is an example. The lead designer on this went back to the drawing board to design the installer. He thought about what a modern OS install experience would be like. He then backed up his ideas with thorough research of existing installers. He built an HTML prototype that looked like the real installer. This prototype was amazingly ‘lifelike’ and even faked a BIOS screen. When he demoed the prototype, he would put the browser in full screen. This made the prototype look completely convincing because the whole screen was taken up by the prototype and it looked like the OS was actually being installed. Two things worked in his favor. First, his research of the other installers was so solid that no one could tell him about an installer because he’d seen them all. Second, the prototype was magical. Everyone knew it was a web page. They could point their browsers at it. Yet, it looked exactly like the real thing. That bit of magic really made the product team pay attention. And when they looked at the prototype, they realized it was exactly what was needed, and the designer had all the credibility he could ask for. The team’s focus then shifted from creating an innovative product to creating the best experience possible.

Another way to establish that the designers have great design skills is to show the product team lots of quality variations. This can convince the rest of the product team to go with the design team’s recommendation.

Java Store R7 home screen

Here is another example: We were designing the Java Store. For the ‘home screen’ we needed a way to let users explore the store content in a “non-linear”, sort of “guided exploration.” Its a bit like wandering down the specials aisle of a physical store. There was general agreement among the product team that we needed an “innovative” animation, but a lot of concern about what that meant. We had to move the conversation to one about “the correct” animation. One of the designers in the team was very very prolific, and could produce lots of animations in short order.Â He teamed up with a visual designer who could keep the visual design quality very high. Between them they created a whole series of animations, some unique, some frivolous, all outstanding.

The design team had two concerns. First, we wanted to show that we could get the “right” animation. Second, we were concerned that the home screen was too busy. There was too much going on, and the users would get overwhelmed. So we showed the product team lots and lots of animations that we worked on. That made it clear that we knew what were doing and that we’d been quite thorough in our exploration. Once that was established, we showed the product team some of the things that concerned us, and they agreed. The animation that the store ended up is a simple flow, sort of like a “river”, in which product icons float by on a stream. Its pleasant, almost soothing. The team loved it. You can try it yourself: download it from the Java Store website.

So when a product team tells you that they want an innovative product, grab that as an opportunity to learn more about what they want, and you won’t be disappointed.

Kartik Mithal

Design Cohesion

Design

The Best Airline Boarding Sign Yet

Why Water Saving Flushes are “Wired” Wrong

How to Confuse ATM Users

What phone ads say about design philosophies

So: Your Product Team Wants “Innovation”