/ Data Science, Design

Computational invaders are attacking the UI designers!

Or are is this just a fantasy?

World of design is rapidly changing. Generative and algorithmic design are names for emerging approaches to optimize user interfaces (UIs), improve customer experience and drive better business. More intelligent applications call for new design guidelines but designers can also benefit from computational techniques in several ways.

I had a chance to talk with Professor Antti Oulasvirta of Aalto University in Finland about how he sees the field right now and in the near future. He has for several years been running a highly successful research group on interface design and has an excellent vista point to the domain.

User interfaces research group at Aalto University. Photo by the group

Antti is especially known for his work in attempting to solve user interface design problems mathematically, through different ways of optimizing. He’s group has in the past demonstrated for instance the Sketchplore proof-of-concept that visioned how designers might receive assistance from an AI. But now we’ve got a chance to hear about the latest and yet unpublished works!

I have personally very recently switched my brain to a model in which we are currently living an era of data-driven design in UI design. This means that we’re all set up for amassing more data and possibly even insights, but automation and digitalization falls short on helping to make decisions or generate new designs. As the next generation, I expect generative (or algorithmic design) in which digital assistants will do much more than collect your data. Generative techniques would further fall into assistive and agentive bins (working together and working independently, respectively) But what does professor Oulasvirta think of my hypothesis?

What do you think of the division into assistive and agentive tech?

To me, data-driven design includes generative design step, the creation of UIs. It is preceded by an inference step, which refers to interpretation of the data. But generative aspects don’t indicate a total lack of control as agentive technology might imply. There are different levels of control in relation to automated systems. For instance, the autonomous vehicles don’t offer total on/off control, but the user (or designer, if the system is about design) will need to steer the process.

Antti Oulasvirta – Aalto University – Photo by Lasse Lecklin

So in other words, the division of assistive and agentive technologies These tools free up designer’s resources to focus on different aspects of design work.

What ’s the need for the artificial intelligence in UI design?

Much of design effort is educated guesswork. That is very ineffective. Models can embody information that designers can’t grasp intuitively or which takes unreasonable amount of effort to fully master. It would be better to let algorithms to solve really complicated problems, such as configuring Microsoft Office menu structure, which have several dimensions of requirements for very heterogeneous users.”

Much of design effort is educated guesswork. That is very ineffective.

In Oulasvirta’s vision, the designer does not drop out of the loop, but their responsibility changes to be a person on look effective data to feed optimization algorithms. Designer becomes a modeler, a data engineer (his words), of sort who takes care that the problems which can be solved get the right inputs. And of course shaping the output of AI design tools to fit their ultimate purpose.

What will happen in next 5 to 10 years?

We can automate the solving of certain design problems. There are three conditions that the design problems must meet in order to be solvable: goals, design space and formalized requirements in what to optimize for,” Antti explains.

He believes that future systems can automate design maintenance and creation of new hypothesis for A/B type of testing. Example, if web page components may, predictive models can forecast how new components will impact the outcome.

Real design spaces are extremely big. It is very difficult to make good guesses of which states would be better than the current one.  Our group is currently building a next generation of Sketchplore. It will take on much bigger problems, more effectively than the original proof of concept. We have already solved several of the very difficult problems on the way. “

Algorithmic design is not free from cold start problems which have traditionally bothered, for instance, recommendation engines.

With current solutions, we can’t yet create totally new layouts and ideas. Modeling needs data and by definition, we can’t have data about solutions that do not yet exist.”

Well, sounds that human designers and physical perspiration will have their place in the near future as well, something that should please even the old school designers.

What can algorithms do for designers right now?

It seems future always starts tomorrow. Is algorithmic design also but a science fiction?

Antti responds: “We can pick balanced color palettes, make sure colors work for color blind users or pick optimal web page templates. But UI optimization is a 20/80 problem, with 20% of work we can get 80% of the functionality. Getting 100% takes, much, much more effort.

Antti continues by stating a case in which their group has been stomping head against the wall to get to the final 100%:

Currently, we’re working with French standardization officials to renew French keyboard layout particularly for special characters that are too difficult to type. This problem is not easy. The size of the problem space is in the degree 10^179. Solving the challenge is further complicated by the demands of data engineering, covering all special use cases. We have mathematical proof that we can solve it which guarantees that our solution is, if not optimal, very close to optimal. This will in future lead to an improved keyboard layout.

problem space is in the degree 10^179

As we talk, again and again, we return to the issue of data. Antti emphasizes that the right, high-quality data is key to unlock to the potential of computational design:

“In most cases, it is not really only about algorithms, but also critically about data. Without the right data, the optimization is impossible.”


Would you like to learn more about the methods Antti sees so important for the future?

That’s possible earlier than you think!

SC5 is organizing a hands-on workshop around UI optimization methods in Helsinki to spread these new design skills in the world. Sign up and join us!

http://ui-optimization.sc5.io