Adobe’s next big bets are on AR and mixed reality software

By the end of this year, designers will be getting three new Adobe apps, which are all part of the company’s vision for the next era of creativity. This next generation of apps will focus on mixing real-life, physical elements with digital ones, and it will emphasize experiences “outside of the glass,” says Adobe chief technology officer Abhay Parasnis. They’ll also be interconnected through mobile and desktop experiences so that users can switch between apps on any device.

“Every app we’re building — Aero, Fresco, Photoshop on the iPad — you will see us push to be cloud native, making collaboration a lot simpler,” Parasnis told The Verge. Perhaps the best example of the future Adobe envisions is another project that’s still in development called Project Glasswing. It’s a mixed-reality display prototype that would bring all of Adobe’s apps into the real world in the form of Photoshop or After Effects layers on a transparent screen in front of real 3D objects.

Parasnis stopped by The Verge to demonstrate all of the apps in development on an iPad Pro, starting with Fresco, the upcoming raster and vector illustration app for the iPad. He drew some brush strokes on the canvas, saved it, then opened the same file in Photoshop for the iPad, where the brush strokes remained. Parasnis says the goal is to make the apps “feel the same, not just in the UI, but in the paradigm of cloud documents.” Files should seem local, while the actual work is happening in the cloud, instantly syncing and saving every change.

Fresco, which is meant to mimic how watercolors and oil paints behave on paper, is the culmination of eight years of research using Adobe Sensei’s AI platform. The first version required high-end GPUs to run the engines, and a big part of the research time was spent making sure that the realistic strokes wouldn’t drain battery life and performance. “To an artist, nothing is worse than feeling like there’s already a level of interaction with a glass and a digital screen,” Parasnis says. For Adobe’s research team, cutting down the latency and getting the brushes and paints to mix naturally in real time represents several breakthroughs.

What Fresco is meant to achieve by bringing the analog painting experience into digital screens, Project Aero aims for the opposite. The augmented reality authoring tool allows for designers to create immersive experiences through the Adobe tools they already know how to use without having to know how to code. According to Parasnis, Aero started with the question of “Can we take digital output of creations from Photoshop, Illustrator, InDesign, and Dimension, and seamlessly bring them to life in physical 3D spaces around us?”

Aero was previewed at Apple’s developer conference last year, where Adobe announced that it would work with an AR file format created by Apple and Pixar. The unified format — USDZ, which is limited to iOS for now — means that files can be exported to show up in iPhones on regular apps like iMessage and Safari, without having to download separate AR viewers.

The app lets artists use drag-and-drop modules to create animations and responsive AR experiences that automatically take into account factors like physics and lighting, which would normally require knowledge of programs like Unity and Apple’s ARKit. Aero also gives artists the ability to fine-tune their AR creations using 3D textures and materials from Substance, the texturing software by Allegorithmic, which Adobe recently acquired. A handful of artists have access to the closed beta of Aero, but the app is taking requests for early access here.

Still, Parasnis recognizes the limits of AR, which binds users to screens and requires devices to look at the world. “For AR and immersive to go mainstream, both the hardware and the software has to happen,” Parasnis says. Adobe’s solution to that is Project Glasswing, a transparent LCD display that could potentially be used in retail environments or public spaces like hotels and museums. The prototype display can blend digital and physical environments without the need for glasses and headsets. “We like it as an alternative form of AR,” he says.

Parasnis stresses that Project Glasswing isn’t a hardware product; instead, it’s a means for Adobe to showcase how its software can be used in real-life applications, especially in retail. Adobe intends to partner with other companies to “take the hardware forward” because, Parasnis says, “We don’t actually want to be in the business of making the display.”

Image: Adobe

For retailers worried about how to compete with Amazon, Parasnis thinks AR has the potential to give them an advantage. “If I can make my physical retail environment and spaces more interactive,” he says, “it gives users a reason to actually leave their screens and go into physical spaces.”

[“source=theverge”]