13 Days with the Vision Pro -- how it feels to see the future

I just happened to be visiting my parents the day the Vision Pro was released, and like a good daughter I made them give up several hours of their day to hang about a suburban Portland shopping center so I could get a demo. When I finally dropped the cash and got one, according to my mom, she hadn't seen me as excited since I got a Barbie Dream House for the holidays when I was seven.

But almost two weeks in, is the headset still dreamy? Yes, but with some caveats. First off, it's not anywhere ready for prime time in terms of being a mass consumer device. Price aside, it's got some bugs and a very small app ecosystem, and while I'm leaning in and using it daily, the average person needs a lot more to be developed for it to be ready for mainstream use. That said, it's going to get there. Here's a few use cases I've tried out and found amazing, and a few that honestly aren't quite there yet.

Spatial video: this just might be the killer app. I've been shooting tons of spatial video, from strolling around my block to capturing my hands doing all sorts of tasks to chatting with friends. The video quality is fantastic and the depth is incredible. The downsides are that it's not easy to view that content many places, but that'll change with the ubiquity of the device. It's also such a good mainstream use case -- everyone loves taking photos and videos and if parents and grandparents can take hands free, super realistic videos of sporting events and birthday parties, that right there is enough to explode a market.

Entertainment: My first try using it on a plane was a bust, because I didn't know about putting the device in travel mode. Also, maybe it's just my account, but for some reason I'm not able to see movies I saved in Apple TV, only TV shows. Still, I'm more than willing to try it again on my next flight, and have been using it to watch TV at home. The lack of a Netflix app isn't great but watching content in the browser is more or less OK.

Fitness: Purpose built apps will solve some of this, but for right now, the Vision Pro is a bust for some types of fitness. I tried doing a yoga video but the battery pack kept getting in the way and the static placement of the browser made it tough to follow the instructor. I tried it walking on a treadmill and found it fine; I didn't try running out of fear of damaging the device.

Work: I've done a few Zoom calls but had some glitches and TBH I hate how my persona looks, but that's likely because I took it on a bad hair day and need to retake the photo. On those calls I was definitely less likely to multitask and get distracted, which is good. I've done other work (writing emails etc) in the headset with a wireless keyboard and found it about the same as writing on my laptop -- not an improvement, really, but not worse.

Overall, I'm still thrilled I got one and it's going to progress really quickly. The future isn't here yet, but it's coming a lot faster.

For spatial computing to power the future of work, we need to keep it simple

Last week I attended the IMSH Society for Simulation in Healthcare conference, and let me tell you, it was quite an experience. Many of the items on display there were not for the faint of heart, and one poor actress in a booth had to pretend to give birth to a fake baby at regular intervals throughout the day. Hope she was paid well.

There was also plenty of spatial computing and extended reality content on offer, much more so than in previous years according to regular attendees. And while that is certainly gratifying to see, a quick stroll of the floor for demos revealed that there are some areas we need to improve in before this technology can scale.

Many of the booths were manned by experienced docents, which is a best practice for trade shows -- unfortunately, this also revealed a huge limitation. I've been working in this space for almost nine years and if you need to stand next to someone like me and tell them where to click and what to do, how is that going to work when a medical student who has never seen a headset before puts one on for the first time. Sure, you can send docents and instructors out to sites, but that's a big cost and barrier to scaling.

I've long advocated that you should design for the least technical person who would reasonably be expected to go through a simulation. If your core audience is young men with gaming backgrounds, by all means, go nuts. But if your audience is broader than this and includes novices, you need to really scale back; otherwise, the frustration and friction is too great. People blame themselves when technology doesn't work, and give up with a negative impression.

What does all this mean? It means using voice or gaze to drive simulations, or limiting the amount of controller actions someone might need to use. It means lots of in-headset on-boarding, in a way that feels natural and allows people to fail early and without consequence. I once worked on a piece for a data warehouse company, and we started people off by having them use the controllers to make a snack in the break room. It allowed them to learn and make mistakes in a low stress environment, rather than having to remember which button to push when you were in the middle of a high stakes situation.

We're still figuring out best practices for all this, and at the end of the day, it's really exciting to see so many people playing in this space. But keeping it simple will be the path forward.

How We Evenly Distribute the Future of Work

OK, look, right now it's still in the nightmare fuel stage of innovation. You stumble into a convenience store on the way home from the bar to grab a late night snack, and you're confronted with the terrifying-looking robot creature you see above, casually refreshing the Red Bulls. The poor remaining human employees have to do a lot of cleanup on aisle five.

But a few years down the road, running into the inventory robot will be a common as seeing a human stocking cans. And there will be a human involved as well -- they'll just be in the comfort of their homes, anywhere in the world, wearing a headset and powering the robot. And as strange as it may seem right now, this could be a big part of the future of work -- and help level the playing field for disadvantaged workers.

Take food delivery, for example. It is extraordinarily difficult and dangerous work, but the demand exists so people rush to fill it. Autonomous delivery robots have thus far been underwhelming, but spatially aware human powered delivery robots could be the answer. And the people powering the robots could be anywhere in the world, thus eliminating some of the economic displacement of communities.

This could be an especially big benefit for women, who are often shut out of industrial work. If the robot is doing all the heavy lifting, the size and strength of the person powering it in headset is totally irrelevant. And if a robot is broken or destroyed, it can simply be fixed or replaced, whereas a human being injured or killed is a tragedy.

There will obviously be bumps along the way and we will need to make sure that people have other outlets for in-person connection if they are no longer in the streets or on factory floors. But the era of spatial computing and robotics could do a lot of good for workers around the world.

The Future of Work is Finally Here

Unless something absolutely catastrophic happens, one year from now, a percentage of the population will be regularly using Apple Vision Pro headsets for work. The size of that population depends on the availability and cost of the initial release of the headset, but rest assured, once Apple enters a market, it is go time. Apple's entry will also push other tech companies to innovate and iterate of face a crisis -- I was recently reminded of this when someone posted a piece about former head of Nokia crying that their company's disruption wasn't his fault (side note: if you haven't read The Innovator's Dilemma, you can just watch this video).

Of course, some companies are already adopting spatial computing and XR for work -- I've worked with Walmart and Amazon and Lowe's, among others, as well as NGOs, governments, and major consultancies. But the vast majority of businesses are still lagging, despite reams of research that using XR for training leads to better outcomes.

Now is the time for companies and industries to start investing in expertise in this space. This isn't fringe technology just for kids and video games anymore -- and even if it was, those kids will be in the workforce in the next few years. Much like designing for the web is very different than designing for print, designing for XR and spatial computing is a different skill set. It is teachable and learning, for sure, but organizations need to make sure they have experts who can do the initial teaching.

Over the course of the next several newsletters, I'll drill deep into the best practices and what folks need to know about how to create for this medium. If you want to get started early, you can check out my courses for Caltech and New Mexico State and reach out about consulting and speaking. Happy New Year!

How We'll Experience Higher Education Next

Two weeks ago I had the great opportunity to attend and speak at the EdX Global Summit and learn about the future of higher education. The industry is at a crossroads -- disrupted by Covid and an ever-increasing number of young people who are deciding college simply isn't worth it. Under political and financial pressure, some schools are cutting departments or simply closing altogether.

On the flipside, growth in online learning has provided a huge opportunity for some schools -- someone the event described MIT's online AI course as a "money printing machine." A sizable population of "Harvard" grads on LinkedIn have never set foot in Cambridge; simply pay for an online class, take a quiz or two, and you've entered the education elite, at least on this platform.

But internet learning only gets you so far. I say this as someone who has taught online classes for several platforms and is about to launch another -- the engagement just isn't great. I don't think I'm the only person who spends Zoom sessions half listening and half catching up on the news, emails, or messaging a friend about dinner plans.

There is a solution for this, and if you've ever read literally anything I've written in the last several years, you can guess what it's going to be -- that's right, VR. Early research out of universities using VR for education point to greater engagement, retention, and grades -- not to mention the leveling effect the technology can have. That lab at MIT or Caltech that has all the most cutting edge equipment -- now people everywhere can have access. You can virtually sit in on a lecture from a leading expert and then break off into virtual dissection sessions with people from all over the world.

VR is going to play a big role in education's transformation over the next several years. We need to let go of the idea of higher education being something you do for four years in your late teens and early twenties, and something that people do throughout the course of their entire life. People need to get comfortable pivoting, reskilling, and moving around -- and immersive education will enable that. If you're interested in learning more, drop me a line.

So, How Exactly Do You Tell a Story That Surrounds Someone?

Over the past several weeks of thinking and iterating, I keep coming back to one phrase when I think about the next phase of extended reality: stories that surround us. I truly wish I could remember when I first encountered this term, but it encapsulates the concept of how we're going to share content in the near future. Rather than being constrained by flat screens and limited activity, we'll be able to be fully immersed in story worlds and have more control over our interactions and participation. As exciting as this is, it also means entering uncharted territory for most people.

Fortunately (puts salesperson hat on) there is a brand new resource for folks who want to learn about this. In partnership with New Mexico State University, I'm proud to announce the launch of an online XR Storytelling class. Over the course of five units, I cover everything from the basics of extended reality to pulling together concepts, storyboards, and scripts, along with tips for filming, building, and distributing content. I also cover some of the research about why this content is so powerful. Students create storyboards and scripts and get feedback, and at the end, even receive a cool badge. The former Girl Scout in me was VERY excited to design that.

As more people and organizations start using XR, being able to design a good story will become an essential skill. I'm really proud of all the work I've done and the projects that will be released next year, and excited to help upskill a new generation of storytellers and creators. Students will also hear from a number of award-winning creators and experts who share their invaluable perspectives on how to make XR that moves the needle.

As an added bonus for my newsletter readers, you can get 25% off if you use the code "25OFFXRSTORY" when you sign up. And for my university affiliated readers, keep your eyes peeled as I am developing a series of educator specific workshops that will launch in 2024.

How We'll Remember 2023: The Contraction Before the Expansion

The older I get, the more I start to see how everything moves in cycles. At the risk of sounding a little too woo-woo, there are phases of contraction and expansion, and we're definitely in an era of contraction right now. It's not just the layoffs, although that's huge part of it -- it's the overall feeling the one era has come to an end but the next hasn't quite started yet.

The internet of the last fifteen or so years is sputtering to a close. Twitter, or whatever they're calling it these days, is losing users and clout. The big D2C brands that defined an era and an aesthetic are losing steam -- sure, you can still buy a Casper mattress, but it's a mattress, not a lifestyle with ubiquitous subway ads. WeWork is bankrupt. Now that money isn't free, services like Uber and DoorDash have raised prices and stopped caring about customer service, and use has slowed.

But while all those things are fading, the next wave hasn't quite crested yet. Web3 and the metaverse is growing, especially among young users who are bailing on Web2 socials for platforms like Roblox, but that technology is nowhere near mass adoption. So we're left in an odd in-between place.

In many ways, it feels like the early 2000s -- not just because of the current global political situation, but because one bubble has burst and another has yet to fully inflate. The dot com crash, for some commentators, spelled the death of the internet, but all those businesses have come stronger than ever. Pets.com and Webvan were just too early, and had to stumble so that Chewy and Instacart could run.

As odd as it sounds, this might be the best time to build. Lots of smart people have been freed from their FAANG golden handcuffs and want to try something new. Interest rates will come back down at some point, and money will start flowing, and hopefully flowing in a more strategic and equitable way.

When we look back at this era, it'll be defined by it laid the groundwork for the next iteration of human connection and communication. Head-mounted devices and avatar driven platforms will be so common we'll wonder how we ever lived without them. The smartest thing folks can do is recognize this start making things happen now, lest they wind up behind the curve.