Smartly dressed experts, most with advanced degrees and some who traveled from other countries, took turns prodding a bell pepper inside a mock hospital room last week, their eyes covered by what looked like a high-tech sleeping mask.
No, this was not a tryout for some sort of culinary competition or a fad cooking class, but rather an exploration of high-tech surgical training. The mask — Apple’s Vision Pro — contained a pair of high-resolution screens, one for each eye, providing a digital canvas. Receiving a digital video feed from an endoscope, the kind commonly used to conduct minimally invasive surgeries, clever software turned the pepper into a human kidney on the headset screens, allowing participants to practice a tactile medical procedure with salad-level risk.
These were the sorts of scenes that unfolded during the inaugural session of San Diego’s Spatial Computing Health Care Summit, a two-day event convened by Sharp HealthCare at its new innovation center in Kearny Mesa.
Hundreds of physicians and technologists from as close as San Diego and as far away as ’s University of Leipzig came to spend Thursday and Friday sharing their experiences using virtual reality to augment health care delivery from surgical procedures to mental health consultations.
Participants shared early wins, such as increases in surgeon comfort during long procedures. Head-mounted displays, it turns out, can be more comfortable than staring at wall-mounted displays when viewing endoscopic cameras inserted inside the body.
There are plenty of plans to go deeper. The higher resolutions allowed by the latest crop of headsets allow specialists to see finer detail than they ever could before. And the latest in software aided by artificial intelligence algorithms allows the immediate juxtaposition of all sorts of health data, from guides on where a surgeon should make an incision to a dashboard of a patient’s full medical history, spread across a doctor or nurse’s entire field of view, allowing faster study of everything from trends in medical tests to three-dimensional examination of full-body scans.
Sharp, looking to make its large network of hospitals and medical offices more efficient, purchased a large number of Apple Vision Pro headsets when they were first released last year, drawing the attention of the Cupertino tech giant. While not sponsoring this week’s event, Apple helped pull it off, dispatching a team of experts to guide participants through demonstrations that included medical applications, including the latest enhancements from Epic, the Wisconsin-based company that has gradually become the market leader in electronic medical records.
The event drew the attention of Susan Prescott, Apple’s vice president of worldwide developer relations and education markets, who personally attended Thursday, speaking to an enthusiastic crowd in the innovation center’s main auditorium.
Asked why this event merited a trip to San Diego, Prescott said she has been impressed by the work that Sharp, and also UC San Diego, have done in using the Vision Pro in health care settings.
“This center is amazing and the people they’ve aggregated together just put the whole thing up at the top,” Prescott said. “So who wouldn’t want to be here today?”
UC San Diego’s Joan and Irwin Jacobs Center for Health Innovation was technically the first in the nation to evaluate the Vision Pro in minimally invasive surgery, launching a clinical trial in the fall of 2024 as Sharp was working on efforts to test the device in multiple settings at the innovation center.
This work caused a fair bit of back-and-forth with Apple, and Prescott said that the interaction has informed the company’s further refinement of the technology and the software that drives it.
“They asked for things that made us go back and think hard about how to incorporate, you know, camera access, how to incorporate object tracking, how to incorporate bar code scanning,” Prescott said.
The nature of health care, which often calls for consultation, one physician weighing in on or observing another’s work, has also driven development.
“There’s definitely a lot of there, and there’s more as you talk about patients, like learning about our guest mode to enable multiple people to use the same device,” Prescott said.
What does all of this add up to for patients?
Many have speculated that using virtual screens to overlay data onto the real world of medicine can improve safety and make busy health care workers more efficient. Whether that translates to less time watching one’s doctor scroll a mouse during office visits, and more time spent listening intently to patient concerns remains to be seen.
Dan Exley, Sharp’s interim chief information and innovation officer, said that these questions are still in the active exploration stage, though there has been enough progress made in the first year to hint that it should not be long before the technology being demoed this week ends up in mainstream use.
“I know that we will start to see our clinicians use these devices to improve the way that they’re managing information and practicing medicine within this year,” Exley said. “How rapidly that scales outward to other caregivers, to our patients, to other use cases? That will be exciting to see.”
