February 1998
Two conferences, organised by the same organisation, ran sequentially at the same venue in San Diego between 27th and 31st January 1998: Medicine Meets Virtual Reality 6 (MMVR) and NextMed: The End of Healthcare? I attended most of both sessions, although some overlap between the two conferences, and a day of parallel sessions at MMVR, meant that I could not see everything.
NextMed was a highly speculative conference, dealing in future projections rather than current realities. As such, there was a great deal of hype and relatively little solid information. An exception to this tendency was the talk by Karl Pribram, one of the best known and most highly regarded experimental psychologists (of the old school) around. Although qualified in medicine and experienced as a neurosurgeon, he is perhaps most generally associated with "Plans and the Structure of Behavior" the book (co-authored with Miller and Galanter) which helped launch the "cognitive revolution" that saved psychology from the behaviourists (but not from the horrors of cognitivism).
Pribram was dismissive of the work in neuropsychology that focuses on nerve impulses, as if nerves were like simple electrical circuits. Dendrites and axons do not work on an "all or nothing", digital basis. Rather they exhibit "all or something" behaviour: what matters are the patterns of evoked potentials, not the effect of impulses switching on or off in the brain. Nerve impulses are thus not the central things, they are only used to communicate the results of computations done through graded potentials. Unfortunately, through our experiences with comparatively simple electronic systems, we seem trapped into talking about impulses rather than fields. He used the metaphor of a piano to attack the popular notion of feature perception built up through the detection of low-level stimuli such as "line detectors". When we hear piano music, we experience music produced by vibration of the piano strings. But there are no intermediate levels needed to produce these results. Music is a pattern of activity, not the end result of a sequence of increasingly complex features. Perception to Pribram seems to be the result of a process analogous to holography. He is also associated with work to re-examine Freud's contribution to psychology, especially his apparently prescient specification of the link between certain psychological states and measurable physiological changes.
George Poste of SmithKline Beecham, one of the world's major pharmaceutical companies, give an overview of commercial interests in molecular medicine, population genetics and individualised care. The basic message was that knowledge about the molecular (genetic) origins of differential susceptibility to disease will revolutionise healthcare towards prevention and customisation to individuals, rather than cures. The focus of this work is genes that determine how other genes express themselves. Most diseases have multigenic origins, and the new discipline of genetic informatics deals with such things as the probabilistic evaluation of disease likelihood in particular individuals. This view sees people as complex systems, with medical informatics in general lying at the intersection of health data, telecommunications and computer systems. The approach requires the development of large-scale population-genetic databanks, and ongoing monitoring of individuals' status, leading to predictable but complex ethical, legal and social issues.
Richard Satava talked about a practical project to develop monitoring technology that could help with such an overall aim. This consisted of a bathroom suite that could automatically take various biological and physiological samples during normal use, and which would be linked to medical information systems. This project has, however, attracted the unfortunate nickname of "The Millennium Toilet".
A disappointing talk by Edythe London, Director of the Brain Imaging Center at the National Institute of Drug Abuse, had the interesting title of "What Can Brain Imaging Tell Us About the Mind and the Soul?". The answer seems to be "little" in the case of minds, and "nothing" in the case of souls, although a claim was made that analysis of brain patterns in adolescents can help predict tendencies towards, for example, cocaine addiction.
On the mechanical side of medical intervention, there were descriptions of work on miniaturisation, and on building replacement parts for the brain. Theodore Berger described interesting, though still very early, work to produce what he termed the neuron-silicon interface. The idea is to develop biologically-realistic neural networks implemented in analogue VLSI, using photonic interconnections to produce extremely small multi-chip modules. These modules are then exposed to cultured nerve cells which "grow over them", being especially attracted to the metal parts of the printed circuits. This growth yields neuron-silicon hybrids that can be implanted in existing neural tissue (i.e. in someone's brain). This approach is being applied to the development of prosthetic implants for the hippocampus, to help remedy defects in long-term memory function (for example, the clinical condition where short term memory functions well, but no new long-term memories are formed - making learning impossible).
The buzz-words of this conference were "interventional genomics" and its synonym "molecular cell engineering", a discipline that is predicted to lead to genetic re-designs of humans, permitting extreme longevity or even (God forbid) immortality. Several speakers described progress in these areas, although at least one pointed to the dangers of virus or prion (as in "Mad Cow Disease") adaptations to these changes leading to epidemics of new diseases. However, there was little discussion of the moral and social impact of the seriously-made prediction that our children's children (and all of their descendants) could live forever. There is no doubt, though, that molecular cell engineering will have an increasingly important role in screening, diagnosis, prevention and treatment of illness. Interventional genomics is coming, whether we like it or not.
Only one speaker, Robert Bohrer, Professor of Law and Director of Biomedical Programs at UCSD, seriously addressed the social aspects of further advances in health care. He pointed out the well known fact that although United States' spending on health is by far the highest per capita in the world, the US ranks 23rd out of 29 countries in the industrial world in terms of average level of health. As most people can probably guess, this is because most of the spending goes on the relatively wealthy and well-insured, whereas those without health insurance have the same low levels of life expectancy to be found in most third world populations. The gap between these two tracks of society is widening, and technology is certainly not tending to narrow it. In the future, thanks to genomics, computational biology and immunology, people with money will increasingly use medicine for enhancement rather than therapy, since they will already be free of major degenerative or infectious disease. The US will then develop into a three-track society: at the top, the rich and genetically-enhanced, living almost endlessly; in the middle, the reasonably well off and healthy, but stuck with unenhanced genes; and at the bottom, an underclass with little money, poor health and short life expectancy.
This, the sixth conference in the series on Medicine Meets VR, was very well attended, with an estimated 1,000 participants, from the US and Canada, Europe (Germany had a noticeable presence), and Asia (Japan, South Korea, Hong Kong, Singapore). This was a "high visibility" conference at which most of the key players were to be seen, presenting updates of their work.
The conference was a mixture of exhibition show and technical paper sessions. The quality of the technical presentations varied greatly, from interesting new technologies to introductory presentations such as "Applications of volumetric rendering in clinical practices". In general, I felt that there were far too many presentations: some 135, over four days. One had to switch madly between parallel sessions to get to hear papers that were arbitrarily classified under "technology" or "education". Paper acceptance is based on abstracts only, which explains the diversity to some extent, although there also seems to be a philosophy of breadth, rather than depth in selecting contributions.
The first day was The Richard Satava Show. Col. Satava, MD, formerly with DARPA and now at Yale Medical School, is in charge of granting money to many projects on medical VR and is therefore at the centre of the community. His humble motto is "I have the vision, you guys realise it". He had two sessions to update the assembly on the progress of his grants: one about DARPA projects and the other about a new wave of funding from Yale-NASA (mostly the same projects that got funding from DARPA now getting the funding from Yale-NASA). They included:
a "smart" T-Shirt that senses the path of the bullet that hit its wearer, monitors his condition and location, etc., so that rescue teams can decide if he's worth rescuing, and be prepared, and combat units can knock out the location from which he was attacked;
various personal monitoring devices, including a wearable system for astronauts;
a Limb Trauma simulator using the PHANToM (by MusculoGraphics, in Boston):
a stretcher with monitoring systems; and
an enhanced dexterity robot called ParaDex, among others.
HT Medical also gets funding from his committee, as well as SRI international, Boston Dynamics, HIT Lab, and others.
In general for the paper sessions, the presentations were too many to describe all in detail, were allocated only 15 minutes, and - surprisingly - no questions were allowed.
There were many endoscopic simulators, for the knee, shoulder, colon, abdomen... And all had some force-feedback that wasn't convincing as real tissue (from what doctors said) but apparently helped in training (from what the engineers said).
Tactile tissue simulation was one of the key phrases. Everybody is trying to figure out how to do it, but I didn't see (feel) any convincing implementation. Force feedback is the latest craze, but the sensitivity to model subtle gradations just isn't there yet. An interesting alternative is to use sound as feedback.
Also, many atlases of the whole human body (and one of a frog) were presented. Most used the Visible Human, but there were others (the Japanese) that had their own data sets.
One interesting point that was raised by the team at SRI is that the key problem in training surgeons is not how to convey the locomotive skills needed to manipulate an endoscope or how to cut using a scalpel, but how to understand patient anatomy. Training the hands how to use an endoscope takes a week or so, but learning how to the interpret a patient's anatomy takes years. I agree with this assessment, and I think that's where rich interaction capabilities combined with real-time volumetric rendering of multimodal data are crucial.
SRI, of Stanford, have tested their telepresence system with live animals using a 200 metre link. Their results are published in the Journal of Vascular Surgery. Dr. John Hill of SRI presented their first attempts to move towards computer-generated graphics training simulators using their telepresence system. They use a set-up similar to the ISS Virtual Workbench, but with their own interaction devices. They are working on simulating suture of tissue and vessels using an Onyx and 2D texture maps.
Dr. Ramin Shahidi, Stanford University Medical Center, is working on SGI-based volume rendering neurosurgery and craniofacial applications. Their graphics didn't include more than one volume at a time. His presentation was an overview of the use of volume rendering vs. surface rendering, and he didn't include the papers in the proceedings.
NASA-Ames and Stanford University have created the National Biocomputation Center: Dr. Muriel Ross was announcing this centre as a resource for collaboration with academics and industry, to promote medical VR. NASA-Ames have an Immersion Workbench (aka Responsive Workbench, aka Immersadesk) and their own visualisation software and are working on craniofacial "virtual" surgery. It appears that they use polygon meshes for their visualisation.
Dr. Henry Fuchs presented work in progress at UNC that uses depth range finders to reconstruct a surface map of the intestines to then guide an endoscope for colonoscopy. All this was added to their augmented reality system, and comprises an interestingly novel approach.
HT Medical presented their VR Simulation of Abdominal Trauma Surgery. They use the PHANToM and some "wet" graphics to remove a kidney. They simulate the "steps" taken by the surgeon. First the surgeon cuts the skin, which then opens, revealing the intestines. A wet graphics effect is used, but this looks more like "cling film" wrapping over everything. The intestines moved quite unconvincingly, in an animation that was slightly under the control of the user (it didn't appear like inverse kinematics were attaching the end-point of the intestines to the user's tool). The kidney was removed by simply "reaching into it" and moving it out. I couldn't quite see the point of such a simulation, frankly, unless trainee doctors don't realise that it's a good idea to cut into the abdomen before trying to remove a kidney.
On the other hand, I was very impressed by a paper from Wegner and Karron of Computer Aided Surgery Inc., which described the use of auditory feedback to guide blind biopsy needle placement. Their audio feedback system generates an error signal in 3D space with respect to a planned needle trajectory. This error signal and the preoperative plan are used to motivate a position sonification algorithm which generates appropriate sounds to guide the operator in needle placement. To put it simply, harmonics versus dissonances are used to convey position information accurately along 6-8 dimensions. A nice example of a synaesthetic medium - using one modality (sound) where one would normally expect another (touch and/or vision). Their approach has wide applicability.
Myron Kreuger is President of Artificial Reality Corporation and a claimant to the title of inventor of VR. He coined the term "Artificial Reality" (AR) in the early 80s but, sadly for him, it didn't catch on - which is perhaps a pity. Here he was describing ARC's work on adding smells to VR. The system he described was a training system for dealing with emergencies, where smells of, for example, petrol or the contents of the lower intestine, can provide valuable information in a hazardous situation. However, this work seems premature. Smells are messy - they involve molecules, not bits - and so tend to linger after they have been turned off..
HT demonstrated CathSim, a simulator that trains nursing students to perform vascular catheterisations. They built a special force feedback device and a some simple graphics to provide visual feedback. It was quite good to guide the needle, but had little (no?) feedback once inside the skin. This seemed like an "technological overkill" since the procedure is easily learned without VR and is not exactly hazardous.
They also demonstrated a Flexible Bronchoscopy simulator developed with a partnership of pulmonologists and pharmacology experts at Merck & Co. (based on the Visible Human Project). They have a way to track the flexible tip of the endoscope ("a secret", I was told when I asked) and they generate nice 2D texture-mapped graphics of the interior throat using an SRI Impact.
Fraunhofer had two demonstrations from their Providence office:
1. TeleInVivo, demonstrating a PC software volume renderer (a few seconds per rendering for small windows areas) attached to an ultrasound probe.
2. Interventional Ultrasound: A guiding system for biopsy needle insertion using an ultrasound tracking system (not much of an implementation at the moment), so it's the old idea of using ultrasound to guide a biopsy needle. They overlay the ultrasound view with the biopsy needle path, something that UNC demonstrated at SIGGRAPH, but without the expensive head gear.
Matthias Wapler, of the IPA branch in Stuttgart, described a robot for precise endoscopy and neurosurgical navigation. They have not yet developed planning software for their system.
The people of Loral were at the Immersion booth, presenting a training system using the Immersion Corp.'s force feedback device. The application lets the surgeon guide an endoscope through the nose of a patient. The simulation was "helpful" to surgeons, although it is rather crude and doesn't feel like the real thing.
A very tall Swede from Prosolvia (Swedish VR company) demonstrated a Virtual Arthroscopy of the shoulder, developed with University Hospital of Linkoping. They used the Immersion Corp. force-feedback system, and their own Oxygen software base. They may be interested in collaborating on VR medical training systems.
Four demonstrations were shown at their booth:
1. The Ophthalmic surgical simulator. This project combines N-Vision US$25,000 stereo display (binoculars with 1280x1024 resolution; there is a cheaper version for VGA graphics at US$15,000) with the PHANToM, and a nice simulation of the feel of an eye. The computer platform is Intergraph. Since the PHANToM doesn't provide torque feedback, I didn't really appreciate the usefulness of the feedback system while cutting around the cornea. Poking the eye was more fun.
2. MusculoGraphics surgery simulation solutions. Their Limb Trauma simulator DIDN'T have force feedback, so the PHANToM was used as a 3D pointer. The simulation consisted in picking up a bullet and stopping bleeding of a blood vessel. I thought the system was unrealistic and completely pointless.
3. Their IV catheter insertion system HAD force feedback, and was quite convincing.
4. Spine Biopsy Simulator, by the Georgetown University Medical School, for educational use. The aim is to mimic an actual spine biopsy procedure and improve overall learning by students. Unfortunately, their demo wasn't working.
This UK company presented two good tools.
1. VolWin, a volume rendering package (US$700) on the PC that is based on the Voxar API. The performance was really good, running on a plain PC. A 256x256x256 volume was rendered at some 5-6 fps, with some aliasing effects, but basically usable quality.
2. A package that tests the surgeon's performance using the Immersion Corp. laparoscopy device. No fancy graphics, the idea being to measure performance in hitting targets. Excellent simple idea for laparoscopy training.
They have produced a CD-ROM with a segmentation of the Visible Male. The package volume renders the views and structures chosen. On PC.
Sense8 medical customers are the National Centre for Biocomputation (NASA, Stanford U), Rutgers U, Center for Neuro-Science, and Iowa School of Dentistry.
A knee simulator was presented. Unfortunately, it broke early in the conference and before I could use it.
Good head mounted display to substitute the microscope. Not head tracked, but it allows the surgeon to look through the microscope and outside. It also allows Picture-in-Picture, so that an endoscope can be used to supplement the microscope.
There was a nice demonstration of 3D sonification from Lake, who were also involved in the 3D sound feedback for biopsy needle placement described briefly above (paper by Wegner and Karron). Using their kit, it is very simple to place sounds in a three-dimensional landscape surrounding the body to the front (as with normal stereo) and to the back (as with cinema surround sound) but using only headphones. I have a diskette containing a demo of this system.
There were some very interesting papers presented in this session, and a few peculiar ones. Unfortunately, few were included in the printed proceedings. It was clear that this is one of the medical areas where VR can most immediately and successfully be applied today. Topics included treatment of phobias, psychological assessment, and cognitive rehabilitation.
The session also provided an opportunity for the launch of the new "CyberPsychology and Behavior" journal, the first number of which includes a useful summary of the use of VR as a therapeutic tool.
Brenda Wiederhold presented a good paper on using VR to go beyond the standard "imaginal" training of phobic patients. The advantages of VR are, first, that fear can be effectively activated (which is necessary to bring about change) but can be controlled (too much fear reinforces the phobia) and, second, physiological measures can be used to control the display. One simple measure of anxiety, first used by Jung, is a drop in skin resistance.
Similar work on claustrophobia and fear of heights was described by Bulligen of the University of Basle. Another paper on acrophobia (fear of heights) by Huang et al. of the University of Michigan described comparisons of real and virtual environments for emotional desensitisation, and questioned the need for a high level of realism. Using the CAVE environment, they compared the same views in VR and in reality. See their Web page for views [http://www.umich.edu/~psychvr].
A rather pleasant system from Japan, the "Bedside Wellness" system by Ohsuga et al, allows bedridden patients to take a virtual forest walk while lying on their backs in bed. An array of three video screens present the unfolding view of the forest as the patient gently steps on two foot pedals. There is also 3D sound of birds, streams and wind in the trees. A slot below the central screen delivers a gentle breeze scented with pine to the "walking" patient.
Rizzo, of the University of Southern California, is using VR to give increased ecological validity to standard tests applied to Alzheimer's Disease patients, such as the mental rotation task (where the patient has to decide if a second figure is a rotated version of an earlier figure, or is different in shape). This Immersadesk application seemed like technological overkill to me. However, a fuller paper by Rizzo et al in the CyberPsychology and Behavior journal, lists several advantages of VR for cognitive and functional assessment and rehabilitation applications:
1. ecologically valid and dynamic testing and training scenarios, difficult to present by other means
2. total control and consistency of administration
3. hierarchical and repetitive stimulus challenges that can be readily varied in complexity, depending on level of performance
4. provision of cueing stimuli or visualisation tactics to help successful performance in an errorless learning paradigm
5. immediate feedback of performance
6. ability to pause for discussion or instruction
7. option of self-guided exploration and independent testing and training
8. modification of sensory presentations and response requirements based on user's impairments
9. complete performance recording
10. more naturalistic and intuitive performance record for review and analysis by the user
11. safe environment, although realistic
12. ability to introduce game-like aspects to enhance motivation for learning
13. low-cost functional training environments
Also on the topic of psychological assessment, Laura Medozzi et al, from Milan, described what seemed to be high quality work to compare traditional tests with VR-based testing. The case of a patient suffering frontal lobe dysfunction several years after a stroke was used to make the point that traditional tests often fail to reveal deficits that can be identified with VR. This is thought to be due to the nonverbal and immersive realism of VR, compared to the presence of a human examiner, in traditional testing, who inadvertently provided surrogate control over higher order faculties - largely through verbal exchanges. The same group, in collaboration with workers under David Rose at the University of East London, described the use of VR to aid cognitive rehabilitation.
Joan McComas of the University of Ottawa described a VR system for developing spatial skills in children. She had carried out a four-condition study where choice of location to move to was either passive or active, as was navigation to that location. The four were then: passenger (passive choice/passive movement) navigator (active choice/passive movement), driver (active choice/active movement) and navigated driver (passive choice/active movement). The task was to find things hidden at locations, but without going to the same location twice. Measures were percent of correct choices and visit of first error. It occurred to me that we could use this sort of approach in studies of exploration in 3D information landscapes. A paper by Weniger also struck a chord by comparing spatial learning (maze navigation) with exercise of the executive function (the maze with pictograms) and with the use of orientation skills (navigation of landscapes).
Riva, from the Applied Technology for Psychology Lab at the Instituto Auxologico Italiano in Verbania also discussed the use of VR for psychological assessment - particularly the development of the Body Image Virtual Reality Scale. Patients chose which virtual body they think matches their own, and which they would prefer to have instead. The difference gives a measure of body image distortions.
An interesting snippet from a paper by another Italian, Dario Alpini, is that normal people, when asked to rotate on the spot, will usually chose a clockwise rotation, unlike abnormals. I wondered whether this holds true in the southern hemisphere or if, as with bath water going down the drain, the direction there is reversed. His paper was concerned with such differences between normals and others, in real and virtual environments,
Mark Riva, of The Flow Network, evangelised on the joys of flowing, derived from the work in art aesthetics by Csikszentmihalyi (who is President of the company). Flow means optimal experience, flow means maximising your self in autotelic fashion. Flow is A GOOD THING, in fact, the best. Sign up now and never have to think again.
In a somewhat similar vein, Rogers, of the Mythseeker Institute, described a very complicated system loosely based on Jung's ideas of archetypes and their role in myths and human psychology. The idea is to use the system (patents pending) to create your own myth to give your life meaning, overcoming the common problem of people unconsciously choosing a myth that is not right for them. Myth choosing included such things as how to experience time, what visual textures are encountered, and so on, in some ways sounding like a way of adjusting the computer environment depending on the individual's personality type (cf. the thesis work by Thomas Ahlmark).
Greene and Heeter, of the Michigan State University Communication Technology Lab, described CD-ROMs that contain VR-like stories of cancer sufferers, particularly in relation to coping with pain. Details can be found at [http://www.commtechlab.msu.edu/products/]. An interesting paper by Hunter reported the finding that VR can be very effective in helping burn-recovery patients cope with the pain of treatment. Patients in the VR condition reported significant pain reduction and less time spent thinking about pain.
Pope described the use of a VR system called "Viscereal" to provide physiological feedback. Users could control the flow of blood to their hands, and hence could warm or cool them at will. It has also been found to be effective in permitting conscious control of bowel activity, easing clinically harmless but distressing conditions such as Irritable Bowel Syndrome.
The Woodburys, a husband and wife team from the Puerto Rican Institute of Psychiatry, mused on modern cosmology and the origins of our three dimensionality. They gave the conference a useful reminder that the 3D world is in our heads, not in the world "out there". Pathological psychological states - especially various psychoses - and altered states of consciousness produced by certain hallucinogenic drugs, make this clear as the world around the experiencer, and his sense of his body and its place in that world, falls apart in typical psychotic panic states. Following Pribram (see report of NextMed above), the Woodburys view the 3D world we know so well as a holographic projection, formed in the brain according to principles established through evolution as aiding survival. While recognising that this world is an illusion, psychiatrists work to restore it in patients whose world has literally collapsed.
The author of a mystifying paper from Italy, Gabriele Optale, suggested that while impotence and premature ejaculation are topics which fascinate women, they terrify men. The latter seems plausible, at least. An animation with unbelievably poor graphics addressed these problems, but it was not clear why watching a bizarre story of chopping through foliage, broken swords, and magic restorative potions would help.
Although not mentioned by presenters, one of the audience, Rita Addison, talked about the use of VR to communicate the reality of mental deficits to other, normal people. Rita has visited the VRLab in Umea and is well-known for her "Detour: Brain Deconstruction Ahead" which reproduces for others her visual problems since a car accident a few years ago. She was also the only person at the conference to use the word "synaesthesia".
February 1998