The Return of the 3D Crystal Ball

Home

Sections

Headlines

News

Articles

About Us

Newsletter Signup

The Return of the 3D Crystal Ball
Thursday, October 26, 2006 - Iddo Genuth
Home >> Articles >> Computer Technology

A true 3D display has been a dream of many for years. Although various three-dimensional technologies have been developed in recent years, few are able to display real time 3D images and video from any angle in the same way an object would appear to us in the real world. This article will present one such technology called Perspecta and explore its current and future capabilities and uses.

Perspecta for medical use

About four years ago, I reviewed a fascinating display called the Perspecta Spatial 3-D System, which was developed by the Massachusetts-based company Actuality Systems. Perspecta is a true three-dimensional display that allows users to view moving objects from any angle with the unaided eye, simply by walking around them as you would if you were looking at real 3D objects. Perspecta consists of a rotating round white polymer screen resting on box containing software, hardware, and an optical system. Moving images that seemingly float inside a crystal ball-like structure are generated by slices of successive 2D images, rapidly projected one after another onto the screen, creating an illusion of a real 3D image.

Perspecta was first revealed to the public as a prototype in 2001 when it was only able to display still 3D images. Since then, Perspecta has come a long way and is now capable of full 768 x 768 3D video. Initially Perspecta was very expensive and, although the current version, Perspecta 1.9, still costs tens of thousands of dollars, Actuality Systems found a few specialized markets in which the benefits of real 3D technology justify this price. Among these niche markets is the medical market where the new Perspecta aids in the treatment of cancer patients receiving radiation therapy. The 3D technology enables doctors to clearly observe the path of focused radiation, thereby minimizing damage to nearby critical organs. Perspecta can also help simulate the 3D structure of complex molecules useful to both chemists and biologists, as well as in drug development. Actuality Systems also claims that it has customers in the gas and oil industry as well as the military, which is interested in Perspecta's ability to display 3D topography.

A CT scan on the Perspecta

The evolution of Perspecta hardware over the past five years has paralleled that of the commercial hardware market. Early Perspecta models used a Texas Instruments 200 MHz processor as CPU, had 16 MB of RAM, and a very early model of DDR SDRAM running at 66 MHz dedicated to graphics. Current Perspecta models are equipped with a modern AMD CPU, high end NVIDIA or ATI GPU, and a Dual Gigabit Ethernet which are used as inputs to connect to a standard PC. This huge jump in hardware capability generated a leap in performance that has made real time 3D animation on Perspecta possible.

When Actuality Systems originally launched its 3D technology, it was suggested that real 3D gaming could be applied to a Perspecta-like screen. Currently the price of the hardware is still too high and the technology remains unsuited for such applications.

Interview with Actuality Systems

In an interview TFOT recently conducted with Gregg Favalora, Actuality Systems' CTO and Founder, this question was raised along with others regarding Actuality Systems' 3D technology and vision of the future.

Q: What are the basic components of the Perspecta system and how do they work to create a 3D image?

The Perspecta Spatial 3D System

A: The basic components of the Perspecta System are the Spatial Visualization Environment, Core Rendering Electronics / Volume Rendering Unit, and 3D projection optics. For example, if a doctor wants to view a CT scan of a patient's heart, the application "Perspecta Medical" loads in the CT data as standard "DICOM" (Digital Imaging and Communications in Medicine) data. Proprietary-rendering algorithms in the Spatial Visualization Environment perform image processing to compute 396 (198 x 2) 2D "slices" or intersections of the data around a central vertical axis. This computation is performed on a high-end NVIDIA GPU within the Volume Rendering Unit, and the results are stored in the Core Rendering Electronics (CRE). The CRE drives three Texas Instruments DMDs (Digital Micromirror Device) at approximately 6,000 frames per second with these slices, which are projected onto a diffuse screen that rotates at 900 rpm. The result is a crisp, bright, 3D image that can be viewed from any angle.

Q: When the Perspecta 3D display was first introduced around 2002, one of your biggest concerns was the device's refresh rate (then around 24 Hz). What is the current status of this issue with current Perspecta models and what should we expect in 2-3 years?
A: The refresh rate has improved from 24 Hz to the current 30 Hz in Perspecta 1.9, which was accomplished by increasing the screen's rotational rate to 900 rpm. Perspecta 1.9 is still a prototype. Within the next 2 years, we anticipate releasing Perspecta 2.0, with a refresh rate of 42 Hz. This improvement entailed a significant optical and mechanical system redesign. The dome and optical assembly have been replaced with one that should have extraordinary image quality. Perspecta 1.9 has very little flicker; in most images it is barely perceptible. However, we want to do even better, so Perspecta 2.0's refresh of approximately 42 Hz should look fantastic. It is worth noting in this context that CRTs require a much higher refresh rate than volumetric displays. We have built 3D displays at 20 Hz that exhibited no flicker.

Discovery Channel video showing the Perspecta in action

Q: The original Perspecta used a 200 MHz Texas Instruments processor and had a very weak graphic processor by today's standards. Where do you stand today in terms of CPU & GPU power and how important is the strength of the hardware in general to the continued improvement of your technology?

Manipulating molecular
models using the Perspecta

A: Perspecta 1.9's graphics performance absolutely flies. Whereas the original Perspecta took 45 minutes to position, scale, and render a medical CAT scan, today's Perspecta can show 4D (animated) CAT scans of breathing lungs at approximately 4 volumes per second! This is an extraordinary performance improvement. In the last two months alone, performance increased by 270%.

We use an NVIDIA graphics card and a custom embedded "voxel router" in our CRE to achieve this. It's easy for the customer, too, because now Perspecta is a network device. It connects to a Windows XP PC using Gigabit Ethernet. This performance is important because it allows our medical collaborators at major hospitals to perform cancer treatment planning and review, interactively. We are working with Chicago's Rush University Medical Center and Philips Medical Systems to improve the accuracy and effectiveness of radiation oncology. (Of the one million Americans treated each year for cancer, 2/3 receive radiation therapy.) In Perspecta, the doctors can position and check multiple radiation beams as they intersect on tumors in the brain, chest, or prostate. We are beginning a large pre-clinical trial this month at three institutions.

Perspecta's basic components

Q: Would you describe how your company focus has changed since 2002 and what about the possibility of a gaming-oriented 3D display that you suggested at the time?
A: Like many development-stage companies, our strategy has evolved over the years. For example, we are focused on more than displays - we develop application software and entire solutions for medical and petroleum-industry applications. We use our Visualization Analysis Platform (VISAP) to: load 3D medical data, integrate it with existing (e.g., Philips) therapy systems, and then review and change treatment.

We are developing additional technology for the entertainment and medical diagnostic industries. I cannot say much about it at this stage, but we are working on exciting, hologram-like displays for the desktop and arcade. They are very different from the Perspecta display.

Q: Recently you added the ability to use Perspecta as a multi-touch screen. Would you briefly describe this new feature?

Perspecta Markers used to create a multi- touch screen

A: The University of Toronto published an award-winning paper that created the first volumetric GUI for MCAD (mechanical computer-aided design).

Our PerspectaRad medical imaging system includes a "virtual ruler" directed by a SensAble Phantom 3D haptic mouse. Doctors can click and drag within the anatomy, and measurements float within the 3D display.