Sight of death — a brief history of military head-mounted display
Although many people today connect head-mounted displays to gaming, virtual reality, and watching movies, the roots of the technology lay in the history of military aviation. The idea of a pilot directing a missile to its target by simply looking at it isn’t new, but it was only during the last few decades that the technology was perfected to a level that could be considered useful.
|
.jpg)
The first reference to a head-mounted display or head-mounted sight (HMD/HMS) in aviation was likely made by Albert Bacon Pratt, who applied for and received a patent for an integrated gun helmet in 1916. At the time, there was no real way to create an actual HMD, and the patent was more or less forgotten. It took six long decades before the first real HMD was developed and brought into service. The Visual Target Acquisition System (VTAS) was developed during the 1960s by the Honeywell Corporation and was introduced by the U.S. Navy into 500 of its F-4N and F-4J Phantoms in 1973. The effectiveness of the VTAS was hampered by the limited capability of the AIM-9P and N Sidewinder missiles. The missiles would track clouds,
|
objects on the ground, and even the pilot wingman instead of the actual target. It was clear that the system was not mature enough, and use of the VTAS HMD was terminated. However, the U.S. military did not abandon the concept of HMD and during the 1980s fielded sights for both fixed-wing aircraft and combat helicopters. Around the same time, other countries, including the Soviet Union and Israel, started to develop HMD programs of their own. The Israeli optical equipment company EL-OP (now part of Elbit systems) pioneered this field, and today it is considered one of the leaders in the world. HMDs have come a long way since the days of the VTAS. So solid is the trust in this technology nowadays that the F-35 Joint Strike Fighter aircraft will be the first jet in 50 years to not include a heads-up display (HUD), which will be completely replaced by a new HMD developed by Vision Systems International (VSI), a U.S.-based sister subsidiary of Elbit.
Lumus and the “size dilemma”
A basic issue with such mobile devices as cell phones, PDAs, and mobile game consoles has always been screen size. On the one hand, a large screen is crucial for watching movies, playing games, and even reading e-mail comfortably. On the other hand, the larger the screen, the bigger the device, which isn’t something you want in a light and portable product.
|
Until recently, this “size dilemma” dictated developing new devices with either a very small screen (it’s not much fun watching 24 on a two-inch cell phone screen) or a much larger footprint, such as ultramobile PCs. But things are about to change, and several new technologies are promising in the very near future small-size products with large displays. Microlaser projectors, such as Explay’s nanoprojector that we recently previewed, are about to enter the market in the next few months. Another interesting solution is the “rollable display,” based on either e-paper or OLED technology. Although prototypes have been demonstrated — Polymer Vision’s Readius device is one recent example — the technology still seems to be a few years away from widespread commercial adoption.
|
Another technology that can help solve the “size dilemma” is HMD. TV glasses and virtual-reality goggles have been around for many years. Early products were usually heavy, expensive, and uncomfortable to use for long periods of time. In the last few years, lighter and more affordable products have started to reach the market, but most of them are still too big and cumbersome to be used for any prolonged period of time. Aesthetics is also a factor when dealing with consumer products. Existing products cannot be used discreetly and would not be used by most people outside their home or office. The Geordi La Forge look isn’t exactly popular.
Now enters an Israeli start-up, Lumus, aiming to take commercial HMDs to a whole new level. Lumus was founded by Dr. Yaakov Amitai, a former Israeli fighter pilot and physicist who worked for years on developing holographic HMDs for the military. In 2000, he finally had a breakthrough idea for creating a new type of microdisplay, and after filing a patent he decided to start his own company to promote the idea.
Almost seven years later, the TFOT team visited the offices of Lumus in the Israeli city of Rehovot, near the Weizmann Institute of Science. After a long and complex development process, the company is now several months away from delivering to the market its first see-through head-mounted display.
Upon our arrival, we were shown two prototypes: one an LCD-based binocular device, and the other an earlier-stage prototype based on an LCoS (liquid crystal on silicon) display. Although the LCoS-based device had more vivid colors, it was less interesting from our point of view, being monocular and bulky. Lumus explained that the main reason for showing it was to convey the direction they are going with regard to increased brightness for all of their products. To that end, the LCD binocular device seemed to be more mature. It was fairly light and felt more or less like a pair of conventional glasses with wires. We were told that the production model will be even lighter, weighing less than 50 grams (1.7 ounces).
|
The wires for the prototype were fairly thick, and although the first production devices will be wired as well, we were told they will have much more flexible and thin wires. As for a wireless version, Lumus made it clear that this is a distinct possibility for the future that will be done on the OEM level and not by Lumus itself, which is instead focusing on the basic display technology. Missing from the prototype we saw was sound. The Lumus prototype was connected to an Archos 404 PMP, but it did not have earpieces. The commercial version will include earpieces, which will probably come as part of the glasses, à la high-tech eyewear Oakley Thump.
|
Using the Lumus glasses was fairly easy, although the prototype did require a small external electronic box between the PMP and the glasses (another thing that will disappear in the commercial product). Quality was pretty good, and we didn’t really need time to adjust our vision to the microdisplays. Resolution, however, is a different matter — currently, Lumus is planning QVGA (320×240) and VGA (640×480) variants. They did mention that they are playing around with higher-resolution versions but, though technically possible, there are no current plans for a full HD version, mainly due to the current lack of micro-HD LCDs and lack of portable devices capable of pushing such high-definition content. Watching a video on LCD goggles gives the effect of watching a much larger screen from several feet away. Lumus claims their product is comparable to watching a 70-inch screen from 9 feet away, and though it is hard to verify, the displays did present a fairly sizable image.
|
As with all displays, ambient light is an issue, and using the glasses to watch videos in broad daylight will likely be a difficult task. However, using the Lumus glasses indoors to watch Finding Nemo seemed to work very well. Lumus demonstrated a dark clip-on that further improves the image, although using it diminishes some of the excitement of using see-through TV glasses. Lumus’s LCoS prototype features higher brightness and contrast, and clip-on sunglasses can be added to provide additional contrast outdoors against ambient sunlight. Lumus mentioned that both clip-on sunglasses and prescription glasses are distinct options for its future products. We didn’t have a chance during our short visit to the Lumus labs to test how comfortable the glasses are when used for several hours, though we hope to review them again closer to their commercial launch.
|
See-through video displays, such as those currently under development by Lumus, have many potential applications. According to Lumus, the biggest market for this technology is the mobile TV, currently suffering from the small size of cell phone screens. Gaming is another huge potential market, especially with the possibility of implementing 3-D into the glasses by sending slightly different images to each eye. Lumus and its technology also have a place in virtual reality (VR), as well as in a related field called augmented reality (AR), in which information is overlaid on or displayed alongside reality (here, the see-through nature of the glasses is a must). AR is an evolving field that may become quite popular, especially among such professionals as mechanics and doctors, but it could also be very useful as a virtual tour guide for cities and places like museums. In the more distant future, AR glasses will help you shop, displaying product information on the items themselves as you walk by in a store.
Other potential applications include both indoor and outdoor GPS and military use. Although details are classified, Lumus has been working with the military and security-related companies for several years and plans an initial launch of its professional, military-grade product this summer. As it turns out, the LCoS-based prototype we examined was being developed primarily for such military projects as “future soldier,” as well as for next-generation HMD helmets for fighter pilots.
|
Before we say a few words about how Lumus’s technology works, we want to mention the topics of design, availability, and price. The prototypes examined were not exactly aesthetically appealing; few prototypes are. Lumus promised that it is already working with OEMs and professional eyeglass manufacturers to combine its technology into a much slicker design. Reducing the size of the projection unit — something Lumus is already doing — should make this a clear possibility. Lumus hopes to have preproduction models out by the end of this year and to release its first consumer binocular product sometime in the first half of 2008. As for pricing, we did not get exact numbers, but we were told that the price range for the consumer product should be somewhere between $300 and $500 at its launch.
Lumus LOE technology
|
To achieve the see-through nature, thin screen and lens structure while maintaining a large field of view, Lumus has developed a unique technology it calls Light-guide Optical Element (LOE). The Lumus video-glasses are connected either wired or wirelessly to a media source, such as a cell phone, video iPod, or laptop, which transfers the video image into a microprojector embedded in the temple of the eyeglasses. The tiny projector can use LCD, LCoS, OLED, or any other kind of future microdisplay technology. From the projector, the image travels through the lens by way of Total Internal Reflection (TIR).
This well-known optical concept is fairly simple to understand and has been in use for many years in telecommunications (fiber optics), the cutting of diamonds, and producing different types of sensors (such as rain sensors for cars). Each time light crosses a boundary between two materials with different refractive indices, some of the light is refracted across the boundary and some is reflected. TIR is an optical phenomenon that occurs when light strikes a medium boundary at a steep angle. If the refractive index is lower on the other side of the boundary, no light can pass through, effectively reflecting all of the light. The critical angle is the angle of incidence, above which the total internal reflection occurs.
In an optical fiber, for example, light from a laser source hits the inside surface of the fiber in a steep angle, steep enough for TIR, causing the beam to bounce forward continuously without losing much of its strength (some of the light signal does degrade over long distances, mostly due to impurities in the internal coating of the fiber). The same principle is used in Lumus’s LOE, but instead of using a hollow tube to bounce the light, Lumus developed a way to create TIR in the glass lens itself. The image is then expanded and coupled out by a set of partial reflectors for viewing by the user.
|
|
Interview with Lumus
TFOT recently interviewed Dr. Yaakov Amitai, CTO and founder of Lumus, and Ari Grobman, Lumus’s business development manager.
Q: How was the idea for the technology born, and was there a eureka moment?
A (Amitai): I remember talking about head-mounted displays (HMD) as far back as the ’70s, when, as an Air Force navigator, I was asked to help define such systems.
After that, I spent many years working with hologram-based HMDs at the Weitzman Institute, Stanford University, and EL-OP. However, after many years of R&D and several patents, I felt as if I had come to a dead end. Bound by the laws of physics, I saw no possibility to combine the three key features that would make HMDs practical, namely: high image quality, large image size, and small form factor.
Only after I had completely abandoned my work on HMDs and left EL-OP did the ultimate solution come to me in a eureka moment. I had been working on a patent in the field of communications when the concept of the LOE came to me. If you want to get really specific, I was taking my son to kindergarten as the idea started fermenting in my head. The second I returned home, I started writing what soon became our first patent. Realizing right away that I had found the holy grail for personal displays, I founded Lumus shortly thereafter.
Q:Could you explain how your technology works?
A (Amitai): To put it in the simplest terms, our patented LOE (Light-guide Optical Element) technology shatters the laws of conventional optics. Traditionally, personal displays suffer from the “size dilemma” — if you want the image to be large, the device has to be large; if you want the device to be small, the image will be small. A small image defeats the purpose of having a portable near-to-eye display, while having a bulky device is uncomfortable, unsightly, and hardly portable.
The LOE is a unique ultrathin lens design that embeds miniature, see-through prisms in front of the eye. A mini projector embedded in the temple of the eyeglasses receives the image content from the mobile device and projects it into the side of the LOE. As the image travels to the center of the lens, it is reflected into the eye via see-through prisms, creating the effect of a virtual large screen.
In addition to the technological breakthroughs of combining a large, high-quality image in an incomparably compact form factor, the transparent lens enables Lumus to offer products with the natural look of standard eyeglasses.
|
Q: The current resolution of your device is QVGA/VGA. Could you go higher, possibly up to HDTV? What limits the resolution?
A (Grobman): Currently, we offer QVGA and VGA for consumer products and SVGA for professional products where higher resolution is required
With regard to resolution increases, this is in fact another great feature of the LOE —resolution and field of view can be increased without any increase to the LOE’s size. The only limiting factor at this point is in the microdisplays (mini LCD, LCoS, OLED, and so forth). Currently, manufacturers of microdisplays don’t offer such products in such high resolutions, but once demand for higher resolutions increases and the manufacturers start to supply them, Lumus will be ready to provide products with increased resolution
It should also be noted that most of the devices primed to interact with personal displays over the next couple years, such as portable media players and next-gen cell phones, don’t provide resolution higher than VGA — in other words, the image sources we are connecting to aren’t providing higher resolution.
Q: Would you say that such devices use lower resolution because they are bound by their small embedded screen? Perhaps with Lumus offering a much larger screen, the device manufacturers would want to support higher resolutions.
A (Grobman): The inherently small screen is definitely one reason. Other reasons include bandwidth constraints when talking about mobile TV and memory constraints on portable devices — the higher the resolution, the more memory required.
Nevertheless, Lumus’s large screen would enable device manufacturers to significantly increase resolution to enhance the viewing experience. Of course, we see this happening as a second step after our QVGA/ VGA video eyeglasses have been successful in market.
Q: Could your technology be used with prescription glasses or sunglasses?
A (Grobman): Because the Lumus image is focused to infinity, customers who are nearsighted will require corrective lenses. As such, customers may order customized clip-on lenses. Additionally, because Lumus’s LOE is so compact, it may be embedded in a prescription lens
Q: There are many companies worldwide developing and selling head-mounted display technologies and products. What is unique about your device?
A (Grobman): I like the way you pose the question — “head-mounted display.” This connotes a heavy, bulky device mounted on one’s head, which is exactly what you have seen on the market so far. You may also refer to them as “goggles” or “visors.” But you cannot truly call them “video eyeglasses,” because they don’t look like eyeglasses; they don’t feel like eyeglasses
Such solutions have been around for years and have failed to become mainstream consumer products. Why? Because mainstream consumers don’t want to look like gadget freaks. Mainstream consumers don’t want to stick out in a crowd; they don’t want people to stop them and say, “What is that?” They don’t want to wear something that is obtrusive, heavy, and uncomfortable
Consumers want a device that is natural looking, discreet, lightweight, and portable. They don’t want people to notice if they’re watching a TV clip, reading an e-mail, or glancing at stock tickers. They want information flow without obstructing their vision so they can carry on their day uninterrupted
Believe it or not, these comments are not my own but rather what we have been hearing from device manufacturers and cellular carriers who have been looking at personal displays for over a decade. Every product you’ve seen, they’ve seen; and nothing has ever taken off … until now. The points mentioned above are the exact reasons why things are developing so quickly with the device manufacturers and carriers we are talking to.So if you ask me who our direct competition is, I shrug my shoulders. When we meet with device manufacturers and cellular-service providers, they don’t ask us what Lumus’s advantage is — they tell us
Q: What are the main applications you see for your product?
A (Grobman): The list keeps growing every day. The most obvious applications for the consumer market are mobile TV, primarily from next-gen cell phones, and video, when you consider the fact that cellular-service providers are spending billions of dollars on infrastructure to support mobile TV, and close to 100 million cellular phones will be video enabled by 2010. These devices can also support 3-D content, gaming, and virtual reality, bringing mobile entertainment to a whole new level
With regard to professionals and consumers alike, viewing such content as text messages, e-mail, attachments, and mobile Internet on a large, hands-free screen has strong appeal. In fact, because our video eyeglasses are inconspicuous, you can read e-mail in the middle of meetings without anybody noticing. One can give a speech without having to look down at his notes. A stockbroker can watch tickers roll across the bottom of his personal screen throughout the day.
PDA manufacturers and content providers can now offer large-screen apps for their small devices, making mobile computing far more efficient. A good example of that is in maintenance applications: instead of a maintenance engineer having to look down at a tiny PDA screen that he has to scroll through to view instructions and diagrams, he can have a large screen right in front of him. You can take that a step further and bring augmented reality into the picture, no pun intended. A diagram of instructions may be superimposed on the actual part he is working on
Museums, tourist organizations, and educational associations contact us regularly for enhanced interactive information delivery. Medical doctors want to be able to keep their eyes on the patients they are working on while having the ability to also receive important information, such as critical patient data, X-rays, or even live images from an endoscope. In aviation, we are finding key applications for pilots in the military, commercial, and recreational markets. Other military applications include personal displays for combat soldiers where the difference between keeping your head up and looking down at a PDA can mean the difference between life and death
In conclusion, I will say that Lumus video eyeglasses enable so many applications that we are inundated daily with new applications and opportunities. Of course, as a small company, it is impossible to pursue all avenues at this point. That said, we believe that our strong relationships with top portable-device manufacturers, tier-one cellular carriers, and key military integrators will serve as key conduits for getting our products out to the market most successfully
Q: Will you have a wireless version, and what about battery life and the weight of the device?
A (Grobman): I believe that our consumer products will definitely have a wireless version. While we haven’t finalized battery life specs, the benchmarks for battery life are already set by first-generation personal displays (the “goggles” we were talking about earlier) — that is somewhere between two and six hours of continuous video. Of course, we’re aiming to be at the higher end, as we know wearers will be using our system extensively during the course of a day
Q: Have you performed tests using the glasses for several hours straight, and did you consider such health risks as eye damage due to prolonged focusing or more exotic issues, such as VR sickness?
A (Grobman): Because of the see-through nature and open peripheral vision inherent in the design of our glasses, motion sickness and dizziness associated with other personal displays is not an issue. Having said that, health is a very important issue that we take quite seriously here at Lumus. We have procedures in place for product-safety validation, such as extensive wear testing, which includes user feedback and the objective medical evaluation of wearers.
Q: What product lines are you going to launch?
A (Grobman): Our consumer product line will be introduced to the market by the device manufacturers, as we discussed earlier. The way things are going and the way we are planning is to start shipping products in 2008. We will start shipping initial quantities of our professional market product, the PD-18, as early as this summer. Most initial orders we have received for the PD-18 are concentrated in military applications. This product has several key differences from the video eyeglasses we discussed earlier: it is a monocular [single-eye], top-down device that typically mounts to a helmet to hang in front of the eye; it is SVGA resolution; and it comes with very high brightness and contrast. This higher-end product is also suitable for other augmented-reality applications, such as maintenance and medical, where the higher resolution helps significantly when reading small text and detailed schematics and graphs.
|
Q: What will be the price of the retail product, and when will we be able to see it in the stores?
A (Grobman): As our consumer products are going to be sold on an OEM basis to major device manufacturers, who will provide the market with their finished products based on our core components, we cannot comment directly on price. We can only guesstimate the product will be no more than a few hundred dollars to the end user.
However, I should add that things will get very interesting when cellular carriers — most likely to be our first distributors — roll out these products to their customers. Because carriers often make a zero margin or even subsidize products that drive more billable network usage, they may apply the same model to the video eyeglasses, offering the products at super-low prices or perhaps even for free when signing up for certain plans. We see how they’ve been doing it with the cell phone for years. Most carriers I have spoken to are already entertaining such options with Lumus video eyeglasses, as they see this as a driver for mobile TV and other offerings.
