It was May 1977 when the world got its first glimpse of a desperate Princess—made entirely of light—make her fervent plea for help in Star Wars. In the scene, R2D2 projects a holographic Leia who utters one of the most iconic lines in movie history: 

“Help me Obi-Wan Kenobi, you’re my only hope.”

This line and the technological marvel accompanying it captivated its viewers just as absolutely as the lightsaber or starships also on display. In some ways, it was made even more tantalizing by seeming far closer to our real-world capabilities than most of the other technology present in the film. Sentient droids and planet-hopping starships still seem complex and far off, but simply projecting a 3D image of a person can’t be that difficult… right? Well, it turns out the answer is Yes and No, but before we can talk about our quest to turn make-believe into reality we need to make sure everyone is on the same page.

When I say the word hologram, what pops into your head? Is it a translucent blue Yoda meeting with the rest of the Jedi Council on Coruscant, the image of Tupac Shakur rapping on stage at Coachella, or maybe one of those metallic photographs where the image changes as you move around? These things don’t have all that much in common, so it might come as no surprise to learn that when people talk about holograms, there are generally three separate categories of phenomena they could be referring to:

An Experiment in Serendipity

The first category I’ll describe is the original scientific definition of a hologram. Simply put, these ‘True Holograms’ are layered variations of an image—each one causing light to reflect in a different way—that gives the subject a three-dimensional appearance. When you look at them it’s almost as if the hologram is a tiny window that you’re looking through to observe a real object beyond. These holograms are basically 3D photographs produced by the interference of two laser beams and they are used for a wide variety of purposes: from making objects like passports, credit cards, and currency more difficult to counterfeit, to making holofoil Pokémon cards.

You may be surprised to learn that the concept of holography was thought up 30 years before Star Wars even existed. In 1947, Hungarian-British physicist Dennis Gabor accidentally discovered it while trying to improve the resolution of electron microscopes. Unfortunately, he couldn’t move forward with his research unless he had the ability to generate a pure and consistent beam of light. He realized that it would take more complex technology than existed in 1947 to create holograms and he dubbed his discovery: “An experiment in serendipity that began too soon.”

He would ultimately be proven right in 1960 with the breakthrough invention of the laser. Lasers were able to emit light that was coherent, meaning it could be focused to a tight spot and stay narrow over long distances. Two years later, the first practical optical holograms that recorded 3D objects were created by Emmett Leith and Juris Upatnieks at the University of Michigan. The holograms were of a toy train and a bird.

The technology didn’t stop there though. In 1972, Lloyd Cross combined White-light transmission holography with conventional cinematography to produce moving images from a sequence of recorded images. When viewed in rotation, the composite images are synthesized by the human brain as a moving 3D image. In 1976 there was even a prototype for a holographic movie!

Around the same time, a revolutionary technique for mass-producing holograms was developed. Known as embossed holograms, MasterCard put the first one on a credit card in 1983 and they’ve been a familiar presence ever since. Unfortunately, as cool and useful as they are, ‘True Holograms’ aren’t the solution to our desire to communicate in real-time with an incorporeal 3D image of another person like in the movies. For that, we need to look elsewhere.

It’s not a trick…it’s an illusion!

The second thing people can mean when they talk about holograms is a fairly large collection of visual effects used to give 2D planar images the illusion of depth. Often called ‘False Holograms’, these include lenticular printing, tomography, and the Pepper’s Ghost illusion. Many of these techniques are centuries old and have found popular use as stage tricks, not being called holograms until well after the term was established.

The most famous and widespread of these techniques is called Pepper’s Ghost, so that’s what we are going to learn about. Named after the English scientist John Henry Pepper (1821-1900), the illusion was first developed in the 17th century and then perfected for stage use in the 19th century by Pepper and the British Engineer Henry Dircks. The technique literally uses smoke and mirrors to project a partial reflection of a real solid object into an empty space, making the subject appear translucent and ghostly. You have likely encountered this illusion without realizing it, either in person or on TV. Its largest and most famous implementation is at The Haunted Mansion and Phantom Manor attractions at Walt Disney Parks and Resorts where the spectral projections of animatronic ghosts fly around an empty ballroom. My personal favorite version is in the original Scooby-Doo, Where are You! episode “Hassle in the Castle”, in which—Do I really need a spoiler alert for a 50-year-old cartoon? —the ex-magician Bluestone the Great uses a version of the illusion to create a spooky ghost that can seemingly walk through walls.

The Pepper’s Ghost illusion is easy to understand once you get a look behind the scenes, but before I can explain exactly how it works, we need to look at the science involved.

Imagine we have a partial mirror that reflects 50% of the light that hits it and lets the other 50% pass-through, like a very reflective window. If we put this partial mirror between two rooms that both have a lamp, let’s just call them room A and room B, then go into room A and look through the mirror towards B what would we see?

Well, that depends on what the lamp situation is in the two rooms. If the lamp is on in room A and off in room B, we will see just see a dim reflection of room A in the partial mirror. You know how you get that ghostly reflection of yourself if you look out a window at night while the lights are still on inside? It’s the same principle at work! Light is bouncing off your face and hitting the window where half of it goes through and the other half bounces off the window back towards you, revealing a reflection that looks less substantial than a normal mirror would show.

Conversely, if the lamp is off in room A and on in room B, we won’t see much of a reflection at all. Instead, we will see straight through the partial mirror into B. The view will just be a bit darker than if there was no partial mirror at all because half of the light coming through to us is bouncing off the partial mirror back into B. This is how “one-way mirrors” work in places like police interrogation rooms. The mirror/window is partially reflective, and the interrogation room is brightly lit while the observation room often isn’t lit at all. This allows people in the observation room to watch what happens in the interrogation room while remaining unseen by anyone inside.

What if we turn on the lamps in both rooms? What we see on the glass will be a combined image where 50% is the stuff in room B and 50% is a reflection of the stuff in room A. This would manifest with the partial mirror showing us a dim view of room B with a semi-transparent reflection of room A overlaid on top of it. This is where “Pepper’s Ghost” comes in.

In order to avoid giving us all headaches, I’ll use the diagram above to help explain. The illusion works by taking advantage of the optical mechanics we just talked about. We begin by putting an actor in a pit that is in front of and below the stage but hidden from the audience. With them in the pit is a bright light that they will use to illuminate themselves and a perfectly reflective mirror facing back towards the stage and angled slightly up. Next, we will place a sheet of glass at an angle in between the stage and the audience. After the light illuminating the actor bounces off the mirror in the pit, it hits the sheet of glass where half of it passes through and the other half reflects back towards the audience. From the perspective of the audience in a dark theater, they see what appears to be the ghostly visage of the actor on the stage!

The Pepper’s Ghost technique saw an increase in relevance a few years back when Digital Domain used an updated version of the illusion for the appearance of Tupac Shakur onstage with Dr. Dre and Snoop Dogg at the 2012 Coachella Music and Arts Festival and Michael Jackson at the 2014 Billboard Music Awards. While they might not be real holograms, the effect they had during these performances was still strikingly realistic and impressive. But there’s something even more impressive out there, and science is just now starting to explore the possibilities.

Luminous beings are we!

The third and final category is what I’m going to call ‘Volumetric Images’. This encompasses basically any technology that forms a visual representation of an object in three physical dimensions, as opposed to planar images that merely simulate depth. These are the holograms envisioned by science fiction writers after they talked to some overly-enthusiastic scientist trying to market their new technology. Unfortunately, the capabilities of our real-life versions fall tragically short so far. They usually require an expensive and complicated set-up, can only be seen from certain points of view (just ask Obi-Wan), or require some sort of special headgear to view. But some scientists are hard at work bringing these science-fiction dreams to life.

In May of 2018, a team of researchers at Queen’s University in Ontario, Canada, unveiled a new 3D display system capable of transmitting a full-size, 360-degree image of a human that can be seen with the naked eye. The display system is called the ‘TeleHuman 2’ and it uses three stereoscopic cameras to take live video of a human subject from every angle. The cameras can record information about the 3D shape of the subject, called ‘lightfield’ by the researchers, and transmit that information over a network or internet connection to the ‘telepod’, a 6-foot-tall cylindrical screen with an external hoop of over 40 projectors, where an image of the subject is displayed.

The projected person appears 3D and real from any angle and can be observed by multiple viewers without issue. If you walk behind the telepod, you’ll simply see the back of the person that appears inside it. Roel Vertegaal, a professor of human-computer interaction at Queen’s University and lead researcher of a study about Telehuman 2, says that he thinks the system will be very useful for telecommunication. According to Vertegaal, a conversation between two different people using the system would only require six times the bandwidth of a modern 2D video call. He is hopeful that the technology will be adopted by conference venues, corporate meeting rooms, and possibly even stadiums where performers can make virtual appearances on stage. But while Vertegaal and his team are aiming to make a practical solution to the world’s desire for holograms, others are taking a more ambitious approach.

Around the same time in 2018, researchers at Brigham Young University in Utah published an article in Nature where they detailed the construction of something they call an Optical Trap Display (OTD). The mechanical marvel traps a small opaque particle of cellulose in mid-air using an invisible laser beam, then moves the particle around a preset path in free space by evenly heating it with the laser. Simultaneously, the particle is illuminated with a second set of red, green, or blue lasers and when the particle rapidly moves around in the air, it creates a solid, 3D, holographic image that can be seen from any angle. If the particle is made to move even faster, it can make the image appear to move.

The group working on this is led by professor Daniel Smalley, who has long dreamed of recreating Star Wars holograms. “We refer to this colloquially as the Princess Leia project,” said Smalley. “Our group has a mission to take the 3D displays of science fiction and make them real. We have created a display that can do that.” Smalley explains that the easiest way to understand what they are doing is to think about the images they create like 3D-printed objects. “This display is like a 3D printer for light. You’re actually printing an object in space with these little particles,” said Smalley.

The variety of images possible with the OTD seems to be almost endless. So far, the researchers have generated images of a butterfly, a prism, the BYU logo, rings that wrap around an arm, the Earth, a Pokémon, and even a lab coat wearing scientist in a very Alderaanian princess-like crouch. The volumetric images created by the OTD are stunningly bright and clearly defined, with the resolution of the images being as high as 1,600 DPI (that’s over 5 times the resolution of most printed photos). Unfortunately, the images are also extremely small, being tinier than a fingertip.

In November of 2019, an article in Nature revealed another group of researchers at the University of Sussex have taken a similar approach and are getting impressive results. Unlike Smalley and his team at BYU who are using lasers, these researchers are using two arrays of ultrasonic transducers that generate sound waves capable of controlling a two-millimeter polystyrene bead in midair. The transducers are able to move the bead up to 20 miles per hour and can trace shapes as large as 10 centimeters across in less than 1/10^th of a second.

Like the Princess Leia Project, these volumetric images exist in 3D space, so they can be viewed from any angle and appear completely solid. But the team over at the University of Sussex have another trick up their sleeve. The transducers that move the bead around can also make it vibrate at frequencies that create sound waves, allowing the team to make tiny figures to appear to speak or add sound effects and musical accompaniments to the images. The same technology also allows for the generation of ultrasonic sound waves that can be felt as a tactile sensation. By placing their hands close enough to the image, someone could actually feel the hologram as if it were really there.

It might not be Star Trek’s holodeck, with its sentient projections and force field technology, but it’s a very good start! The next big hurdle to overcome is the size limitations of this technology. If we want to get bigger images, we need to figure out how to manipulate more than one particle at a time. Smalley has said that his group is already experimenting with creating larger volumetric images using multiple particles, which if successful, would allow for many new ways for these images to be used and the team at Sussex is working on expanding the size of their array. There’s still a long way to go before we have real Sci-fi level holograms, but maybe the future isn’t as far off as we thought.