ToF: Understand Current Assisted Photonic Demodulator (CAPD)

For 4D (Sensing) and 4A (AI, AR, AV and AIoT)

6 min readJan 18, 2021

Time of Flight technology works by measuring the time delay of light traveling from a light source, reflecting off a scene, and into a sensor. The specific type of Time of Flight the Sony IMX556 CMOS sensor uses is called continuous wave (CW) modulation, also known as CW phase-shift or Indirect Time of Flight. Instead of measuring the time delay of a single light pulse, the Helios camera emits continuously modulated light, and the phase-shift between this emitted light and the reflected light is used to calculate distance. In order to accurately calculate the phase-shift, the IMX556 utilizes a current assisted photonic demodulator (CAPD) pixel structure that samples incoming light synchronously with emitted light modulation. CAPD allows for an alternating voltage inside each pixel’s photodiode, creating drift fields that divide and pull electrons to alternating detector junctions. In the simplified example below, modulated light is emitted from the VCSEL (Vertical Cavity Surface Emitting Laser) diode. The light is reflected back into the pixel photodiode where it is converted into electrons and divided between the alternating detector junctions. This allows for better light collection efficiency.

Above: CAPD’s alternating voltage creates an alternating drift field that divides electrons between two detection junctions (b1, b2). This alternating drift field is synchronized to the modulation frequency of the VSCEL laser diodes. The two junctions are 180° out of phase in order to fully capture the reflected light. This is how the sensor can compare the emitted light to the reflected light and calculate the phase shift between the two.

Side Note

Sony IMX556 DepthSense ToF sensor specs:

Backside Illuminated CMOS
640 x 480 px
Global shutter
10.0µm pixel size
1/2″ sensor size

Side Note

The CAPD ToF image sensor technology was first introduced in 2006 from the founding members of the Belgium based company Softkinetic, acquired by Sony in 2015. Below: CAPD diagram from “Time-of-flight Optical Ranging Sensor Based on a Current Assisted Photonic Demodulator” by D. Van Nieuwenhove, W. Van der Tempel, R. Grootjans and M. Kuijk.

Divide and Conquer

Once the photons are converted to electrons within the photodiode they must quickly be pulled towards the detector junctions for the camera to accurately calculate the phase shift. Located in each detection junction are positive (p+) and negative regions (n+). When a current is introduced to produce a drift field, the electron holes (h+) move towards the lowest voltage p+ region and the electrons (e-) move in the opposite direction towards the highest voltage region (n+). CAPD is extremely fast and effective. It is able to quickly alternate drift field polarity producing very high demodulating contrast. A high contrast ratio means more electrons are directed to the right junction based on their return time, allowing for higher quality phase shift calculations. This results in better accuracy and precision with the DepthSense sensor. In addition, CAPD’s drift field is able to reach deep into the photodiode accelerating as many electrons as possible towards the detection junction. This, combined with Sony’s backsided technology helps produce a high quantum efficiency of 57% at 850 nm. As a result, the Helios camera can achieve an accuracy of less than 5 mm and a precision of less than 2 mm at 1 m camera distance.

Above Left: Cross-section of a CAPD’ showing electrons (e-) accelerating towards the highest voltage region (n+) while the electron holes (h+) move towards the lowest voltage region (p+) Above Right: CAPD helps produce high QE (57% at 850nm) and high demodulation contrast.

1 Frame, 4 Phases

The speed of CAPD allows the IMX556 to sample incoming light with 4 different phases for each depth frame. Each sample, also known as a micro-frame, is phase-stepped by 90° (0°, 90°, 180°, 270°) with each one having 3 stages: reset, integration, and readout. During the reset stage, the pixel voltage is reset back to baseline. The Integration stage occurs when the drift-fields are introduced by the current to direct the electrons towards the detection junctions. During the readout stage all pixel data is read on the sensor. After the readout stage the sensor enters idle time to reduce power and lower heat levels.

It is only necessary to have the 0° and 90° micro-frames to calculate depth. As the distance of an object changes, the values of b1 and b2 in the 0° and 90° micro-frames will change. Ambient light is also easily removed by subtracting the values of b2 from b1. However, even if the collected photons are the same for b1 and b2, there are always slight fluctuations (caused during photon to electron conversion and electron to voltage conversion). It is important to cancel out these fluctuations and their influence on b1 and b2 values. By having 2 additional micro-frames that are 180° offset from 0° and 90° (which effectively reverses the roles of b1 and b2) and by adding the values of b1 and b2 to their opposite 180° micro-frame (and vis-versa) the camera cancels out these fluctuations in each pixel.

Subtracting b2 from b1 cancels out the ambient light in each micro-frame.
Adding b1 of the 0° micro-frame to b2 of the 180° micro-frame (and vis-versa) gives two new quantities that are free of any sensitivity discrepancies in b1 and b2 (discrepancies that are caused during conversion stages such as dark offset, gain, etc.)

CAPD + Backside Illuminated CMOS

CAPD provides an effective way of demodulating and collecting electrons inside the photodiode, but reflected light must first reach the sensor’s photodiode unobstructed. Sony’s backside illuminated (BSI) technology ensures the maximum amount of light reaches the photodiode. Unlike traditional front-side CMOS sensor design, backside illuminated sensors place the wiring layer below the photodiode improving light sensitivity. This placement removes any wiring or circuit obstructions that might otherwise block some of the incoming light.

Reducing unwanted image artifacts is another benefit of the IMX556. Because the IMX556 is a CMOS sensor it performs better against smearing and blooming versus CCD ToF sensors. Smearing and blooming can cause erroneous 3D data and is created when the charge overflows the pixel well capacity, spilling into adjacent pixels. This is usually caused by intense light sources or reflective surfaces. In many ToF use cases, it is important to minimize these factors. The IMX556 CMOS used in the Helios camera is much more forgiving in reducing these image artifacts.