The Dark Side of Image Sensors

An ideal image sensor is always desired in astronomical imaging, which has negligible read noise and dark current, with almost 100% quantum efficiency. That means your SNR is only limited by your background sky noise and exposure time. The reality is, most of the time, far from being ideal.

In the last decade, the fierce competition for consumer camera market share and the investment into R&D perfected the CMOS sensor to a point far exceeding the performance of CCDs. Here let’s looked at the imperfections left in some of these famous CMOS sensors.

Recently I ported the “Dark Current Enable Tool” to the 3rd generation Nikon DSLRs. This made it possible to evaluate the image sensor more accurately and conveniently.

Uneven Bias Level

In the imaging pipeline, the first calibration step is subtraction of master bias frame. This will even up the pedestal for all the pixels before any scaling step. The master bias is usually created by combining dozens of bias frames. Now just by looking at the master bias we can check how good the image sensor and read out circuit are made.

D700-ISO200

Portion of bias frame in NC81338L (D3, D700) at ISO200

The above bias frame is from D700. Due to its use of 6 discrete AD9974 dual channel ADCs from analog readouts, the 12 column per cycle of uneven bias level is clearly visible even without frequency analysis. The Sony sensors, like the IMX028 in D3X, employed the column parallel ADC. The column-wise irregularity is much smaller due to the voltage comparators share a common ramping DAC reference. There should also be calibration circuits before readout to cancel the differences between these column circuits.

D3X

Bias pattern in D3X at ISO 1600 and ISO 100

However, I do notice a weird global pattern in the IMX028 bias frame. At first I thought it’s due to a light leak. But I quickly ruled it out since it decreases with increasing ISO and is absent in 2 Bayer channels. It cannot be a low frequency noise in power circuit either as the same pattern is observed in individual bias frames and different ISO. Could this be individual sensor issue? I don’t know. But it’s not observed in any other camera I had data access to.

Read Noise

Most calibration program only outputs a master bias frame. During this process, read noise for each pixel can be calculated by sample standard deviation without much effort. (In statistics, sample SD is a biased estimator, but none the less, consistent if sample size is large)

ReadNoise

Read noise in D700, D3X, D5100 and D800

It is clear how the sensor design had an effect on the read noise distribution. Unlike the 4T stand alone pixel design in NC81338L, all Sony CMOS inside Nikon used 2.5T shared pixel. That’s what give you the pairs of noisy pixels.

Photo Response Non-Uniformity

PRNU is very difficult to estimate unless a light source more uniform than the sensor response itself is used. Some form of PRNU, such as stitching artifact is more apparent.

Stitching artifact in IMX028

These artifacts are caused by multiexposure photolithography. Since the stepper does not have an imaging circle large enough to cover a full frame image sensor die, it has to be stitched like making a panorama. IMX028 has only one seam while the NC81338L has 3 due to a much smaller mask used.

IMX071 Characteristics

The IMX071 is the CMOS imaging sensor on Nikon D7000 and D5100. After almost 2 years of work, we have found the register controlling the black level and the patch is available for both D5100 and D7000. As expected, the register is part of the EXPEED Image Processor, not on sensor. With more tweaking on the EXPEED preprocessing switch, we discovered the optical black region is also filtered to get rid of hot pixel and pattern noise. Digital gain above ISO1000 and color channel scaling could also be turned off.

Much work has been done to understand the sensor SPI protocol as well. So far, a register controlling the average black level after digitalization is discovered. There’s another set of 4 registers controlling the analog gain for each color channel. The camera faithfully issues the same gain setting for each ISO no matter the exposure length, thus linearity should be ideal.

Based on these work, a more accurate test on the sensor can be done again. Here’s a sensor characteristic test using data obtained by service mode hack and later disabling the internal image pre-processing.

Area Definition

The mod will also turn on image sensor overscan, where the periphery dummy pixels and optical black pixels are included in the NEF image.

Total Area: 0, 0 – 5054, 3357

Virtual Dummy: 0, 0 – 5039, 11 (Blank ADC runs, very low readout noise)

Dummy Pixels: 0, 12 – 5039, 35 & 4960, 36 – 5039, 3357 (Pixel readout without dark current, for bias level estimation)

Optical Black: 0, 36 – 4959, 3357 except 2, 70 – 3, 3357 & 4956, 70 – 4957, 3357 (Dummy pixels) and effective pixels

Effective Pixel: 4, 70 – 4955, 3357

Offset to non-overscan: 8, 74

The column 5040..5054 are the horizontal blanking region and it outputs a constant value. The dummy pixels don’t seem to have a photodiode, hence not dark current but only read noise is recorded. However, dummy pixels should not be used for read noise estimation. The optical black pixels are actual pixels with light shielding, and they can be used for read noise and dark current estimation.

Testing Procedure

First, a pair of images is taken with same exposure setting against a flat lighting source. A LCD panel display with 4 layer of parchment paper is used as uniform light source. Then the RAW image is split into 4 separate CFA channel and a center 256 x 256 uniform region is cropped.

Image pair

The average pixel value and standard deviation are recorded.

Subtract

Then the second image is subtracted from the first one, taking away the variance caused by PhotoResponse Non-Uniformity (PRNU). The rest of the variance will be the double of summation from photon shot noise, dark current shot noise and read noise. Since the shutter speed is relative fast – 1/6s, thus we ignore the dark current.

2x Std EV

The read noise can be calculated from the optical black region with the same subtraction. Then the variance of read noise is subtracted from the rest.

Test Result

Gain

The following table illustrates the more accurate sensor characteristics under ISO100 and ISO320. Note that with the patch, sensor is using the default analog gain setting instead of calibrated ones. Thus ISO100 is more like ISO 80 compared to factory setting. Also note that “DEV 2”s are the summation of 2 deviation resulting from image subtraction.

D7000-Gain

The sensor pretty much behaves “ISO-less” with identical read noise. A little elevated read noise at base ISO is likely due to quantilization error from ADC at such a low gain. PRNU may not be accurate enough since I did not use a diffuser. Flatness is achieved by manually picking a 512×512 section from image with the most uniformity.

Read Noise

To further characterize read noise at ISO320, I used an in-house tool to take 64 bias frames without shutter actuation, much like the in camera Long Exposure Noise Reduction (LENR). The images are then converted to TIFF and imported into R Studio for analysis. For one channel CCD, all pixel went through a single output amplifier, thus read noise is identical. However this is not true for CMOS where each pixel has its own amplifier, and each column has its own CDS and ADC circuit. To address this, I calculated the standard deviation for each pixel and plot its distribution. Ideally, each pixel will have a different gain in a best mathmatical model. But measuring it with the current setting is difficult. Under the assumption that gain is the same for each channel, we have this:

RN Distribution

2.5x is the median value for the read noise distribution, meaning that half of the pixels have read noise less than 2.55e- RMS in that channel. Albeit a bit higher than scientific CMOS, which has median read noise of around 1e-, this is a lot better than CCDs.

When we save the read noise as a FITs image, more interesting results are revealed!

071_RN

Notice that the pixels having high read out noise always occur in vertical pairs, why? Well I guess the answer is due to the vertical 2 pixels read out sharing structure. Namely, 2.5T pixel. It’s possible the elevated noise in the common floating diffusion or the transistors cause elevated read noise appear in both photodiode.

Dark Current

To measure the dark current, I first checked the linearity of the dark count with exposure time. A bracketing set of exposures was taken after 40 minutes of thermo stabilization. And the average count in optical black and white point in effective pixels is checked. The test was done before the hack was available, but from the subtracted black level the linearity is nicely followed.

Dark_linearity

Dark_linearity_OB 

Now the actual dark current can be computed. A series of 5 minutes exposures were taken continuously. And the dark count can be illustrated using the following diagram. The dark current gradually increase from 0.13eps to 0.34eps at the end. It never reaches equilibrium even after 100 minutes as the battery continues to heat up.

Dark_current

As dark current accumulates with increased temperature, the distribution shifts right and becomes even dispersed due to dark current shot noise and hot pixels. dark_current

Recently, 2 NTC thermistors were discovered inside the camera, one of which resides on the sensor PCB. The good news is, there’s a PTP command to read the resistance value although it’s not reported in EXIF. The thermistor is not in direct contact with the sensor heat sink nor the metal mounting frame, but none the less it’s the best representation of sensor temperature so far.

Dark-Temp

The above dark current – temp diagram was constructed using 146 30sec dark frames and immediate NTC thermistor read out. 5% of extreme pixels on both end were discarded. 0.14eps at 25°C.

Regression

With linear regression, I could better tell the doubling temperature. A 4.8°C doubling temp was recorded around RT. Below 10°C, cooling the sensor give dimishing return as doubling temp goes to 6.8°C. But none the less, only 0.005eps of dark current is accumulating at freezing. That’s only 3 electrons for 10 minutes exposure!

For more sensor related information, view this post.

Updated 2015/2/27

LPS Filter for D7000

Just got my Hutech clip in filter for D7000, here’s a unboxing post of the LPS-P2-N4 filter. LPS stands for Light Pollution Suppression I guess. Overlaying the glass is a special interference coating that rejects the unwanted wavelength from most sodium lamps, while passing the majority of nebula emission light.

Package

ADs

A brochure of advertisement, including the spectrum of the filter

Filter in Box

The filter packed inside a SD card box, rolled in bubble wrap

Installed on D7000

Clip the filter into the lens mount

Now it’s time to mount the lens. It’s said on their website that lens with greater than 8.7mm protrusion are not compatible. The Samyang 14mm 2.8 AE works pretty well with less than 8mm of protrusion. And it clears my biggest doubt on using wide angle lens with interference filter. Here’s 2 images taken with filter on, absence of any color vignetting seen on front mounted LPS.

Cloudy day with LPS

Indoor shot with LPS 

The indoor shot showing distinctive spectrum from 2 fluorescent tube

And as expected, the AF 50 1.8D cannot be mounted because of an outstanding “Aperture Indexing Post” which transmit the maximum aperture mechanically. This is only required on old mechanical SLR like this one:

Aperture Indexing Post

The Aperture Indexing Post couples with the small black tab in the bottom of the mount on camera side

Since DSLR communicate with the lens electronically, it makes no sense to preserve this protruding post. One way is to simply shave it down, but the rear element could be damaged in the process and it will leave dust inside. During my last repair with the plastic mount on my 18-105 DX lens, I’ve found out that the plastic ring hosting the “Aperture Indexing Post” could be disassembled independently from the mount. And it turned out to be the same with 50 1.8D lens.

Plastic Ring

Disassembled plastic ring from the metal mount

If you unscrew 3 black ones seen on the metal mounting ring and 2 on the side holding the electronic contacts, the black plastic ring would comes out easily. Then just screw the contact post back onto the metal ring. The lens could be used with the clip in filter easily.

The 180 2.8D telephoto lens was my biggest concern. The screws on the filter holder scraped the 2 corners of the plastic baffled tube. It seems the baffled tube can be disassembled, yet it would present a problem when light from the last element deep inside shines onto the metal mount leading to stray light. This lens is unlike the 50 1.8D where the last optical element is outside the mount, while 180 has the last one and aperture blades deep recessed inside the mount.

180 Mount Scrape

2 corners next to the electronic contact post are scraped on 180 2.8D

 

Star Field Test

 

 

Spectrum

The spectrum measured fits reasonably well with the advertisement. The spectrophotometer used only has 2nm FWHM, meaning that the graph below is the transmission of 5nm moving average.

LPS-P2 Spectrum

Spectrum of LPS-P2 at 0 incident angle (Measuring interval 2nm)

Compilation of libgphoto2 under Cygwin

Gphoto2, and its library libgphoto2 is a Linux application enables controlling cameras and downloading images through USB PTP or serial cable. It is of importance if you would like to build a remote controlled camera or automate the time lapse photography with advanced setting, such as altering the shutter speed during sunset with a predefined value, or make exposure at precise moment for solar eclipse. Another feature will be turning on service mode to enable uncooked RAW image download for Nikon DSLRs. Cameras from almost all vendor are supported.

But this package is for Linux, we will need a emulator to work in Windows. Cygwin is one great linux emulator with a core cygwin1.dll to link basic windows API with the Linux API. Application compiled in Cygwin will be saved as a Windows executable (.exe), and can be run from Cygwin command line or directly in windows with DLLs in the same folder. To help newbie who don’t know much about Linux and want quick compilation, here’s a list of package you need to install in order to get a working libgphoto2.

I’m working with the following combination. Unlike Windows and Mac, Linux is a collection of open source package, compatibility is really a big issue. It’s like in the old days when they quote TIFF as Thousands of Incompatible File Format!

Cygwin Setup v2.774 & libgphoto2 v2.5.1.1

 

autoconf

libtool

pkg-config

GNU make

libusb

gcc-g77

libiconv

gettext

 

Except for “GNU make”, make sure you installed the source code for the other packages. Then it should be fine to follow the install procedure.

First test of converted D7000

Right now the camera is fully capable of capturing the scarce H-Alpha light from the deep universe.

Last night I went out for a test aiming the camera with 180mm lens at the center of Cygnus. Aperture is set to F3.5 to limit chromatic aberration. Tracking is done with AstroTrac TT-320X. Total exposure 33 minutes.

Picture saved with settings embedded.

新机场

不得不表示对livespace的关闭无语,好在发现Wordpress在国内外都可以访问

那么重新开新的机场,过段时间把之前的日志转进来