National Park Time Lapse – Tranquility

Since my last astrophotography road trip in California two and half years ago, I really haven’t spent anytime writing on travel and photography. Amidst the camera project and my PhD, I somehow have accumulated a pile of decent photography yet to be processed or released. But anyway, all those hard work serves to produce better images wouldn’t they. So I took a break in the previous weeks to finish off some of those leftover photo work.

Please enjoy my second time lapse compilation – Tranquility

Included are some of the time lapses I took in Big Bend NP, Mojave National Preserve, Death Valley, Picture Rock Lakeshore and Shenandoah NP. Then there’s also the Jiuzhai Valley in Sichuan, China!

In terms of astrophotography, I only got a few left in hard drive for release. The road trips I cover recently were on the east coast. With light pollution and bad weather along the way, there really weren’t many stars to be seen. Let alone for deep space imaging.

Cygnus

Wide Field Milky Way Center shot in Death Valley

As for 360 panorama, it becomes a routine for me now as the pipeline for 3×6 stitching is well established. In the meantime I start to incorparate the floor image in the stitching process.

Carlsbad CavernsThe WindowWhite SandBig BendPorcupine MountainTybee Island LighthouseShenandoahDeath Valley

Mouse over for location, Click for 360 View

The link to my first time laspe compilation is here:

Advertisements

The Whitney Challenge

I don’t prefer physical challenges. Yet the Mt. Whitney stood there a very special place. It’s the highest point in contiguous United States. Beyond that, it’s also one of few accessible through a trail. This makes it a huge difference comparing to, say the highest point in China – Everest, where professional mountaineering skill is essential plus huge bucks.

I’ve been tempting to summit this mountain for a while. So back in 2014 when planning a road trip in California, I decided to stop near the lone pine for a distant look on the mountain and get more detailed information. So here it is, a map from National Geography and a picture.

247530505892932681

HAO_4649

The center is actually the line pine peak, Whitney is right under the flag pole

It wasn’t until I got home that I found out the image was the wrong target. And the Whitney of course is so far way. This gives you a glimpse on the length of trail, a whopping 22 miles round trip! I gradually realized this wasn’t an easy task. Each year, some people got injured along the way or worse, lost their lives here. The complication is partly due to the altitude sickness after exertion. Other times, lack of buddy to overlook each other when accident happens.

In 2016, I met Weichen, a post-doc and an avid amateur mountaineer here in Michigan. This was the perfect chance to accept this challenge. Another big push to me was so much was going on with my life that I just want to forget. In May this year we didn’t get the lottery for an overnight permit. A slot in late July was the only option for two people.

So began our training, I picked a staircase with elevator next to it. And every other day, we set off a 99 floors of simulated hike with weight. Then the next day ran for cardio. The gradual push made me comfortable with even more weights. Three days before we left for Whitney, we peaked at 11 kilometer.

We packed in all gears needed for the hike. For your reference, head light and sufficient battery is critical as you don’t want to get lost in the middle of night or fall off cliff! Trekking poles can be helpful crossing the creek and on the way down. Lastly, don’t forget to bring food, candy and water. On the website it suggests 3 liters per person per day and that really is the minimal. There is water supply along the way if you bring your filtration device. But if not, bring plenty water!

The Great Sand Dunes

Monument Valley

Navajo Mountain and Colorado-San Juan River Junction

Lake Powell, Page, Glen Canyon Dam, Horseshoe Bend and Antelope Canyon

Grand Canyon

At that time, flight to Las Vegas was the most affordable. We arrived the previous day at noon. After getting the car and filling ourselves full with buffet, we headed toward California through the Death Valley National Park.

Badwater Basin

 

Stovepipe Wells and Sunset

By the time we got to lone pine, it was already dark. We retrieved the permit from a small locker next to the visitor center. Then we immediately checked into motel just to get enough sleep for the hike next day.

Day use permit at Whitney

Sky was clear in high Sierra early next day. We left the motel at 3:30AM before dawn. There was 20 minutes’ drive from lone pine to the Whiney Portal. From there we started the actual hike at 4:15. Just after the third switchback, we encountered the first trouble at the north fork of lone pine creek. Water level was unusually high that made us to trek it through with bare foot. The standing rocks were mostly submerged.

Moon and Venus, looking back down the valley

The trail stayed on one side of mountain for the next half hour of hike until you hit the Lone Pine Lake. The first crossing has tree log bridges. After a while, we hit another submerged section without any bridge. For once more, we took off our boots to cross it.

\

Me crossing the log bridge near Lone Pine lake

In the next hour, we saw the first sign of snow. The unusual heavy precipitation last winter at California has left so much snow to melt. And of course, the flooding alone the Whitney trail.

Daybreak above the Lone Pine Lake

Alone the route the trees recedes rapidly. After passing the outpost campground below the mirror lake, landscape transforms into barren rock faces. Occasionally, there’s patches of flower scatter between the melting snow.

The traverse is at the top of this huge snow patch

Then we encountered the first challenge. A part of trail completely covered in snow and we don’t have any ice climbing equipment as we didn’t expect this so late in the season. For each step forward, we made sure the trekking pole is deep solid in the snow and the other foot wasn’t slipping at all. I was scared since this was my first time attempting something so steep on ice.

The red arrow indicates the traverse covered in ice

At 10AM we finally arrived at the trail camp upstream of Consultation Lake. As we watched the huge massif and icy slope, chances of making to top at noon faded away in a flash. Altitude had already taken effect on my body when I saw my fingernails starts turning purple. We spent 15 minutes recovering ourselves with water and candy. Then we put on the sunscreen before march ahead to the endless 99-switchbacks that will quickly rise from 12,000 feet to trail crest at 13,600.

Looking back down at the Consultation Lake (right) and the Trail Camp Pond (left)

The last switchback, traversing a 40 degree ice slope

Had I succumbed to the previous scary icy traverse, then I probably wouldn’t have to suffer this near heart stopping endeavor at 13500 feet. After the last switchback, we had to traverse two more continuous patches of melting snow just before the trail crest. OMG! A 500 meter 40 degree slope is waiting for your slip at any single step. I tried not looking down. But at every step, I have to just to make sure I secured my foot at the right place.

Exhausted at the trail crest

This little guy wants my food!

At 4100 meters, I started to feel the pain in my stomach. I know I was exhausted, so was my buddy. But there was no appetite to eat. The weird feeling of thin air here is taking a toll! I essentially forced the lunch into my mouth. With the projected timing, we will at most arrive after 2PM. We dropped some of our backup supplies at the John Muir Trail Junction and we pushed on.

The Hitchcock Lakes in the Sequoia National Park

The final 2 miles to the summit was a completely different experience. Here traversing across the west flank of peaks leading to Whitney, it was dry and hot unlike the icy north slope of the valley we trekked up. For this portion, we were left with a single bottle of water for both of us. So we had to preserve as much as we can.

A final few steps toward the victory!

Oh I forgot, we brought two cans of beer! Cheers!

Finally, we made it at 2:20PM. And cheers! We brought two cans of beer, but only for photography. I sip once cause I knew alcohol would make you dehydrate even quickly. So we just left the other can inside the hut. We log our name into history and then for some serious view!

The highest 360 panorama in contiguous United State

(click for 360 link)

I took a 360 panorama on the top of Whitney, with love. There was absolutely no time to shoot a time lapse. We headed down immediately.

The needles next to the last 2 miles up

Started our way down at 3PM

At the trail crest, I felt we probably would not make it to the top. Now I had a sensation of dying due to dehydration before reaching the trail crest. Half way down, we consumed the last drop of water. It was a race against time until we dehydrate or make it back to the trail junction. There we have 1 more liter for both of us.

Some of the trail section is dangerously close the cliff edge

But to get more water, we would have to make it back down to 12000 feet at the trail camp, where most campers were. So I still have to think about that icy traverse again. We saw some daring climbers slid down the icy slope. That was really a dangerous move given melting snow with low friction and exposed rocks.

As the trail camp got closer, the hope of surviving finally overtook the fear of death. We met someone alone the route who kindly gave us filtered water. The fresh melting ice water was cold, yet so refreshing! The sun already falls behind the massif. We would have to made it faster to get back to the Whitney Portal before it’s dark again.

We skid down the frozen ice of lone pine creek

To conserve time, we follow the fresh trace of others alone the ice on the lone pine creek. We stayed away from the rocks where cracks most often formed. But in the end, there’s a moment we had to get back on the trail when getting close to the snow line. Sunset happened by the time we got close to the Mirror Lake. We got back onto the trail, turned on our head lamps and continued down the mountain.

We followed the river on the way down. GPS record

The flooding was even worse after a day of melting snow washing down in torrent. But this time, we trekked right across the creek. The cold ice actually made a beneficial effect like pain killer to our heating and swelling ankle.

The milky way came out, for the first time I’ve ever seen in high Sierra. It was so clear. With the calm air at high altitude, those stars barely twinkled! I decided to sit down for a five minute rest, just enjoy the sound of nature from waterfalls, insects, birds and the starry nights through the gaps between those tall pine trees.

We finally got back to the portal. Both cellphones battery went dry. Buddy’s GPS watch also stopped recording between the Mirror Lake and the Lone Pine Lake. It was fulfilling feeling after such an achievement. 22miles/36km round trip in snow icy condition without proper gear. It was close to midnight when we checked into hotel. I took a shower and fell asleep immediately when I touched bed. My leg still hurt the second day. That morning, I finished three plates of breakfast while watching sun light shine on the Mt. Whitney.

Sun set behind the Sierra mountain range at Owen Lake

We left the lone pine before sunset. I really need to treat myself with star gazing. Next stop was the Dante’s View over watching the Badwater Basin of Death Valley.

Along the high way, shimmering lights filled the distant void of desert. I wondered, would I one day stand on the top of that peak again or I might never wish to challenge myself with some like this. But one thing was certain, I had just left my lingering love on the top. Good night, Mt. Whitney!

The Milky Way Arch over Badwater Basin

(Click for 360)

Written on Oct 15, 2017

CMOS Camera – P6: First Light

In July I finally got the UV/IR cut filter for this camera. I designed a simple filter rack and 3D printed it. The whole thing now fits together nicely in front of the sensor. IR cut is necessary due to a huge proportion of light pollution in the near-infrared spectrum.

Filter rack

UV/IR cut taped to the plastic rack.

With all the hardware in place, I added a single trigger exposure mode in the camera firmware. And accordingly a protocol command to issue a release on the PC software.

70SA

The camera is then attached to a SkyRover 70SA astrograph. In the camera angle adjuster, there’s a 12nm bandwidth Ha filter. This would allow me to easily reject light pollution while imaging in front of my house. Focusing through the Ha filter is extremely difficult. I chose a bright star and pulled the exposure time to maximum during liveview for focusing. Finally, before the battery pack went dry (supplying both AZ-EQ6 mount and my camera), I managed to obtain 15 frames with 5 minutes each.

NGC7000

No dark frame was used for the first light image and guiding performance was exceptional. This foiled the kappa sigma algorithm for hot pixel removal and makes the background very noisy. Anyway, NGC7000 already shows rich details!

Remarks

1. This sensor has higher dark current than Sony CMOS. Somewhat >4 folds more at the same temperature. However, doubling temperature is small. In another word, its dark current reduces quickly with cooling. Last time I observed no dark noise at –15C. Thus imaging the horsehead during winter would be brilliant here in Michigan!

2. Power issue. The sensor consumes ~110mA @5V during long integration comparing to ~400mA for continuous readout, which is minimal. However, the Zynq SoC + Ethernet PHY consumes much more than a full running CMOS sensor. Thus some power saving technique can be employed. CPU throttling during long integration/standby, powering down the fabric during standby mode, move the bulk of RTOS to OCM instead of using DDR, etc. But many of these require substantial work.

 

Anyway, I’m going to use this during the solar eclipse here in USA!

CMOS Camera – P5: Ethernet Liveview

To make camera control easier, I spent the last several weeks making a control scheme based on Ethernet. The camera will be a server with LWIP tasks running on a freeRTOS operating system. The client will be my computer of any OS platform. The only thing connects the two will be a 1G Ethernet cable. To speed things up, the client demo program is written in python3.

image

Client application based on TKinter

Once the RTOS is boot up, a core task will set up the network and instantiate a listening port. On the client side, all control commands are sent through TCP protocol once connection is established. On the application layer, there’s really not much protocol going here. I chose to decode the command using a magic code followed by actual command id. Four commands are established so far:

1. Send Setting

2. Start Capture (RTOS will create the CMOS run task)

3. Halt Capture

4. Send Image

Once TCP handshake is done, client could send 1 and 2 to begin video capture with defined setting. During this time, command 4 will retrieve the latest image to decode and display on GUI. The camera setting includes exposure time and gain, frame definition and on chip binning, shutter mode and ADC depth, as well as many other readout related registers.

The image are transferred in RAW data, which is linear. Thus numpy functions become very helpful here to implement the level control and post readout binning. RAW image can be written to disk as RAW video given a fast enough I/O.

Several ongoing improvements are under progress. First and foremost is the Ethernet performance. In a direct point to point connection, there really should be reliability issue. And according to test, TCP could achieve ~75MB/s on a GigaETH. UDP will be even fast might need to with potential packet drop. But anyway, TCP will be able to handle 24FPS 1080P liveview. But both server and client needs optimization. Other issue includes file saving task on RTOS and better long exposure control.

Update 6/24

Some updates on the board operation system.

1. By modifying on socket API, I incorporated  the zero copy mode of TCP operation. Thus pointer to data memory is passed directly to EMAC task and no stack memcpy is involved. This provides a 15% bandwidth gain under TCP operation. Top speed is around 70MB/s for payload.

2. I added in an interrupt event on SDIO driver to avoid polling the status register. Thus IO will not waste CPU cycle and the single core can perform EMAC listening task. As a result, SD file I/O can be performed simultaneously along the video liveview. 

Microscopic survey of recent image sensors

Last year through cooperation with ChipMod studio, we obtained multiple die shot images of recent Sony sensors. And in this post we’re going to show some of them. Most of our device identification is based on teardown from various reliable source, such as Chipworks and manufacture repair manual. Or from direct microscopic imaging. For inference, it has to be relied on die signature such as number of bond pad and their relative location, or referred as “bond pad signature”.

 

Let’s begin. The first one, IMX038AQE from Pentax K-x/K-r. It’s the same silicon die as the AQQ variant seen in Nikon D90 and D5000 DSLR.

SONY and device marking code IMX038

Layer number during photolithography of Bayer pattern and on chip lens (OCL)

Factory die level testing left probe scratch on the test pad

Next, let’s take a look at the IMX071AQQ from D5100.

No device marking was found on the die except “SONY”

Bayer layer mark. PAC appears to be Photo Activated Chemical based on patents

Factory test pads

Finally we have the IMX094AQP from D800/D800E. The first image shows the alignment mark near the die boundary. It’s interesting that Nikon customized the cover glass to be a quartz anti-moiré layer. As advertised by Nikon, both D800 and E variant included the vertical separation glass. The glass appeared to be specially AR coated only in the image area, not on the whole plate level. We had never seen this on other Sony sensor, not even on IMX128.

Alignment marks shows duplicated image in vertical direction

Edge of the multilayer AR coating shows uneven gradient

Similar to 071, Sony did not imprint the device marking in the corner. However, I found a pair of mask number related to this device. MM094L and MM094R on the long edge of the silicon die. This pairs of mark appears on Sony full frame sensors only. We later found the pair on IMX235 and IMX128 as well. Based on their location, I realized that it could be a mask code for a stitching pair. A full frame sensor was just too big to fit in the circle of stepper imaging field of view. Thus to make a full sensor, a pair of mask has to be used just like taking your panorama. This was the case for IMX028 when I discovered the non-uniformity on its flat field image.

The microscope I had access to has a 40x objective. However its working distance is too short to prevent direct imaging through the sensor cover glass. With the permission and request by ChipMod studio, I’ll show some more enlarged image onto the pixels themselves.

One interesting sensor was the X-pro1 CMOS harboring a Sony marking code. Again no actual device code.

Xpro-1 IMX165

Xpro-1 IMX165

The corner of Fujifilm X-trans array

Through the central opening on the back of PCB, the package marking register X165A?. The second character is presumably a R or P or F.  It’s possibly IMX165AFE based on IC searching where many distributer had an entry on their listing. Sony usually used the second letter to denote Bayer type, with Q for RGB Bayer and L for mono. F would naturally mean a different pattern like X-trans. The die itself, appears to be the same as the 16MP IMX095 found in Sony NEX F3 and Pentax K-01.

Fujifilm CMOS PCB

IMX095AQE-K-01

Pentax K-01 uses CLCC version IMX095AQE

It’s possible that Sony fixed the underlying circuit, only altering the last few steps in their back end of line (BEOL) to pattern a different color filter array. This would significantly reduces cost by avoiding making a new sensor. So the question is, when will we see a native monochromatic CMOS in APS-C or larger format?

Next we will have a big one, the IMX235AQR in the Sony A7S, which harbors a 12MP full frame at around 8.5um pixel pitch. ChipMod obtained the following image during mono chip mode. In essence, scraping away the microlens and Bayer layer. The pixel opening is super wide given 55% area fill factor on the metal three layer.

50x objective view of the Metal 3 layer after Bayer removal

IMX235

The microlens array appears to shift towards top left of pixel boundary

We also surveyed the IMX183 BSI sensor. Surprisingly, BSI sensor also has a grid on the light sensitive side. After some literature search, the presence of this grid could reduce color crosstalk between adjacent pixels. This is because on BSI sensor light can easily pass to the collecting well in the next pixel when fill factor gets larger and incident angle gets smaller. It is also the reason to employ microlens array to focus light rays on to the pixel center.

IMX183

IMX183 BSI pixel boundary grid

At the end, we take a look at the old school interline CCDs. ICX413 in Pentax K-100.

And ICX493 using rotated horizontal-vertical transfer registers.

 

ICX493 employed four phase CCD, with two pixels covering a period. Thus readout is interlined. Charge on odd and even columns are transferred upward then right or downward and left to their respective HCCD (organized vertically) on each side for read out. Then the same is repeated for interline rows.

Cooled CMOS Camera – P4: Lens Mount

Things have been going slowly recently. Instead of improving the image acquisition pipeline, I decided to apply some mechanical touch to make it more stable. The PCI-E connector is without a doubt, the weakest link for the entire structure. Also I need to actually make this a camera by mounting a lens on it, instead of just several pieces of PCBs.

Drawing_1Drawing_2

3D Visualization with PCBs

Notice that the linkage of side plate consists of three slots instead of holes. This was designed for tuning the flange distance from the focal plane. Both PCBs are mounted on M3x0.5mm standoffs just like your motherboard in a computer case.

ASM_BackASM_FrontMount

View through the lens mount

An EF macro extension tube is used to mounting the lens. The flange distance is approximately 44mm. The electrical contacts are left float for now. I attached a 50mm 1.8D lens using a mount adapter.

50mm Lens

First image this camera sees through my window.

No AA filter? More of a marketing hype

Back in 2012 when D800 was released, Nikon did a bit tweaking on its antialiasing filter which led to the higher resolution D800E. A pair of birefringent crystal is organized in the parallel 180 degree to cancel out the effect. But were they worth it? As we had disassembled more camera, I decided to write a post on how these filter stack is organized.

ChipMod sent me a pair of filters on the Nikon D600. The IMX128 was scraped during monochromatic mod.

Filter set

Filters from D600: UV-IR, CMOS Cover Glass, Color Correction Stack

Back on D7000, I had shown the filter set consists of an antialiasing layer with UV-IR coating and an ICF stack sandwiched from a wave plate, a color correction glass and an other AA layer. Upon receiving the filter, I initially suspect the same. After closer examination, I found the color correction glass was actually just a single thin layer. No wave plate was glued to it. On a micrometer, it registered 0.484mm thick.

Without a wave plate, it’s impossible to spread a point into four, since the two light rays are in orthogonal polarized directions. I thought a workaround was to cut the AA filter at 45 degree instead of 0 or 90. (Here I refer to the orientation to the direction where two light rays separate. The AA filter is always cut perpendicular to the optical axis, or Z-axis, of the birefringent crystal) As such, the blue color could be mixed with red. However, upon inspection under a microscope, this was again rebutted. It turned out, the first UV-IR layer is only blurring on the vertical direction, leaving moiré as is in the horizontal direction.

AA under Microscope

Calibration slide between objective and AA1mm in 100 division

Stage setup with micrometer ruler in the vertical direction

The spread from this filter is around 5 micron and wider than that in D7k. This corresponds to a thicker crystal at 0.8mm. Now we know for sure D600 only blurs vertically. This gives the advantage to gain a bit higher resolution in the horizontal direction. The DPreview had an excellent resolution test confirming the case. D600 resolve horizontally well beyond 36, albeit accompanying color moiré. But it blurs out at around 34 in vertical directions.

Any other cameras also do this? It turns out that many other cameras follow this trend. To name a few: Sony A7Rii, Nikon D5100, and possibly other low end DSLRs all had a single AA glued to a color correction filter. One possibility is to suppress the already strong false color during video live view rising from row skipping. However, I would still argue the effect of this is minimal given the spread distance close to pixel pitch.

The material for AA filter and wave plate is usually crystalline quartz glass. Many website cites lithium niobate and that is incorrect. An argument floats around that quartz has too small a birefringent value and it requires a thick slice. This is true during the early days of digital imaging where pixel pitch were huge! (>10um) Once a proper calculation is done, the above 0.8mm thick material happens to give a close to 5um displacement. Should lithium niobate be used, it would be way too thin to manufacture. Another interesting property with quartz, or fused silica, is its UV transparent property. Based on the above transmission spectrum scan, the AA substrate material permits UV to pass when measured at corner. Lithium niobate would absorb strongly in UV just like those ICFs. Notice that without any coating, the glass itself reflects 10% of light. Again, for emission nebula imaging, you could keep the UV-IR filter.

Cooled CMOS Camera – P3: Image Quality

In the previous post I successfully obtained the test pattern with custom VDMA core. The next step will be to implement an operating system and software on host machine. In order to obtain real time live view and control, both software should be developed in parallel. Thus in this post, let’s take a look at the image quality with a simple baremetal application.

The sensor is capable for 10FPS 14Bit, 30FPS 12Bit, or 70FPS at 10bit ADC resolution. For astrophotography, 14bit provides the best setting for dynamic range and achieves unity gain at default setting. The sensor IR filter holder and the camera mounting plate are still in design. I will only provide a glimpse into some bias and dark images at this moment.

To facilitate dark current estimation, the cover glass protective tape was glued to a piece of cardboard. The whole sensor was then shielded from light with metal can lid. Lastly, the camera assembly was placed inside a box and exposed to -15°C winter temperature. During the process, my camera would continuously acquire 2min dark frames for 2 hours, followed by 50 bias frames.

Bias Hist

Pixel Intensity distribution for a 2×4 repeating block (Magenta, Green, Blue for odd rows)

The above distribution reflects a RAW bias frame. It appears each readout bank has different bias voltage in its construction. The readout banks assignment is a 2 rows by 4 columns repeating pattern, each color for each channel. A spike in the histogram at certain interval implies a scaling factor is applied to odd rows post-digitalization to correct for uneven gain between top and bottom ADCs.

Read Noise Distribution

Read Noise – Mode 3.12 Median 4.13 Mean 4.81

The read noise distribution is obtained by taking standard deviation among 50 bias frames for each pixel. Then I plot the above distribution to look at the mode, median and mean. The result is much better compared to a typical CCD.

Dark_current_minus_15

Finally the dark current in a series of 2-minute exposures is measured by subtracting master bias frame. Two interesting observations: 1. The density plot gets sharper (taller, narrower) as temperature decreases corresponding to even lower dark generation rate at colder temperature. 2. The bias is drifting with respect to temperature. This could be in my voltage regulator or in the sensor, or a combination of two.

The bias drift is usually compensated internally by the clamping circuit prior to ADC. But I had to turn this calibration off due to a specific issue with this particular sensor design. I will elaborate more in a later post. Thus to measure dark generation rate, I have to use FWHM of the noise distribution and compare against that in a bias frame. At temperature stabilization, FWHM was registered at 8.774, while a corrected bias is 8.415 e-. For a Gaussian distribution, FWHM is 2.3548 of sigma. Thus the variance for the accumulated dark current is 1.113 given the independent noise source. As such, the dark generation rate at this temperature is less than 0.01 eps. Excellent!

Preliminary Summary

The sensor performs well in terms of noise. For long exposure, the dark generation rate in this CMOS is more sensitive to temperature change than CCDs. The dark current is massively reduced when cooled below freezing point. The doubling temperature is below 5°C.

LEXP_001

An uncorrected dark frame after 120s exposure showing visible column bias and hot pixels