Sunday, September 8, 2013

Data Compression: Curiosity's “Focus-Merge” Images...

The Curiosity rover currently exploring Mars has an interesting image manipulation capability on board that is effectively a form of data compression, albeit a very unconventional one.

Since early in the days of digital cameras, photographers have realized that there are ways to combine multiple digital images of the same scene to provide a better composite image.  One of the first uses of this technique was to increase the camera's effective depth-of-field with a technique called focus stacking or focus bracketing.  The idea is simple enough: you take several images of precisely the same scene, but each with different focus settings.  Then you combine the parts of each image that are in focus, so that the entire resulting image is in focus.

Focus stacking is particularly useful in macro (close-up) photography, because the lenses used have very shallow depth of field.  Digital photographers routinely use focus stacking today, generally with software such as Photoshop.  You might start with images taken at 4 or 5 focus settings, then combine them carefully to get one image as a result.  This is especially useful with studio or product photography, where your subject is stationary.

From the perspective of data size, this is a very inefficient process.  To get a single final image, you have to move several full size images from the camera to your focus stacking software.  For an earthbound digital photographer, that's a few seconds of time as the memory card gets read.  For Curiosity, millions of miles away on Mars, it's an entirely different story – it has to send all those images over a very slow communications link.

Now Curiosity has a macro camera (the MAHLI imager), and it has a major depth-of-field challenge.  It's designed to capture images of rocks for geologists to study, and those rocks generally have surfaces that are far from flat.  But sending multiple images for focus stacking over that slow data link is a more bandwidth than they'd like to use.  The solution: implement focus stacking right on the Curiosity rover, so that only the final, all-in-focus image has to be transmitted.  The MAHLI team calls this technique “focus-merge”

How do they accomplish this feat, normally done with human manipulation?

First, the MAHLI team built an in-focus sensing capability into the camera that extends across the entire image.  On every image, the camera knows which pixels are in focus and which are not.  This is similar to (but more extensive than) the multiple-areas autofocus capability of most high-end digital SLRs today.  This in-focus data is stored with each image taken.

Then for any given rock that MAHLI is imaging, they'll take several photos (2 to 8, depending on the rock's depth variation) at different focus settings.  Software on board Curiosity then scans all the images pixel-by-pixel, choosing the image with the best focus for each pixel.  It also creates a visualization map showing the image selected at each pixel (using 2 to 8 gray levels, with blacker being more distant).

To anyone who has done much macro photography (as I have, with wildflowers), the result approximates magic: extreme close-up photos with essentially infinite depth-of-field.  I really can't use this technique with wildflowers, because the darned things are highly unlikely to be still at microscopic scales.  But I can envy the result nonetheless.

Here's a recent Curiosity focus-merge image of a rock's surface (on top) and it's matching focus map (at bottom):




Links of the Day...

Armchair parkour.  The helmet camera really brings it home...

I don't think there's a practical cure.  You don't have to know much history to understand that the safest time in human history is right now.  It never ceases to amaze me how few people seem to understand this rather basic (and quite verifiable) fact.  This ignorance, unfortunately, drives a lot of public policy (both directly and indirectly, via elections)...

I'm a big supporter of gun rights, but...  Iowa is granting concealed carry permits for blind people.  Their laws require it.

A majority of low information voters can't be good.  I'm glad to see leaders taking note of this problem.  Retired Supreme Court Justice Sandra Day O'Connor:  “Less than one-third of eighth-graders can identify the historical purpose of the Declaration of Independence, and it's right there in the name.”

Sad cat diary.  If you own a cat, this will resonate :)

Tyrants rule us.  What else can you call a government that won't even let you know how many security-related inquiries it makes?  Yahoo! reports that it supplied customer content (presumably email and/or chat) to the government for 4,604 requests (each of which may encompass information on multiple individuals).  Yahoo! states: “The U.S. Government does not permit us to disclose additional details regarding the number of requests, if any, under national security authorities at this time, or even to separate them in aggregate from other requests.”  We're not allowed to know how our own government works.  That's tyranny.

At this point, color me completely unsurprised.  The Obama administration secretly removed Bush-era restrictions on NSA spying on American citizensWe can't believe anything these bozos tell us!

Speed riding on Mont Blanc.  Via reader Simi L.  Be sure to go full screen...

Jury to decide whether science data is accurate?  A judge has ruled that Michael Mann's defamation suit against the National Review (prompted by this Mark Steyn post) can proceed.  To prove defamation, Mann will have to prove that the allegations in Steyn's post are false – which means we may end up having a jury decide on the “truth” of the science.  In my opinion, that's a very bad idea...

I wondered about these...  I once saw a giant concrete arrow, in a remote area of the Ruby Mountains in northern Nevada, and I could not figure out what the heck it was for.  It's all the Post Office's fault!  

Ouch.  Conrad Black, writing in the New York Sun:
“But for this administration to redeem its credibility now would require a change of direction and method so radical it would be the national equivalent of the comeback of Lazarus: a miraculous revolution in the condition of an individual (President Obama), and a comparable metamorphosis (or a comprehensive replacement) of the astonishingly implausible claque around him.”
I think the first chance for Lazarus to return is after the next election – and I'm not too optimistic about that, either.  Especially after the 2012 elections...

Wow!  Now this is some juggling.  A team of jugglers!

Embarrassed, repelled, and outraged.  How does our military leadership feel about the Obama administration's preparations for intervention in Syria?  Robert Scales, retired U.S. Army Major General and former commandant of the U.S. Army War College, writing in the Washington Post:
They are embarrassed to be associated with the amateurism of the Obama administration’s attempts to craft a plan that makes strategic sense. None of the White House staff has any experience in war or understands it. So far, at least, this path to war violates every principle of war, including the element of surprise, achieving mass and having a clearly defined and obtainable objective.

They are repelled by the hypocrisy of a media blitz that warns against the return of Hitlerism but privately acknowledges that the motive for risking American lives is our “responsibility to protect” the world’s innocents. Prospective U.S. action in Syria is not about threats to American security. The U.S. military’s civilian masters privately are proud that they are motivated by guilt over slaughters in Rwanda, Sudan and Kosovo and not by any systemic threat to our country.

They are outraged by the fact that what may happen is an act of war and a willingness to risk American lives to make up for a slip of the tongue about “red lines.” These acts would be for retribution and to restore the reputation of a president. Our serving professionals make the point that killing more Syrians won’t deter Iranian resolve to confront us. The Iranians have already gotten the message.
I find it very plausible that Scales is well plugged into our military leadership's thinking, and what he reports is even more plausible...

How much does a bullet cost? Between the time Nidal Hasan (the Fort Hood Shooter) killed 13 and wounded 30, and his sentencing next week, U.S. taxpayers will have paid him over $300,000.  We've also paid over $650,000 just to house him.  A bullet certainly would be less expensive.  Seems like a win for process over common sense.  Just sayin'...

Why Old Men Don't Get Hired...

Via reader Jim M.:
An old man went to a job interview, given by the human resources manager.  The conversation went like this:

Manager: “What is your greatest weakness?”
Old Man: “Honesty.”
Manager: “I don't think honesty is a weakness!”
Old Man: “I really don't give a shit what you think.”