imagej MER basics - Page 7

Previous 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 Next
Author Message
hortonheardawho


Posts: 3465

Reply: 121



PostPosted: May 13, 2012 11:00 PM 

If anyone is interested ( I'm looking at you Winston ) I will post PDF_RAD_Reader, an imagej plugin that I wrote today for loading RAD files into 32 bit images with the radiance offset and scaling applied.

I validated the code by hand calculating several images - so I think it's correct.

I am still working on a procedure that may under limited circumstances allow mapping JPG images with known exposure data into approximate reflectance spectra.

dx


Posts: 1661

Reply: 122



PostPosted: May 15, 2012 2:17 AM 

h>>>

I tried the BIG macro-from your first post-on Oppy Sol 2938 as L234567 and got hung up on the little '8' anomaly...so I merged L257 and various other L's and came up with the following:

...and the Blue rocks are gone. Can you please tell me that the Blue berries name is just a name given because of the Blue in the merging images as first appeared 8 years ago, and why wouldn't they be the natural color of brown, black, red??? I am confused on this issue. Since you mentioned that there are NO blue rocks on Mars...I would certainly agree with that statement-so why are they shown blue in the merges?
I want to remove the Blue from the rocks so the blue also is removed from the berries...??? Is this correct?
Thanks for explaining.

BTW>>>I did replace the 'smiley' with an '8' and '' came up and imageJ stopped the merge process of the 6 L images. 3 out of the 6 merged-all images same type. So I tried changing the '8' as -8; - 8 ; in varying formats and the macro still hung up. Were the merged image to be in 'color' or B/W?

yt
dx

dx


Posts: 1661

Reply: 123



PostPosted: May 15, 2012 2:20 AM 

h>>>

...a piece of missing data should have read.

I did replace the 'smiley' with an '8' and '' came up and imageJ stopped the merge process

yt
dx

dx


Posts: 1661

Reply: 124



PostPosted: May 15, 2012 2:23 AM 

h>>>

Strange...it didn't post again!!!replacing the 8 with the written word.

maybe this will work.

yt
dx
delete post 123 and 124 if you wish, rhanks


dx


Posts: 1661

Reply: 125



PostPosted: May 15, 2012 8:36 AM 

h>>>

Oppy Sol 1915...Would be great to see your take on this image. Thanks.

yt
dx

hortonheardawho


Posts: 3465

Reply: 126



PostPosted: May 15, 2012 12:47 PM 

dx, if you want to see if I have done a particular image - just search for the image sol in my Flickr collection of images.

Here is the results for searching for 1915 in my images.

I was surprised that I hadn't done a full filter version of the image. The other filters may have been posted later.

So I did a full left filter ( L234567 ) "natural color", high dynamic range, maximum acuity, white balanced and exposure balanced image from the RAD data available from the Analyst's Notebook site:

The image is about as good as I can do with my amateur tools and ability.

After downloading the original data in RAD format the imagej processing steps were:

1) load the RAD .img files into 32 bit images using the plugin PDS_RAD_Reader ( which I wrote. ) for filters L2, ... L7.

2) brightness / contrast adjust each filter image individually to contain most of the data and convert to 16 bits.

3) Translation register all the images to the L5 filter image using the plugin Turboreg and stack the results.

4) create the color channels R = min ( Stack[1]...Stack[3]), G = Stack[4], B = min ( Stack[5], Stack[6] )

5) perform a square root transform on the R,G,B images ( for High Dynamic Range ).

6) create the base image from the R,G,B images.

7) white balance the RGB image using the rightmost edge of the rover. ( Is is "really" black. You can use other grey parts of the rover to do the white balance with similar results. )

Cool brightness / contrast adjust the image ignoring the specular reflections on the IDD. ( I also truncated a small percent of the highlights in the rover tracks. )

9)sharpen the image using the oddly named imagej command Process / Filters / Unsharp Mask ( using a radius of 1 )

Thanks for your continuing interest in imagej MER processing.

It keeps me engaged and looking for new ways to do things.

hortonheardawho


Posts: 3465

Reply: 127



PostPosted: May 15, 2012 12:52 PM 

The smiley face, as you can surmise from the previous post, is really an 8 'Right Parenthesis'.

LWS


Posts: 3062

Reply: 128



PostPosted: May 15, 2012 1:52 PM 

dx and hort;

Somehow I prefer the art in dx's rendering of the sol 1915 image to the science in hort's.

That dx image looks really beautiful to me even though it would not pass muster as being photorealistic or politically correct.

But who knows, perhaps when a man gets there several decades from now he might find that the traditionalists were wrong?

Winston

dx


Posts: 1661

Reply: 129



PostPosted: May 15, 2012 2:18 PM 

h>>>

WOW on your 126...beautiful science merge. I'm nowhere near accomplishing your method, but the amount of schooling you have taught me so far has opened up my mind a great deal, and I thank you for your patience.

I know what I'm doing wrong and glad you posted the difference. I'm still on Earth looking with an Earth eyeballs, you're on Mars looking with Martian eyes. That's where I want to be, then I can have some fun with the images as you are.

Thanks for mentioning to check your images over, I'll keep that in mind as a search.

LWS>>>

You may have read h's response to my question regarding the color cameras on Curiosity. I read every one of h's links on those cameras and looking forward to the results. I have a feeling there will be calibrations to the colored images thus far produced-for its always bright on Mars, nowhere to hide from that sunlight!

OK h>>>right parenthesis...somehow the left parenthesis would not show up here [123-124 above] in the post for imageJ sent back 'left8right' parenthesis...and I did not know what to make of it. I'll change it to 8> and give it another go. Thanks for setting that straight for me.[I'm not a programmer as you know]

Somehow when I tried the RGB for each image in the 'square root transform' [your No. 5 in 126] my images turned 'BLACK'!!!Did the data disappear? Shocked

yt
dx

dx


Posts: 1661

Reply: 130



PostPosted: May 15, 2012 3:09 PM 

h>>>

Trust you won't mind the image merge of 125+126, but I had to try it out. Your Martian work caught my Earth eye. LOL

I placed a comment on flickr.

yt
dx


hortonheardawho


Posts: 3465

Reply: 131



PostPosted: May 15, 2012 3:13 PM 

Er, sorry dx. I forgot an important step in using the square root transform - multiply by 255 after the square root is applied to the 16 bit image.

Maybe I need to step one step further back:

To work with 8 bit images converted to 16 bit images:

change the type of image from 8 bit to 16 bit and then multiply the image by 255. That stretches the 8 bit brightness range 0-255 to 0-65025.

The reason you have to multiply the image by 255 after the square root transform is to stretch the brightness range back to the 16 bit range.

Er, actually I don't think of the "natural color" image as a "science" image. Much better "science" can be seen with the false color image - or the saturated false color image. Or best yet - the one I am developing now based on the radiometric images. But that is for another day.

dx


Posts: 1661

Reply: 132



PostPosted: May 15, 2012 3:36 PM 

h>>>

Thank you for that clarification in 131.

yt
dx

dx


Posts: 1661

Reply: 133



PostPosted: May 17, 2012 12:45 AM 

h>>>

I produced this image using your formula on post 17-perhaps something more complicated, like my favorite Sad R, G, B ) = ( max( R3,R4,R5,R6,R7), avg( R1,R2), R1 )

That process took some time to accomplish with open and closed image windows so as not to get too confused with the merges and stacks. Wild stuff indeed. What do you mean by 'max' I could not identify where that was in imageJ? That was the only thing I did not understand.

Oppy PanCam Sol 2730

yt
dx


LWS


Posts: 3062

Reply: 134



PostPosted: May 17, 2012 7:10 AM 

Hort; I didn't see your 121 until just now.

Of course im interested in the plugin. Grateful if you would post it.

Winston

hortonheardawho


Posts: 3465

Reply: 135



PostPosted: May 17, 2012 1:04 PM 

//copy and paste to file named PDS_RAD_Reader.java in the imagej Plugin directory
//Then compile the file with the imagej command Plugins / Compile and Run...

import ij.plugin.*;
import java.awt.*;
import java.io.*;
import ij.*;
import ij.io.*;
import ij.process.*;
import ij.measure.*;

/** Opens and displays RAD PDS images. Does not work with compressed images */
public class PDS_RAD_Reader extends ImagePlus implements PlugIn {

private static final String TITLE = "PDS RAD Reader";
private String directory, fileName;
private DataInputStream f;
private StringBuffer info = new StringBuffer(512);
private double bscale, bzero;
String keyword, value, line = "", sampleType, encodingType="",mapScale="";
private int recordBytes,bitsPerPixel;
double RAD_Scale=0.0, RAD_Offset=0.0; // RAD value offset and scaling
public void run(String arg) {
OpenDialog od = new OpenDialog("Open PDS...", arg);
directory = od.getDirectory();
fileName = od.getFileName();
if (fileName==null)
return;
IJ.showStatus("Opening: " + directory + fileName);
FileInfo fi = null;
try {
if (!checkFileType(directory+fileName)) {
IJ.showMessage(TITLE, "This does not appear to be a PDS image file.");
return;
}
fi = getInfo();
} catch (IOException e) {
IJ.showMessage(TITLE, ""+e);
return;
}
if (fi!=null && fi.width>0 && fi.height>0) {
FileOpener fo = new FileOpener(fi);
ImagePlus imp = fo.open(false);
ImageProcessor ip = imp.getProcessor();
setProcessor(fileName, ip);
setCalibration(imp.getCalibration());
setProperty("Info", getHeaderInfo());
// added to convert to a 32 bit radiance image--------------------
if (RAD_Scale>0.0) {
ip=ip.convertToFloat();
setProcessor(fileName,ip);
ip.multiply(RAD_Scale);
ip.add(RAD_Offset);
}
//----------------------------------------------------------------------------
if (arg.equals("")) show();

} //else
//IJ.error("This does not appear to be a PDS file.");
IJ.showStatus("");
}

FileInfo getInfo() throws IOException {
FileInfo fi = new FileInfo();
fi.fileName = fileName;
fi.directory = directory;
fi.width = 0;
fi.height = 0;
fi.offset = 0;
BufferedReader f = new BufferedReader(new FileReader(directory+fileName));
while (!line.trim().equals("END")) {//read label until we get to the END statement

line = f.readLine();
//System.out.println(line);
//IJ.write(line);
if (line == null) break;
line = line.replace('"',' ');
if (line.trim().equals("")) continue;
if (line.indexOf("=") > 0) {
keyword = line.substring(0,line.indexOf("=")).trim();
value = line.substring(line.indexOf("=")+1,line.length()).trim().toUpperCase();
}
else continue;
if (value.length() == 0) continue;
// added to extract the radiance paramaters --------------------
if (keyword.equals("RADIANCE_OFFSET")) {
RAD_Offset = getFloat(value);
//IJ.write(value);
//IJ.write(String.valueOf(RAD_Offset));
continue;
}
if (keyword.equals("RADIANCE_SCALING_FACTOR")) {
if (value.indexOf("<")>0) value = value.substring(0,value.indexOf("<"));
RAD_Scale = getFloat(value);
//IJ.write(value);
//IJ.write(String.valueOf(RAD_Scale));
continue;
}
//----------------------------------------------------------------------------
if (keyword.equals("LINES")) {
fi.height = getInteger(value);
continue;
}
if (keyword.equals("LINE_SAMPLES")) {
fi.width = getInteger(value);
continue;
}
if (keyword.equals("RECORD_BYTES")) {
recordBytes = getInteger(value);
continue;
}
if (keyword.equals("^IMAGE")) { //will only work if image pointer follows record bytes keyword
//System.out.println("imagepointer="+value);
try {fi.offset = Integer.parseInt(value);}
catch (NumberFormatException e) {
if (value.indexOf("(") >= 0) {
fi.fileName = value.trim().substring(value.indexOf("(")+2,value.lastIndexOf(","));
fi.offset = Integer.parseInt(value.substring(value.indexOf(",")+1,value.lastIndexOf(")")));
}
else {
fi.fileName = value;
fi.offset = 0;
}
}
// fi.fileName = fi.fileName.substring(0,fi.fileName.lastIndexOf(".")) +".IMG"; }
fi.offset = (fi.offset-1) * recordBytes;
//System.out.println("offset="+String.valueOf(fi.offset));
continue;
}
if (keyword.equals("SAMPLE_BITS")) {
bitsPerPixel = getInteger(value);
//System.out.println("samplebits="+value);
continue;
}
if (keyword.equals("SAMPLE_TYPE")) {
sampleType = value;
//System.out.println("sampletype="+value);
continue;
}
if (keyword.equals("SCALING_FACTOR")) {
bscale = getFloat(value);
continue;
}
if (keyword.equals("OFFSET")) {
bzero = getFloat(value);
continue;
}
if (keyword.equals("ENCODING_TYPE")) {
encodingType = value;
continue;
}
if (keyword.equals("MAP_SCALE") ||
keyword.equals("MINIMUM_LATITUDE") ||
keyword.equals("MAXIMUM_LATITUDE") ||
keyword.equals("MINIMUM_LONGITUDE") ||
keyword.equals("MAXIMUM_LONGITUDE") ||
keyword.equals("EASTERNMOST_LONGITUDE") ||
keyword.equals("WESTERNMOST_LONGITUDE") ||
keyword.equals("MAP_PROJECTION_TYPE")
) {
IJ.write(keyword +" = " + value); // display map scale value
continue;
}
} // end of label parsing

if (bitsPerPixel==8) fi.fileType = FileInfo.GRAY8;
else if (bitsPerPixel==16) {
fi.fileType = FileInfo.GRAY16_UNSIGNED;
if (sampleType.equals("VAX_INTEGER")
|| sampleType.equals("PC_INTEGER"))
fi.intelByteOrder = true;
}
else if ((bitsPerPixel==32) && sampleType.equals("UNSIGNED_INTEGER"))
fi.fileType = FileInfo.GRAY32_INT;
else if ((bitsPerPixel==32) && sampleType.equals("PC_REAL")){
fi.fileType = FileInfo.GRAY32_FLOAT;
fi.intelByteOrder = true;
}
else if ((bitsPerPixel==-32) && sampleType.equals("REAL"))
fi.fileType = FileInfo.GRAY32_FLOAT;
else {
IJ.showMessage(TITLE, "bitsPerPixel="+bitsPerPixel+"sampleType="+sampleType+" SAMPLE_BITS must be 8, 16, 32 or -32 (float).");
f.close();
return null;
}
line = f.readLine();
if (encodingType.length() > 0 && !encodingType.equals("NONE")) {
IJ.showMessage(TITLE, "Cannot open PDS compressed images.");
f.close();
return null;
}
f.close();
if (fi.fileType==FileInfo.GRAY16_SIGNED && !(bscale==1.0&&bzero==32768.0)) {
double[] coeff = new double[2];
coeff[0] = -32768.0;
coeff[1] = 1.0;
fi.calibrationFunction = Calibration.STRAIGHT_LINE;
fi.coefficients = coeff;
fi.valueUnit = "gray value";
}
return fi;
}

String getString(int length) throws IOException {
byte[] b = new byte[length];
f.read(b);
return new String(b);
}

int getInteger(String s) {
//s = s.substring(10, 30);
//s = s.trim();
return Integer.parseInt(s);
}

double getFloat(String s) {
//s = s.substring(10, 30);
//s = s.trim();
Double d;
try {d = new Double(s);}
catch (NumberFormatException e){d = null;}
if (d!=null)
return(d.doubleValue());
else
return 0.0;
}

String getHeaderInfo() {
return new String(info);
}
boolean checkFileType(String path) throws IOException {
InputStream is;
byte[] buf = new byte[132];
is = new FileInputStream(path);
is.read(buf, 0, 132);
is.close();
int b0=buf[0]&255, b1=buf[1]&255, b2=buf[2]&255, b3=buf[3]&255;

// PDS ("CCSD", XXCD, NJPL, XXNJ, PDSX)
if ((b0==67 && b1==67 && b2==83 && b3==68) ||
(b2==67 && b3==67) ||
(b0==78 && b1==74 && b2==80 && b3==76) ||
(b2==78 && b3==74) ||
(b0==80 && b1==68 && b2==83))
return true;
else
return false;
}

}

hortonheardawho


Posts: 3465

Reply: 136



PostPosted: May 17, 2012 2:07 PM 

I tested the copy, paste, save, compile and run of this code and it works as advertised.

To download a RAD file from the Analyst's Notebook, after clicking on a file in the Search Results list, click on the "Derived Products" in the Product menu and then click on the download icon for the RAD Product.

The numbers in the 32 bit image are in spectral radiance units of W/m**2/nm/sr ( Watts per meter squared per namometer per steradian )

To convert this to an absolute spectral ratio that you can use to search a spectral library you need to divide this number by the input spectral radiance.

You can use this blackbody calculator and the temperature of 5777K and the wavelength of the Pancam filter to approximate the sun's input spectral radiance.

I have been doing this last step in an Excel spreadsheet, but am working on a mod to the RAD Reader to calculate this and put it in the 32 bit radiance image.

LWS


Posts: 3062

Reply: 137



PostPosted: May 17, 2012 6:27 PM 

Hi Hort; I downloaded the plugin, compiled and ran it. It worked OK except that after compilation I got 1 error that didn't seem to affect its running properly.

Thanks.

Winston

hortonheardawho


Posts: 3465

Reply: 138



PostPosted: May 17, 2012 6:45 PM 

Glad to hear it worked for you. What exactly was the error?

After the plugin is complied it should appear in the plugin menu after you restart imagej. You only have to compile it once.

I am trying to locate a published pancam spectra that I can use as a test of my procedures.

I think there are additional necessary multiplicative factors to turn the results into "science". I have worked out simple factors for the lighting geometry and the Martian date - but haven't tackled the sky's effect on the spectra at all. It looks like there is quite a drastic attenuation in the green - blue - UV part of the spectra. That is of course consistant with the Mars is red - really red mantra.

I think I now understand why the sundial is photographed along with every pancam image. I think that the color chips and the sundial rings are used to calibrate the light source spectra so science can be done.

dx


Posts: 1661

Reply: 139



PostPosted: May 19, 2012 3:29 AM 

h>>>

...working on merging image clarity and just now noticed the damp Martian soil!!!

yt
dx

dx


Posts: 1661

Reply: 140



PostPosted: May 20, 2012 8:09 PM 

h>>>

More work on this image to lesson the blue.

yt
dx

Previous 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 Next


Join the conversation:















Very Happy Smile Sad Surprised
Shocked Confused Cool Laughing
Mad Razz Embarassed Crying or Very Sad
Evil or Very Mad Twisted Evil Rolling Eyes Wink
Powered by MTSmileys