AFM release (sort of)

The Atomic Force Microscope (AFM) images that were taken on the Phoenix mission have been released to the Planetary Data Systems (PDS) archive. Unfortunately, they are contained in .dat binary files apparently as byte arrays rather than in the standard image formats, so they are not easy to view.

"
Mars Phoenix Lander Mission Data
The NASA Planetary Data System announces the third and final release of data from the Mars Phoenix Lander mission. This release includes raw and derived data products from Sols 91 through 152, July 27 - October 29, 2008.

Data are being released for the following experiments:

* ASE - EDRs only
* MET LIDAR and Pressure-Temperature Sensors (includes reprocessing of data from Releases 1 and 2.) MECA (TECP, AFM, and WCL only; AFM RDRs to be released by approximately June 1, 2009) TEGA Telltale Anemometer

For OM, RAC and SSI data, please check the Imaging Node site for release status.

PDS offers two services for searching the Phoenix archives:

* The Planetary Image Atlas at the Imaging Node allows selection of Phoenix data by specific search criteria.
* The Phoenix Analyst's Notebook at the Geosciences Node allows searching and browsing of Phoenix data in the context of mission events, for the expert user.
"

http://phoenix.lpl.arizona.edu/05_20_09_pr.php

Phoenix update.

Anyone already got a look at the pressure/temperature data of the first Sols - the Sols of that neat blobs (droplets) observation on the landers struts?

Well, I went, I looked for, and got lost as usual, there were no options for AFM, after selecting phoenix, the PDS is rather huge, can someone post a link to the AFM bits?

My post was just set aside for approval ??
This hasn't happened for years ???

Right, I know now, not to fill in completely the Name, Email and URL, or I will be set aside. Just Name and email only. Here's what I just tried to say.

=====================
Well, google to the rescue again,

The EDRs are all mixed up together. e.g. AFM in with TECP and WCL data mixed together, I don't know yet how to decipher the filename here, files begin with fs, ps ws, and xs, looking for the descrip docs for that still. In the RDRs they are separated into dif dirs.

The afm data is in the AF dir.

EDRs
http://pds-geosciences.wustl.edu/geo/phx-m-meca-2-niedr-v1/phxmec_0xxx

The above directories, such as this one,
http://pds-geosciences.wustl.edu/geo/phx-m-meca-2-niedr-v1/phxmec_0xxx/data/sol004/

contain only *.dat files.

RDRs
http://pds-geosciences.wustl.edu/geo/phx-m-meca-4-nirdr-v1/phxmec_1xxx

The above directories, such as this one,
http://pds-geosciences.wustl.edu/geo/phx-m-meca-4-nirdr-v1/phxmec_1xxx/data/sol004/

contain *.txt, *.lbl and *.tab files.

The first calibration scan was done on Sol 4. Most Sols do not have AF data. Bit of a search without knowing which ones I want.

This file,
meca_afm_report_sol004.txt

found in this directory,
http://pds-geosciences.wustl.edu/geo/phx-m-meca-4-nirdr-v1/phxmec_1xxx/data/sol004/af/

Contents of file,
------------------------------------
Sol: 004
Scan # for this sol: 1 of 1
Sample: calibration set
Substrate ID: OM16
Substrate type: AFM tip finder

Intent of scan: Determination of AFM tip location relative to OM frame
Target type: alignment mark

OM context image before scan: OS004EFF896566361_11040MBM1.IMG
OM context image after scan: NONE

-------------------------------------

The first *.lbl file is about 12 KB
It has a *.tab file that goes with it.
40 MB.

Ah, it appears that files beginning with fs are the AFM files. The RDRs appear huge compared to the EDR's, isn't it usually the other way around?

How to get 80 MB of data out of 0.5 MB ?

Is there a method or software, that allows the viewing of this data? Either EDR or RDR.

Hello Mr Admin.
I have been here for years and now my posts are being set aside?

Why? What did I do?

What was in the post that caused it, very frustrating that some go and some don't.
Bleuch.

> can someone post a link to the AFM bits?

Here is a recipe for getting to them:

1. Go to http://an.rsl.wustl.edu/phx

2. Click on Search, then Search the Data Products.

3. For Instrument select MECA AFM.
For Category select Science.
For Product Type select EM2.
Click on Search.

4. That gives you the AFM SCAN data.

Now you can select one, say the one from Sol 004 and you can click on Derived Products and download. You can euther download zipped or tarred files, or the actual file. You can also play around with other selections above.

Hi Barsoomer, I tried to post much of what you just told me, during my last post, it seems that due to the URL links I included, I have now been set aside for administrator approval, this has not happened for ages. Hang on, I'll try something here, I've recently been using my name, Kevin, instead of my usual, vk3ukf.
I'll post this and try my findings again using vk3ukf.

Hey, my original post has just been inserted. I'll try my postings test somewhere else.

I would expect the .dat files to be the ones that contain the image-like data. Actually, the closest to image data would be topographic maps where the "pixels" are, I think, height data for that region of the image. To get an image, one would need to convert elevation data into a simulated image. There is some hint that such might eventually be placed in pds.

Any GIS software, open source, or at a cost, that might be available to integrate with the file and binary data types for some imaging construction? There are a variety of new GIS type software packages starting to become distributed world-wide. Unfortunately, the list of hundreds of file types exceeds the number of software's available at any point in time, and no central filtering site exists bringing GIS and these archives together.
With very limited investment, what are the alternatives?

> With very limited investment, what are the alternatives?

From elevation data, one might be able to (painstakingly) construct a 3-dimensional model out of putty or something, and then take a photo of it. Or write a program from scratch to simulate the light reflections that would result from illuminating the 3-d object from a particular direction. Maybe just displaying a color coding of the height data would allow the eye to discern the form of the object. (Just grasping at straws here.)

In the announcement, there was an item saying that RDS files of the AFM data would be released later at approximately June 1, 2009. Maybe that will include image files.

If the AFM data can be read into imagej then a surface plot can be generated which will reconstruct a 3D view of the surface.

I will have a look at the AFM data and see if I can get it into imagej.

The SDD .tab files do seem to be the ones that might have the most accessible information.

For example, from

http://an.rsl.wustl.edu/phx/solbrowser/downloadFile.aspx?type=d&f=\\pds-cluster\pds-san\phoenix-web\phx-an-data\release\data\f_meca_afm\004\fs004sdd_001_4e0111040000a0.tab

one can download the file fs004sdd_001_4e0111040000a0.tab and when I open this in emacs, I see that starting at character 79872 there are comma-separated lines of numbers like +0.0000E+000,+0.0000E+000,...

I assume each number is an elevation value, and each comma-separated line is a scan line of such values. There are 2048 lines each of which has 19968 comma-separated numbers.

Hope this helps.

Correction to reply 14:

Each line has 19968 CHARACTERS, which seems to boil down to only 1536 comma-separated numbers.

The entries in the table look like this:

...,+0.0000E+000,+2.7756E-017,+3.9796E+000,+0.0000E+000,+2.7756E-017,+4.0722E+000,+0.0000E+000,+2.7756E-017,+4.1647E+000,...

Notice that the pair +0.0000E+000,+2.7756E-017 seems to get repeated between every "ordinary" number like +3.9796E+000. I assume this is some kind of coding where only every third number actually denotes a raster value. Thus, there seem to be actually only 512 "real" values in any line of values. The interspersed pairs are dummy values, it appears.

So the topographic images may actually be something like 2048 x 512 pixel images. Also each line begins with an extra +0.0000E+000

On the second line, the "dummy" pair seems to be +1.0980E-001,+2.7756E-017 while on the third it is +2.1961E-001,+2.7756E-017. So there is also a regular progression of the dummy values. Hmm..maybe these are not just dummies but represent location coordinates of some kind for the elevation values.

Thhe SIS document

http://an.rsl.wustl.edu/phx/solbrowser/documentation/missionDocs/fpwx_meca/meca_rdr_sis.htm

describes the format of the .tab files. Quote:

The scan data follows the header as four sequential ASCII table objects, forward scan error, forward scan height data, backward scan error and backward scan height data. Within each table, three columns (X,Y,Z) are repeated 512 times to specify the X and Y position and the Z value (height in microns or error in volts) of each point in a 512 x 512 matrix. Typically, scans will be square with 256 x 256 rows and columns but can be as little as 8 x 8 or up to 512 x 512. The data will always be presented in the 512 x 512 configuration, padded out with zeros where necessary. All four sets of data, representing one AFM scan, and the header information will be stored in the same RDR file with a unique filename in the format described in 0. All scans run on the same day will be grouped together in folders labeled by sol number.

So the scan data, beginning at line 5, consists of:

(1) 512 lines of 512 ...,X,Y,Z,... triples
denoting forward error data.
(2) 512 lines of 512 ...,X,Y,Z,... triples
denoting forward scan data.
(3) 512 lines of 512 ...,X,Y,Z,... triples
denoting backward error data.
(4) 512 lines of 512 ...,X,Y,Z,... triples
denoting backward scan data.

It seems only the Z values are useful for extracting an image. I think the X,Y values give the position in the OM image where the AFM scan was performed.

Here is the first AFM image from Mars on sol 45. Perhaps it would serve as a good test for reading and displaying the data.

According to Dr Pikes BBC Phoenix Dairy nearly one hundred AFM images were taken!

Reading the diary closely reveals what the "forward and reverse scans" are - two independent images of the same scan area.

Other than this first one and the "pea in the antechamber" picture ( explained in the dairy ), and one shown briefly during a, er, Phoenix briefing, I have never seen another Phoenix AFM image.

I have found the "hooks" needed to write a plugin to import an arbitrary file formatinto imagej - but I'm not yet ready to give up on finding an easier way...

sol 1920 capture magnet and 3D surface plot:

The z values are not **really** height data - but it illustrates the power of the imagej plugin Surfacej, available in here in the BIJ plugin. The plugin is interactive in creating the 3D -- and has a zillion awesome options.

Now to get the data into imagej...

Sigh.

Why does NASA make everything so difficult?

There is an imagej plugin called openMI that opens the AFM file formats of quite a few manufacturers.

The FAMARS instrument is one-of-a-kind.

Perhaps the FAMARS developers would consider extending the OPEN_MI to open the FAMARS dat files...