3D and 2D Bubbles In Rock

Use Computer Vision and Machine Learning to Discover and Segment Chondrules in Meteorites

Abstract

Many meteorites record the early accretion of small solids into larger rocks. So do the planets, but we can study the process better in meteorites. Many have small (<1 mm) spherical rocks called chondrules (not "bubbles") with fine grained mineral dust matrix in between. We image small chunks of meteorites in 3D using synchrotron computed tomography (CT), and then we slice them with saws to make polished surfaces we can map in 2D. From the 3D and 2D data we try to measure the sizes of the chondrules in the meteorite by segmenting (outlining) them. Once they are segmented, we can get more information about each object (see Mineral Mapping challenge). Until now, the only way to segment the objects has been by hand (ugh!). Can we segment the roundish chondrules in Semarkona automatically? In CT, we have 3D data but not much chemical data. In 2D we have chemical information. Here is a map of the elements Mg, Ca, Fe element x-ray maps in red-green-blue composite:

Data

The CT data are in tiff stacks where each tiff is a consecutive slice through the rock. Here is the raw rock with a flat face on the left size in the Y-Z plane (Z is up), and two frames of CT data parallel to Y-Z:

A movie in the Y-Z direction is here: http://research.amnh.org/~debel/meteorites/Semarkona/SEm2_YZ_15fps.avi

The 2D chemical maps record the chemical abundances of elements in each pixel, and are combined into 3-element red-green-blue composites, like this Mg-Ca-Al map:

The round things are frozen magnesium-silicate rocks called "chondrules" - how would you recognize one with computer vision? We also collect maps of back-scattered electrons (BSE). Electrons "bounce" more readily off denser material, like iron-rich grains. Here is a BSE map of sample 1:

Sub challenge: Find the frame in the tomography - what plane in the full tomography volume best matches the BSE map of a flat polished surface cut from the original rock?

Methods: Detail

Synchrotron computed tomography (CT) was done at beamline 13BM at the Advanced Photon Source, Argonne National Lab, Illinois, a Department of Energy user facility.

CT yields interpretable tiff stacks of 3D density structure. Each volume element (voxel) has a value for its x-ray attenuation. Each cubic voxel is 17 micron on each edge. These data were collected in 12-bit, rendered into 16-bit for reconstruction (the "computed" part), and output as either 8- or 16-bit tiff stacks. A ~1 cubic cm. chunk of the meteorite called Semarkona (LL3.0 type) is the target for segmentation: http://research.amnh.org/~debel/meteorites/Semarkona/Sem2_YZ_8a/ (8-bit tiff stack in Y-Z plane) http://research.amnh.org/~debel/meteorites/Semarkona/Sem2_Z_16a/ (16-bit tiff stack in Z plane) The chondrules are the round things. This is a grayscale image analysis problem we approached in 2004 (http://adsabs.harvard.edu/abs/2004M%26PSA..39.5153E) using an easier meteorite to segment.

X-ray intensity maps were made with wavelength dispersive spectrometers (WDS) and an energy dispersive spectrometer (EDS) on the Cameca SX100 electron probe microanalyzer (EPMA) at the AMNH. The EPMA is a kind of electron microscope, where the electron gun makes an about 1 micron beam on the flat sample surface, and each element there emits x-rays of characteristic wavelength. The x-ray intensity at an element's wavelength at each pixel in a raster is directly proportional to the weight fraction of each element in that pixel. Chemical maps record the chemical abundances of elements at each 1 micron pixel, where pixels are 8 microns apart for this project. Raster maps 512 x 512 pixels in size (8-bit depth) are stitched together and made into color-balanced red-gree-blue (RGB) composite mosaics. The round things are frozen magnesium-silicate rocks called "chondrules" - how would you recognize one with computer vision?

---

Solutions

A 3D model showing nucleus (blue), mitochondrion (purple), and multicolored microtubules

Above is an example of the final product we hope to achieve (in this case, entirely adjusted manually). We would like to see solutions that automate some of the manual efforts and make it easier for us to produce a model like this. Streamlining any stages of the process, or in an ideal world, combining a number of these automations together into a single tool, would solve the challenge.

Sub-challenge: Find the frame that maps to the mineral map

Some needed solutions include:

• Selecting candidate cells. As of now, the TIFF images captured by the TEM are first checked by human eye, then candidate specimens are selected for further adjustment. Is it possible to choose candidate specimens using computer vision?

• Aligning slices. Register the orientation of the slices so that changing shapes line up. What do we focus on? The contour of the membrane of the organism; other cells around the organism can be helpful as well. Can you utilize an existing tool or create a new tool that accomplishes this?

• Correcting distortions. Specimens from the Museum's transmission electron microscope (TEM) are cut very thinly, and distort in the process. Once slices are aligned with one another, they may need to be adjusted to compensate for nonlinear compression, skew, and other distortions. This could also include image adjustment of contrast and brightness; slices can also have oil spots, staining artifacts, dust, oil, or other image artifacts to clean up.

Animation made from six cross-section TIFF images. The images have been rotated to match one another, but the different distortions in each image are apparent.

---

Resources

Be sure to check the Online Resources and Data Sets page to see if there might be any general purpose code or utilities you might use, especially for computer vision and image processing.

• Challenge repository files: A few publicly available files (unprocessed images, processed images, Adobe Illustrator files, and Maya ASCII files) for one of the protists is available in this repository.
• Additional images and files are available for two other protists are available on a hard drive. Teams that wish to work on this challenge can ask hackathon organizers for access to the hard drive to copy files locally.

**Please note: because some of the files for this challenge are not yet published we ask that you not keep the files listed as "under NDA" on the hard drive after the hackathon is over. As with all challenges, there will be opportunities for those interested in continuing to work on their projects after the hackathon! **

2D image processing:

• ImageJ2: The lab is currently experimenting with ImageJ to try and do some of the pre-processing of the 2D TEM images, including automated alignment/registration of each cell.
• FIJI - Fiji Is Just ImageJ: This version of ImageJ may be easier to get started with, but may also not have the same capabilities as the full toolkit.
• CellProfiler: This toolkit may have some capabilities of identifying structures within cells or otherwise contributing to 2D image processing.

3D image manipulation:

• Point Cloud Library: PCL has a bunch of tools to deal with 3D point cloud data and might be used to match or segment objects. See: http://docs.pointclouds.org/trunk/group__registration.html

Maya:

• Maya scripting documentation: Official documentation that explains Maya MEL scripting language and how to use Python in Maya scripting
• SimplyMaya scripts: Example MEL scripts from SimplyMaya

---

Challenge owner: Denton Ebel