Implement mag1c filter

[sarahmakki12]

Resources

Paper: https://ieeexplore.ieee.org/document/9034492
PyTorch implementation: https://github.com/markusfoote/mag1c

Notes

• If filter is too computationally expensive, we can find an approximation, possibly with lesser results.
• Try to integrate the above function into our code base and adjust it to be compatible with our data. Try to convert it to Tensorflow.
• helps the models get the segmentation mask more accurate than just the hyperspectral images
• run a linear regression model on the hyperspectral images to get

Prahar: As a starting point, you can get the mag1c filter working on un-cropped images. You can compare the output you get to the one in the folder (mag1c.tiff). No need to do this for the entire dataset, but if you can show it works on 30-40 images, that'll put us in a good spot

Progress:

1. Found out we need a 2 column text file, (wavelength in nm and reflectance or absorption). Why? Mag1c uses this file to detect pixels that match the methane gas spectral signature across wavelengths. It compares each pixel’s spectrum in the image against the template.

I reread parts of the starcop research paper and realized that the template they used for mag1c was derived from a strong AVIRIS-detected methane plume and extracted its spectrum.

Since mag1c helps our model to detect the methane gas in our satellite images, we would need too get a pretty accurate template if we want a more accurate detection of methane gas from our model. Though I’m not completely sure how much the template can impact our accuracy.

The spectral signature of methane with is a vector of reflectance or absorption values will be across the wavelengths between 2100 and 2400 nanometer.

Since the starcop paper didn’t release that template, we can get a synthetic template that mimics a Gaussian shaped methan absorption curve around 2310nm. It works well with AVIRIS and EMIT imagery and can be our starting point.

Or we can extract our own methane signature from the Starcop labeled plumes by finding a large confidently labeled methane plume in our dataset, select pixels from that plume area, and average the 16 band vectors to form our template. This would help with higher accuracy.

1. The mag1c tool only accepts ENVI format inputs which means a binary image file, .img, and a header file, .hdr. ENVI is a standard format in remote sensing. So the code i’m working on will read the .tif files, which were given to us from the starcop dataset, and read it with rasterio and write it back as ENVI. Later we will need to load the .img and .hdr output (output of mag1c) into a NumPy array or TF tensor so that we can normalize it, and concatenate it with the satellite image before inference or training.

High level of what the code will do:

1. Stack the 16 .tif bands into a single multi-band array (mag1c expects a single multi-band hyperspectral image in ENVI format)
2. Save the stacked array as ENVI format
3. Run mag1c once on the stack .img file

[michellejyao]

New updates:

1. https://github.com/markusfoote/mag1c/blob/master/mag1c/ch4.hdr is not the methane spectral signature template. The file ch4.hdr is an ENVI header file and contains the metadata for the ch4.img associated binary file. At the top of the ch4.hdr file, it write:
   samples = 7
   lines = 1
   bands = 31800
   This means the file describes a 7-pixel wide, 1 line, 31800 band hyperspectral image. We know the mag1c function expects a 2 column txt file from reading the mag1c.py file that says:
   with open(args.spec) as f:
   lines = f.readlines()
   template = [float(l.strip().split()[-1]) for l in lines]

The code splits each line and tasks the last column representing the absorption value. Then it collects it into a python list which becomes the vector that gets compared to each pixel in the input ENVI file.

Also the mag1c README file says: "--spec TEMPLATE.TXT: a file describing the expected spectral signature of the target gas."

That means we need to create a custom methane spectral signature template txt file. Methane absorbs strongly around 2310nm. We will model that as a dip in reflectance from 1.0 (no absorption/strong reflectance) to 0.2 (strong absorption/no reflectance), using Gaussian distribution. In the future, we can extract the real spectrum from the plume labels.

[michellejyao]

Road block #1:
Running mag1c on: ./data/raw_data/STARCOP_train_easy\ang20190922t192642_r10240_c0_w512_h512
Beginning processing of src/mag1c_outputs\mag1c_ang20190922t192642_r10240_c0_w512_h512
Using compute device: cpu.
Computing with precision float64. Output will be written as float32.
Opening radiance data file.
Loading target spectrum from src/methane_template.txt
Target spectrum loaded successfully.
Checking that the provided target is compatible with the data file wavelengths...
Traceback (most recent call last):
File "C:\Users\itsmi\Documents\GitHub\SatML\src\testmag1c.py", line 127, in
mag1c_result_path = generate_mag1c_from_datagen(sub_dir, template_path, output_base)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\itsmi\Documents\GitHub\SatML\src\testmag1c.py", line 86, in generate_mag1c_from_datagen
mag1c_output = run_mag1c(output_base, template_path, output_base)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\itsmi\Documents\GitHub\SatML\src\testmag1c.py", line 67, in run_mag1c
mag1c_module.main()
File "C:\Users\itsmi\Documents\GitHub\SatML\env\Lib\site-packages\mag1c\mag1c.py", line 653, in main
if wavelengths.shape[0] != target.shape[0]:
~~~~~~~~~~~~~~~~~^^^
IndexError: tuple index out of range

More to update