Splice for Fourier log deconvolution?

[maclariz]

I can't find any function for splicing low loss and high loss datasets. This is pretty key to how the Glasgow group does their work. We tend to fit against experimental standards where possible, for our most quantitative work. I would like to gradually transition away from dependence on Digital Micrograph, but some of their nice functions are not obvious in Hyperspy. I have Fourier_ratio_deconvolution working but we would prefer Fourier_log_deconvolution if we can get it. One step towards this is having a good splice function. One nice step towards would be plotting the two spectra together on same axes with the appropriate scaling applied, and the splice region highlighted. And then from that choosing a splice range and calculating a scaling of the high loss to make it fit the low.

I am prepared that we might have to build this ourselves, but need to check first if there isn't already something there I could use for this.

[ericpre]

I have Fourier_ratio_deconvolution working but we would prefer Fourier_log_deconvolution if we can get it.

The Fourier ratio deconvolution is also implemented:
https://hyperspy.org/exspy/reference/signals.html#exspy.signals.EELSSpectrum.fourier_log_deconvolution

exspy/exspy/signals/eels.py

Lines 873 to 978 in 6444392

	def fourier_log_deconvolution(self, zlp, add_zlp=False, crop=False):
	"""Performs fourier-log deconvolution.
	Parameters
	----------
	zlp : EELSSpectrum
	The corresponding zero-loss peak.
	add_zlp : bool
	If True, adds the ZLP to the deconvolved spectrum
	crop : bool
	If True crop the spectrum to leave out the channels that
	have been modified to decay smoothly to zero at the sides
	of the spectrum.
	Returns
	-------
	An EELSSpectrum containing the current data deconvolved.
	Raises
	------
	NotImplementedError
	If the signal axis is a non-uniform axis.
	Notes
	-----
	For details see: Egerton, R. Electron Energy-Loss
	Spectroscopy in the Electron Microscope. Springer-Verlag, 2011.
	"""
	self._check_signal_dimension_equals_one()
	if not self.axes_manager.signal_axes[0].is_uniform:
	raise NotImplementedError(
	"This operation is not yet implemented for non-uniform energy axes"
	)
	s = self.deepcopy()
	zlp_size = zlp.axes_manager.signal_axes[0].size
	self_size = self.axes_manager.signal_axes[0].size
	tapped_channels = s.hanning_taper()
	# Conservative new size to solve the wrap-around problem
	size = zlp_size + self_size - 1
	# Calculate optimal FFT padding for performance
	complex_result = zlp.data.dtype.kind == "c" or s.data.dtype.kind == "c"
	size = optimal_fft_size(size, not complex_result)
	axis = self.axes_manager.signal_axes[0]
	z = np.fft.rfft(zlp.data, n=size, axis=axis.index_in_array)
	j = np.fft.rfft(s.data, n=size, axis=axis.index_in_array)
	if self._lazy or zlp._lazy:
	j1 = z * da.log(j / z).map_blocks(np.nan_to_num)
	else:
	j1 = z * np.nan_to_num(np.log(j / z))
	sdata = np.fft.irfft(j1, axis=axis.index_in_array)
	s.data = sdata[
	s.axes_manager._get_data_slice(
	[
	(axis.index_in_array, slice(None, self_size)),
	]
	)
	]
	if add_zlp is True:
	if self_size >= zlp_size:
	if self._lazy:
	_slices_before = s.axes_manager._get_data_slice(
	[
	(axis.index_in_array, slice(None, zlp_size)),
	]
	)
	_slices_after = s.axes_manager._get_data_slice(
	[
	(axis.index_in_array, slice(zlp_size, None)),
	]
	)
	s.data = da.stack(
	(s.data[_slices_before] + zlp.data, s.data[_slices_after]),
	axis=axis.index_in_array,
	)
	else:
	s.data[
	s.axes_manager._get_data_slice(
	[
	(axis.index_in_array, slice(None, zlp_size)),
	]
	)
	] += zlp.data
	else:
	s.data += zlp.data[
	s.axes_manager._get_data_slice(
	[
	(axis.index_in_array, slice(None, self_size)),
	]
	)
	]
	s.metadata.General.title = (
	s.metadata.General.title + " after Fourier-log deconvolution"
	)
	if s.tmp_parameters.has_item("filename"):
	s.tmp_parameters.filename = (
	self.tmp_parameters.filename + "_after_fourier_log_deconvolution"
	)
	if crop is True:
	s.crop(axis.index_in_axes_manager, None, int(-tapped_channels))
	return s

One step towards this is having a good splice function. One nice step towards would be plotting the two spectra together on same axes with the appropriate scaling applied, and the splice region highlighted. And then from that choosing a splice range and calculating a scaling of the high loss to make it fit the low.

Off the top of my head, there isn't a function that splice the low and high loss spectra. It would be good to have it.
Making it interactive with a SpanROI shouldn't fairly straightforward because there are other functionalities that can be used as an example. Just ask and we can give the relevant pointer.
I am wondering if this is something that should be go into hyperspy, if this is generic enough and potential useful for other extension? @jlaehne, what do you think?

[maclariz]

I realise Fourier_Log_Deconvolution is implemented it, it is just about pipelining data into this function.

[jlaehne]

I am wondering if this is something that should be go into hyperspy, if this is generic enough and potential useful for other extension? @jlaehne, what do you think?

Indeed, the splicing part for a Signal1D could be interesting also for other signal types than EELS. Actually, we have implemented a kind of splicing procedure in the lumispy function join_spectra. The integration times of the different spectra are usually same or similar. It is meant for the case that the full spectrum does not fit the on the detector and one wants to stitch several spectra together - and there will usually be small offsets so that the data needs to be scaled and cannot be simply lined up in a single vector.

[francisco-dlp]

Wouldn't it be possible to use hs.stack for splicing spectra setting the axis argument to the Energy Loss axis?