tools.dewow behavior at beginning/end of trace

[jankae]

I stumbled across something in tools.dewow which I believe is a bug (or if it is intended behavior, I do not understand the reasoning behind it).

Dewow removes a moving average from a trace. For the purposes of this example, assume that one trace has 300 samples and the window size for dewow is set to 101. For each sample in the trace, dewow calculates the average of the 101 samples around it (50 in both directions) and subtracts that average from the sample. At least that is how I understood the function to work.

But there is a problem for the first and last window/2 samples (in this example, the first and last 50 samples) in each trace: the averaging window would extend beyond the available samples.

The usual solution I am aware of is to shrink one side of the window to keep it contained within the trace samples. Keeping with this example for sample number 20, the window should cover samples -30 to 70. Since not all these samples exist, a smaller window from samples 0 to 70 is used instead.

Here is a very simple example how this would look like in Matlab/Octave:

clear;
original = 0:1:299
dewow = original - movmean(original, 101);
plot(original);
hold on;
plot(dewow);
xlabel("Sample (#)");
ylabel("Value");
legend("Original data", "After moving average");

However, GPRPy does it differently: For the first window/2 samples, is always uses the window from 0 to window/2. This means even for sample 49, which could arguably use a window from 0 to 99, dewow is still just subtracting the average from 0 to 50.
A similar example with GPRPy therefore gives a different result:

import gprpy.gprpy as gp
import numpy as np
import matplotlib.pyplot as plt

data = np.matrix([range(300)]).transpose()
data_dewow = gp.tools.dewow(data, 101)

original = data.transpose().getA()[0]
dewowed = data_dewow.transpose().getA()[0]

fig, ax = plt.subplots()
ax.plot(original, label="Original data")
ax.plot(dewowed, label="After dewow")
ax.set(xlabel='Sample (#)', ylabel='Value')
plt.legend()
plt.show()

There is a comment in there that makes me think that this might actually be the intended behavior instead of a bug:

GPRPy/gprpy/toolbox/gprpyTools.py

Line 71 in 69609ea

# For the first few samples, it will always be the same

This seems weirdly different to other moving average implementations I am aware of. If this is indeed not a bug, what was the reason for this implementation?

The same issue just along the other dimension is probably also present in tools.remMeanTrace (just by looking at the code, I did not test that yet).

[jankae]

Exactly the same behavior with remMeanTrace:

import gprpy.gprpy as gp
import numpy as np
import matplotlib.pyplot as plt

data = np.matrix([range(300)])
data_remMeanTrace = gp.tools.remMeanTrace(data, 101)

original = data.getA()[0]
remMeanTrace = data_remMeanTrace.getA()[0]

print(remMeanTrace)

fig, ax = plt.subplots()
ax.plot(original, label="Original data")
ax.plot(remMeanTrace, label="After remMeanTrace")
ax.set(xlabel='Sample (#)', ylabel='Value')
plt.legend()
plt.show()