What is the ideal FIR length, given a specific pulse length?

Question

I am trying to ascertain what the 'ideal' FIR filter length should be, given the pulse length $T_p$ of a windowed sinusoid in noise that I seek to filter.

As parameters into an FIR filter that I design, I have:

1. $F_c = 15 \text{ kHz}$, The center frequency. (This is the carrier frequency of the signal). I know this.

2. Since this is a BPF FIR, I specify the pass band as $F_{c} - \frac{1}{T_p}$ to $F_{c} + \frac{1}{T_p}$. This is because the bandwidth of the windowed sinusoid is $\frac{2}{T_p}$

3. The last parameter that I do not know how exactly to specity, is the length of this FIR... this is where I am lost. What is the ideal length here, (if any?)... Should it just be the length of the pulse (in samples of course), thereby making it something akin to a matched filter? Does this mean I have no further gains in increasing filter length?

---

As further context, I am seeking this 'ideal' length, should it exist, because I am trying to filter out as much noise as possible, but also try my best to retain the sharp transients. This is what led me to ask, is there an ideal filter length to start from. For example, in the following plot below, I have filtered a noisy version of my signal, with filters of length 11 (red) and 171 (black) respectively. They are shown below:

As you can see, while the black result is 'smoother', you can see that it is also more 'smudged' in as far as its transients go. In contrast, the red still retains some noise, but transients are not as affected.

The plot below shows the spectrums of the above filters:

TLDR: So, is there an 'ideal' length for FIR filters, in so far as that further increasing the filter length does not buy you any more noise immunity, but might actually smear your transients even more than needed?

EDIT:

I have added two new images. The first one has filter of length 11, (red), filter of length 171, (black), and filter of length 901, (blue). The thick-blue is the spectrum of the data.

Here is the corresponding results for the filter of length 11, (red), and the new filter of length 901, (black).

Answer 1

I'm converting some of my comments from above into this answer:

If you are trying to detect the presence of some pulse shape $p(t)$ on the AWGN channel, then the optimum detector uses a filter that is matched to the pulse shape $p(t)$ (aptly named the matched filter); this maximizes the signal-to-noise ratio (SNR) at the output of the filter and thus provides the best detection statistic. This approach is equivalent to a sliding cross-correlation of the pulse shape with the observed signal, expressed mathematically as follows:

$$ d(t) = x(t) * p(t) $$

where $x(t)$ is the observed signal and $d(t)$ is the resulting detection statistic.

The main problem, therefore, consists of selection of an appropriate threshold that can be used to determine where the pulses of interest occur in $x(t)$. Specifically, one would indicate a detection when $d(t) > T$, where $T$ is a threshold used to balance between two opposing performance metrics: probability of detection $P_d$ and probability of false alarm $P_{fa}$. Here is a link to a previous answer where I talk about the tradeoff a bit more. Target values for these metrics would be chosen according to the requirements for your specific application.

For this case, we can come up with general expressions for $P_d$ and $P_{fa}$ pretty easily:

• Probability of false alarm: A "false alarm" indicates a case where the detector reports the presence of the target pulse $p(t)$ when it is in fact not present. Since we've defined the channel to be AWGN, that means that for a false alarm, the input signal $x(t)$ is a white Gaussian noise (WGN) process. Without loss of generality, we will assume that noise to be zero-mean with variance $\sigma^2$.

  To determine $P_{fa}$, we are concerned with what the signal at the output of the matched filter looks like. Recall that $d(t)$ is defined as:

  $$ d(t) = x(t) * p(t) = \int_{-\infty}^{\infty} x(\tau) p(t-\tau) d\tau $$

  Assuming that $x(t)$ is a white Gaussian noise process with variance $\sigma^2$, it can be shown that $d(t)$ will also be Gaussian, with a variance equal to:

  $$ \sigma_d^2 = \sigma^2 \int_{-\infty}^{\infty} |p(t)|^2 dt $$

  That is, the variance at the output of the matched filter is just scaled by the total energy of the pulse waveform $p(t)$. The detection statistic $d(t)$ is then just a Gaussian process with variance $\sigma_d^2$. The probability of a false alarm at any given time $t$ is equal to the probability that the detection statistic exceeds the threshold $T$. Using the properties of the Gaussian distribution, we can write this as:

  $$ \begin{align} P_{fa} &= P(d(t) > T\ | \text{ no signal present}) \\ &= 1 - F_d(T) \\ &= Q\left(\frac{T}{\sigma_d}\right) \end{align} $$

  where $F_d(d)$ is the cumulative distribution function (CDF) of the Gaussian distribution and $Q(x)$ is the Q-function.

• Probability of detection: This case differs from the false-alarm case in that the signal of interest is present. Specifically, we examine the situation where the matched filter is perfectly aligned with the pulse of interest. The same noise component that we analyzed previously is present, but the autocorrelation of the desired pulse shape gives it a non-zero mean. This mean is equal to the total energy of the pulse waveform:

  $$ \mathbb{E}\left(d(t)\right) = \int_{-\infty}^{\infty} |p(t)|^2 dt $$

  The probability of detection, therefore, is the probability that the detection statistic exceeds threshold:

  $$ \begin{align} P_{d} &= P(d(t) > T\ | \text{ signal present}) \\ &= 1 - F_d(T) \\ &= Q\left(\frac{T - m_d}{\sigma_d}\right) \end{align} $$

The design process would then look like this:

• Select an operating range for your detector, defining a minimum signal-to-noise ratio (or equivalently, a maximum noise variance $\sigma^2$) at which you will operate.

• Assuming the worst-case conditions (i.e. the maximum noise level), select a threshold $T$ that meets either your required $P_d$ or $P_{fa}$ (whichever is more important to you).

• Plug the resulting value for $T$ into the other equation to determine your predicted performance metrics.

This is a pretty high-level treatment of the problem, and if you get down to business trying to build something that works practically, you'll run into some other details of note:

• One thing that may be relevant for your problem of detections sinusoids is that it's likely that the received pulses will be at some unknown starting phase $\phi$ relative to your template pulse shape $p(t)$. You would then observe a reduction in the correlation peak based on the amount of phase offset, which will wreak havoc with your detector's performance. If that is the case, a noncoherent detector is a better approach:

  $$ d_1(t) = x(t) * p(t) $$ $$ d_2(t) = x(t) * p_Q(t) $$ $$ d(t) = \sqrt{d_1^2(t) + d_2^2(t)} $$

  where $p_Q(t)$ is a 90-degree shifted (or quadrature) version of the template sinusoidal pulse. The statistics of this case are slightly different and are left as an exercise for the interested reader.

• The above treatment assumes that the pulse is received in $x(t)$ at the same power level as the template $p(t)$, which is almost certain to be untrue. Two ways to approach this complication come to mind: either use some sort of automatic gain control (AGC) process in order to steer the received power level to what you would expect, or you can make $T$ an adaptive threshold that adjusts itself relative to the observed signal (e.g. you could try to estimate the background noise variance $\sigma^2$ and then set $T$ appropriately).

Answer 2

In order to obtain the "best" filter you need to go through a few steps

1. Determine the spectrum of your signal: the spectrum of a windowed sine pulse is basically given by the window function convolved with the carrier. Since the carrier is a sine wave the convolution simply shifts the window spectrum. However the shape of the window is important: is it raised cosine, hanning, hamming, kaiser, etc. This determines how wide the bandwidth is and if there are strong side lobes that need to be maintained.
2. Determine the spectrum of the noise: that's easiest done by looking at the spectrum at areas in the signal where there is only noise
3. Look at signal and noise spectrum. Useful frequencies are the ones where the signal energy is larger than the noise. "Bad" frequencies are the ones where the noise is bigger.
4. Create a filter specification that passes the "useful" frequencies and "rejects" the bad ones. The amount of rejection can be optimized using an approach called "Wiener Filter" http://en.wikipedia.org/wiki/Wiener_filter. In a pinch a bandpass will do as well.
5. The filter spec should have at least center frequency, bandwidth and roll off steepness. More filter coefficients typically makes the pass band flatter and the roll off steeper. Again looking at a signal to noise graph will typically tell you what roll off makes sense

Answer 3

The FIR filter length will determine the accuracy of the FIR filter. The FIR filter is an approximation of the optimum filter response (which is for you to decide) and shouldn't be tied to the "pulse" duration. The longer the FIR filter, the more coefficients you have, the more accurately the filter magnitude response will conform to your specifications. There are two other issues to consider. You have to consider the phase response because that can distort the shape of the demodulated pulse. You may already be doing this, but I think you probably want your FIR filter to have linear phase. Additionally as you increase the length of the FIR filter, you will add delay from input to output, so you don't want to just shoot for the moon in terms of filter length. So in terms of optimal length, you have to find a happy medium between accuracy and response time.