Presence detection#

This presence detector measures changes in the data over time to detect motion. It is divided into two separate parts:

Intra-frame presence – detecting (faster) movements inside frames

For every frame and depth, the intra-frame deviation is based on the deviation from the mean of the sweeps

Inter-frame presence – detecting (slower) movements between frames

For every frame and depth, the absolute value of the mean sweep is filtered through a fast and a slow low pass filter. The inter-frame deviation is the deviation between the two filters and this is the base of the inter-frame presence. As an additional processing step, it is possible to make the detector even more sensitive to very slow motions, such as breathing. This utilizes the phase information by calculating the phase shift in the mean sweep over time. By weighting the phase shift with the mean amplitude value, the detection of slow moving objects will increase.

Both the inter- and the intra-frame deviations are filtered both in time and depth. Also, to be more robust against changing environments and variations between sensors, normalization is done against the noise floor. Finally, the output from each part is the maximum value in the measured range.

Presence detected is defined as either inter- or intra-frame detector having a presence score above chosen thresholds.

How to use#

Tuning the sensor parameters#

First, the range of detection needs to be determined. Based on the start range, start_m, a best fit for the profile is calculated. The profile is set to the biggest profile with no direct leakage in the chosen range. This is to maximize SNR. The shortest start range needed for the different profiles can be found in Table 13:

Table 13 Minimum start range for different profiles.#

Profile

Start range

1

0 m

2

0.14 m

3

0.28 m

4

0.38 m

5

0.64 m

Note

To maximize SNR in long range detections, the start range needs to be set to at least 0.64 m.

For each profile a half power pulse width can be calculated based on the pulse length. We choose the step_length to not exceed this value, while still having it as long as possible. We want the step length as long as possible to reduce power consumption, but short enough to get good SNR in the whole range. Choosing a high number of hwaas will increase SNR. However, it will also affect the power consumption. Choose the highest possible HWAAS that still fulfills your power requirements. A good starting point is to use the deault value. For better use of the intra-frame presence detector, increase the number of sweeps_per_frame. This will improve the sensitivity.

Tuning the detector parameters#

To adjust overall sensitivity, the easiest way is to change the thresholds. There are separate thresholds for the inter-frame and the intra-frame parts, inter_detection_threshold and intra_detection_threshold. If only one of the motion types is of interest, the intra-frame and inter-frame presence can be run separately, otherwise they can be run together. The detection types are enabled with the inter_enable and intra_enable parameters.

For slow motion detection, there is the possibility to use inter_phase_boost to increase sensitivity. This will increase detection for someone sitting still and breathing, even if the sensor is not placed in an optimal position. However, have in mind that it will increase detection of all slow moving objects.

If a stable detection and fast loss of detection is important, for example when a person is leaving the sensor coverage, the inter_frame_presence_timeout functionality can be enabled. If the inter-frame presence score has declined during a complete timeout period, the score is scaled down to get below the threshold faster.

Advanced detector parameters#

Antoher way to adjust overall sensitivity is to change the output time constants. Increase time constants to get a more stable output or decrease for faster response.

Fast motions - looking for a person walking towards or away from the sensor

The intra-frame part has two parameters: intra_frame_time_const and intra_output_time_const.

Look at the depthwise presence plot in the GUI. If it can’t keep up with the movements, try decreasing the intra frame time constant. Instead, if it flickers too much, try increasing the time constant. Furthermore, if the presence score output flickers too much, try increasing the intra output time constant, while on the other hand decreasing it will give faster detection.

Slow motions - looking for a person resting on a sofa

For the base functionality, the inter-frame part has four parameters: inter_frame_slow_cutoff, inter_frame_fast_cutoff, inter_frame_deviation_time_const, and inter_output_time_const.

The inter-frame slow cutoff frequency determines the lower frequency cutoff in the filtering. If it is set too low, unnecessary noise might be included, which gives a higher noise floor, thus decreasing sensitivity. On the other hand, if it is set too high, some very slow motions might not be detected.

The inter-frame fast cutoff frequency determines the higher bound of the frequency filtering. If it is set too low, some faster motions might not be detected. However, if it is set too high, unnecessary noise might be included. Values larger than half the frame_rate disables this filter. If that is not enough, you need a higher frame rate or to use the intra-frame part.

Inter-frame phase boost

To increase detection of very slow motions inter_phase_boost can be enabled.

Inter-frame timeout

For faster loss of detection, inter_frame_presence_timeout can be used. This regulates the number of seconds needed with decreasing inter-frame presence score before the score starts to get scaled down faster. If set to low, the score might drop when a person sits still and breathes slowly. If set very high, it will have no effect.

Detailed description#

The sparse IQ service service returns data frames in the form of \(N_s\) sweeps, each consisting of \(N_d\) range distance points, see Frames, sweeps and subsweeps. We denote frames captured using the sparse IQ service as \(x(f,s,d)\), where \(f\) denotes the frame index, \(s\) the sweep index and \(d\) the range distance index.

Intra-frame detection basis#

For very fast motions and fast detection we have the intra-frame presence detection. The idea is simple – for every frame we depthwise take the deviation from the sweep mean and low pass (smoothing) filter it.

Let \(N_s\) denote the number of sweeps, and let the deviation from the mean be:

\[s_\text{intra_dev}(f, d) = \sqrt{\frac{N_s}{N_s - 1}} \cdot \frac{1}{N_s} \sum_s |x(f, s, d) - y(f, d)|\]

where the first factor is a correction for the limited number of samples (sweeps).

Then, let the low pass filtered (smoothened) version be:

\[\bar{s}_\text{intra_dev}(f, d) = \alpha_\text{intra_dev} \cdot \bar{s}_\text{intra_dev}(f-1, d) + (1 - \alpha_\text{intra_dev}) \cdot s_\text{intra_dev}(f, d)\]

The smoothing factor \(\alpha_\text{intra}\) is set through the intra_frame_time_const parameter.

The relationship between time constant and smoothing factor is described under Calculating smoothing factors.

The intra-frame deviation is normalized with a noise estimate and, when appropriate, a depth filter is applied, both are discussed in later sections.

Inter-frame detection basis#

In the typical case, the time between frames is far greater than the time between sweeps. Typically, the frame rate is 2 - 100 Hz while the sweep rate is 3 - 30 kHz. Therefore, when looking for slow movements in presence, the sweeps in a frame can be regarded as being sampled at the same point in time. This allows us to take the mean value over all sweeps in a frame, without losing any information. In the basic part of the inter frame presence, we only use the amplitude value. Let the absolute mean sweep be denoted as

\[y(f, d) = |\frac{1}{N_s} \sum_s x(f, s, d)|\]

We take the mean sweep \(y\) and depthwise run it though two exponential smoothing filters (first order IIR low pass filters). One slower filter with a larger smoothing factor, and one faster filter with a smaller smoothing factor. Let \(\alpha_\text{fast}\) and \(\alpha_\text{slow}\) be the smoothing factors and \(\bar{y}_\text{fast}\) and \(\bar{y}_\text{slow}\) be the filtered sweep means. For every depth \(d\) in every new frame \(f\):

\[ \begin{align}\begin{aligned}\bar{y}_\text{slow}(f, d) = \alpha_\text{slow} \cdot \bar{y}_\text{slow}(f-1, d) + (1 - \alpha_\text{slow}) \cdot y(f, d)\\\bar{y}_\text{fast}(f, d) = \alpha_\text{fast} \cdot \bar{y}_\text{fast}(f-1, d) + (1 - \alpha_\text{fast}) \cdot y(f, d)\end{aligned}\end{align} \]

The relationship between cutoff frequency and smoothing factor is described under Calculating smoothing factors.

From the fast and slow filtered absolute sweep means, a deviation metric \(s_\text{inter_dev}\) is obtained by taking the absolute deviation between the two:

\[s_\text{inter_dev}(f, d) = \sqrt{N_s} \cdot |\bar{y}_\text{fast}(f, d) - \bar{y}_\text{slow}(f, d)|\]

Where \(\sqrt{N_s}\) is a normalization constant. In other words, \(s_\text{inter_dev}\) relates to the instantaneous power of a bandpass filtered version of \(y\). This metric is then filtered again with a smoothing factor, \(\alpha_\text{inter_dev}\), set through the inter_frame_deviation_time_const parameter, to get a more stable metric:

\[\bar{s}_\text{inter_dev}(f, d) = \alpha_\text{inter_dev} \cdot \bar{s}_\text{inter_dev}(f-1, d) + (1 - \alpha_\text{inter_dev}) \cdot s_\text{inter_dev}(f, d)\]

This is the basis of the inter-frame presence detection. As with the intra-fram deviation, it’s favorable to normalize this with the noise floor and, if relevant, apply a depth filter. Both are discussed in later sections.

Inter-frame phase boost#

To increase detection of very slow motions, we utlize the phase information in the Sparse IQ data. The first step is to calculate the phase shift over time. Let \(u(f, d)\) be the mean sweep:

\[u(f, d) = \frac{1}{N_s} \sum_s x(f, s, d)\]

The mean sweep is low pass filtered and the smoothing factor, \(\alpha_\text{for_phase}\), is set from a fixed and quite high time constant, \(\tau_{for_phase}\), of 5 s:

\[\bar{u}_\text{for_phase}(f, d) = \alpha_\text{for_phase} \cdot \bar{u}_\text{for_phase}(f-1, d) + (1 - \alpha_\text{for_phase}) \cdot u(f, d)\]

When a new frame is sampled, we take the mean sweep and calculate the phase shift between this mean sweep and the previous low pass filtered mean sweep. We define the phase shift to never exceed \(\pi\) radians by adding \(2\pi k\) for some integer \(k\):

\[\phi(f, d) = |angle(u(f, d)) - angle(\bar{u}_\text{for_phase}(f, d)) + 2\pi k|\]

In open air where only noise is measured, the phase will jump around. To amplify the phase shift boost for human breathing, while at the same time decreasing it for open air, the phase shift is weighted with the amplitude. For a more stable weighting, the mean sweep is low pass filtered before the amplitude is calculated:

\[\bar{u}_\text{for_amp}(f, d) = \alpha_\text{inter_dev} \cdot \bar{u}_\text{for_amp}(f-1, d) + (1 - \alpha_\text{inter_dev}) \cdot u(f, d)\]
\[A(f, d) = |\bar{u}_\text{for_amp}(f, d)|\]

The amplitude is noise normalized(see next section) and truncated to reduce unwanted detections from very strong static objects:

\[A(f, d) = \max(A(f, d), 15)\]

Before the final output is generated, the depthwise inter-frame presence score is multiplied with the phase and amplitude weight:

\[\bar{s}_\text{inter_dev}(f, d) = \bar{s}_\text{inter_dev}(f, d) \cdot \phi(f, d) \cdot A(f, d)\]

Noise estimation#

To normalize detection levels, we need an estimate of the noise power generated by the sensor. We assume that from a static channel, i.e., a radar signal with no moving reflections, the noise is white and its power is its variance. However, we do not want to rely on having such a measurement to obtain this estimate.

Since we’re looking for motions generated by humans and other living things, we know that we typically won’t see fast moving objects in the data. In other words, we may assume that high frequency content in the data originates from sensor noise. Since we have a relatively high sweep rate, we may take advantage of this to measure high frequency content.

Extracting the high frequency content from the data can be done in numerous ways. The simplest to implement is possibly a FFT, but it is computationally expensive. Instead, we use another technique which is both robust and cheap.

First, to remove any trends from fast motion in the frame, we differentiate over the sweeps \(N_\text{diff}=3\) times:

\[x'(f, s, d) = x^{(1)}(f, s, d) = x(f, s, d) - x(f, s - 1, d)\]
\[...\]
\[x^{(N_\text{diff})}(f, s, d) = x^{(N_\text{diff} - 1)}(f, s, d) - x^{(N_\text{diff} - 1)}(f, s - 1, d)\]

Then, take the mean absolute deviation:

\[\hat{n}(f, d) = \frac{1}{N_s - N_\text{diff}} \sum_{s=1 + N_\text{diff}}^{N_s} | x^{(N_\text{diff})}(f, s, d) |\]

And normalize such that the expectation value would be the same as if no differentiation was applied:

\[n(f, d) = \hat{n}(f, d) \cdot \left[ \sum_{k=0}^{N_\text{diff}} \binom{N_\text{diff}}{k}^2 \right]^{-1/2}\]

Finally, apply an exponential smoothing filter with a smoothing factor \(\alpha_\text{noise}\) to get a more stable metric:

\[\bar{n}(f, d) = \alpha_\text{noise} \cdot \bar{n}(f-1, d) + (1 - \alpha_\text{noise}) \cdot n(f, d)\]

This smoothing factor is set from a fixed time constant of 10 s.

Both the intra-frame deviation, \(\bar{s}_\text{intra_dev}(f, d)\), and the inter-frame deviation, \(\bar{s}_\text{inter_dev}(f, d)\), as well as the amplitude in the ínter-frame phase boost is normalized by the noise estimate, \(\bar{n}(f, d)\), as:

\[\bar{s}(f, d) = \frac{ \bar{s}(f, d) }{ \bar{n}(f, d) }\]

Depth filtering#

If we choose profile and step length in a way that the reflection spans several depth points, we apply a depth filter with length \(n\) on both the noise normalized intra-frame deviation, and the noise normalized inter-frame deviation. If the depth filter length is odd we have:

\[n' = \frac{n - 1}{2}\]
\[z(f, d) = \frac{1}{2n' + 1} \sum_{i=-n'}^{n'} \bar{s}(f, d + i)\]

and if the depth filter length is even we have:

\[n' = \frac{n}{2}\]
\[z(f, d) = \frac{1}{2n'} \sum_{i=-n'}^{n' - 1} \bar{s}(f, d + i)\]

where the signal \(\bar{s}\) is zero-padded, i.e.:

\[\bar{s}(f, d) = 0 \text{ for } d < 1 \text{ or } d > N_d\]

Output and distance estimation#

The outputs from the noise normalized and depth filtered intra-frame deviation and inter-frame deviation are the maximum scores of the respective deviation:

\[v(f) = \max_d(z(f, d))\]

As a final step, the outputs are low pass filtered:

\[\bar{v}(f) = \alpha_\text{output} \cdot \bar{v}(f-1) + (1 - \alpha_\text{output}) \cdot v(f)\]

The smooting factors for the outputs are set through the intra_output_time_const and the inter_output_time_const parameters.

When both detectors are enabled, presence is defined as either the intra-frame or the inter-frame being over the threshold. If both have detection, the faster nature of intra-frame presence compared to inter-fram presence makes it best practise to use this score to estimate distance. If only one part has detection we will use this for the distance estimate. The estimate is based on the peak value in the data. Let \(p\) be the “present”/”not present” output and \(d_p\) be the presence depth index output:

\[p = v > v_\text{threshold}\]
\[d_p = \arg\max_d(z(f, d))\]

Inter-frame timeout#

For faster decline of the inter-frame presence score, an exponential scaling of the score starts after \(t\) seconds determined by the inter_frame_presence_timeout parameter. We track the number of frames with declining score, \(n\). With the fram rate defined as \(f_f\), the scale factor, \(C_\text{inter}\), is calculated as:

\[C_\text{inter} = \exp\left(\frac{\max(n - (t \cdot f_f), 0)}{t \cdot f_f}\right)\]

And the inter-frame presence score is scaled as:

\[\bar{v}_\text{inter}(f) = \frac{\bar{v}_\text{inter}(f)}{C_\text{inter}}\]

To reduce the effect of the inter-frame phase boost when the score is scaled, the time constant, \(\tau_\text{for_phase}\), controlling the smoothing factor \(\alpha_\text{for_phase}\), is scaled in a similar way. With scale factor \(C_\tau\), the time constant, \(\tau_\text{scaled}\), is calculated as:

\[C_\tau = \exp\left(\frac{\max(n - (t \cdot f_f), 0) \cdot \tau_\text{for_phase}}{t}\right)\]
\[\tau_\text{scaled} = \frac{\tau_\text{for_phase}}{C_\tau}\]

Graphical overview#

digraph { pad=0.1 ranksep=0.3 nodesep=0.3 bgcolor="#ffffff00" fontname=sans fontsize=12 style=rounded node [fontname=sans, fontsize=12, shape=record, style=rounded] subgraph cluster { style=invis subgraph cluster_inter { label="Inter-frame\npresence" style=rounded abs_sweep_mean -> {fast_lpf, slow_lpf} -> fast_slow_dev -> abs_dev -> inter_lpf -> inter_norm -> inter_depth_filter inter_depth_filter -> {inter_out_argmax, phase_boost} phase_boost -> inter_out_max -> inter_lp_out -> timeout -> timeout_scaling -> inter_threshold {rank = same; inter_threshold; inter_out_argmax} } subgraph cluster_intra { label="Intra-frame\npresence" style=rounded sweep_dev -> abs_dev_intra -> intra_lpf -> intra_norm -> intra_depth_filter intra_depth_filter -> intra_out_argmax intra_depth_filter -> intra_out_max -> intra_lp_out -> intra_threshold } subgraph cluster_noise { label="Noise estimation\ndepthwise" style=rounded noise_diff -> noise_dev noise_dev -> noise_comp -> noise_lpf } subgraph cluster_phase_boost { label="Inter-frame phase boost\ndepthwise" style=rounded {for_phase_lpf, phase_mean} -> phase_shift for_amp_lpf -> amp_mean -> amp_norm -> amp_trunc {amp_trunc, phase_shift} -> phase_boost_weight -> phase_boost {rank = same; for_phase_lpf; for_amp_lpf} } } in -> {abs_sweep_mean, sweep_dev, noise_diff, for_phase_lpf, phase_mean, for_amp_lpf} {inter_threshold, intra_threshold} -> presence {inter_out_argmax, intra_out_argmax} -> distance {presence, distance} -> out noise_lpf -> {amp_norm, intra_norm, inter_norm} in [shape=Mdiamond, label="Input frame"] abs_sweep_mean [label="Absolute mean\nover sweeps"] fast_lpf [label="Fast LPF"] slow_lpf [label="Slow LPF"] fast_slow_dev [label="Difference"] abs_dev [label="Absolute\nvalue"] inter_lpf [label="LPF"] inter_norm [label="Noise\nnormalization"] inter_depth_filter [label="Depth filter"] inter_out_max [label="max\nover depths"] inter_lp_out [label="LPF"] timeout [label="Compare against\nprevious score"] timeout_scaling [label="Timeout scaling"] inter_threshold [label="Compare\nagainst\nthreshold"] inter_out_argmax [label="argmax\nover\ndepths"] sweep_dev [label="Deviation\nover sweeps"] abs_dev_intra [label="Absolute\nvalue"] intra_lpf [label="LPF"] intra_norm [label="Noise\nnormalization"] intra_depth_filter [label="Depth filter"] intra_out_max [label="max\nover\ndepths"] intra_lp_out [label="LPF"] intra_threshold [label="Compare against\nthreshold"] intra_out_argmax [label="argmax\nover\ndepths"] noise_diff [label="Differentiate\nover sweeps\n3 times"] noise_dev [label="Absolute\ndeviation"] noise_comp [label="Compensate\nfor\ndifferentiation"] noise_lpf [label="LPF"] phase_mean [label="Mean\nphase\nover\nsweeps"] phase_shift [label="Phase\nshift"] for_phase_lpf [label="LPF"] for_amp_lpf [label="LPF"] amp_mean [label="Mean amplitude\nover sweeps"] amp_norm [label="Noise\nnormalization"] amp_trunc [label="Truncate values"] phase_boost_weight [label="Multiplication"] phase_boost [label="Phase\nboost\nweighting"] presence [label="Combine presence\ndetection"] distance [label="Combine distance\nestimate"] out [shape=Msquare, label="Output"] }

Calculating smoothing factors#

Instead of directly setting the smoothing factor of the smoothing filters in the detector, we use cutoff frequencies and time constants. This allows the configuration to be independent of the frame rate.

The symbols used are:

Symbol

Description

Unit

\(\alpha\)

Smoothing factor

1

\(\tau\)

Time constant

s

\(f_c\)

Cutoff frequency

Hz

\(f_f\)

Frame rate

Hz

Going from time constant \(\tau\) to smoothing factor \(\alpha\):

\[\alpha = \exp\left(-\frac{1}{\tau \cdot f_f}\right)\]

The bigger the time constant, the slower the filter.

Going from cutoff frequency \(f_c\) to smoothing factor \(\alpha\):

\[\begin{split}\alpha = \begin{cases} 2 - \cos(2\pi f_c/f_f) - \sqrt{\cos^2(2\pi f_c/f_f) - 4 \cos(2\pi f_c/f_f) + 3} & \text{if } f_c < f_f / 2 \\ 0 & \text{otherwise} \\ \end{cases}\end{split}\]

The lower the cutoff frequency, the slower the filter. The expression is obtained from setting the -3 dB frequency of the resulting exponential filter to be the cutoff frequency. For low cutoff frequencies, the more well known expression \(\alpha = \exp(-2\pi f_c/f_f)\) is a good approximation.

Read more: time constants, cutoff frequencies.

Configuration parameters#

class acconeer.exptool.a121.algo.presence._detector.DetectorConfig(*, start_m: float = 0.3, end_m: float = 2.5, profile: Optional[Profile] = None, step_length: Optional[int] = None, frame_rate: float = 12.0, sweeps_per_frame: int = 16, hwaas: int = 32, inter_frame_idle_state: IdleState = IdleState.DEEP_SLEEP, intra_enable: bool = True, intra_detection_threshold: float = 1.3, intra_frame_time_const: float = 0.15, intra_output_time_const: float = 0.3, inter_enable: bool = True, inter_detection_threshold: float = 1, inter_frame_fast_cutoff: float = 6.0, inter_frame_slow_cutoff: float = 0.2, inter_frame_deviation_time_const: float = 0.5, inter_output_time_const: float = 2, inter_phase_boost: bool = False, inter_frame_presence_timeout: Optional[int] = 3)#
start_m: float#

Start point of measurement interval in meters.

end_m: float#

End point of measurement interval in meters.

profile: Optional[Profile]#

Sets the profile. If no argument is provided, the highest possible profile without interference of direct leakage is used to maximize SNR.

step_length: Optional[int]#

Step length in points. If no argument is provided, the step length is automatically calculated based on the profile.

frame_rate: float#

Frame rate in Hz.

sweeps_per_frame: int#

Number of sweeps per frame.

hwaas: int#

Number of HWAAS.

inter_frame_idle_state: IdleState#

Sets the inter frame idle state.

intra_enable: bool#

Enables the intra-frame presence detection used for detecting faster movements inside frames.

intra_detection_threshold: float#

Detection threshold for the intra-frame presence detection.

intra_frame_time_const: float#

Time constant for the depthwise filtering in the intra-frame part.

intra_output_time_const: float#

Time constant for the output in the intra-frame part.

inter_enable: bool#

Enables the inter-frame presence detection used for detecting slower movements between frames

inter_detection_threshold: float#

Detection threshold for the inter-frame presence detection.

inter_frame_fast_cutoff: float#

Cutoff frequency of the low pass filter for the fast filtered absolute sweep mean. No filtering is applied if the cutoff is set over half the frame rate (Nyquist limit).

inter_frame_slow_cutoff: float#

Cutoff frequency of the low pass filter for the slow filtered absolute sweep mean.

inter_frame_deviation_time_const: float#

Time constant of the low pass filter for the inter-frame deviation between fast and slow.

inter_output_time_const: float#

Time constant for the output in the inter-frame part.

inter_phase_boost: bool#

Enables the inter-frame phase boost. Used to increase slow motion detection.

inter_frame_presence_timeout: Optional[int]#

Number of seconds the inter-frame presence score needs to decrease before exponential scaling starts for faster decline.

Detector result#

class acconeer.exptool.a121.algo.presence._detector.DetectorResult(*, intra_presence_score: float, intra_depthwise_scores: ndarray[Any, dtype[float64]], inter_presence_score: float, inter_depthwise_scores: ndarray[Any, dtype[float64]], presence_distance: float, presence_detected: bool, processor_extra_result: ProcessorExtraResult, service_result: Result)#
intra_presence_score: float#

A measure of the amount of fast motion detected.

intra_depthwise_scores: numpy.ndarray[Any, numpy.dtype[numpy.float64]]#

The depthwise presence scores for fast motions.

inter_presence_score: float#

A measure of the amount of slow motion detected.

inter_depthwise_scores: numpy.ndarray[Any, numpy.dtype[numpy.float64]]#

The depthwise presence scores for slow motions.

presence_distance: float#

The distance, in meters, to the detected object.

presence_detected: bool#

True if presence was detected, False otherwise.