Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

audio metrics: SNR, SI_SDR, SI_SNR #292

Merged
merged 126 commits into from
Jun 22, 2021
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
126 commits
Select commit Hold shift + click to select a range
74b1b9b
add snr, si_sdr, si_snr
quancs Jun 13, 2021
a653056
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 13, 2021
70f82b4
format
quancs Jun 13, 2021
9e30f76
Merge branch 'audio-dev' of https://github.com/quancs/metrics into au…
quancs Jun 13, 2021
0331dd0
add noqa: F401 to __init__.py
quancs Jun 13, 2021
fcc827e
Update torchmetrics/functional/audio/si_sdr.py
quancs Jun 13, 2021
ef19181
Update torchmetrics/functional/audio/si_sdr.py
quancs Jun 13, 2021
80e6c53
Update torchmetrics/functional/audio/si_sdr.py
quancs Jun 13, 2021
32c0ce0
Update torchmetrics/functional/audio/si_snr.py
quancs Jun 13, 2021
489dba0
Update torchmetrics/functional/audio/snr.py
quancs Jun 13, 2021
249d848
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 13, 2021
017ba3a
remove types in doc, change estimate to preds, remove EPS
quancs Jun 13, 2021
c8aa372
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 13, 2021
c82a094
update functional.rst
quancs Jun 13, 2021
a33ff6e
update CHANGELOG.md
quancs Jun 13, 2021
282681d
switch preds and target
quancs Jun 13, 2021
2ba693c
switch preds and target in Example
quancs Jun 13, 2021
b0a8382
add SNR, SI_SNR, SI_SDR module implementation
quancs Jun 13, 2021
4e08d16
Merge branch 'audio-dev' of https://github.com/quancs/metrics into au…
quancs Jun 13, 2021
db57b13
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 13, 2021
fcfe0ac
add test
quancs Jun 13, 2021
1d77709
add module doc
quancs Jun 13, 2021
6a2d886
Update torchmetrics/audio/SI_SDR.py
quancs Jun 13, 2021
44a4a17
Update torchmetrics/audio/SI_SDR.py
quancs Jun 13, 2021
56ee58e
Update torchmetrics/audio/SI_SNR.py
quancs Jun 13, 2021
0f551b0
Update torchmetrics/audio/SI_SNR.py
quancs Jun 13, 2021
2a62352
Update torchmetrics/audio/SNR.py
quancs Jun 13, 2021
c4bd0c5
Update torchmetrics/audio/SNR.py
quancs Jun 13, 2021
3bd1e7b
Update torchmetrics/functional/audio/si_snr.py
quancs Jun 13, 2021
e028cd3
use _check_same_shape
quancs Jun 13, 2021
96c579a
to alphabetical order
quancs Jun 13, 2021
0848743
Merge branch 'audio-dev' of https://github.com/quancs/metrics into au…
quancs Jun 13, 2021
19e0f0b
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 13, 2021
d7b5b0d
update test
quancs Jun 13, 2021
b179eca
Merge branch 'master' into audio-dev
SkafteNicki Jun 14, 2021
ea3aee4
Update docs/source/references/modules.rst
quancs Jun 14, 2021
fe6e6bc
move Base to the top of Audio
quancs Jun 14, 2021
67467cb
Merge branch 'master' into audio-dev
mergify[bot] Jun 14, 2021
58734ed
add soundfile
quancs Jun 14, 2021
f776693
Merge branch 'audio-dev' of https://github.com/quancs/metrics into au…
quancs Jun 14, 2021
868a9d3
gcc
Borda Jun 15, 2021
6ec17a8
fix cyclic import
Borda Jun 15, 2021
1b0d379
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 15, 2021
37a812f
pysndfile
Borda Jun 15, 2021
cd51a6c
Merge branch 'audio-dev' of https://github.com/quancs/metrics into au…
Borda Jun 15, 2021
7423b6e
v0.4.5
Borda Jun 15, 2021
0bc3956
pl
Borda Jun 15, 2021
587b26e
clang
Borda Jun 15, 2021
fdceaf4
Add FID metric (#213)
SkafteNicki Jun 15, 2021
01b5c2e
fix cyclic import `_reduce_stat_scores` (#296)
Borda Jun 15, 2021
b4ee3f8
Merge branch 'master' into audio-dev
Borda Jun 15, 2021
d500663
update test_snr
quancs Jun 15, 2021
3501dd0
update test_si_snr
quancs Jun 15, 2021
7ae7778
new snr: use torch.finfo(preds.dtype).eps
quancs Jun 15, 2021
e7dfbee
update test_snr.py
quancs Jun 15, 2021
de67913
new si_sdr imp
quancs Jun 15, 2021
6de666f
update test_si_sdr
quancs Jun 15, 2021
5fdbb0d
update test_si_snr
quancs Jun 15, 2021
608b21a
remove pb_bss_eval
quancs Jun 15, 2021
aea0091
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 15, 2021
0add577
add museval
quancs Jun 15, 2021
74d0550
update test files
quancs Jun 16, 2021
f7249b1
remove museval
quancs Jun 16, 2021
8f3e032
add funcs update return None annotation
quancs Jun 16, 2021
e876520
Merge branch 'master' into audio-dev
mergify[bot] Jun 16, 2021
be1ca0b
add 'Setup ffmpeg'
quancs Jun 16, 2021
4ce6b02
Merge branch 'audio-dev' of https://github.com/quancs/metrics into au…
quancs Jun 16, 2021
134d8cd
update "Setup ffmpeg"
quancs Jun 16, 2021
794d1ce
use setup-conda@v1
quancs Jun 16, 2021
2da22cd
multi-OS
Borda Jun 16, 2021
e786ed2
update atol to 1e-5
quancs Jun 16, 2021
cd9504e
Merge branch 'audio-dev' of https://github.com/quancs/metrics into au…
quancs Jun 16, 2021
662cf28
Apply suggestions from code review
Borda Jun 16, 2021
d44be30
change atol to 1e-2
quancs Jun 16, 2021
fe3385b
Merge branch 'audio-dev' of https://github.com/quancs/metrics into au…
quancs Jun 16, 2021
5e875ce
update
quancs Jun 16, 2021
5e3f5c8
fix 'Setup Linux' not activated
quancs Jun 16, 2021
f08916d
add sudo
quancs Jun 16, 2021
05226eb
Merge branch 'master' into audio-dev
mergify[bot] Jun 16, 2021
2f26cd6
reduce Time to 100 to reduce the test time
quancs Jun 17, 2021
ad593c7
increase timeoutInMinutes to 40
quancs Jun 17, 2021
2b2946b
Merge branch 'audio-dev' of https://github.com/quancs/metrics into au…
quancs Jun 17, 2021
377d4a4
install ffmpeg
quancs Jun 17, 2021
41fddfb
timeout-minutes to 55
quancs Jun 17, 2021
53a23c8
+git
quancs Jun 17, 2021
e8a6b6d
show-error-codes
quancs Jun 17, 2021
70a655e
.detach().cpu().numpy() first
quancs Jun 17, 2021
96e7280
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 17, 2021
a23e165
add numpy
quancs Jun 17, 2021
ddbd966
Merge branch 'audio-dev' of https://github.com/quancs/metrics into au…
quancs Jun 17, 2021
23b4683
numpy
Borda Jun 17, 2021
d726ba3
ignore_errors torchmetrics.audio.*
quancs Jun 17, 2021
0861c6e
solve mypy no-redef error
quancs Jun 17, 2021
1848f9e
remove --quiet
quancs Jun 17, 2021
8469dc8
pypesq
Borda Jun 17, 2021
4f3d41f
apt
Borda Jun 18, 2021
9358bfd
Inception Score (#299)
SkafteNicki Jun 17, 2021
fd3aa0a
Apply suggestions from code review
justusschock Jun 18, 2021
05e0f19
add # type: ignore
quancs Jun 18, 2021
a351c8d
try without test_si_snr & test_si_sdr
quancs Jun 18, 2021
67da485
test_import_speechmetrics
quancs Jun 18, 2021
64ffd9a
test_speechmetrics_si_sdr
quancs Jun 18, 2021
0b5fb27
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 18, 2021
85d9f72
test_si_sdr_functional
quancs Jun 18, 2021
f5afe0f
test audio only
quancs Jun 18, 2021
6e11386
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 18, 2021
fe9a7cf
install libsndfile1
quancs Jun 18, 2021
9491551
add sisnr sisdr test
quancs Jun 18, 2021
2236698
test all & add quiet & remove test_speechmetrics
quancs Jun 18, 2021
f5a0411
remove sudo & install libsndfile1
quancs Jun 18, 2021
3858397
apt
Borda Jun 18, 2021
3eb3d65
Update torchmetrics/functional/audio/si_sdr.py
quancs Jun 21, 2021
4c18540
[feat] Add _apply_sync to nn.Metric (#302)
tchaton Jun 21, 2021
3408927
v0.4.0rc0
Borda Jun 21, 2021
267380d
[pre-commit.ci] pre-commit suggestions (#306)
pre-commit-ci[bot] Jun 21, 2021
69c8fbc
adding KID metric (#301)
SkafteNicki Jun 21, 2021
e7753e1
Apply suggestions from code review
Borda Jun 22, 2021
e82bc96
SRMRpy
Borda Jun 22, 2021
bb045c9
pesq
Borda Jun 22, 2021
70687f1
gcc
Borda Jun 22, 2021
050bb7b
comment
Borda Jun 22, 2021
de034a3
env
Borda Jun 22, 2021
2fa46af
Merge branch 'master' into audio-dev
Borda Jun 22, 2021
cda3c6b
env
Borda Jun 22, 2021
47e0cf8
Merge branch 'audio-dev' of https://github.com/quancs/metrics into au…
Borda Jun 22, 2021
fec3598
env
Borda Jun 22, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions torchmetrics/functional/audio/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Copyright The PyTorch Lightning team.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from torchmetrics.functional.audio.si_sdr import si_sdr
from torchmetrics.functional.audio.si_snr import si_snr
from torchmetrics.functional.audio.snr import snr
60 changes: 60 additions & 0 deletions torchmetrics/functional/audio/si_sdr.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Copyright The PyTorch Lightning team.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import torch

quancs marked this conversation as resolved.
Show resolved Hide resolved

def si_sdr(target, estimate, zero_mean=False, EPS=1e-8):
quancs marked this conversation as resolved.
Show resolved Hide resolved
quancs marked this conversation as resolved.
Show resolved Hide resolved
""" scale-invariant signal-to-distortion ratio (SI-SDR)

Args:
quancs marked this conversation as resolved.
Show resolved Hide resolved
target (Tensor): shape [..., time]
estimate (Tensor): shape [..., time]
quancs marked this conversation as resolved.
Show resolved Hide resolved
zero_mean (Bool): if to zero mean target and estimate or not
EPS (float, optional): a small value for numerical stability. Defaults to 1e-8.

Raises:
TypeError: if target and estimate have a different shape

Returns:
Tensor: si-sdr value has a shape of [...]

Example:
>>> from torchmetrics.functional.audio import si_sdr
>>> target = torch.tensor([3.0, -0.5, 2.0, 7.0])
>>> estimate = torch.tensor([2.5, 0.0, 2.0, 8.0])
>>> si_sdr_val = si_sdr(target,estimate)
>>> si_sdr_val
tensor(18.4030)

References:
[1] Le Roux, Jonathan, et al. "SDR half-baked or well done." IEEE International Conference on Acoustics, Speech
and Signal Processing (ICASSP) 2019.
"""

if target.shape != estimate.shape:
raise TypeError(f"Inputs must be of shape [..., time], got {target.shape} and {estimate.shape} instead")

if zero_mean:
target = target - torch.mean(target, dim=-1, keepdim=True)
estimate = estimate - torch.mean(estimate, dim=-1, keepdim=True)

α = torch.sum(estimate * target, dim=-1, keepdim=True) / (torch.sum(target**2, dim=-1, keepdim=True) + EPS)
target_scaled = α * target

noise = target_scaled - estimate

si_sdr_value = torch.sum(target_scaled**2, dim=-1) / (torch.sum(noise**2, dim=-1) + EPS)
si_sdr_value = 10 * torch.log10(si_sdr_value + EPS)

return si_sdr_value
47 changes: 47 additions & 0 deletions torchmetrics/functional/audio/si_snr.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Copyright The PyTorch Lightning team.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import torch

from .si_sdr import si_sdr
quancs marked this conversation as resolved.
Show resolved Hide resolved


def si_snr(target, estimate, EPS=1e-8):
quancs marked this conversation as resolved.
Show resolved Hide resolved
""" scale-invariant signal-to-noise ratio (SI-SNR)

Args:
target (Tensor): shape [..., time]
estimate (Tensor): shape [..., time]
EPS (float, optional): a small value for numerical stability. Defaults to 1e-8.

Raises:
TypeError: if target and estimate have a different shape

Returns:
Tensor: si-snr value has a shape of [...]

Example:
>>> from torchmetrics.functional.audio import si_snr
>>> target = torch.tensor([3.0, -0.5, 2.0, 7.0])
>>> estimate = torch.tensor([2.5, 0.0, 2.0, 8.0])
>>> si_snr_val = si_snr(target,estimate)
>>> si_snr_val
tensor(15.0918)

References:
[1] Y. Luo and N. Mesgarani, "TaSNet: Time-Domain Audio Separation Network for Real-Time, Single-Channel Speech
Separation," 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018, pp.
696-700, doi: 10.1109/ICASSP.2018.8462116.
"""

return si_sdr(target=target, estimate=estimate, zero_mean=True, EPS=EPS)
58 changes: 58 additions & 0 deletions torchmetrics/functional/audio/snr.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Copyright The PyTorch Lightning team.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import torch
from torch import Tensor


def snr(target, estimate, zero_mean=False, EPS=1e-8) -> Tensor:
quancs marked this conversation as resolved.
Show resolved Hide resolved
""" signal-to-noise ratio (SNR)

Args:
target (Tensor): shape [..., time]
estimate (Tensor): shape [..., time]
zero_mean (Bool): if to zero mean target and estimate or not
EPS (float, optional): a small value for numerical stability. Defaults to 1e-8.

Raises:
TypeError: if target and estimate have a different shape

Returns:
Tensor: snr value has a shape of [...]

Example:
>>> from torchmetrics.functional.audio import snr
>>> target = torch.tensor([3.0, -0.5, 2.0, 7.0])
>>> estimate = torch.tensor([2.5, 0.0, 2.0, 8.0])
>>> snr_val = snr(target,estimate)
>>> snr_val
tensor(16.1805)

References:
[1] Le Roux, Jonathan, et al. "SDR half-baked or well done." IEEE International Conference on Acoustics, Speech
and Signal Processing (ICASSP) 2019.
"""

if target.shape != estimate.shape:
raise TypeError(f"Inputs must be of shape [..., time], got {target.shape} and {estimate.shape} instead")

if zero_mean:
target = target - torch.mean(target, dim=-1, keepdim=True)
estimate = estimate - torch.mean(estimate, dim=-1, keepdim=True)

noise = target - estimate

snr_value = torch.sum(target**2, dim=-1) / (torch.sum(noise**2, dim=-1) + EPS)
snr_value = 10 * torch.log10(snr_value + EPS)

return snr_value