[Refactor] Remove additional condition

[tchaton]

Before submitting

Remove addition from my previous PR: #302

• Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure to update the docs?
• Did you write any new necessary tests?

What does this PR do?

Fixes # (issue).

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

[codecov]

Codecov Report

Merging #311 (57d4718) into master (c8f81b1) will increase coverage by 0.05%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #311      +/-   ##
==========================================
+ Coverage   96.46%   96.52%   +0.05%     
==========================================
  Files         106      106              
  Lines        3450     3450              
==========================================
+ Hits         3328     3330       +2     
+ Misses        122      120       -2     

Flag	Coverage Δ
Linux	75.84% <0.00%> (ø)
Windows	75.84% <0.00%> (ø)
cpu	96.46% <100.00%> (ø)
gpu	96.42% <100.00%> (?)
macOS	96.46% <100.00%> (ø)
pytest	96.52% <100.00%> (+0.05%)	⬆️
python3.6	95.55% <100.00%> (ø)
python3.8	96.46% <100.00%> (ø)
python3.9	96.37% <100.00%> (ø)
torch1.3.1	95.55% <100.00%> (ø)
torch1.4.0	95.65% <100.00%> (ø)
torch1.9.0	96.37% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
torchmetrics/metric.py	95.51% <100.00%> (ø)
torchmetrics/functional/regression/spearman.py	97.77% <0.00%> (+4.44%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c8f81b1...57d4718. Read the comment docs.