Releases: adeptex/whispers
2.3.0
Whispers 2.3.0 release notes
-
New Feature: 💫 Static Code Analysis 💫 is now supported!
- The present release complements classic Whispers' structured text analysis with Semgrep's AST generator for common programming languages like Python, PHP, Java/Scala/Kotlin, JavaScript/TypeScript, Go, etc etc.
- New argument
--ast
for enabling this feature via the CLI (it is disabled by default) - New setting
ast: true
for enabling this feature via a custom config file (set toast: false
by default) - Replaced
astroid
Python AST generator withsemgrep
-
Detection rule improvements
- Known API keys
- AWS account ID
- Passwords
- Creditcards
-
Drop end-of-life Python support
- Versions 3.6 and 3.7 are no longer supported. Oldest supported version is Python 3.8.
- Last release that supports Python 3.6 and 3.7 is Whispers 2.2.1
-
Dependency tracking improvements
- New
requirements-dev.txt
file allows Dependabot updates for dev dependencies - Modified
setup.py
to read fromrequirements.txt
andrequirements-dev.txt
- Updated build CI to use Python 3.12.3
- New
-
Debugging and troubleshooting
- Modified
config.yml
to exclude known false positives - Fixed
Dockerfile
to work withdocker build -t whispers .
or the samemake image
- New arg
--dump
for generating an AST of a file:whispers --dump src/example.ts
- Modified
💫 New Feature: Static Code Analysis 💫
With the release of Whispers 2.3, it is now possible to accurately apply Whispers' secret detection techniques for structured text to static code. Before this release, Whispers only supported structured text formats, such as JSON or XML. Semgrep is an open source SAST tool, which has a built-in feature for generating Abstract Structure Trees (ASTs) for many common programming languages. Generating an AST for static code yields an accurate structured text representation, which can be checked for secrets with Whispers' rules and plugins. As such, generating ASTs requires an additional "format conversion" step, which naturally affects runtime speed. When AST is enabled it will take longer to scan the same scope if any source code files are present. The increased amount of runtime time would be however long it takes to run the following command on all static code files in scope:
semgrep scan --metrics=off --quiet --dump-ast --json --lang $LANG $SRCFILE
Consider the following benchmarks:
time whispers -F " " tests/fixtures
# 313 detected secrets
# 0,51s user 0,03s system 99% cpu 0,540 total
# 0,60s user 0,04s system 99% cpu 0,642 total
time whispers -a -F " " tests/fixtures
# 421 detected secrets
# 2,20s user 0,40s system 100% cpu 2,589 total
# 2,32s user 0,46s system 100% cpu 2,772 total
AST conversion is disabled by default - semgrep
will not execute at all unless explicitly enabled. Custom config files that are missing ast: false
or ast: true
will default to false
.
ast: true # enable AST in config.yml
whispers --ast target/dir/or/file # enable AST in CLI
❌ Breaking changes ❌
❌ Replaced astroid
with semgrep
❌
Before Whispers 2.3, only Python AST scanning was natively supported by astroid
, and integrated via plugins/python.py
. With the release of Whispers 2.3, this functionality is superseded by semgrep
, and integrated via plugins/semgrep.py
. As a base line, the new semgrep
plugin detects the same findings as the astroid
plugin, but supports more programming languages.
Unfortunately semgrep
has telemetry enabled by default, but can be turned off via --metrics=off
. In any case, semgrep
will not execute unless explicitly enabled via args or config.
semgrep
does not support Windows OS natively, and can only be installed through WSL. As such, compiled Whispers PE32+ executable comes without Static Code Analysis support. Installing Whispers on Windows via WSL with pip3 install whispers
does have Static Code Analysis support.
Changelog
Date | Version | Release notes |
---|---|---|
2021-12-07 | 2.0.0 | https://github.com/adeptex/whispers/releases/tag/2.0.0 |
2022-07-29 | 2.1.0 | https://github.com/adeptex/whispers/releases/tag/2.1.0 |
2023-10-23 | 2.2.0 | https://github.com/adeptex/whispers/releases/tag/2.2.0 |
2024-06-16 | 2.3.0 | https://github.com/adeptex/whispers/releases/tag/2.3.0 |
What's Changed
- Bump lxml from 5.2.0 to 5.2.1 by @dependabot in #130
- 2.2.2 by @adeptex in #132
- Bump typing-extensions from 4.10.0 to 4.11.0 by @dependabot in #131
- 2.3.0 by @adeptex in #139
- Bump jellyfish from 1.0.3 to 1.0.4 by @dependabot in #136
- Bump lxml from 5.2.1 to 5.2.2 by @dependabot in #134
Full Changelog: 2.2.1...2.3.0
2.2.1
What's Changed
- Bump wrapt from 1.15.0 to 1.16.0 by @dependabot in #114
- Bump jellyfish from 1.0.1 to 1.0.3 by @dependabot in #115
- Bump typing-extensions from 4.8.0 to 4.9.0 by @dependabot in #116
- Bump astroid from 2.15.6 to 3.0.2 by @dependabot in #117
- Bump lazy-object-proxy from 1.9.0 to 1.10.0 by @dependabot in #118
- Bump lxml from 4.9.3 to 4.9.4 by @dependabot in #119
- Bump lxml from 4.9.4 to 5.0.0 by @dependabot in #120
- Bump lxml from 5.0.0 to 5.1.0 by @dependabot in #121
- Bump beautifulsoup4 from 4.12.2 to 4.12.3 by @dependabot in #122
- Bump astroid from 3.0.2 to 3.0.3 by @dependabot in #124
- Bump astroid from 3.0.3 to 3.1.0 by @dependabot in #126
- Bump typing-extensions from 4.9.0 to 4.10.0 by @dependabot in #125
- 2.2.1 by @adeptex in #123
- Bump lxml from 5.1.0 to 5.1.1 by @dependabot in #128
- Bump lxml from 5.1.1 to 5.2.0 by @dependabot in #129
Full Changelog: 2.2.0...2.2.1
2.2.0
Whispers 2.2.0 release notes
- License change
- Compatibility improvements
- Standardize severity levels
- Minor code refactor
- Detection improvements
- Add XML cases
- Add
apikey-maybe
rule
💫 Licensing changes (again) 💫
Version 2.1 was released under GNU General Public License v3.0, which is intended to guarantee your freedom to share and change all versions of a program--to make sure it remains free software for all its users.
Version 2.2 is released under BSD 3-Clause License, which is a permissive license that prohibits others from using the name of the copyright holder or its contributors to promote derived products without written consent.
This change removes source code disclosure requirement 🕵️
❌ Breaking changes ❌
❌ Severity levels ❌
Severity level names have been adapted to a more common format. For example, BLOCKER
is replaced by Critical
and so on. The full list is as follows:
Version 2.1 (before) | Version 2.2 (now) |
---|---|
BLOCKER |
Critical |
CRITICAL |
High |
MAJOR |
Medium |
MINOR |
Low |
INFO |
Info |
Instead of
whispers -s BLOCKER dir/or/file
use
whispers -s Critical dir/or/file
See README for details and examples.
Changelog
What's Changed
- Bump astroid from 2.12.9 to 2.12.10 by @dependabot in #62
- Bump typing-extensions from 4.3.0 to 4.4.0 by @dependabot in #63
- Bump astroid from 2.12.10 to 2.12.11 by @dependabot in #64
- Bump astroid from 2.12.11 to 2.12.12 by @dependabot in #65
- Bump astroid from 2.12.12 to 2.12.13 by @dependabot in #68
- Bump lxml from 4.9.1 to 4.9.2 by @dependabot in #69
- Bump astroid from 2.12.13 to 2.13.2 by @dependabot in #71
- Bump lazy-object-proxy from 1.7.1 to 1.9.0 by @dependabot in #70
- Bump astroid from 2.13.2 to 2.13.3 by @dependabot in #72
- Bump astroid from 2.13.3 to 2.14.1 by @dependabot in #73
- Bump beautifulsoup4 from 4.11.1 to 4.11.2 by @dependabot in #74
- Bump astroid from 2.14.1 to 2.14.2 by @dependabot in #75
- Bump typing-extensions from 4.4.0 to 4.5.0 by @dependabot in #77
- Bump soupsieve from 2.3.2.post1 to 2.4 by @dependabot in #76
- Bump wrapt from 1.14.1 to 1.15.0 by @dependabot in #79
- Bump astroid from 2.14.2 to 2.15.0 by @dependabot in #80
- Bump beautifulsoup4 from 4.11.2 to 4.12.0 by @dependabot in #81
- Bump astroid from 2.15.0 to 2.15.1 by @dependabot in #82
- Bump jellyfish from 0.9.0 to 0.11.0 by @dependabot in #83
- Bump astroid from 2.15.1 to 2.15.2 by @dependabot in #84
- Bump jellyfish from 0.11.0 to 0.11.2 by @dependabot in #85
- Bump beautifulsoup4 from 4.12.0 to 4.12.1 by @dependabot in #86
- Bump beautifulsoup4 from 4.12.1 to 4.12.2 by @dependabot in #87
- Bump astroid from 2.15.2 to 2.15.3 by @dependabot in #88
- Bump astroid from 2.15.3 to 2.15.4 by @dependabot in #90
- Bump soupsieve from 2.4 to 2.4.1 by @dependabot in #89
- Bump astroid from 2.15.4 to 2.15.5 by @dependabot in #91
- Bump typing-extensions from 4.5.0 to 4.6.1 by @dependabot in #94
- Bump typing-extensions from 4.6.1 to 4.6.2 by @dependabot in #95
- Bump typing-extensions from 4.6.2 to 4.6.3 by @dependabot in #96
- Bump jellyfish from 0.11.2 to 1.0.0 by @dependabot in #98
- Bump typing-extensions from 4.6.3 to 4.7.1 by @dependabot in #101
- Bump lxml from 4.9.2 to 4.9.3 by @dependabot in #102
- 2.2.0 by @adeptex in #106
- UTF8 Locale by @adeptex in #112
Full Changelog: 2.1.5...2.2.0
2.1.5
2.1.4
What's Changed
- Added nginx.conf parser and detection
- Improved Shell variables detection
- Updated Github Workflows to compile binaries as UTF-8
- Bump astroid from 2.12.5 to 2.12.8 by @dependabot in #56
- Bump astroid from 2.12.8 to 2.12.9 by @dependabot in #57
Full Changelog: 2.1.3...2.1.4
2.1.3
What's Changed
- Added detection for default values in Shell variables
- Added detection for known Grafana keys
- Added SHA256 checksums for compiled binaries
- Bump astroid from 2.12.2 to 2.12.4 by @dependabot in #50
- Bump astroid from 2.12.4 to 2.12.5 by @dependabot in #52
Full Changelog: 2.1.2...2.1.3
2.1.2
In order to compile the package with pyInstaller some minor adjustments were required. There are no functional changes in this release.
It is now possible to download and use Whispers as a binary for Linux, OSX and Windows. See releases page.
Full Changelog: 2.1.1...2.1.2
2.1.1
2.1.0
Whispers 2.1.0 release notes
❌ Breaking changes ❌
❌ Arguments ❌
Several arguments have been modified and/or adapted to improve usability.
-
Human readable output is shown in logs (2.1),
-H
and--human
(2.0) are removed. -
Version can be shown with
--version
(2.1),-v
(2.0) is removed. -
Extended help can be shown with
--info
(2.1),-i
(2.0) is removed. -
Debug mode can be enabled with
--debug
(2.1),-d
(2.0) is removed. -
Logs can be redirected to a file with
--log log.txt
(2.1), constant/tmp/whispers.log
(2.0) is removed. -
Configuration template can be created with
--init
(2.1),--print_config
(2.0) is removed.
❌ Logging ❌
Version 2.0: Opt-in logging for tracing execution flow, useful only for debugging. Results printed to stdout
using print()
as a JSON dict, one result per line. Enabling logging required adding the --log
argument.
Version 2.1: Logging is used to alert identified secrets during execution with WARNING
level. Results are written to stdout
as a JSON list at the end. This improves results parseability as a JSON list, while maintaining live results display that was previously achieved by printing secrets as JSON one per line.
✅ New features ✅
✅ Results as JSON list ✅
To improve integration and downstream processing, Whispers now outputs results as a JSON list of dictionaries with all detected secrets together (2.1), instead of one JSON dictionary per line (2.0). This list is directly loadable and parsable as JSON.
2.0.6
This release addresses two reported bugs, as well as dependency updates.
- Install
dataclasses
only for Python 3.6 to avoid dependency conflicts with higher versions (Fixes Skyscanner/whispers#107) - Improve hardcoded secret detection for values starting with
$
(Fixes Skyscanner/whispers#108)
What's Changed
- Bump typing-extensions from 4.0.1 to 4.1.1 by @dependabot in #21
- Bump lxml from 4.7.1 to 4.8.0 by @dependabot in #22
- Update issue templates by @adeptex in #23
- Bump astroid from 2.9.3 to 2.10.0 by @dependabot in #24
- Bump wrapt from 1.13.3 to 1.14.0 by @dependabot in #25
- Bump astroid from 2.10.0 to 2.11.0 by @dependabot in #26
- Bump astroid from 2.11.0 to 2.11.1 by @dependabot in #27
- Bump astroid from 2.11.1 to 2.11.2 by @dependabot in #28
- Bump soupsieve from 2.3.1 to 2.3.2 by @dependabot in #29
- Bump beautifulsoup4 from 4.10.0 to 4.11.0 by @dependabot in #30
- Bump beautifulsoup4 from 4.11.0 to 4.11.1 by @dependabot in #31
- Bump soupsieve from 2.3.2 to 2.3.2.post1 by @dependabot in #32
- Bump typing-extensions from 4.1.1 to 4.2.0 by @dependabot in #33
- Bump astroid from 2.11.2 to 2.11.3 by @dependabot in #34
- Bump wrapt from 1.14.0 to 1.14.1 by @dependabot in #35
- 2.0.6 by @adeptex in #36
Full Changelog: 2.0.5...2.0.6