Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to check issues against Virus Total #945

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/handle-comments.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ jobs:
needs: comments
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VIRUS_TOTAL_API_KEY: ${{ secrets.VIRUS_TOTAL_API_KEY }}
COMMENT_BODY: ${{ github.event.comment.body }}
# Becasue comment steps take way shorter time than installing
# packages, we can run these in parallel
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/new-issue.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ jobs:
runs-on: windows-latest
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VIRUS_TOTAL_API_KEY: ${{ secrets.VIRUS_TOTAL_API_KEY }}

steps:
- uses: actions/[email protected]
Expand Down
72 changes: 72 additions & 0 deletions scripts/private/api/virus-total/Get-VirusTotalResults.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
function Get-JsonResults {
param(
$inputObject
)

try {
# we first try normal serialization
return $inputObject | ConvertFrom-Json
}
catch {}

try {
# next we try to serialize as hash table
return $inputObject | ConvertFrom-Json -AsHashtable
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-AsHashtable may not be supported in PS5. We're using windows-latest in our tasks, but I think the way we're running it may take us to PowerShell rather than Pwsh. May be wrong! I don't know what we're targetting here, just noting as we normally dive into backwards compatibility.

That may also be fine, as this is being used post-IRM, which (I thought) should convert from JSON anyway, and this may just fall down to returning the object as-is. Looking at the API, though, it's... hm, fun.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-AsHashtable may not be supported in PS5.

You are right, it is not supported in PS5. We are however explicitly telling GitHub action to use Pwsh, so that is fine. Backwards compatibility is not a concern as these files aren't supposed to be ran on a users computer.

Also to mention, from what I remember when this was added the entire thing would fall over if -AsHashtable wasn't used.

}
catch {}

# at the end we just return the object as is
$inputObject
}

function Get-VirusTotalResults {
param(
[Parameter(Mandatory = $true)]
[string]$filePath
)

if (!(Test-Path Env:\VIRUS_TOTAL_API_KEY)) {
return @{
Status = "NoApiKey"
Flagged = 0
TotalCount = 0
Url = $null
}
}

Write-Host ([StatusCheckMessages]::virusTotalCheck)
$checksum = (Get-FileHash $filePath -Algorithm SHA256 | % Hash).ToLowerInvariant()
$apiUrl = "https://www.virustotal.com/api/v3/files/{0}" -f $checksum
$headers = @{
"x-apikey" = $env:VIRUS_TOTAL_API_KEY
}

try {
$response = Invoke-RestMethod -UseBasicParsing -Uri $apiUrl -Method Get -Headers $headers
$response = Get-JsonResults -inputObject $response

$stats = $response.data.attributes.last_analysis_stats

Write-Host ([StatusMessages]::virusTotalResultsAvailable)

[int]$antiVirusCount = $stats.malicious + $stats.suspecious + $stats.harmless + $stats.undetected + $stats.timeout + $stats.failure + $stats.'confirmed-timout'

@{
Status = "Found"
Flagged = ($stats.malicious + $stats.suspecious)
TotalCount = $antiVirusCount
Url = "https://www.virustotal.com/gui/file/{0}" -f $checksum
}
}
catch {
# We will assume this means that no virus results are available.
# VirusTotal seems to return 404 when none are uploaded
Write-Host ([StatusMessages]::noVirusTotalStatusAvailable)
@{
Status = "NotFound"
Flagged = 0
TotalCount = 0
Url = "https://www.virustotal.com/gui/file/{0}" -f $checksum
}
}
}
8 changes: 8 additions & 0 deletions scripts/private/messages.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ class StatusCheckMessages {
static [string]$fileDownloadCheck = "Downloading file from direct download URL for testing.";
static [string]$fileDownloadedTest = "Testing downloaded file";
static [string]$issueUsesRFPOrRFMTitle = "Checking if user have prefixed title withe RFP or RFM";
static [string]$virusTotalCheck = "Checking for existing VirusTotal results";
}

class StatusMessages {
Expand All @@ -60,6 +61,8 @@ class StatusMessages {
static [string]$userNotMarkedRFPTitleCorrectly = "User have not marked that he/she have prefixed the issue title with RFP";
static [string]$userNotProvidedPackageSourceUrl = "User seems to not have provided any URL to the package source (or one do not exist).";
static [string]$userNotProvidedPackageUrl = "User seems to not have provided any URL to the package";
static [string]$noVirusTotalStatusAvailable = "No VirusTotal results are available";
static [string]$virusTotalResultsAvailable = "Found VirusTotal check results"
}

class StatusLabels {
Expand Down Expand Up @@ -129,6 +132,11 @@ class ValidationMessages {
static [string]$maintainerContactedDateMissingError = "We could not detect when you contacted the maintainer. Please add this information to the *'Date the maintainer was contacted:'* part of the template in the format of ``year-month-day`` (example with current date ``$(Get-Date -Format 'yyy-MM-dd')``.";
static [string]$maintainerContactedMethodMissingError = "We could not detect how you contacted the maintainer. Please add this information to the `'How the maintainer was contacted'* part of the template.";
static [string]$triageProcessNotFollowedError = "We could not detect that you have completed the Package Triage Process for this request. Please head over to the [Package Triage Process documentation](https://docs.chocolatey.org/en-us/community-repository/users/package-triage-process#the-triage-process) and come back to this request when you have completed the process.";

# VirusTotal messages, only informational messages
static [string]$noVirusTotalResults = "There are no VirusTotal results available for this software. Please upload the binary file to [VirusTotal](https://www.virustotal.com/gui/home/upload).";
static [string]$virusTotalResultsCount = "{0}/{1} anti-virus softwares flagged this software on [VirusTotal]({2})";
static [string]$virusTotalResultsNone = "No anti-virus softwares have flagged this software on [VirusTotal]({0})";
}

class WarningMessages {
Expand Down
25 changes: 21 additions & 4 deletions scripts/private/validations/Get-NewPackageValidationResult.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -73,13 +73,15 @@ function Get-NewPackageValidationResult() {
Add-ValidationMessage -validationData $validationData -message ([ValidationMessages]::githubSearchNotMarkedError -f $validationData.repository, ($validationData.packageName -replace ' ', '%20')) -type ([MessageType]::Error)
}

$fileOutput = "$([System.IO.Path]::GetTempPath())software.tmp"

try {
if ($downloadUrl) {
Write-Host ([StatusCheckMessages]::fileDownloadCheck)
Get-RemoteFile -url $downloadUrl -filePath "$env:TEMP/software.tmp"
Get-RemoteFile -url $downloadUrl -filePath $fileOutput

Write-Host ([StatusCheckMessages]::fileDownloadCheck)
$result = Test-ValidFile "$env:TEMP/software.tmp"
$result = Test-ValidFile $fileOutput

if ($result -eq "missing") {
Write-ErrorMessage ([ErrorMessages]::toolingMissing)
Expand All @@ -101,8 +103,6 @@ function Get-NewPackageValidationResult() {
$msg += "```````n</details>"
Add-ValidationMessage -validationData $validationData -message $msg -type $msgType
}

Remove-Item "$env:TEMP/software.tmp" -ea 0
}
else {
Write-WarningMessage ([WarningMessages]::noDownloadUrlFound)
Expand All @@ -114,5 +114,22 @@ function Get-NewPackageValidationResult() {
$validationData.newLabels += [StatusLabels]::upstreamBlocked
}

if (Test-Path $fileOutput) {
$results = Get-VirusTotalResults -filePath $fileOutput

if ($results.Status -eq "NoApiKey" -or $results.Status -eq "NotFound") {
Write-Host ([StatusMessages]::noVirusTotalStatusAvailable)
Add-ValidationMessage -validationData $validationData -message ([validationMessages]::noVirusTotalResults) -type ([MessageType]::Info)
}
elseif ($results.Flagged -gt 0) {
Add-ValidationMessage -validationData $validationData -message ([ValidationMessages]::virusTotalResultsCount -f $results.Flagged, $results.TotalCount, $results.Url) -type ([MessageType]::Info)
}
else {
Add-ValidationMessage -validationData $validationData -message ([ValidationMessages]::virusTotalResultsNone -f $results.Url) -type ([MessageType]::Info)
}

Remove-Item $fileOutput -ea 0
}

return $true
}
21 changes: 19 additions & 2 deletions scripts/public/Test-NewIssue.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -133,18 +133,35 @@ function Test-NewIssue {
$warnings = $validationData.messages | Where-Object type -EQ ([MessageType]::Warning)
if ($warnings) {
$commentBody += "`n`n" + [ValidationMessages]::noticesHeader
$detailsComment = ""
$warnings | ForEach-Object {
$commentBody += "`n- $($_.message)"
# When using the <details> section, special handling is needed
if ($_.message -match "\<details\>") {
$detailsComment = $_.message
}
else {
$commentBody += "`n- $($_.message)"
}
}
$commentBody += "`n`n$detailsComment"
}
}

$infos = $validationData.messages | Where-Object type -EQ ([MessageType]::Info)
if ($infos) {
$commentBody += "`n`n" + [ValidationMessages]::maintainersHeader
$detailsComment = ""
$infos | ForEach-Object {
$commentBody += "`n- $($_.message)"
# When using the <details> section, special handling is needed
if ($_.message -match "\<details\>") {
$detailsComment = $_.message
}
else {
$commentBody += "`n- $($_.message)"
}
}

$commentBody += "`n`n$detailsComment"
}

$commentBody += [ValidationMessages]::commentBodyFooter
Expand Down