Restructure action names + Improved status

Changed

• Change the structure of the CLI actions
  • Renamed the "show-status" action to "filters status"
  • Renamed the "update-filters" action to "filters update"
  • Renamed the "remove-filters" action to "filters remove"
  • Renamed the "list-spam-domains" action to "domains list"
  • Renamed the "find-spam-domains" action to "domains find"
• The spam-protection status calculation is now based on the number of spam domain names that are blocked by the existing filters - opposed to the number of filters which are up-to-date.
• Print status updates when removing the spam-filters from an account

Added

• Add a logo for ga-spam-control
• Add animations/screencasts that illustrate the available actions and usage scenarios for ga-spam-control

Manual referrer spam review

This release replaces the machine-learning based spam detection with a manual review process.

Unfortunately the machine-learning based approach for detecting new referrer spam domain names in the analytics data of Google Analytics accounts did not work reliably enough for websites with different usage patterns.

The find-spam-domains action will now download the last 90 days of analytics data from the specified Google Analytics account and list a referrer domain names which are currently NOT recognized as spam.

If you find a domain name which is spam, you uncomment the line. When you are finished with the review, you save the file and close the editor. All domain names which were uncommented are added to your personal referrer spam domain list and henceforce combined with the list of known referrer spam domain names that are provided by the community.

Added

• Add a roadmap to the README.md
• Add "Support for manual segment creation" to the roadmap
• Open the Google Analytics oAuth authorizatin URL in the users' default browser

Changed

• Replaced the automatic machine-learning based referrer spam domain detection with a manual review process

Fixed

• The staticSpamDomains provider did not respect HTTP status codes

Combine multiple referrer domain sources

Change the spam domain provider implementation to join multiple
static referrer spam domain provider lists with dynamically determined
referrer spam domain names using a machine learning web service.

Introduced shards for spam filters based on the first letter of the
referrer spam domain. This makes updates easier.

Other changes:

• Don't display unchanged domains during domain name updates
• Fixed the update command. Updates did not work before.
• Add logging for the update command
• Add package documentation
• Display status as a percentage instead of a text-based status
• Add documentation for public functions
• Fix template newline handling
• Fix the filesystem token store. Create the directory if it does not exist.
• Introduce a quite flag for domain updates
• Display domain updates as a list
• Include the total number of domains in the status overview
• Introduce a new list-spam-domains action
• Remove the global status ... it doesn't make much sense
• Allow to specify the number of days for the find-spam action
• Change the filter names prefix to "Referrer Spam Block Segment"

v0.3.0 Detect referrer spam with Azure Machine Learning

The detect-spam action analyses the last 30 days of analytics data for referrer spam and prints out all referrer domains which were identified as spam.

Usage:

ga-spam-control detect-spam <accountID>

Sample response:

Domainname                            Probability
free-video-tool.com                   0.97
keywords-monitoring-your-success.com  1.00
rank-checker.online                   1.00
uptime.com                            1.00

v0.2.0 Require accountID argument for sub-actions

In the previous version of ga-spam-control all operations were performed on all available Google Analytics accounts. Now you must specify the account ID on which you want to perform the operation.

I have introduced this change because you might not always want to update all accounts that you have access to. Or you simply want more control over when you update or remove spam-control filters.

update and remove actions

Add accountID argument to update and remove commands update and remove operations will only be performed on specified accounts.

ga-spam-control update <accountID>
ga-spam-control remove <accountID>

status action

• Add --quiet or -q flag to status command. The quiet flag instructs the status command to print the account status overview in a parsable format that simplifies using ga-spam-control with tools like awk. Example: Print account IDs of accounts that have the spam-control status of "not-installed"

  ga-spam-control status -q | grep "not-installed" | awk '{print $1}'

• Add optional accountID argument to status command. If an account ID is given to the status command ga-spam-control will only return the status for that given account. If no account ID is specified the status of all available accounts will be displayed. Usage:

  ga-spam-control status [<accountID>]

v0.1.0 First basic version of ga-spam-control cli

This basic version can:

• sign you into your Google Analytics account using oAuth
• print the current spam control status
• create or update the spam filters for all your accounts using the
  latest list of referrer spam domains
• remove all spam control filters from your accounts

The oAuth token is stored in ~/.ga-spam-control. Delete it to sign out.

Display current spam control status:

ga-spam-control status

Create or update your spam control filters:

ga-spam-control update

Remove all spam control filters:

ga-spam-control remove

Print the usage information:

ga-spam-control --help