Tasks
Tasks allow you to perform CRUD operations against results. This section will describe the different types of tasks provided, task views, task execution and grouping, as well as how to use saved restul filters and saved event filters to provide a more limited scope of results for your tasks.
Scumblr ships with three task types:
- Sync
- Security
- Generic
- Maintenance
- Search (legacy)
Sync tasks are used to create results via a variety of sources. Scumblr includes the following Sync tasks:
The Github sync task allows you to specify a Github organization (as well as its members) or a specific user and sync all of their repositories in to Scumblr. If you have supplied an API key you can also use this to load private repositories as well.
The Open Grok sync task allows you to sync repositories indexed in your Open Grok application.
Security tasks are used to perform security checks or metadata collection against Results.
The curl security task is the swiss army knife of Scumblr. The curl task allows you to specify a curl command and grep the http response for certain strings, regular expressions, or http status codes.
The github security task leverages the Github API to allow you to search for specific strings in files or paths. This can be useful for monitoring for secrets, aws keys, bad XML parser settings, etc.
The Brakeman security task performs static code analysis against Ruby on Rails projects.
The Bandit security task performs static code analysis against Python projects.
Generic tasks are used to perform non-specific tasks related to results.
The Sketchy task is used to generate screenshots for Results. This task calls the Sketchy service (a separate application from Scumblr) and Sketchy asynchronously calls back with a screenshot, text scrape, and dom capture of the Result. A status code is also captured which can be useful for Result Filters or event filters where you'd like to monitor when Results status code changes.
Maintenance Tasks are used to perform maintenance operations against Results, Events, or other models within Scumblr as needed.
The event cleaner will allow you to remove old events. You can setup the number of days to keep events for.
This task ensure every result as a vulnerablity object which can allow you to manually create vulnerabilities in results. This is useful for Results which have no vulnerabilities identified by Security Tasks.
Search Tasks are used to sync results from sources such as Twitter, Facebook, and Google.
The following Search tasks are provided:
- iTunes Store
- Certificate Transparency
- Ebay
- Google Play
- RSS Feeds (useful for full disclosure searches)
- YouTube
You can navigate to /tasks view and click the "New Task" button.
Tasks have a number of required and optional fields. The most important ones being the Task Name and Task Group.
Task Groups provide a way to run asynchronous batches of Tasks. Tasks in Group 1 must completed before tasks in Group 2 will execute. This can be useful if you have tasks that are dependent on each other. As an example you could imagine the following:
Group 1
Github - Find Rails Projects
Group 2
Brakeman - Rails Static Code Analyzer
When creating tasks, you may have results you don't want to run Security tasks against. You can use Result Filters to seed your Task which will speed up Task execution time and ensure only results you care about are being checked. Similarly event filters can be used to seed Tasks.
You can list tasks by navigating to the /tasks route of your Scumblr instance. You can click the Arrow dropdowns to expand out more details on the Task.
You can also click on the Task Name on this view to get more information.
This contains details on if the task had any error or warning messages when executing. In addition, it will inform you which results it created or updated.
This contains the last status message, how long the task took to complete, and when it last executed.
A simple overview on how you have configured your task.
Events will show you associated Events with the task.
You can execute tasks by navigating to the /tasks route of your Scumblr instance. You can select the checkboxes on tasks you want to select and click the Run action or if you'd like to execute all tasks you can click Run All Enabled Tasks.
In order to allow Scumblr to automatically run tasks and send email notifications, you may want to setup cron jobs using the appropriate rake tasks.
This task will run all the enabled tasks in groups as they are configured in Scumblr and then send out email notifications:
rake run_tasks_and_email_updates
There are also two rake tasks available to run tasks and send notifications independently:
rake run_tasks # run all tasks
rake send_email_updates # send notifications