Add JSON Output Feature

[Ice3man543]

Hi,
First of all congratulations for building such an awesome tool. I mean it's damn fast and does the work nicely.
I though it would be nice for gobuster to have a JSON output flag which would allow us to output the results in JSON format. This would allow for integration of gobuster in automated scripts.
The current output format is not well enough for integration in automated scripts.

Thanks

[mzpqnxow]

@Ice3man543 FYI I just started on this an hour ago.. right now it is only stdout for dir mode, but it shouldn't take more than another hour or two to have stdout+file output for dns and dir mode

I don't have the most time right now so if you want to work on it, check out https://github.com/mzpqnxow/gobuster/tree/json-logging

[mzpqnxow]

Output currently looks like this:

$ ./gobuster.git -J -w ../raft-medium-words-lowercase.txt -u http://localhost -X OPTIONS -e 
{"Expanded":"true","Mode":"dir","Output file":"","Status":"starting","Status codes":"200,204,301,302,307","Threads":10,"URL":"http://localhost/","level":"info","msg":"Scan initializing","time":"2018-03-04T02:16:40-05:00"}
{"URL":"http://localhost/javascript","level":"info","msg":"HIT","size":-1,"status":301,"time":"2018-03-04T02:16:40-05:00"}
{"URL":"http://localhost/out","level":"info","msg":"HIT","size":-1,"status":200,"time":"2018-03-04T02:16:40-05:00"}
{"URL":"http://localhost/.","level":"info","msg":"HIT","size":-1,"status":200,"time":"2018-03-04T02:16:40-05:00"}
{"URL":"http://localhost/httpd","level":"info","msg":"HIT","size":-1,"status":200,"time":"2018-03-04T02:16:40-05:00"}
{"URL":"http://localhost/test2","level":"info","msg":"HIT","size":-1,"status":200,"time":"2018-03-04T02:16:40-05:00"}
{"URL":"http://localhost/test1","level":"info","msg":"HIT","size":-1,"status":200,"time":"2018-03-04T02:16:40-05:00"}
{"URL":"http://localhost/redirector","level":"info","msg":"HIT","size":-1,"status":200,"time":"2018-03-04T02:16:40-05:00"}
...

[Ice3man543]

Hi @mzpqnxow, I will take a look at it.

[Ice3man543]

Hi @mzpqnxow, Actually the result should be in an array. This would make it easy to move through the response and parse it. The response should be something like this :

{
    "results":[
    {
         "URL":"http://localhost/admin",
         "level":"info",
         "size":-1,
         "status":200
    },
    {
         "URL":"http://localhost/config",
         "level":"info",
         "size":-1,
         "status":302
    }
    ]
}

Currently, I am somewhat busy so I can't help you out much.
Thanks.

[mzpqnxow]

@Ice3man543 this is meant to be consumed line by line at runtime (in real-time) by a third party tool. If the user wants full results at the end they should be using the flag to log results to disk.

[Ice3man543]

Yes @mzpqnxow, that's what I had originally intended by creating this issue. BTW, no problem. I was able to use -n flag to get just directories.

[duraki]

@OJ: Can we merge #119 PR by @itsbriany? nmap and alike tools (see, Kali Linux) implemented it so we can play between each report easily and multi-diagonally (through toolset of use).

[OJ]

At this stage, it's not ready for merging. I'll add commentary to the PR shortly.

[maheshkukreja1]

Hi @OJ, any updates on the JSON output functionality?

[jayrod]

no json??

[aress31]

@Ice3man543 is correct from a semantic standpoint, an array of results makes more sense.

Is there any update on this feature?

[ocervell]

I disagree, JSON lines output is better for live consuming (you can pipe with JQ) and is different for structured JSON output that usually goes in a file, so no need to structure it as an array ... Should we assign someone to continue the work by @Ice3man543 ?