Growing together!

[rfc-st]

What is 'humble' missing?; being reasonable :):

• What HTTP header checks do you miss?.
• What totals do you think should be reported?.
• What results export format do you need and why?.

Let's use this issue to talk about the shortcomings of 'humble' (without preventing specific issues to be created for specific topics), ideas to make it more interesting and accessible for everyone. With total freedom and regardless of your technical knowledge,

I have the healthy habit of thanking all the contributions that result in an improvement of this tool: https://github.com/rfc-st/humble/?tab=readme-ov-file#acknowledgements .

Thanks for your time!.

[jdelamo3]

it would be a good idea to register a Humble formula in HomeBrew, which is one of the most popular package managers for macOSX and Linux, and which also greatly simplifies the compilation and installation of libraries and dependencies.
https://brew.sh

[rfc-st]

Hi, @jdelamo3

I have “zero” experience in the macOS and HomeBrew worlds :); but I promise to take a look and see what the process is like.

Regards,

[rfc-st]

Hi, @jdelamo3

Reviewing the necessary steps to make your suggestion possible ... we will keep in touch.

Best regards,

[cr4zyfish]

check for security headers via meta tags.
Deprecated HTTP Response Headers/Protocols and Insecure Values ====> separate these results

in this report for Deprecated HTTP Response Headers/Protocols and Insecure Values the value from server response is Deny so only the advice should be printed.

 X-Frame-Options (Incorrect Values)
 The only values allowed for this header are 'DENY' or 'SAMEORIGIN'.
 Advice: Replace this header with the CSP 'frame-ancestors' directive.
 Ref: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Frame-Options

[rfc-st]

Hello, @cr4zyfish

Good suggestions, thanks!. I will take them into account.

• Regarding the first one, it seems (https://developer.mozilla.org/en-US/docs/Web/HTML/Element/meta#http-equiv) that the definition of attributes associated with HTTP directives would really be limited to three of them, at the security level: 'content-security-policy', 'content-type' and 'x-ua-compatible' ... at the HTTP response header level 'humble' has checks for all three ... I have to evaluate whether it is worth retrieving and parsing the response body for those meta directives with the added parsing time that might entail.

• Regarding separating the results of “Deprecated HTTP Response Headers/Protocols and Insecure Values” require modifying quite a lot of code (global/per URL statistics, grade of analysis, export to CSV/JSON, etc ...) but yes, it could be interesting to have specific sections for them.

• Regarding the value and messages of “X-Frame-Options”, checking the standards gives me the impression that the only valid values of this header are exactly 'DENY' or 'SAMEORIGIN' ('ALLOW-FROM' is deprecated): https://www.rfc-editor.org/rfc/rfc7034#section-2.1, https://html.spec.whatwg.org/multipage/document-lifecycle.html#the-x-frame-options-header and https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Frame-Options#syntax. Hence the need to indicate that any other value is not correct, and also indicate the suggestion to use a more current option, such as 'frame-ancestors' from CSP.

Regards,

[rfc-st]

Hi, @cr4zyfish

Check out this commit: aeabd4b

'humble' checks, thanks to your suggestion, the values of headers defined via meta tags. For now two of them; I will improve this functionality.

I've added you at https://github.com/rfc-st/humble/?tab=readme-ov-file#acknowledgements.

Thanks & regards!.

[cr4zyfish]

I scanned a target it has this header:
Cache-Control: no-cache, private

The result:

[3. Deprecated HTTP Response Headers/Protocols and Insecure Values]

 The following headers/protocols are deprecated or their values may be considered unsafe:


 Cache-Control (Recommended Values)
 Enable 'no-cache', 'no-store', and 'must-revalidate' if there are sensitive data.
 Ref: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control

why there is a warning here? is anything wrong with no-cache, private ?

this header reported in [1. Missing HTTP Security Headers] is not part of OWASP secure headers:

 NEL
 Enables web applications to declare a reporting policy to report errors.
 Ref: https://scotthelme.co.uk/network-error-logging-deep-dive/

is it a necessary header?

May be more explanation is needed for user why X-XSS-Protection: 1; mode=block is unsafe Value.
because not everybody know why 1 is unsafe.

 X-XSS-Protection (Unsafe Value)
 To mitigate XSS attacks, restrictively use the "Content-Security-Policy" header.
 Ref: https://auth0.com/blog/defending-against-xss-with-csp/ 

These headers are in my target:

Keep-Alive: timeout=5, max=100
Connection: Keep-Alive

but in reports says:

[3. Deprecated HTTP Response Headers/Protocols and Insecure Values]

 The following headers/protocols are deprecated or their values may be considered unsafe:
 
 Keep-Alive (Ignored Header)
 This header is ignored if the value of the "Connection" header is not 'keep-alive'.
 Ref: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Keep-Alive

The value of Connection is Keep-Alive but still this result shows up in scan.

[rfc-st]

Hi, @cr4zyfish

Regarding 'Cache-Control' the warning is more of a recommendation (hence the indication of 'Recommended Values'), to completely avoid caching resources that may be associated with sensitive information, if indeed there is any in the URL being analyzed.

Regarding NEL, 'humble' not only analyzes the set of HTTP response headers considered secure by OWASP, but is much strict and comprehensive, extending checks to more headers, including experimental ones: https://github.com/rfc-st/humble/?tab=readme-ov-file#checks-deprecated-headersprotocols-and-insecure-values & https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/NEL

Regarding 'X-XSS-Protection', indeed, I will include an additional reference indicating why '1; mode=block' is not recommended: in some cases XSS protection can create XSS vulnerabilities (https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-XSS-Protection) and also this header is deprecated in the three major web browsers.

(You can always use the -s parameter in 'humble', excluding the HTTP headers of your choice from the analysis).

And finally, regarding 'Keep-Alive', it seems the value for that header should be exactly 'keep-alive', according to https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Connection.

Thanks & regards!.

[rfc-st]

Hi, @cr4zyfish

Check out this commit: 07fcb54

Thanks to your suggestion, I have improved the message and the reference associated with checking for values other than '0' in the 'X-XSS-Protection' header.

Regards!

[rfc-st]

Hi, @cr4zyfish

Check out this commit: 7bf8632

You were absolutely right: now 'humble', when checking for missing HTTP headers (e.g. 'NEL'), shows whether the header is Experimental or not, based on https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers:

It's a start; I will think of ways to improve these messages.

Regards!

[rfc-st]

Hi, @cr4zyfish

Take a look at this.

I think 'humble' reflects now more clearly everything related to experimental HTTP headers :).

Regards!

[vincentcox]

I have a humble feature request. Is it possible to show which headers are present (like in "test passed")? Would be nice to have that in Json output too!

Thanks!

[rfc-st]

Hi, @vincentcox

Thanks for your suggestion: on which specific HTTP response headers would you want to see if they have passed or not? humble, currently, has 113 checks (focused on insecure values or those considered deprecated), 14 checks (focused on missing headers) and 1186 checks (focused on fingerprint headers).

Do you have an example, in text, of what you would like to see displayed?

Additionally, with the '-r' parameter humble shows the HTTP response headers enabled on the analyzed URL.

Regards!,

[vincentcox]

@rfc-st Thanks for the reply.

To show you what I mean, I can best show it by the tool shcheck, a similar tool like yours.

docker run -it --rm shcheck -j https://seqr-byte.be will give something like this:

It would be nice to see which security headers are present in this tool too. The -r flag you mentioned shows all the headers including caching headers, cookie headers, content-encoding which are not relevant in a security context.

It would be nice to have this too in this tool. I find this tool more reliable than shcheck, because this tool can properly follow redirections and if you check www.google.com this tool shows the proper results while the other one falls over Strict-Transport-Security.

I don't know how hard it is to implement, but if it's not much work it would be highly appreciated!

I think this is easiest done in the print_response_headers(). Something along the lines of:

def print_response_headers():
    # List of security-relevant headers to display
    security_headers = [
        'Strict-Transport-Security',
        'Content-Security-Policy',
        'X-Content-Type-Options',
        'X-Frame-Options',
        'X-XSS-Protection',
        'Permissions-Policy',
        'Referrer-Policy'
    ]

    # Dictionary to hold headers for JSON output
    json_headers = {}

    # Print a newline for formatting
    print(linesep.join([''] * 2))
    print_detail_r('[0headers]')
    
    pdf_style = STYLE[6] if args.output == 'pdf' else ""
    for key, value in sorted(headers.items()):
        if not args.p or (args.p and key in security_headers):
            json_headers[key] = value  # Collect headers for JSON output
            print(f" {pdf_style}{key}:", value) if args.output else \
                print(f" {STYLE[2]}{key}:", value)
    
    print('\n')  # Print a newline after the headers
    
    # If JSON output is specified, write headers to JSON format
    if args.output == 'json':
        with open('output.json', 'w') as json_file:
            dump(json_headers, json_file, indent=4)

In this pseudo code the present headers will be given when the flag "-p" is given. It will also output it if json output is used.

[rfc-st]

Hi, @vincentcox

Allow me to show you a couple of WIP screenshots (brief, detailed analysis and JSON export) :):

Let's see if I can finish everything today (I have to check the formatting when exporting to html, json, csv), etc.

I'll let you know!.

Regards,

[rfc-st]

Hi, @vincentcox

I do not see, at least today, how to make this functionality possible :).

I've been stuck for quite some time with the formatting of this new feature, especially when it comes to exporting it to HTML/PDF ... I hope to have progress in the next few days,

Regards,

[rfc-st]

Hi, @vincentcox,

Take a look at this commit 473cce4 ... I think you might be interested! :)

I have mentioned you in the acknowledgements ... great idea!

Regards.

[vincentcox]

Hello @rfc-st,

Amazing, exactly how I imagined! The Json output also provides the present header responses.

I made a small pull request (#24 ) because a syntax error is making the script (at least in docker) fail.

Thanks for the work and also thanks for the acknowledgement!

[rfc-st]

Hi @vincentcox,

Thanks for the PR!, accepted.

Regards,