Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

normalizing the response headers causes confusion #922

Closed
ErikOwen opened this issue Jun 6, 2024 · 0 comments · Fixed by #924
Closed

normalizing the response headers causes confusion #922

ErikOwen opened this issue Jun 6, 2024 · 0 comments · Fixed by #924
Assignees
Labels
Status: Completed Nothing further to be done with this issue. Awaiting to be closed. Type: Bug Inconsistencies or issues which will cause an issue or problem for users or implementors.
Milestone

Comments

@ErikOwen
Copy link
Contributor

ErikOwen commented Jun 6, 2024

katana version: v1.1.0

Current Behavior:

When crawling a site with the -jsonl flag, the response headers are lowercased, and - characters are converted to _. This is misleading, because technically both Some-Header and Some_Header are both valid headers. The normalization of the response header prevents the end user from being able to identify what the actual response header was.

But lowercasing the response header makes sense, because the headers are case-insensitive.

Expected Behavior:

Response headers in Kebab-Case are preserved, so Content-Type is reflected as content-type in the JSON output.

Steps To Reproduce:

Run this command:

katana -u https://projectdiscovery.io -d 1 --jsonl --crawl-scope https://projectdiscovery.io --omit-raw --omit-body

See that the report_to HTTP header doesn't match the actual response header from the server (report-to).

Anything else:

Offending code: https://github.com/projectdiscovery/katana/blob/main/pkg/navigation/response.go#L23

@ErikOwen ErikOwen added the Type: Bug Inconsistencies or issues which will cause an issue or problem for users or implementors. label Jun 6, 2024
@dogancanbakir dogancanbakir self-assigned this Jun 7, 2024
@dogancanbakir dogancanbakir linked a pull request Jun 10, 2024 that will close this issue
@Mzack9999 Mzack9999 added the Status: Completed Nothing further to be done with this issue. Awaiting to be closed. label Jun 11, 2024
@dogancanbakir dogancanbakir added this to the katana v1.1.1 milestone Oct 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: Completed Nothing further to be done with this issue. Awaiting to be closed. Type: Bug Inconsistencies or issues which will cause an issue or problem for users or implementors.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants