Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

disable headless auto form filling #918

Merged
merged 5 commits into from
Jun 11, 2024
Merged

disable headless auto form filling #918

merged 5 commits into from
Jun 11, 2024

Conversation

dogancanbakir
Copy link
Member

Closes #876

```console
$ go run . -u http://testphp.vulnweb.com -hl -aff -j -o test.json
...

$ grep -r 'method":"POST"' test.json | grep -o '"raw":.*' | cut -c 1-50
"raw":"POST /search.php?test=query HTTP/1.1\r\nHos
"raw":"POST /search.php?test=query HTTP/1.1\r\nHos
"raw":"POST /userinfo.php HTTP/1.1\r\nHost: testph
"raw":"POST /userinfo.php HTTP/1.1\r\nHost: testph
"raw":"POST /secured/newuser.php HTTP/1.1\r\nHost:
"raw":"POST /secured/newuser.php HTTP/1.1\r\nHost:
"raw":"POST /guestbook.php HTTP/1.1\r\nHost: testp
"raw":"POST /guestbook.php HTTP/1.1\r\nHost: testp

@dogancanbakir dogancanbakir self-assigned this Jun 5, 2024
@ehsandeep ehsandeep linked an issue Jun 5, 2024 that may be closed by this pull request
Copy link
Member

@Mzack9999 Mzack9999 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like we should review the proposed solution based upon #876 (comment). Probably form filling should be reconsidered/disabled as a whole for headless navigation as the way we are doing it is totally wrong, as it should happen via headless actions within the page context (ex. click on submit button).

@dogancanbakir dogancanbakir changed the title use request to construct a new one make standard request when it's form Jun 6, 2024
@alban-stourbe-wmx
Copy link
Contributor

Hello,
Do you think it would be a good idea to create a function that automatically fills out forms using the rod package?

Is it possible that for each request launched by page.Navigate, we search through xpaths/selector for forms and content to fill them again with predefined data and click? If this were possible, the request would be hijacked and we wouldn't have to filter requests by tag, and the navigateRequest function would suffice on its own.

I don't realize how hard it would be to add such a feature

@dogancanbakir
Copy link
Member Author

Disabled headless auto form filling for now. I'll create a follow-up ticket to track this.

@dogancanbakir dogancanbakir changed the title make standard request when it's form disable headless auto form filling Jun 7, 2024
@Mzack9999 Mzack9999 merged commit 7808bcc into dev Jun 11, 2024
13 checks passed
@Mzack9999 Mzack9999 deleted the use_request_to_ctor branch June 11, 2024 08:20
@Mzack9999 Mzack9999 added the Type: Bug Inconsistencies or issues which will cause an issue or problem for users or implementors. label Jun 11, 2024
@Miracles666
Copy link

hi @alban-stourbe-wmx Have you successfully solved this problem?

@alban-stourbe-wmx
Copy link
Contributor

hi @alban-stourbe-wmx Have you successfully solved this problem?

Hello @Miracles666, to fix the issue, the automatic-form has been disabled with headless mode. For my case, i detect forms with the option with the standard mode only.

If we really want to use automatic-form with headless mode, a new feature need to be develop to detect form and send data through the web browser.

I hope you understand

@Miracles666
Copy link

@alban-stourbe-wmx standard mode is very few things that can be crawled up ,I think this is a very serious bug。

@alban-stourbe-wmx
Copy link
Contributor

is very few things that can be crawled up

That's why if you think a bit, we can use the SDK Katana to crawl your website with the headless and the standard mode together. After you can merge and the two outputs to obtain all informations ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Bug Inconsistencies or issues which will cause an issue or problem for users or implementors.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Katana JSONL file Issue on raw request field
4 participants