-
Notifications
You must be signed in to change notification settings - Fork 635
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feature: added jsluice based javascript parsing to default #492
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With the latest changes:
./katana -u https://www.hackerone.com -j > output.txt
Max RSS: 1507 MB # memory uses
Sys Time: 728.727µs
User Time: 382.145µs
Actual Time: 37.17816025s
Voluntary Context Switch (nvcsw): 7083
$cat output.txt | jq .request.endpoint | wc # total endpoint count
527
$cat output.txt | jq .response.status_code | dup # endpoint by staus_code
338 200
146 429
35 null
8 404
With the previous version:
katana -jc -u https://www.hackerone.com -j > output.txt
Max RSS: 194 MB # memory uses
Sys Time: 204.804µs
User Time: 367.684µs
Actual Time: 34.285477s
Voluntary Context Switch (nvcsw): 6352
$cat output.txt | jq .request.endpoint | wc # total endpoint count
517
$cat output.txt | jq .response.status_code | dup # endpoint by staus_code
321 200
135 429
44 404
16 null
1 500
Note:
- high memory uses (jumped from 150MB to 2GB)
- improved endpoint discovery
- headless mode still needs to be tested.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm implementation! - marking as request changes
as docs need to be added.
Note: High memory consumption might require upstream library modification
Adds jsluice based javascript parsing to gather links from pages on default runs. The functionality is enabled by default and allows gathering various kinds of javascript links.
An example run on public-crawl-maze javascript interactive section
Command to run:
TODOS: