Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: added jsluice based javascript parsing to default #492

Merged
merged 8 commits into from
Jul 16, 2023

Conversation

Ice3man543
Copy link
Member

@Ice3man543 Ice3man543 commented Jun 26, 2023

Adds jsluice based javascript parsing to gather links from pages on default runs. The functionality is enabled by default and allows gathering various kinds of javascript links.

An example run on public-crawl-maze javascript interactive section

Command to run:

./katana -u https://security-crawl-maze.app/javascript/interactive/  -v -d 3
Screenshot 2023-06-26 at 1 49 50 PM

TODOS:

  • Improve parsing
  • Add docs to readme

@Ice3man543 Ice3man543 self-assigned this Jun 26, 2023
Copy link
Member

@ehsandeep ehsandeep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the latest changes:

./katana -u https://www.hackerone.com -j > output.txt

Max RSS: 1507 MB # memory uses
Sys Time: 728.727µs
User Time: 382.145µs
Actual Time: 37.17816025s
Voluntary Context Switch (nvcsw): 7083

$cat output.txt | jq .request.endpoint | wc # total endpoint count
     527

$cat output.txt | jq .response.status_code | dup # endpoint by staus_code
 338 200
 146 429
  35 null
   8 404

With the previous version:

katana -jc -u https://www.hackerone.com -j > output.txt

Max RSS: 194 MB # memory uses
Sys Time: 204.804µs
User Time: 367.684µs
Actual Time: 34.285477s
Voluntary Context Switch (nvcsw): 6352

$cat output.txt | jq .request.endpoint | wc # total endpoint count
     517

$cat output.txt | jq .response.status_code | dup # endpoint by staus_code

 321 200
 135 429
  44 404
  16 null
   1 500

Note:

  • high memory uses (jumped from 150MB to 2GB)
  • improved endpoint discovery
  • headless mode still needs to be tested.

Copy link
Member

@Mzack9999 Mzack9999 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm implementation! - marking as request changes as docs need to be added.
Note: High memory consumption might require upstream library modification

@ehsandeep ehsandeep added the Status: Review Needed The issue has a PR attached to it which needs to be reviewed label Jun 27, 2023
@ehsandeep ehsandeep merged commit c6f08ed into dev Jul 16, 2023
@ehsandeep ehsandeep deleted the add-jsluice-default branch July 16, 2023 20:42
@ehsandeep ehsandeep removed the Status: Review Needed The issue has a PR attached to it which needs to be reviewed label Jul 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants