Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

skip if the maximum depth is exceeded #799

Merged
merged 1 commit into from
Mar 13, 2024
Merged

skip if the maximum depth is exceeded #799

merged 1 commit into from
Mar 13, 2024

Conversation

dogancanbakir
Copy link
Member

Closes #529

before:

$ go run . -u www.apple.com -kf all -v

   __        __                
  / /_____ _/ /____ ____  ___ _
 /  '_/ _  / __/ _  / _ \/ _  /
/_/\_\\_,_/\__/\_,_/_//_/\_,_/                                                   

                projectdiscovery.io

[INF] Current katana version v1.0.5 (latest)
[INF] Started standard crawling for => https://www.apple.com
[file] [GET] https://www.apple.com/wtgh/re/
[file] [GET] https://www.apple.com/working-with-apple-services/
[file] [GET] https://www.apple.com/watch/close-your-rings/
[file] [GET] https://www.apple.com/usergroups/
[file] [GET] https://www.apple.com/watch/battery/
[file] [GET] https://www.apple.com/watch/cellular/
[file] [GET] https://www.apple.com/watch/
[file] [GET] https://www.apple.com/watch/why-apple-watch/
[file] [GET] https://www.apple.com/wallet/
[file] [GET] https://www.apple.com/watchos/watchos-10/
[file] [GET] https://www.apple.com/us/search/product-red?f=watchultra&fh=246098&sel=accessories
[file] [GET] https://www.apple.com/watch/compare/
[file] [GET] https://www.apple.com/us/search/product-red?f=wireless&fh=30b5&sel=accessories
[file] [GET] https://www.apple.com/us/search/product-red?f=watch45mm&fh=96d651&sel=accessories
[file] [GET] https://www.apple.com/us/search/product-red?f=watch49mm&fh=47a111&sel=accessories
[file] [GET] https://www.apple.com/us/search/product-red?f=watchse&fh=4d1d&sel=accessories
[file] [GET] https://www.apple.com/us/search/product-red?f=watch40mm&fh=4a7b&sel=accessories
[file] [GET] https://www.apple.com/us/search/product-red?sel=accessories
[file] [GET] https://www.apple.com/us/search/product-red?f=watch38mm&fh=44d1&sel=accessories
[file] [GET] https://www.apple.com/us/search/product-red?f=watch42mm&fh=44d2&sel=accessories
[file] [GET] https://www.apple.com/us/search/product-red?f=watch44mm&fh=4a7c&sel=accessories
[file] [GET] https://www.apple.com/us/search/product-red?f=series7&fh=9ceacd&sel=accessories
[file] [GET] https://www.apple.com/us/search/product-red?f=watchultra2&fh=53c158&sel=accessories
[file] [GET] https://www.apple.com/us/search/product-red?f=series8&fh=2653b6&sel=accessories
[file] [GET] https://www.apple.com/us/search/product-red?f=silicone&fh=2d07&sel=accessories
[file] [GET] https://www.apple.com/us/search/product-red?f=series9&fh=7f9533&sel=accessories
[file] [GET] https://www.apple.com/us/search/product-red?f=watch41mm&fh=14313&sel=accessories
^C[INF] - Ctrl+C pressed in Terminal
[INF] Creating resume file: /Users/fortytwo/.config/katana/resume-cno3g27a2uac1oveep70.cfg

after:

$ go run . -u www.apple.com -kf all -v

   __        __                
  / /_____ _/ /____ ____  ___ _
 /  '_/ _  / __/ _  / _ \/ _  /
/_/\_\\_,_/\__/\_,_/_//_/\_,_/                                                   

                projectdiscovery.io

[INF] Current katana version v1.0.5 (latest)
[INF] Started standard crawling for => https://www.apple.com
[file] [GET] https://www.apple.com/wtgh/re/
[file] [GET] https://www.apple.com/working-with-apple-services/
[file] [GET] https://www.apple.com/usergroups/
[file] [GET] https://www.apple.com/watch/battery/
[file] [GET] https://www.apple.com/watch/why-apple-watch/
[file] [GET] https://www.apple.com/watch/cellular/
[script] [GET] https://www.apple.com/v/working-with-apple-services/b/built/scripts/main.built.js
[file] [GET] https://www.apple.com/wallet/
[script] [GET] https://www.apple.com/v/watch/battery/b/built/scripts/main.built.js
[file] [GET] https://www.apple.com/watchos/watchos-10/
[file] [GET] https://www.apple.com/watch/
[script] [GET] https://www.apple.com/v/watch/cellular/g/built/scripts/main.built.js
[file] [GET] https://www.apple.com/watch/close-your-rings/
[script] [GET] https://www.apple.com/ac/pricing/latest-1/scripts/autopricing.built.js
[script] [GET] https://www.apple.com/v/watch/cellular/g/built/scripts/head.built.js
[script] [GET] https://www.apple.com/global/ac_retina/ac_retina.js
[script] [GET] https://www.apple.com/v/wallet/g/built/scripts/main.built.js
[file] [GET] https://www.apple.com/watch/compare/
[link] [GET] https://www.apple.com/v/watch/bk/built/styles/overview.built.css
[script] [GET] https://www.apple.com/v/watch/close-your-rings/d/built/scripts/head.built.js
[link] [GET] https://www.apple.com/v/watch/close-your-rings/d/built/styles/overview.built.css
[link] [GET] https://www.apple.com/v/watch/compare-strip/h/built/styles/overview.built.css
[script] [GET] https://www.apple.com/ac/libs/lottie/5.6.3/lottie.min.js
[file] [GET] https://www.apple.com/us/search/product-red?sel=accessories
[script] [GET] https://www.apple.com/ac/ac-films/6.9.0/scripts/autofilms.built.js
[script] [GET] https://www.apple.com/v/watchos/watchos-10/c/built/scripts/main.built.js
[script] [GET] https://www.apple.com/v/watch/compare/ae/built/scripts/head.built.js
[script] [GET] https://www.apple.com/v/watch/close-your-rings/d/built/scripts/main.built.js
[link] [GET] https://www.apple.com/v/watch/compare/ae/built/styles/overview.built.css
[script] [GET] https://www.apple.com/v/watch/compare/ae/built/scripts/main.built.js
...
^C[INF] - Ctrl+C pressed in Terminal
[INF] Creating resume file: /Users/fortytwo/.config/katana/resume-cno3fl7a2uac0sjp1dvg.cfg

@dogancanbakir dogancanbakir self-assigned this Mar 12, 2024
@Mzack9999 Mzack9999 merged commit 24c31f4 into dev Mar 13, 2024
13 checks passed
@Mzack9999 Mzack9999 deleted the fix_kf_crawl branch March 13, 2024 18:50
@Mzack9999 Mzack9999 linked an issue Mar 13, 2024 that may be closed by this pull request
-kf appears to only scan the known files, not add it to the spider #529
Closed
@chenrui333 chenrui333 mentioned this pull request Mar 26, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Mzack9999 Mzack9999 Mzack9999 approved these changes

Assignees

@dogancanbakir dogancanbakir

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

-kf appears to only scan the known files, not add it to the spider
2 participants
@dogancanbakir @Mzack9999