pre-condition in code , fuzz and other misc updates

[tarunKoyalwar]

Proposed Changes

• fuzz: rename 'filters' -> 'pre-condition' (cc: @princechaddha )
• fuzz: rename 'filters-condition' -> 'pre-condition-operator'
• add support for pre-condition in code protocol
• OS + Arch Detection Helper Function is Pre Condition for Code Protocol
• fix/update scrapefuncs to generate mintlify dsl helper function page (https://projectdiscovery-nuclei-v3-2-3-updates.mintlify.app/templates/reference/js-helper-functions#code-protocol)
• only recover on third-party package panics ( goja / go-rod) , removed others (Investigation on inappropriate use of panic-recover paradigm #4886)
• closes Code Protocol should implement a prerequisite DSL condition #4888

id: pre-condition-code

info:
  name: example code template
  author: pdteam
  severity: info


self-contained: true

variables:
  OAST: "{{interactsh-url}}"

code:
  - pre-condition: IsLinux()
    engine:
      - sh
      - bash
    source: |
      echo "$OAST" | base64
    
    matchers:
      - type: dsl
        dsl:
          - true

[tarunKoyalwar]

• due to variable priority in flow / multi-protocol , array was not flattened in some cases . since we always flatten slice if its size is 1 . for cases where length is not known using _slice suffix along with required key ensures that it always returns a slice ( note: slice/array are only meant to be used in flow )

id: extracted_value

info:
  name: Extracted value does not work in flow
  author: pascal-sun
  severity: info

flow: http(1) && http(2)

http:
  - method: GET
    path:
      - "{{BaseURL}}"
      - "{{BaseURL}}/{{test}}"

    extractors: 
      - type: regex
        name: test
        group: 1
        regex:
          - 'This (domain) is for use in illustrative examples in documents.'
        internal: true

    matchers:
      - type: status
        internal: true
        status:
          - 200
 
  - method: GET
    path:
      - "https://example.org/{{test}}"
    matchers:
      - type: status
        status:
          - 404

Before

 nuclei -t a.yaml -u https://example.com -v -debug-req 

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v3.2.2

		projectdiscovery.io

[VER] Started metrics server at localhost:9092
[INF] Current nuclei version: v3.2.2 (latest)
[INF] Current nuclei-templates version: v9.8.0 (latest)
[WRN] Scan results upload to cloud is disabled.
[INF] New templates added in latest release: 85
[INF] Templates loaded for current scan: 1
[WRN] Loaded 1 unsigned templates for scan. Use with caution.
[INF] Targets loaded for current scan: 1
[INF] [extracted_value] Dumped HTTP request for https://example.com

GET / HTTP/1.1
Host: example.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36 Edg/117.0.2045.47
Connection: close
Accept: */*
Accept-Language: en
Accept-Encoding: gzip

[VER] [extracted_value] Sent HTTP request to https://example.com
[INF] [extracted_value] Dumped HTTP request for https://example.com/[domain]

GET /[domain] HTTP/1.1
Host: example.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:56.0) Gecko/20100101 Firefox/56.0
Connection: close
Accept: */*
Accept-Language: en
Accept-Encoding: gzip

[VER] [extracted_value] Sent HTTP request to https://example.com/[domain]
[INF] [extracted_value] Dumped HTTP request for https://example.org/domain

GET /domain HTTP/1.1
Host: example.org
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20100101 Firefox/13.0.1
Connection: close
Accept: */*
Accept-Language: en
Accept-Encoding: gzip

[VER] [extracted_value] Sent HTTP request to https://example.org/domain
[extracted_value] [http] [info] https://example.org/domain

Now

$  ./nuclei -t a.yaml -u https://example.com -v -debug-req

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v3.2.2

		projectdiscovery.io

[VER] Started metrics server at localhost:9092
[INF] Current nuclei version: v3.2.2 (latest)
[INF] Current nuclei-templates version: v9.8.0 (latest)
[WRN] Scan results upload to cloud is disabled.
[INF] New templates added in latest release: 85
[INF] Templates loaded for current scan: 1
[WRN] Loading 1 unsigned templates for scan. Use with caution.
[INF] Targets loaded for current scan: 1
[INF] [extracted_value] Dumped HTTP request for https://example.com

GET / HTTP/1.1
Host: example.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36 Edg/107.0.1418.26
Connection: close
Accept: */*
Accept-Language: en
Accept-Encoding: gzip

[VER] [extracted_value] Sent HTTP request to https://example.com
[INF] [extracted_value] Dumped HTTP request for https://example.com/domain

GET /domain HTTP/1.1
Host: example.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/604.2.4 (KHTML, like Gecko) Version/9.1.2 Safari/604.2.4
Connection: close
Accept: */*
Accept-Language: en
Accept-Encoding: gzip

[VER] [extracted_value] Sent HTTP request to https://example.com/domain
[INF] [extracted_value] Dumped HTTP request for https://example.org/domain

GET /domain HTTP/1.1
Host: example.org
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.0 Safari/605.8.25
Connection: close
Accept: */*
Accept-Language: en
Accept-Encoding: gzip

[VER] [extracted_value] Sent HTTP request to https://example.org/domain
[extracted_value] [http] [info] https://example.org/domain

[tarunKoyalwar]

Fixed Incorrect Template Loading Counters

• due to logic of excluding template being after parsing template load and executing template count were always incorrect. this is now fixed

Before

After

$ ./nuclei -dast -u example.com -t ~/Codebase/nuclei-templates/ -v -itags fuzz

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v3.2.2

		projectdiscovery.io

[VER] Started metrics server at localhost:9092
[WRN] Excluded 82 template[s] with known weak matchers / tags excluded from default run using .nuclei-ignore
[INF] Current nuclei version: v3.2.2 (latest)
[INF] Current nuclei-templates version: v9.8.0 (latest)
[WRN] Scan results upload to cloud is disabled.
[INF] New templates added in latest release: 85
[INF] Templates loaded for current scan: 21
[WRN] Executing 21 unsigned templates for scan. Use with caution.
[INF] Targets loaded for current scan: 1

cc: @ehsandeep