[BUG] Shared variables are overwritten when the execution forks during a workflow

[tovask]

Is there an existing issue for this?

• I have searched the existing issues.

Current Behavior

During a workflow, when a template has multiple matches for the same target (and the subtemlate is under a Matcher Name based condition check), therefore the execution forks and continues for each match separatelly, the extracted shared variables are overwritten between these fork execution flows in deeper subtemplates. Seems they share the same context in deeper templates between all the matches.

Expected Behavior

When there are multiple matches, the subsequent execution context of each match is separate.

Steps To Reproduce

Create a workflow that has deep subtemplates, a Matcher Name based condition check that will be hit multiple times, extract variables from the first template with different value for each match, try to refer them in deeper templates.

debug_workflow_deep.yaml

id: debug_workflow_deep
info:
  name: Debug workflow
  author: Levente Kovats
  severity: info

workflows:
  - template: debug_first.yaml
    matchers:
      - name: always_true # <- this will match mutiple times, and the execution of subtemplates will continue separatelly for each match
        subtemplates:
          - template: debug_between.yaml
            subtemplates:
              - template: debug_between.yaml
                subtemplates:
                  - template: debug_between.yaml
                    subtemplates:
                      - template: debug_last.yaml

debug_first.yaml

id: debug_first

info:
  name: Debug
  author: Levente Kovats
  severity: info

http:
  - method: GET
    path:
      - '{{BaseURL}}'
      - '{{BaseURL}}' # perform a second request to have multiple matches

    matchers:
      - type: dsl
        name: always_true
        dsl:
          - true
    extractors:
      - type: regex
        name: debug_shared
        regex:
          - '"uuid": "(.*)-'
        group: 1

debug_between.yaml

id: debug_between

info:
  name: Debug
  author: Levente Kovats
  severity: info

http:
  - method: GET
    path:
      - '{{BaseURL}}'

    matchers:
      - type: dsl
        name: always_true
        dsl:
          - true && print_debug(debug_shared)

debug_last.yaml

id: debug_last

info:
  name: Debug
  author: Levente Kovats
  severity: info

http:
  - method: GET
    path:
      - '{{BaseURL}}'

    matchers:
      - type: dsl
        name: always_true
        dsl:
          - true
    extractors:
      - type: kval
        kval:
          - debug_shared

Execution

nuclei -target https://httpbin.org/uuid -workflows debug_workflow_deep.yaml -v

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v3.3.1

                projectdiscovery.io

[INF] Current nuclei version: v3.3.1 (outdated)
[INF] Current nuclei-templates version:  (latest)
[WRN] Scan results upload to cloud is disabled.
[INF] Workflows loaded for current scan: 1
[INF] Executing 5 signed templates from unsigned
[INF] Targets loaded for current scan: 1
[INF] print_debug value: [[7a39b227-86b3-4220-a8b2]]     <- 1. subtemplate, first value
[INF] print_debug value: [[6447b7b2-6d26-4f31-816a]]     <- 1. subtemplate, second value
[INF] print_debug value: [[6447b7b2-6d26-4f31-816a]]     <- 2. subtemplate: the first value is overwritten by the second
[INF] print_debug value: [[6447b7b2-6d26-4f31-816a]]     <- 2. subtemplate
[INF] print_debug value: [[6447b7b2-6d26-4f31-816a]]     <- 3. subtemplate
[INF] print_debug value: [[6447b7b2-6d26-4f31-816a]]     <- 3. subtemplate
[debug_last:always_true] [http] [info] https://httpbin.org/uuid ["[6447b7b2-6d26-4f31-816a]"]
[debug_last:always_true] [http] [info] https://httpbin.org/uuid ["[6447b7b2-6d26-4f31-816a]"]

It's visible that in the first subtemplate receive separate values, but after its execution of the second match, it overwrites the first value in subsequence subtemplates.

It is usually not an issue for workflows where is only one level of subtemplates:
debug_workflow_flat.yaml

id: debug_workflow_flat
info:
  name: Debug workflow
  author: Levente Kovats
  severity: info

workflows:
  - template: debug_first.yaml
    matchers:
      - name: always_true
        subtemplates:
          - template: debug_last.yaml

$ nuclei -target https://httpbin.org/uuid -workflows debug_workflow_flat.yaml
                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v3.3.1

                projectdiscovery.io

[INF] Current nuclei version: v3.3.1 (outdated)
[INF] Current nuclei-templates version:  (latest)
[WRN] Scan results upload to cloud is disabled.
[INF] Workflows loaded for current scan: 1
[INF] Executing 2 signed templates from unsigned
[INF] Targets loaded for current scan: 1
[debug_last:always_true] [http] [info] https://httpbin.org/uuid ["[46875354-5b4c-4b22-813d]"]
[debug_last:always_true] [http] [info] https://httpbin.org/uuid ["[98a916a3-a4cd-45b7-993a]"]

I guess the overwriting happens here as well but since there's no more subtemplates it's not causing a problem.

Relevant log output

$ nuclei -target https://httpbin.org/uuid -workflows debug_workflow_deep.yaml -v



                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v3.3.1

                projectdiscovery.io

[VER] Started metrics server at localhost:9092
[INF] Current nuclei version: v3.3.1 (outdated)
[INF] Current nuclei-templates version:  (latest)
[WRN] Scan results upload to cloud is disabled.
[INF] Workflows loaded for current scan: 1
[INF] Executing 5 signed templates from unsigned
[INF] Targets loaded for current scan: 1
[VER] [debug_first] Sent HTTP request to https://httpbin.org/uuid
[VER] [debug_first] Sent HTTP request to https://httpbin.org/uuid
[VER] [debug_between] Sent HTTP request to https://httpbin.org/uuid
[INF] print_debug value: [[b5c3a64e-b9d7-4213-b3fa]]
[VER] [debug_between] Sent HTTP request to https://httpbin.org/uuid
[INF] print_debug value: [[f99a8ca0-4bee-444d-bd3c]]
[VER] [debug_between] Sent HTTP request to https://httpbin.org/uuid
[INF] print_debug value: [[b5c3a64e-b9d7-4213-b3fa]]
[VER] [debug_between] Sent HTTP request to https://httpbin.org/uuid
[INF] print_debug value: [[b5c3a64e-b9d7-4213-b3fa]]
[VER] [debug_between] Sent HTTP request to https://httpbin.org/uuid
[INF] print_debug value: [[b5c3a64e-b9d7-4213-b3fa]]
[VER] [debug_last] Sent HTTP request to https://httpbin.org/uuid
[debug_last:always_true] [http] [info] https://httpbin.org/uuid ["[b5c3a64e-b9d7-4213-b3fa]"]
[VER] [debug_between] Sent HTTP request to https://httpbin.org/uuid
[INF] print_debug value: [[b5c3a64e-b9d7-4213-b3fa]]
[VER] [debug_last] Sent HTTP request to https://httpbin.org/uuid
[debug_last:always_true] [http] [info] https://httpbin.org/uuid ["[b5c3a64e-b9d7-4213-b3fa]"]

Environment

- OS: Linux or Windows (happens on both)
- Nuclei: v3.3.1

Anything else?

It is really good that the workflow execution fork when there are multiple matches and the subtemlates are executed for each one, please don't remove this feature! 🙂