Handle python session requests/response in modify_chunk_response plugin

[ananth-sivam]

Discussed in #1351

^{Originally posted by ananth-sivam September 27, 2023}
@abhinavsingh
I have used the modify_chunk_response plugin to modify an upstream response header and publish it to client.
Here the client is using python requests.Session() to do GET/POST/PUT via https.

Below is the UT that I have to mimick the client and the last assert is failing as assert 'https://httpbin.org/get' == 'https://httpbin.org/post'

UT
session = requests.Session()
r1 = session.get( "https://httpbin.org/get", proxies={"https": "http://127.0.0.1:8899"}, verify=False, )
assert r1.json().get("url") == "https://httpbin.org/get"
r2 = session.post( "https://httpbin.org/post", proxies={"https": "http://127.0.0.1:8899"}, verify=False, )
assert r2.json().get("url") == 'https://httpbin.org/post'

OR below was executed after starting the python -m proxy --plugins proxy.plugin.ModifyChunkResponsePlugin

r1 = session.get( "http://httpbin.org/get", proxies={"http": "http://127.0.0.1:8899"}, verify=False, );print("R1: ",r1.json().get("url"));r2 = session.post( "http://httpbin.org/post", proxies={"http": "http://127.0.0.1:8899"}, verify=False, );print("R2: ",r2.json().get("url"));

Below is the print output
R1: http://httpbin.org/get
R2: http://httpbin.org/get

proxy version: 2.4.3

Facing a similar issue on the pipeline_request side also. Client published a GET call followed with a POST method call using requests.Session.
But proxy passing the GET request to the handle_client_request method for the POST call also.
Any help is appreciated.

[noorul]

@abhinavsingh Can we confirm that this is a known issue?

[abhinavsingh]

@noorul @ananth-sivam This will need some investigation. Last time I checked, pipelines requests were well supported. But afaik, there is no exhaustive test suite to keep pipeline support intact and it "may" have gone broke. I'll try and dig into it this month.

[ananth-sivam]

Thank you @abhinavsingh. For now I have overwritten the request/response state to httpParserStates.INITIALIZED and request.headers to None, after each request/response completion in the pipeline