Multipart formdata reader decode error if filename is not specified

[Yanpas]

Long story short

I have a client that uses libcurl and a server which uses aiohttp. The client sends multipart formdata where one of the forms is binary (image). aiohttp tries to decode it as string

If you use curl utility curl -v -s -X POST -F "image=@img" http://myserver.com it automatically substitutes filename. libcurl API on the other hand doesn't let you specify filename unless you upload real file as well as content-type.

So there is no way to pass image using libcurl if this image doesn;t exist as a file on filesystem

Link to the line where exception occurs

aiohttp/aiohttp/web_request.py

Line 525 in cc2a522

value = value.decode(charset)

Expected behaviour

Every form field is represented as bytes (or Bytestream) with headers.

Your environment

centos 7 on both sides

PS this requires API modification

[aio-libs-bot]

GitMate.io thinks the contributor most likely able to help you is @asvetlov.

Possibly related issues are #3242 (Error), #3064 (multipart BodyPartReaderPayload filename wrong), #2201 (Client: FormData._gen_form_data ignores filename, explodes), #3090 (Remove reader parameter from request.multipart()), and #67 (fix header encoding error).

[asvetlov]

Could you provide a minimal reproducible example for demonstrating the problem?
It can save a lot of time of the issue investigating.

[Yanpas]

server:

import asyncio
import aiohttp
from aiohttp import web

async def hnd(request: web.Request):
	data = await request.post()
	img = data['img']
	if hasattr(img, "file"):
		cont = img.file.read()
	else:
		cont = img
	print(cont[:100])

app = web.Application()
app.add_routes([web.post('/', hnd)])
web.run_app(app)

client:

#include <string>
#include <fstream>
#include <streambuf>
#include <curl/curl.h>

using namespace std;

int main(int ac, char**av) {
	ifstream f(av[1]);
	string cont{istreambuf_iterator<char>(f), istreambuf_iterator<char>()};
	auto req = curl_easy_init();
	curl_easy_setopt(req, CURLOPT_PORT, 8080);
	curl_easy_setopt(req, CURLOPT_URL, "localhost");
	curl_httppost *post=nullptr, *last=nullptr;
	curl_formadd(&post, &last,
		CURLFORM_PTRNAME, "img",
		CURLFORM_PTRCONTENTS, cont.c_str(),
		CURLFORM_CONTENTLEN, cont.size(),
		CURLFORM_END
	);
	curl_easy_setopt(req, CURLOPT_HTTPPOST, post);
	// curl_easy_setopt(req, CURLOPT_POST, post);

	auto res = curl_easy_perform(req);
	printf("curl res %i\n", res);

	curl_easy_cleanup(req);
}

Compile client: g++ req.cpp -o req -lcurl
Run: ./req req

Error:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa8 in position 128: invalid start byte
Unhandled exception
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/aiohttp/web_protocol.py", line 447, in start
    await resp.prepare(request)
  File "/usr/lib/python3.7/site-packages/aiohttp/web_response.py", line 353, in prepare
    return await self._start(request)
  File "/usr/lib/python3.7/site-packages/aiohttp/web_response.py", line 667, in _start
    return await super()._start(request)
  File "/usr/lib/python3.7/site-packages/aiohttp/web_response.py", line 410, in _start
    await writer.write_headers(status_line, headers)
  File "/usr/lib/python3.7/site-packages/aiohttp/http_writer.py", line 112, in write_headers
    self._write(buf)
  File "/usr/lib/python3.7/site-packages/aiohttp/http_writer.py", line 67, in _write
    raise ConnectionResetError('Cannot write to closing transport')
ConnectionResetError: Cannot write to closing transport

[Dreamsorcerer]

The request is missing a content-type. It works fine when adding:

        CURLFORM_CONTENTTYPE, "image/jpeg",