URI Too Large Errors

[pthurlow]

Came across this issue when our security tags started getting large. It appears that the headers sent to algolia are being inlined as query params in the url. This can be seen on line 74 here. When I removed those and put them in the request header I only received a 400 when the header x-algolia-agent was removed from the query params. Leaving just that one in the query params and moving the rest to request headers solved my issue.

I'd like to understand why these headers are being inlined into the url so that I might be able to supply a patch for this issue. Also any documentation on required headers/query params would be awesome as I could not find any.

[vvo]

Hi @pthurlow, could you maybe send here a CURL like command that results in an error when your security tags are too big? If this is not OK given the security of your application, please send this to [email protected] (and mention me and algolia javascript client)

You can get a CURL like command from the network tab in chrome, right clicking a failed request and "copy as CURL".

We recently updated the way we provide security at Algolia and you can now always use secured API keys: https://www.algolia.com/doc/guides/security/api-keys.

Those keys will then be sent in POST when doing searches if the key is too big.

Since then, I am interested in seing what's inside your query.

As for the reasoning, we only put in the HTTP headers, headers that are considered as "simple HEADERS" given the XHR spec: https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS#Simple_requests

That way, we can avoid doing an extra OPTIONS request in browsers.

Let me know how it goes.

[pthurlow]

I sent an example request header to your support. When you say "Those keys will then be sent in POST when doing searches if the key is too big" do you just mean the request is made as a POST rather than a GET? If that is the case this problem will still persist because this is nginx or other server limits on URI request length. Ideally these keys would be inserted into the body to avoid this issue altogether, but as I mentioned it appears the js client is simply inlining all security tags and api keys into the url query string.

[pthurlow]

After a bit more digging I think I've found the issue. The logic which determines whether to stick the apiKey into the body seen here assumes that the body object contains a params field. However this is not the case for non fallback requests as there is a query request array on the body with the params field on each array element. This is done here

If I remove the check for body.params it correctly inserts the api key into the POST body. So maybe you can advise on the best course of action for a fix.

as a related issue, we also use the setSecurityTags() method to send what we thought were required security tags by the api so it can compare it to the encoded api key. This also causes issues because it appears this header cannot be stuck in the post body like the api key (i tried but it didn't recognize any names i guessed, heh). since security tags are long just like the api key this causes the same URI too large issue but there is no discernible way to put them into the body. Of course, when I tried just not sending the security tags it still worked! so I supposed those are not required after all (maybe you can comment on that)

[vvo]

After a bit more digging I think I've found the issue. The logic which determines whether to stick the apiKey into the body seen here assumes that the body object contains a params field. However this is not the case for non fallback requests as there is a query request array on the body with the params field on each array element. This is done here

It seems not to be an issue with "fallback requests" but an issue being that requests done using client.search() will not have their API keys inlined when too big.

If I remove the check for body.params it correctly inserts the api key into the POST body. So maybe you can advise on the best course of action for a fix.

I will fix this and release.

Of course, when I tried just not sending the security tags it still worked! so I supposed those are not required after all (maybe you can comment on that)

Yes there's no more a need to use setSecurityTags if you already generated a secured API key with all the tags. All our security features are now provided by secured API keys and can be found here: https://www.algolia.com/doc/guides/security/api-keys.