Error 400 - Ambiguous URI Empty Segment

[Justvuur]

Jetty Version
12.0.3

Jetty Environment
ee8

Java Version
JavaSE-17

Question
I just migrated from Jetty 10.0.15 to 12.0.3 and I keep getting the following error:
URI: /badURI
STATUS: 400
MESSAGE: Ambiguous URI empty segment
CAUSED BY: org.eclipse.jetty.http.BadMessageException: 400: Ambiguous URI empty segment

Any ideas what could be causing this?

[joakime]

What is your URI?

[lachlan-roberts]

@Justvuur It means your URI has an empty segment (//) which makes it ambiguous. This is because it could be an attempt to bypass some security constraints.

You can set the URI compliance mode in the HttpConfiguration.

See https://eclipse.dev/jetty/javadoc/jetty-12/org/eclipse/jetty/http/UriCompliance.Violation.html#AMBIGUOUS_EMPTY_SEGMENT

When allowing this Violation, the application developer/deployer must ensure that the application behaves as desired when it receives a URI path containing //. Specifically, any URI pattern matching for security concerns needs to be carefully audited.

[yokotaso]

If any vaiolation of UriCompliance exists at entry point of ServletContextRequest , Jetty12 using Servlet returns 400 Bad Request.
So, Whether HttpConfiguration is set or not, any violation is not allowed implicitly with using Servlet.
It might be good that UriCompliance check should be taken into account about value of HttpConfiguration#getHttpCompliance

https://github.com/jetty/jetty.project/blob/jetty-12.0.x/jetty-ee10/jetty-ee10-servlet/src/main/java/org/eclipse/jetty/ee10/servlet/ServletContextRequest.java#L195-L208

If you need shorthand workaround, make EmptySegmentHandler and make it handle before ServletContextHandler

public class UriComplianceCheckHandler extends Handler.Wrapper {

    public UriComplianceCheckHandler(Handler handler) {
        super(handler);
    }

    @Override
    public boolean handle(Request request, Response response, Callback callback) throws Exception {
        if (request.getHttpURI().hasViolations()) {
            return super.handle(rewriteRequest(request), response, callback);
        } else {
            return super.handle(request, response, callback);
        }
    }

    Request rewriteRequest(Request request) {
        log.warn("There are HttpURI Violation exists. HttpURI:{}. Violations:{}", request.getHttpURI(),
                request.getHttpURI().getViolations());
        if (request.getHttpURI().hasViolation(UriCompliance.Violation.AMBIGUOUS_EMPTY_SEGMENT)) {
            HttpURI rewrite = rewriteHttpURL(request);
            return Request.serveAs(request, rewrite);
        }
        return request;
    }

    private HttpURI rewriteHttpURL(Request base) {
        String param = base.getHttpURI().getParam();
        String query = base.getHttpURI().getQuery();
        List<String> segments = Arrays
                .stream(base.getHttpURI().getPath().split("\\/"))
                .filter(v -> !v.isEmpty())
                .toList();
        String newPath = "/" + StringUtils.join(segments, "/");

        return HttpURI.build(base.getHttpURI(), newPath, param, query).asImmutable();
    }
}

[Justvuur]

URI

http://localhost:9998/static/

[Justvuur]

Ok, I managed to get it working but using http.setUriCompliance(UriCompliance.LEGACY); but I noticed that something somewhere is appending the extra "/". Even though I navigate to "http://localhost:9998/static/", it ends up being "http://localhost:9998/static//".

Any ideas why?

[joakime]

If you see http://localhost:9998/static// with UriCompliance.LEGACY, then that means that is exactly the path as it was sent on the HTTP request from your User-Agent (HTTP Client).
That would also explain the "400: Ambiguous URI empty segment" you were seeing.
That last segment // is valid per URI rules, and it means a segment with no value (hence ambiguous).
It also invalid per Servlet rules, as it cannot match a context-path, url-pattern, or constraint.

To address this problem you have to address what is going on with your HTTP Client.
You can confirm this easily with a network traffic capturing tool like wireshark.

Jetty does not "clean up" these kinds of URIs as there are some libraries that rely on this ambiguity (esp security frameworks).

[salacr]

I might have similar problem:

Jetty Version
12.0.9

Jetty Environment
ee10

Java Version
JavaSE-17

My request look like this: http://localhost//something.js and I'm also getting 400: Ambiguous URI empty segment

I have set UriCompliance.LEGACY but it doesn't work for me any chance how to get it working?

Thanks!

[gregw]

@salacr Are you sure you have set LEGACY mode?

Note that there is also the CompactPathRule that can be added to the RewriteHandler that will replace '//' with '/'

[salacr]

@gregw I have unfortunately in ServletContextRequest.java
there is the
// TODO we should check if current compliance mode allows all the violations?

And it still returns the

return new ServletApiRequest.AmbiguousURI(this, msg.toString());

I will check the CompactPathRule then thanks.

[salacr]

Unfortunately, even when CompactPathRule and the URI are rewritten the violations are already set so I'm still getting same error. Any other advice will be highly apriciated

[chiqiu]

Same here, I changed it to LEGACY in jetty.xml , but when call getSerlvetPath() , it still throws 400 because AmbiguousURI was returned

[joakime]

@chiqiu as pointed out in the Servlet spec, those paths are ambiguous and lead to all kinds of bugs in the servlet spec.
These bugs have been present since the beginning of the Servlet spec (and are not new to ee10 and Servlet 6), and no manner of bug fixing in the spec or the containers can address them properly.
The servlet spec instead decided that these kinds of requests are bad and unsupported.

The AmbiguousURI class properly throws an exception in the case of .getServletPath() and .getPathInfo() as it is not possible to support an ambiguous path with those servlet methods. (there are similar failures for ambiguous paths found in other places in the servlet spec as well. eg: getRequestDispatcher and sendRedirect APIs)

Just using LEGACY without fixing the paths before sending them into the Servlet API will always result in these kinds of issues.

The changes in PR #12306 should help with most, but not all, ambiguous path related fixes that the CompactPathRule is capable of addressing. (other ambiguous path situations need to be addressed with custom code that is specific to your webapp).

[gregw]

@gregw I have unfortunately in ServletContextRequest.java there is the // TODO we should check if current compliance mode allows all the violations?

That TODO and that section of the code is all about if ambiguous URIs will be returned from the servlet API getPathInfo and getServletPath methods. By default, even if the compliance mode allows ambiguous URI, those methods will throw if you use them. You need either get the getRequestURI API or call servletHandler.setDecodeAmbiguousURIs(true) so that the servlet API will allow bad URIs to be returned.

I've tested that we do act correctly with the following test added to org.eclipse.jetty.ee10.servlet.ComplianceViolations2616Test

    @Test
    public void testAmbiguousSlash() throws Exception
    {
        String request = """
            GET /dump/foo//bar HTTP/1.1\r
            Host: local\r
            Connection: close\r
            \r
            """;

        String response = connector.getResponse(request);
        assertThat(response, containsString("HTTP/1.1 400 Bad"));

        connector.getConnectionFactory(HttpConnectionFactory.class).getHttpConfiguration().setUriCompliance(UriCompliance.RFC3986.with("test", UriCompliance.Violation.AMBIGUOUS_EMPTY_SEGMENT));

        response = connector.getResponse(request);
        assertThat(response, containsString("HTTP/1.1 200 OK"));
    }

[gregw]

I also tested with servletAPI calls in the servlet and used:

    @Test
    public void testAmbiguousSlash() throws Exception
    {
        String request = """
            GET /dump/foo//bar HTTP/1.1\r
            Host: local\r
            Connection: close\r
            \r
            """;

        String response = connector.getResponse(request);
        assertThat(response, containsString("HTTP/1.1 400 Bad"));

        connector.getConnectionFactory(HttpConnectionFactory.class).getHttpConfiguration().setUriCompliance(UriCompliance.RFC3986.with("test", UriCompliance.Violation.AMBIGUOUS_EMPTY_SEGMENT));
        server.getContainedBeans(ServletHandler.class).stream().findFirst().get().setDecodeAmbiguousURIs(true);

        response = connector.getResponse(request);
        assertThat(response, containsString("HTTP/1.1 200 OK"));
        assertThat(response, containsString("GET /dump/foo//bar"));
    }

[salacr]

Thanks a lot @gregw that made it working. We will start process of migration to valid urls but it will take quite a time :/ so in meantime this will help a lot

[amar9292]

@salacr what is the fix for this issue. I am having the same issue with jetty 12.0.14 and tried to add UriCompliance.LEGACY which didn't work and also tried

connector.getConnectionFactory(HttpConnectionFactory.class).getHttpConfiguration().setUriCompliance(UriCompliance.RFC3986.with("test", UriCompliance.Violation.AMBIGUOUS_EMPTY_SEGMENT));

I am initializing jetty programmatically. what is the solution for this duplicate // in the url. I want to fallback to the LEGACY compliance.

[joakime]

Some advice.

• First, try to just use LEGACY as-is.
• Next, make sure you set the UriCompliance on the HttpConfiguration when you create the connector, Don't attempt to modify it after the fact, as those (old) settings have likely propagated deep into child code before you set it.
• If your endpoint is a Servlet, make sure you enable the setDecodeAmbiguousURIs on that ServletContext. (needed for ServletContextHandler and WebAppContext alike)
• If you need to customize the LEGACY mode, use UriCompliance.from(String) (see javadoc)

UriCompliance myCustomUriCompliance = UriCompliance.from("LEGACY,BAD_UTF8_ENCODING");

See examples of this in action:

• jetty-example/ee10-servlet-ambiguous-paths Server, with LEGACY connector, with ServletContext that accepts ambiguous paths
• jetty-example/ee10-servlet-ambiguous-paths Test cases

[amar9292]

@joakime
thanks for you reply. I have tried all the options. the AMBIGUOUS_EMPTY_SEGMENT error is gone, but I am still getting 404 for the url's with double slashes (//) is request path. is there any other setting that avoids getting into 404 ?

[joakime]

@joakime thanks for you reply. I have tried all the options. the AMBIGUOUS_EMPTY_SEGMENT error is gone, but I am still getting 404 for the url's with double slashes (//) is request path. is there any other setting that avoids getting into 404 ?

What is the URL you are using?
Can you setup a testcase and share it as a repo on github?
Or you can just fork jetty-examples, make the changes to the URL in that ee10-servlet-ambiguous-paths project, and demonstrate the issue with your fork.

[amar9292]

@joakime
my application is a spring-web-mvc with embedded jetty. Server is getting Initialized through spring beans(@bean).

I am able to replicate the behaviour with this minimum setup.

`public static void main(String[] args) throws Exception {
    new Main().startJetty(DEFAULT_PORT);
 }

public Server startJetty(int port) throws Exception {
    logger.info("Starting server at port {}", port);
    Server server = new Server();
    server.setHandler(getWebAppContext(getContext()));
    ServerConnector connector = new ServerConnector(server, getHttpConfig());
    connector.setPort(port);
    server.addConnector(connector);
    server.start();
    logger.info("Server started at port {}", port);
    server.join();
    return server;
}

private HttpConnectionFactory getHttpConfig(){
    HttpConfiguration httpConfig = new HttpConfiguration();
    httpConfig.setUriCompliance(UriCompliance.LEGACY);
    HttpConnectionFactory httpConnectionFactory = new HttpConnectionFactory(httpConfig);
    return httpConnectionFactory;
}

public WebAppContext getWebAppContext(WebApplicationContext webApplicationContext){
    WebAppContext webAppContext = new WebAppContext();
    webAppContext.setBaseResourceAsString("webapp");
    webAppContext.addServlet(new ServletHolder(new DispatcherServlet(getContext())),MAPPING_URL);
    webAppContext.setContextPath(CONTEXT_PATH);
    webAppContext.getServletHandler().setDecodeAmbiguousURIs(true);
    return webAppContext;
}

private static WebApplicationContext getContext() {
    AnnotationConfigWebApplicationContext context = new AnnotationConfigWebApplicationContext();
    context.setConfigLocation(CONFIG_LOCATION);
    return context;
}`

and one simple rest controller

` @RestController
public class TestController {

@GetMapping("/rest/topic/get/{id}")
public String getTopic(@PathVariable("id") int id){
    return "Hello";
}

}`

when I access from browser the url http://localhost:8080/rest/topic/get/1 return "Hello".
and the URL http://localhost:8080/rest/topic//get/1 returns
URI: /badURI
STATUS: 400
MESSAGE: Ambiguous URI empty segment

I have added
httpConfig.setUriCompliance(UriCompliance.LEGACY);
and also setDecodeAmbiguous(true)

and the URL http://localhost:8080/rest/topic//get/1 returns 404.

URI: /rest/topic//get/1
STATUS: 404
MESSAGE: No endpoint GET /rest/topic//get/1.
SERVLET: org.springframework.web.servlet.DispatcherServlet-1b083826

[joakime]

Congrats, you have successfully enabled AMBIGUOUS_EMPTY_SEGMENT.

Your Servlet layer is now successfully receiving the empty path segments.

Empty path segments are 100% legal (and common) concept on the web.
It's defined as such in both the HTTP specs and the URL/URI specs.
The core Jetty handlers support this concept easily.

Unfortunately, the Servlet spec has never handled them properly.
And countless years of efforts to bend and twist the servlet spec to allow it to use these kinds of ambiguous and suspicious paths has lead the servlet spec leaders to recognize that it is not possible for the Servlet spec to support ambiguous paths without breaking backward compatibility in horrid ways.

So it was written into the Servlet spec that ambiguous paths are unsupported by the Servlet spec.

See the Servlet Spec, point 10 "Rejecting Suspicious Sequences" in Section 3.5.2. URI Path Canonicalization

All you have done with the work in this issue is to bypass the Servlet spec requirement "If suspicious sequences are discovered during the prior processing steps, the request must be rejected with a 400 bad request rather than dispatched to the target servlet."

The empty path segment is not changed as a result of the configurations you have enabled in here.

The sequence is still bad and ambiguous and the Servlet spec is supposed to reject it instead of operating in a broken mode that causes various issues in existing Servlet APIs (many of which are security related).

Also of note, your REST API is following the REST Spec TCK.
Those suspicious sequences are not supported by your REST API as well.

• TCK Challenge: Rest TCK should not use "suspicious sequence" in URI testing jakartaee/rest#1138

Even Jersey has removed support for suspicious path sequences.
Spring itself has also removed support for suspicious path sequences in many of its components. (Even newer spring client code will throw an exception if you attempt to use a suspicious path without extra configuration to disable that check).

At this point, you have only 1 option, will have to adjust the broken path segments BEFORE they get sent to the Servlet contexts you have.

‼ Important Notice: the steps outlined below is in violation of the HTTP and URL/URI specs! There are webapps that expect and use empty path segments, doing the following will break those webapps.

1. Enable jetty-rewrite.
2. Make sure you have a rewrite ruleset that includes CompactPathRule somewhere early in its list.

That will cause the path to be compacted before being submitted to whatever handler is being wrapped by the RewriteHandler (eg: a Servlet Context).