Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MemoryResource #8451

Merged
merged 3 commits into from
Aug 12, 2022
Merged

MemoryResource #8451

merged 3 commits into from
Aug 12, 2022

Conversation

gregw
Copy link
Contributor

@gregw gregw commented Aug 11, 2022

Added MemoryResource to simplify favicon handling.
Using the ResourceFactory mechanism results in an entire jar mounted as a ZipFS in memory that is only needed for favicon.ico

Added MemoryResource to simplify favicon handling
@gregw gregw requested review from lorban and joakime August 11, 2022 07:45
Copy link
Contributor

@lorban lorban left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea, but I'm very concerned by this implementation's shortcomings. They're solvable, so I think we should not cut corners.

@Override
public Resource resolve(String subUriPath)
{
throw new UnsupportedOperationException();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolve() should never throw, otherwise we're going to have to add try / catch blocks everywhere we call it. I'm not even sure we should let it return null, so it feels like the right thing to do here is to return this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the wrong thing to return.
We don't want these resources slipping into general usage (not unless we implement an in memory filesystem... and hey that is what ZipFS is), so I'm OK with an unchecked exception. If one of these resources ever ends up being used for a resolve, then that is a bad state and should fail.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It has a URI, so I'm now just defering to the default resolve. You wont get back a memory resource, but that is fine.

@Override
public Path getPath()
{
return null;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So far, code has been written that assumes Resource.getPath() cannot return null, and maybe there's some code that still calls File.newInputStream(resource.getPath()) instead of resource.getInputStream().

I think we should arrange for this implementation of getPath() to never return null, maybe by creating a simple in-memory filesystem?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe by creating a simple in-memory filesystem?

heh, uh, this is a BIG undertaking.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lorban Well every time I see anybody saying that getPath cannot return null, I've said "oh yes it can". If getPath could always return non-null then we would have succeeded in replacing Resource with Path as there would be a 1:1 mapping. But there is not. ResourceCollection is one such resource that cannot always return a Path.

I've been on the lookout for code that makes that kind of assumption (eg getting the Path and using it to get last modified, rather than just asking the resource). We need to remove such assumptions. We should javadoc that getPath can return null.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe by creating a simple in-memory filesystem?

heh, uh, this is a BIG undertaking.

Yeah - that's why I want this to be just a single resource - no lists, no directories, no resolve.

try
{
_uri = Objects.requireNonNull(url).toURI();
_bytes = IO.readBytes(url.openStream());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're leaking a file descriptor.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe IO closes it, by I'll use a try with resources.

@joakime
Copy link
Contributor

joakime commented Aug 11, 2022

Whatever you decide for favicon.ico should also be done for jetty-dir.css they exist in the same jar and have the same basic issues you are attempting to solve here.

The way it's handled in Jetty 12 now is that those exist as mounts on the Server object, which already has the jetty-server-<ver>.jar mounted for other things. (it gets accessed for every webapp for the bytecode scanning steps for example)

@gregw
Copy link
Contributor Author

gregw commented Aug 11, 2022

Whatever you decide for favicon.ico should also be done for jetty-dir.css they exist in the same jar and have the same basic issues you are attempting to solve here.

I'm doing this for Server.newResource, so we can change the mechanism. I think the stylesheet already uses that same mechanism.

Updates from review
@gregw gregw requested a review from lorban August 11, 2022 22:12
@gregw gregw mentioned this pull request Aug 11, 2022
@joakime
Copy link
Contributor

joakime commented Aug 11, 2022

Wouldn't it be easier to copy those to the work dir and serve them via a normal file?

@gregw
Copy link
Contributor Author

gregw commented Aug 11, 2022

Wouldn't it be easier to copy those to the work dir and serve them via a normal file?

Potentially, but remember our start times are important as well. We don't want to mount a zip file, extract favicon, unmount it, mount it again, extract style sheet, unmount etc. So that ends up with the result we want but will take a lot of CPU cycles at start time.

These are classloader resources already available via the classloader getResource mechanism... unfortunately via a URL. So we just want to quickly grab the contents via the existing JVM mechanism and make them available.

@joakime
Copy link
Contributor

joakime commented Aug 11, 2022

The unmount only occurs at server.stop() for those resources

@gregw
Copy link
Contributor Author

gregw commented Aug 11, 2022

@joakime if we use the zipFS bases mechanism to get these resources, then either we leave it mounted whilst running (wasting memory) or we unmount after extracting (wasted CPU). There is already a perfectly good (modulo URL API) mechanism to get classpath resources. We should use that and not re-crack open the jar file. The zipFS mechanism behind Resource is great and we should use it... but only when appropriate (eg the deployer has asked to not unpack a war file). We should not use it when a better mechanism already exists.

@gregw
Copy link
Contributor Author

gregw commented Aug 11, 2022

@joakime @lorban I'm inclined to merge this, even if not perfect. The current favicon stuff has been breaking the build for a few days due to mount leaks, so this will at least avoid that problem.

@gregw gregw merged commit e249d39 into jetty-12.0.x Aug 12, 2022
@gregw gregw deleted the jetty-12.0.x-memoryResource branch August 12, 2022 00:36
@lorban
Copy link
Contributor

lorban commented Aug 12, 2022

This is adequate for now, but we should come up with an agreement for the long term. That discussion should happen in #8443.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants