Fix handling of time exceeded exception in fetch phase #116676
Merged
+268
−20
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
javanna
added
:Search Foundations/Search
elasticsearchmachine
added
the
Team:Search Foundations
|
Pinging @elastic/es-search-foundations (Team:Search Foundations) |
|
Hi @javanna, I've created a changelog YAML for you. |
original-brownbear
approved these changes
Nov 12, 2024
server/src/main/java/org/elasticsearch/search/fetch/FetchPhaseDocsIterator.java
Outdated
Show resolved
Hide resolved
server/src/test/java/org/elasticsearch/action/search/FetchSearchPhaseTests.java
Outdated
Show resolved
Hide resolved
The fetch phase is subject to timeouts like any other search phase. Timeouts may happen when low level cancellation is enabled (true by default), hence the directory reader is wrapped into ExitableDirectoryReader and a timeout is provided to the search request. The exception that is used is TimeExceededException, but it is an internal exception that should never be returned to the user. When that is thrown, we need to catch it and throw error or mark the response as timed out depending on whether partial results are allowed or not.
javanna
force-pushed
the
fix/time_exceeded_fetch
branch
from
5ad7e3e
to
4f38193
Compare
javanna
commented
Nov 12, 2024
| @@ -75,7 +75,7 @@ public void shardResult(SearchHits hits, ProfileResult profileResult) { | |||
|
|
|||
| private static boolean assertNoSearchTarget(SearchHits hits) { | |||
| for (SearchHit hit : hits.getHits()) { | |||
| assert hit.getShard() == null : "expected null but got: " + hit.getShard(); | |||
| assert hit == null || hit.getShard() == null : "expected null but got: " + hit.getShard(); | |||
There was a problem hiding this comment.
@original-brownbear I am not clear if this is introducing a potential bug. With a timeout, we now may have null items in the SearchHit array. I am not entirely sure whether that may cause issues down the line. I see also a similar assertion in the deallocate method that checks that all items are not null. Shall I relax that too or do you have a different idea in mind?
There was a problem hiding this comment.
Right, thanks for pointing this out + sorry for not spotting this.
Currently, this would sort of work but the output is slightly confusing and we'd have to add null checks to the deallocate method. Also UX-wise, the x-content serialization will contain JSON null values in the hits array. We don't need that or even want that even do we?
Maybe we should just keep it simple and in case we run into a timeout create a new array with just the non-null elements? That seems like the low-risk simple solution that doesn't change anything about the SearchHits instances and their users?
There was a problem hiding this comment.
This works well now, I also added a test to cover for the scenario where partial results are returned, those fetched until the timeout error was thrown.
javanna
added a commit
to javanna/elasticsearch
that referenced
this pull request
Nov 18, 2024
The fetch phase is subject to timeouts like any other search phase. Timeouts may happen when low level cancellation is enabled (true by default), hence the directory reader is wrapped into ExitableDirectoryReader and a timeout is provided to the search request. The exception that is used is TimeExceededException, but it is an internal exception that should never be returned to the user. When that is thrown, we need to catch it and throw error or mark the response as timed out depending on whether partial results are allowed or not.
javanna
added a commit
to javanna/elasticsearch
that referenced
this pull request
Nov 18, 2024
The fetch phase is subject to timeouts like any other search phase. Timeouts may happen when low level cancellation is enabled (true by default), hence the directory reader is wrapped into ExitableDirectoryReader and a timeout is provided to the search request. The exception that is used is TimeExceededException, but it is an internal exception that should never be returned to the user. When that is thrown, we need to catch it and throw error or mark the response as timed out depending on whether partial results are allowed or not.
javanna
added a commit
that referenced
this pull request
Nov 18, 2024
The fetch phase is subject to timeouts like any other search phase. Timeouts may happen when low level cancellation is enabled (true by default), hence the directory reader is wrapped into ExitableDirectoryReader and a timeout is provided to the search request. The exception that is used is TimeExceededException, but it is an internal exception that should never be returned to the user. When that is thrown, we need to catch it and throw error or mark the response as timed out depending on whether partial results are allowed or not.
javanna
added a commit
that referenced
this pull request
Nov 18, 2024
The fetch phase is subject to timeouts like any other search phase. Timeouts may happen when low level cancellation is enabled (true by default), hence the directory reader is wrapped into ExitableDirectoryReader and a timeout is provided to the search request. The exception that is used is TimeExceededException, but it is an internal exception that should never be returned to the user. When that is thrown, we need to catch it and throw error or mark the response as timed out depending on whether partial results are allowed or not.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The fetch phase is subject to timeouts like any other search phase. Timeouts may happen when low level cancellation is enabled (true by default), hence the directory reader is wrapped into ExitableDirectoryReader and a timeout is provided to the search request.
The exception that is used is TimeExceededException, but it is an internal exception that should never be returned to the user. When that is thrown, we need to catch it and throw error or mark the response as timed out depending on whether partial results are allowed or not.