feat: Image folder processing and load balancing. #103
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Only added to v4. Also some smaller fixes of typos and typing.
You can now specify multiple redact_url parameter, and each will have a parallel_jobs number of parallel processes run against it.
A temporary file was not created in the intended directory.
When handling video as image folders batching is used, with 1500 images by default. Also fixes for: * Cleaning of output directory when unfinished. * Creation of output tar in correct temp directory.
georgkoester
commented
Apr 11, 2024
| @@ -37,14 +38,16 @@ def redact_folder( | |||
| service: ServiceType, | |||
| job_args: Optional[JobArguments] = None, | |||
| licence_plate_custom_stamp_path: Optional[str] = None, | |||
| redact_url: str = settings.redact_url_default, | |||
| redact_url: Union[str, List[str]] = [settings.redact_url_default], | |||
There was a problem hiding this comment.
Hi @pkern90 , here you can see that the new multi-url feature for client-side load balancing was implemented in a backwards-compatible fashion.
Before it was generally applied, now the limit of 2 post requests applies to the host:port combination, allowing for more parallel requests if client-side load-balancing is employed.
|
Document the impact on consumption of using batches: Every batch counts, that means aborting before all batches are finished leaves you with some consumption having already been recorded. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add processing for videos provided as image folders. Only added to v4.
You can now specify multiple redact_url parameter, and each
will have a parallel_jobs number of parallel processes run
against it. Only added to v4.
Also some smaller fixes of typos and typing.