feat: Add default OutputAdapter filters (post 2.0) #7078
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why:
Let's keep this PR open and continue collecting useful filters that keep repeating. Whenever you find yourself reusing some filter over and over again - add it here.
With these filters, developers can seamlessly adjust values being passed between components, ensuring the Haystack components receive data in suitable formats.
The first batch of standard filters includes:
What:
The pull request introduces the
output_adapter_filters.py
module that offers a set of custom filters for theOutputAdapter
. TheALL_FILTERS
dictionary contains these newly-defined filters and is included intest_output_adapter.py
.How can it be used:
The filters can be employed in pipelines by adding custom filters to the
OutputAdapter
instance incustom_filters
. All filters can be used simultaneously or selectively. When anOutputAdapter
instance is serialized with custom filters, the serialization includes custom filter definitions, ensuring the filters' persistency across various execution environments.How did you test it:
Unit tests were executed to examine the serialization and deserialization of an
OutputAdapter
instance with theALL_FILTERS
dictionary added as custom filters. Assertions validate the custom filters' existence in both the serialized and deserializedOutputAdapter
instances.Notes for the reviewer:
When utilizing filters in production environments, sanitize and validate inputs carefully, especially if data originates from untrusted sources, to minimize potential security concerns.