Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add default OutputAdapter filters (post 2.0) #7078

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

vblagoje
Copy link
Member

@vblagoje vblagoje commented Feb 23, 2024

Why:

Let's keep this PR open and continue collecting useful filters that keep repeating. Whenever you find yourself reusing some filter over and over again - add it here.

With these filters, developers can seamlessly adjust values being passed between components, ensuring the Haystack components receive data in suitable formats.

The first batch of standard filters includes:

  • 'change_role' - adjusting chat message roles
  • 'prepare_fc_params' - preparing function call parameters
  • 'tojson' - converting JSON strings to Python dictionaries.

What:

The pull request introduces the output_adapter_filters.py module that offers a set of custom filters for the OutputAdapter. The ALL_FILTERS dictionary contains these newly-defined filters and is included in test_output_adapter.py.

How can it be used:

The filters can be employed in pipelines by adding custom filters to the OutputAdapter instance in custom_filters. All filters can be used simultaneously or selectively. When an OutputAdapter instance is serialized with custom filters, the serialization includes custom filter definitions, ensuring the filters' persistency across various execution environments.

How did you test it:

Unit tests were executed to examine the serialization and deserialization of an OutputAdapter instance with the ALL_FILTERS dictionary added as custom filters. Assertions validate the custom filters' existence in both the serialized and deserialized OutputAdapter instances.

Notes for the reviewer:

When utilizing filters in production environments, sanitize and validate inputs carefully, especially if data originates from untrusted sources, to minimize potential security concerns.

@github-actions github-actions bot added topic:tests 2.x Related to Haystack v2.0 type:documentation Improvements on the docs labels Feb 23, 2024
@coveralls
Copy link
Collaborator

coveralls commented Feb 23, 2024

Pull Request Test Coverage Report for Build 8171735291

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage decreased (-0.06%) to 89.98%

Totals Coverage Status
Change from base Build 8156947262: -0.06%
Covered Lines: 5298
Relevant Lines: 5888

💛 - Coveralls

@vblagoje vblagoje force-pushed the add_default_adapter_filters branch from 58ae2f5 to 72eca8a Compare March 6, 2024 10:08
@vblagoje vblagoje added this to the 2.1.0 milestone Mar 6, 2024
@vblagoje vblagoje changed the title feat: Add default OutputAdapter filters feat: Add default OutputAdapter filters (post 2.0) Mar 6, 2024
@vblagoje vblagoje marked this pull request as ready for review March 6, 2024 12:04
@vblagoje vblagoje requested review from a team as code owners March 6, 2024 12:04
@vblagoje vblagoje requested review from dfokina and masci and removed request for a team March 6, 2024 12:04
@masci masci modified the milestones: 2.1.0, 2.2.0 Apr 7, 2024
@masci masci marked this pull request as draft May 15, 2024 19:57
@masci masci removed request for dfokina and masci May 15, 2024 19:57
@masci masci removed this from the 2.2.0 milestone May 25, 2024
@vblagoje vblagoje marked this pull request as ready for review June 21, 2024 14:57
@vblagoje vblagoje marked this pull request as draft June 21, 2024 14:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.x Related to Haystack v2.0 topic:tests type:documentation Improvements on the docs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants