v1.10.0-rc.3 🦩

What's Changed

• also intersect the universe for searchOnAttributes by @dureuill in #4858
• Don't remove facet value when multiple original values map to the same normalized value by @dureuill in #4864
• Fix perf regression facet strings by @dureuill in #4845
• Infer locales from index settings by @ManyTheFish in #4881

v1.10.0-rc.2

⚠️ Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports and feedback about new features.

What's Changed

Bugs fixes 🐞

Use a fixed date format regardless of features

If you were compiling Meilisearch yourself, we found out that using one version compiled from the root of the repository and then one from the meilisearch directory could cause database corruption.

Fixed by @dureuill in #4850

Fix edit by function deletion of documents

When deleting documents from the new edit by function route, if you had more than one thread, some documents would randomly not be deleted.

Fixed by @dureuill in #4853

Maintainance 🐢

• Add OpenAI tests by @dureuill in #4846

Full Changelog: v1.10.0-rc.1...v1.10.0-rc.2

v1.10.0-rc.1 🦩

⚠️ Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports and feedback about new features.

Bug fixes 🪲

• Add missing PUT and GET /indexes/{indexUid}/settings/localized-attributes routes by @dureuill in #4836

Other improvements

• Errors emitted at the main level of the Meilisearch binary are now logged with level ERROR by @dureuill in #4835

Full Changelog

v1.10.0-rc.0 🦩

Warning

Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports and feedback about new features.

With Meilisearch v1.10 we keep innovating by introducing the really demanded federated search! You can now apply multi-search requests and get one single list of results 🎉. This version also includes a setting to define your index languages (even if multiple languages are in your documents!), and new experimental features like the CONTAINS operator and the ability to update a subset of your dataset by using a simple function.

New features and updates 🔥

Federated search

By using the POST /multi-search endpoint, you can now return a single search result object, whose list of hits is built by merging the hits coming from all the queries in descending ranking score order.

curl \
 -X POST 'http://localhost:7700/multi-search' \
 -H 'Content-Type: application/json' \
 --data-binary '{
  "federation": {
    "offset": 5,
    "limit": 10,
 }
  "queries": [
 {
      "q": "Batman",
      "indexUid": "movies"
 },
 {
      "q": "Batman",
      "indexUid": "comics"
 }
 ]
}'

Response:

{
  "hits": [
    {
      "id": 42,
      "title": "Batman returns",
      "overview": "..",
      "_federation": {
        "indexUid": "movies",
        "queriesPosition": 0
      }
    },
    {
      "comicsId": "batman-killing-joke",
      "description": "..",
      "title": "Batman: the killing joke",
      "_federation": {
        "indexUid": "comics",
        "queriesPosition": 1
      }
    },
    ...
 ],
  processingTimeMs: 0,
  limit: 20,
  offset: 0,
  estimatedTotalHits: 2,
  semanticHitCount: 0,
}

If federation is empty ({}) default values of offset and limit are used, so respectively 0 and 20.
If federation is null or missing, a classic multi-search will be applied, so a list of search result objects for each index will be returned.

To customize the relevancy and the weight applied to each index in the search result, use the federationOptions parameter in your request:

curl \
 -X POST 'http://localhost:7700/multi-search' \
 -H 'Content-Type: application/json' \
 --data-binary '{
  "federation": {},
  "queries": [
    {
      "q": "apple red",
      "indexUid": "fruits",
      "filter": "BOOSTED = true",
      "_showRankingScore": true,
      "federationOptions": {
        "weight": 3.0
      }
    },
    {
      "q": "apple red",
      "indexUid": "fruits",
      "_showRankingScore": true,
    }
  ]
}'

weight must be positive (>=0)

• if < 1.0, the hits from this query are less likely to appear in the results.
• if > 1.0, the hits from this query are more likely to appear in the results.
• if missing, the default value is applied (1.0)

📖 More information about the merge algorithm on the here.

Done by @dureuill in #4769.

Experimental: CONTAINS filter operator

Enabling the experimental feature will make a new CONTAINS operator available while filtering on strings.
This is similar to the SQL LIKE operator used with %.

Activate the experimental feature:

curl \
 -X PATCH 'http://localhost:7700/experimental-features/' \
  -H 'Content-Type: application/json' \
 --data-binary '{
    "containsFilter": true
 }'

Use the newly introduced CONTAINS operator:

curl \
 -X POST http://localhost:7700/indexes/movies/search \
  -H 'Content-Type: application/json' \
 --data-binary '{
      "q": "super hero",
      "filter": "synopsis NOT CONTAINS spider"
 }'

🗣️ This is an experimental feature, and we need your help to improve it! Share your thoughts and feedback on this GitHub discussion.

Done by @irevoire in #4804.

Languages settings

You can now set up the language of your index in your settings and during the search. This will prevent users from using alternative Meilisearch images we were separately created until now (like for Swedish and Japanese)

Done by @ManyTheFish in #4819.

Index settings

Use the newly introduced localizedAttributes setting (here is an example of handling multi-language documents):

curl \
  -X PATCH 'http://localhost:7700/indexes/movies/settings' \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "localizedAttributes": [
      {"locales": ["jpn"], "attributePatterns": ["*_ja"]},
      {"locales": ["eng"], "attributePatterns": ["*_en"]},
      {"locales": ["cmn"], "attributePatterns": ["*_zh"]},
      {"locales": ["fra", "ita"], "attributePatterns": ["latin.*"]},
      {"locales": [], "attributePatterns": ["*"]}
    ]
}'

locales is a list of language codes to assign to a pattern, the supported codes are: epo, eng, rus, cmn, spa, por, ita, ben, fra, deu, ukr, kat, ara, hin, jpn, heb, yid, pol, amh, jav, kor, nob, dan, swe, fin, tur, nld, hun, ces, ell, bul, bel, mar, kan, ron, slv, hrv, srp, mkd, lit, lav, est, tam, vie, urd, tha, guj, uzb, pan, aze, ind, tel, pes, mal, ori, mya, nep, sin, khm, tuk, aka, zul, sna, afr, lat, slk, cat, tgl, hye.

attributePattern is a pattern that can start or end with a * to match one or several attributes.

"locales": [], "attributePatterns": ["*"] means the is the default rule.

Notes:

• if an attribute matches several rules, only the first rule in the list will be applied
• if the locales list is empty, then Meilisearch is allowed to auto-detect any language in the matching attributes
• These rules are applied to the searchableAttributes, the filterableAttributes, and the sortableAttributes.

At search time

The search route accepts a new parameter, locales allowing the end-user to define the language used in the current query:

curl \
 -X POST http://localhost:7700/indexes/movies/search \
  -H 'Content-Type: application/json' \
 --data-binary '{
  "q": "進撃の巨人",
  "locales": ["jpn"]
}'

The locales parameter overrides eventual locales in the index settings.

Experimental: edit documents by using a function

You can edit documents by executing a Rhai function on all the documents of your database or a subset of them that you can select by a Meilisearch filter.

Activate the experimental feature:

curl \
 -X PATCH 'http://localhost:7700/experimental-features/' \
  -H 'Content-Type: application/json' \
 --data-binary '{
    "editDocumentsByFunction": true
 }'

By indexing this movies dataset, you can run the following Rhai function on all of them. This function will uppercase the titles of the movies with an id > 3000 and add sparkles around it. Rhai templating syntax is applied here:

curl http://localhost:7700/indexes/movies/documents/edit \
 -H 'content-type: application/json' \
  -d '{
 "filter": "id > 3000",
 "function": "doc.title = `✨ ${doc.title.to_upper()} ✨`"
 }'

📖 More information here.

🗣️ This is an experimental feature and we need your help to improve it! Share your thoughts and feedback on this GitHub discussion.

Done by @Kerollmops in #4626.

Experimental AI-powered search: quality of life improvements

For the purpose of future stabilization of the feature, we are applying changes and quality-of-life improvements.

Done by @dureuill in #4801, #4815, #4818, #4822.

⚠️ Breaking changes - changing the parameters of the REST API

The old parameters of the REST API are too numerous and confusing.

Removed parameters: query , inputField, inputType, pathToEmbeddings and embeddingObject.
Replaced by

• request : A JSON value that represents the request made by Meilisearch to the remote embedder. The text to embed must be replaced by the placeholder value “{{text}}”.
• response: A JSON value that represents a fragment of the response made by the remote embedder to Meilisearch. The embedding must be replaced by the placeholder value "{{embedding}}".

Before:

// v1.9 (old) version
{
  "source": "rest",
  "url": "https://localhost:10006",
  "query": {
    "model": "minillm",
 },
  "inputField": ["prompt"],
  "inputType": "text",
  "embeddingObject": ["embedding"]
}

// v1.10 (new) version
{
  "source": "rest",
  "url": "https://localhost:10006",
  "request": {
    "model": "minillm",
    "prompt": "{{text}}"
 },
  "response": {
    "embedding": "{{embedding}}"
 }
}

Caution

This is a breaking change to the configuration of REST embedders.
Importing a dump containing a REST embedder configuration will fail in v1.10 with an error: "Error: unknown field query, expected one of source, model, revision, apiKey, dimensions, documentTemplate, url, request, response, distribution at line 1 column 752".

Upgrade procedure (Cloud):

1. Remove any embedder with source "rest"
2. Follow the usual steps described [here in the documentation](https://www.meilisearch.com/docs/lear...

v1.8.4 🪼

Improvements

• Generate vectors in dumps by @dureuill in #4796
  • this version provides an upgrade path to v1.9 without regenerating embeddings for autogenerating embedders.
  • to upgrade from v1.8.x to v1.9.0 without regenerating embeddings, please follow the following procedure:
    1. Upgrade from v1.8.x to v1.8.4 by simply restarting Meilisearch v1.8.4 on you v1.8.x DB. No dump is necessary at this step
    2. Create a dump.
    3. Import the created dump in a Meilisearch v1.9

Full Changelog

v1.9.0 🦎

Meilisearch v1.9 includes performance improvements for hybrid search and the addition/updating of settings. This version benefits from multiple requested features, such as the new frequency matching strategy and the ability to retrieve similar documents.

🧰 All official Meilisearch integrations (including SDKs, clients, and other tools) are compatible with this Meilisearch release. Integration deployment happens between 4 to 48 hours after a new version becomes available.

Some SDKs might not include all new features. Consult the project repository for detailed information. Is a feature you need missing from your chosen SDK? Create an issue letting us know you need it, or, for open-source karma points, open a PR implementing it (we'll love you for that ❤️).

New features and updates 🔥

Hybrid search updates

This release introduces multiple hybrid search updates.

Done by @dureuill and @irevoire in #4633 and #4649

⚠️ Breaking change: Empty _vectors.embedder arrays

Empty _vectors.embedder arrays are now interpreted as having no vector embedding.

Before v1.9, Meilisearch interpreted these as a single embedding of dimension 0. This change follows user feedback that the previous behavior was unexpected and unhelpful.

⚠️ Breaking change: _vectors field no longer present in search results

When the experimental vectorStore feature is enabled, Meilisearch no longer includes _vectors in returned search results by default. This will considerably improve performance.

Use the new retrieveVectors search parameter to display the _vectors field:

curl \
  -X POST 'http://localhost:7700/indexes/INDEX_NAME/search' \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "q": "SEARCH QUERY",
    "retrieveVectors": true
  }'

⚠️ Breaking change: Meilisearch no longer preserves the exact representation of embeddings appearing in _vectors

In order to save storage and run faster, Meilisearch is no longer storing your vector "as-is". Meilisearch now returns the float in a canonicalized representation rather than the user-provided representation.

For example, 3 may be represented as 3.0

Document _vectors accepts object values

The document _vectors field now accepts objects in addition to embedding arrays:

{
  "id": 42,
  "_vectors": {
    "default": [0.1, 0.2 ],
    "text": {
      "embeddings": [[0.1, 0.2, 0.3], [0.4, 0.5, 0.6]],
      "regenerate": false
    },
    "translation": {
      "embeddings": [0.1, 0.2, 0.3, 0.4],
      "regenerate": true
    }
  }
}

The _vectors object may contain two fields: embeddings and regenerate.

If present, embeddings will replace this document's embeddings.

regenerate must be either true or false. If regenerate: true, Meilisearch will overwrite the document embeddings each time the document is updated in the future. If regenerate: false, Meilisearch will keep the last provided or generated embeddings even if the document is updated in the future.

This change allows importing embeddings to autoembedders as a one-shot process, by setting them as regenerate: true. This change also ensures embeddings are not regenerated when importing a dump created with Meilisearch v1.9.

Meilisearch v1.9.0 also improves performance when indexing and using hybrid search, avoiding useless operations and optimizing the important ones.

New feature: Ranking score threshold

Use rankingScoreThreshold to exclude search results with low ranking scores:

curl \
 -X POST 'http://localhost:7700/indexes/movies/search' \
 -H 'Content-Type: application/json' \
 --data-binary '{
    "q": "Badman dark returns 1",
    "showRankingScore": true,
    "limit": 5,
    "rankingScoreThreshold": 0.2
 }'

Meilisearch does not return any documents below the configured threshold. Excluded results do not count towards estimatedTotalHits, totalHits, and facet distribution.

⚠️ For performance reasons, if the number of documents above rankingScoreThreshold is higher than limit, Meilisearch does not evaluate the ranking score of the remaining documents. Results ranking below the threshold are not immediately removed from the set of candidates. In this case, Meilisearch may overestimate the count of estimatedTotalHits, totalHits and facet distribution.

Done by @dureuill in #4666

New feature: Get similar documents endpoint

This release introduces a new AI-powered search feature allowing you to send a document to Meilisearch and receive a list of similar documents in return.

Use the /indexes/{indexUid}/similar endpoint to query Meilisearch for related documents:

curl \
  -X POST /indexes/:indexUid/similar
  -H 'Content-Type: application/json' \
  --data-binary '{
    "id": "23",
    "offset": 0,
    "limit": 2,
    "filter": "release_date > 1521763199",
    "embedder": "default",
    "attributesToRetrieve": [],
    "showRankingScore": false,
    "showRankingScoreDetails": false
  }'

• id: string indicating the document needing similar results, required
• offset: number of results to skip when paginating, optional, defaults to 0
• limit: number of results to display, optional, defaults to 20
• filter: string with a filter expression Meilisearch should apply to the results, optional, defaults to null
• embedder: string indicating the embedder Meilisearch should use to retrieve similar documents, optional, defaults to "default"
• attributesToRetrieve: array of strings indicating which fields Meilisearch will include in the response, optional, defaults to ["*"]
• showRankingScore: boolean indicating if results should include ranking score information, optional, defaults to false
• showRankingScoreDetails: boolean indicating if results should include detailed ranking score information, optional, defaults to false
• rankingScoreThreshold: Excludes search results with a ranking score lower than the defined number, optional, defaults to null.

/indexes/{indexUid}/similar supports GET and POST routes. Use URL query parameters to configure your GET request, or include your parameters in the request body if using the POST route. Both offer identical functionality.

Done by @dureuill in #4647

New feature: frequency matching strategy

This release adds a new matching strategy, frequency. Use it to prioritize results containing the least frequent query terms:

curl \
 -X POST 'http://localhost:7700/indexes/{index_uid}/search' \
 -H 'Content-Type: application/json' \
 --data-binary '{
    "q": "cheval blanc",
    "matchingStrategy": "frequency"
 }'

Done by @ManyTheFish in #4667

Set distinctAttribute at search time

This release introduces a new search parameter: distinct which you can use to specify the distinct attribute at search time:

curl \
 -X POST 'http://localhost:7700/indexes/{index_uid}/search' \
 -H 'Content-Type: application/json' \
 --data-binary '{
    "q": "kefir le double poney",
    "distinct": "book.isbn"
 }'

If a distinct attribute is already defined in the settings it'll be ignored in favor of the one defined at search time.

Done by @Kerollmops in #4693

Improve indexing speed when updating/adding settings

Meilisearch now limits operations when importing settings by avoiding unnecessary writing operations in its internal database and reducing disk usage.

Additionally, when changing embedding settings, Meilisearch will now only regenerate the embeddings for the embedders whose settings have been modified, instead of for all embedders. When only the documentTemplate is modified, embeddings will only be regenerated for documents where this modification leads to a different text to embed.

Done by @irevoire, @Kerollmops, @ManyTheFish and @dureuill in #4646, #4680, #4631 and #4649

Other improvements

• Speed up filter ANDs operations during the search (#4682) @Kerollmops
• Speed up facet distribution during the search (#4713) @Kerollmops
• Improve language support (#4684) @ManyTheFish @Soham1803 @mosuka @tkhshtsh0917
  • Add new normalizer to normalize œ to oe and æ to ae
  • Fix chinese-normalization-pinyin feature flag compilation
• Prometheus experimental feature: Use HTTP path pattern instead of full path in metrics (#4619) @gh2k
• ⚠️ Remove exportPuffinReport experimental feature. Use logs routes and logs modes instead (#4655) @Kerollmops

Fixes 🐞

• All fields now have the same impact on relevancy when searchableAttributes: ["*"]. Consult the GitHub issue for a detailed breakdown of these changes (#4631) @irevoire
• Fix searchableAttributes behavior when handling nested fields. Consult the GitHub issue for more information (#4631) @irevoire
• Fix security issue in dependency: bump Rustls to non-vulnerable versions (#4622) @Kerollmops
• Reset other embedding settings when changing the source of an embedder. This prevents misleading error messages when configuring the embedders (#4649) @dureuill
• Fix panic in hybrid search when removing all embedders from the DB (#4715) @irevoire
• Hybrid search now respects the offset and limit parameters when returning keyword results earl...

v1.9.0-rc.5 🦎

⚠️ Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports and feedback about new features.

Bug fixes 🪲

• Fix hybrid search limit offset by @dureuill in #4746
• Make embeddings optional and improve error message for regenerate by @irevoire in #4740

❤️ Thanks to @inventor123 for first reporting #4745

Full Changelog

v1.9.0-rc.4 🦎

⚠️ Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports and feedback about new features.

Bug fixes 🪲

• Fix memory leak @dureuill in #4710 -- This issue was also fixed in v1.8.3
• Fix panic in hybrid search when removing all embedders from the DB @irevoire in #4715

Improvements

• Update mini-dashboard to 2.14 by @curquiza in #4712
• Speed up facet distribution by @Kerollmops in #4713

❤️ Thanks to @sam-ulrich1 for reporting the panic in hybrid search in #4588

Full Changelog

v1.8.3 🪼

Fixes 🪲

• Fix memory leak @dureuill in #4707

❤️ Thanks @irevoire for reproducing the issue and checking fixes

❤️ Thanks to Tater, Hannsr, Martin from the Discord thread, and @doutatsu on GH for the report and helping with the investigation

v1.9.0-rc.3 🦎

⚠️ Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports and feedback about new features.

Bug fixes

• Fix a meilisearch freeze that could happen under heavy search loads by @dureuill in #4681 -- Note that this bug is already fixed in Meilisearch v1.8.2

Breaking changes

• The _vectors field is not returned anymore when retrieving documents; you must use the retrieveVector parameter instead
• When retrieving the _vectors field with the retrieveVector parameter, their embeddings are not returned "as-is"; they'll always be returned with the maximum precision
• When specifying or retrieving vectors, the userProvided field has been removed in favor of a new regenerate field that better represents your intent. When set to true it means the embeddings will be regenerated on every change to the document (default behavior). If set to false the embeddings will never be updated by the engine.
• Dumps with embeddings created from previous RCs cannot be imported into the new RC

Improvements

• Speed Up Filter ANDs operations by @Kerollmops in #4682
• Speedup the vector store and reduce the size of the database by @irevoire and @dureuill in #4649
• Define your distinct attributes at search time by @Kerollmops in #4693

Misc

• Fix ci tests by @ManyTheFish in #4685

Full Changelog: v1.9.0-rc.2...v1.9.0-rc.3