v1.12.8 🦗

What's Changed

• Accept the max readers param by env var and reduce rayon tasks by @Kerollmops with the help of @dureuill in #5294

Full Changelog: v1.12.7...v1.12.8

v1.13.0-rc.0

v1.13.0 release changelogs

Warning

Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports and feedback about new features.

Meilisearch v1.13 introduces several significant improvements, including stabilizing the Vector Store feature for seamless hybrid search integration! 🎉 This version also simplifies the Meilisearch upgrade process: you don't need a dump for migrating anymore!

New features and updates 🔥

Stabilize Vector Store feature

After its initial release in v1.3.0, we have now fully stabilized the Vector Store feature. The experimental feature no longer requires manual activation, and the vectorStore field is no longer displayed or accepted by the /experimental-features route. This makes it even simpler to utilize our hybrid search capability, which delivers significantly better search relevance by combining traditional full text search with AI-powered semantic search.

Breaking Changes before Stabilization

• Accepted Ollama URLs can only end with /api/embed and /api/embeddings.
• Modified error codes:
  • invalid_embedder has been split into invalid_search_embedder and invalid_similar_embedder. These codes are returned when the embedder parameter of a search (resp. similar) request refers to a non-existing embedder configuration or is not a string.
  • invalid_hybrid_query has been renamed to invalid_search_hybrid_query. It is returned when the hybrid parameter is invalid: contains unknown keys or is not either null or an object.

Done by @dureuill in #5232 & #5234

Experimental Dumpless Upgrade: Ease upgrading to the next version without a dump

From now on you can upgrade any database in the v1.12 version or more to the latest version without using a dump.
That means it's an in-place, way faster upgrade process that consumes only the minimal amount of RAM or disk required.

To upgrade your v1.12 database to v1.13 runs:

./meilisearch --experimental-dumpless-upgrade

Read more about it here.

Done by @irevoire and @dureuill in #5264

Other improvements

• Improve the auto-batching of tasks by exposing a way to limite the total size of a batch by @Kerollmops in #5223
• Related to Prometheus the experimental feature: Add Prometheus metrics to measure the task queue latency by @takaebato in #5178
• Speed up the request to list the indexes by @irevoire in #5166
• Add support for GITHUB_TOKEN authentication in installation script by @Sherlouk in #5216

Fixes 🐞

• Improve the error message when an attributes is not filterable by @jameshiew in #5135

Misc

• Dependencies updates
  • Upgrade dependencies and fixes the idna severity issue by @Kerollmops in #5218
• CIs and tests
  • Split Meilisearch crate tests in separate file by @K-Kumar-01 in #5134
  • Split tests in separate file by @K-Kumar-01 in #5171
  • Remove obsolete test code by @K-Kumar-01 in #5173
  • Fix the flaky batches test by @irevoire in #5175
  • Split tests for option crate meilisearch in a separate test file by @K-Kumar-01 in #5174
  • Remove hard coded task IDs to prevent flaky tests by @mhmoudr in #5182
• Misc
  • Guide people to create custom reports on the benchboard by @Kerollmops in #5029
  • Fix typo in a comment by @eltociear in #5184
  • Replace hardcoded string with constants by @Gnosnay in #5169
  • Refactor the index-scheduler by @irevoire in #5199
  • Refactor indexer by @dureuill in #5168
  • Auto-generate the OpenAPI spec by @irevoire in #4867 & #5231
  • Merge bitmaps by using Extend::extend by @Kerollmops in #5221
  • Fix corrupted task queue errors on index creation by @irevoire in #5239

❤️ Thanks again to our external contributors:

• Meilisearch: @dureuill, @Kerollmops, @takaebato, @irevoire, @Sherlouk, @jameshiew, @K-Kumar-01, @mhmoudr, @eltociear, @Gnosnay.

v1.12.7 🦗

What's Changed

• Fix the stuck indexation due to the internal BBQueue capacity by @Kerollmops and @dureuill in #5276

Full Changelog: v1.12.6...v1.12.7

v1.12.6 🦗

What's Changed

• Unify facet strings by their normalized value by @dureuill and @Kerollmops in #5258

Full Changelog: v1.12.5...v1.12.6

v1.12.5 🦗

Fixes 🪲

• Dump export no longer fails when the task queue contains enqueued or processing tasks, by @dureuill and @ManyTheFish in #5246
• Dumps containing enqueued tasks no longer fail to import, also in #5246
• Dumps exported in v1.12 reset the batch_uid of all tasks, as batches are not persisted in dumps in v1.12. See #5247 for details.

Full Changelog: v1.12.4...v1.12.5

v1.12.4 🦗

Fixes 🪲

• The indexing process no longer starts an infinite loop in some circumstances by @dureuill in #5242
• The indexing process no longer fails task when attempting to delete keys that are not in the DB, also in #5242

Full Changelog: v1.12.3...v1.12.4

v1.12.3 🦗

What's Changed

• Field distribution is no longer miscomputed when sending incremental updates to documents via the PUT route by @dureuill in #5224
• Improve the panic message when deleting an unknown entry by @Kerollmops in #5229

Full Changelog: v1.12.2...v1.12.3

v1.12.2 🦗

🪲 Fixes

• Fix a performance regression introduced in v1.12.0 when updating a small number of documents in indexes that contain sortableAttributes or filterableAttributes. By @dureuill in #5205

Full Changelog: v1.12.1...v1.12.2

v1.12.1

Fixes

There was a bug in the engine when adding an empty payload, it was making the batch fails.
Fixed by @irevoire in #5192

Full Changelog: v1.12.0...v1.12.1

v1.12.0 🦗

Meilisearch v1.12 introduces significant indexing speed improvements, almost halving the time required to index large datasets. This release also introduces new settings to customize and potentially further increase indexing speed.

🧰 All official Meilisearch integrations (including SDKs, clients, and other tools) are compatible with this Meilisearch release. Integration deployment happens between 4 to 48 hours after a new version becomes available.

Some SDKs might not include all new features. Consult the project repository for detailed information. Is a feature you need missing from your chosen SDK? Create an issue letting us know you need it, or, for open-source karma points, open a PR implementing it (we'll love you for that ❤️).

New features and updates 🔥

Improve indexing speed

Indexing time is improved across the board!

• Performance is maintained or better on smaller machines
• On bigger machines with multiple cores and good IO, Meilisearch v1.12 is much faster than Meilisearch v1.11
  • More than twice as fast for raw document insertion tasks.
  • More than x4 as fast for incrementally updating documents in a large database.
  • Embeddings generation was also improved up to x1.5 for some workloads.

The new indexer also makes task cancellation faster.

Done by @dureuill, @ManyTheFish, and @Kerollmops in #4900.

New index settings: use facetSearch and prefixSearch to improve indexing speed

v1.12 introduces two new index settings: facetSearch and prefixSearch.

Both settings allow you to skip parts of the indexing process. This leads to significant improvements to indexing speed, but may negatively impact search experience in some use cases.

Done by @ManyTheFish in #5091

facetSearch

Use this setting to toggle facet search:

curl \
  -X PUT 'http://localhost:7700/indexes/books/settings/facet-search' \
  -H 'Content-Type: application/json' \
  --data-binary 'true'

The default value for facetSearch is true. When set to false, this setting disables facet search for all filterable attributes in an index.

prefixSearch

Use this setting to configure the ability to search a word by prefix on an index:

curl \
  -X PUT 'http://localhost:7700/indexes/books/settings/prefix-search' \
  -H 'Content-Type: application/json' \
  --data-binary 'disabled'

prefixSearch accepts one of the following values:

• "indexingTime": enables prefix processing during indexing. This is the default Meilisearch behavior
• "disabled": deactivates prefix search completely

Disabling prefix search means the query he will no longer match the word hello. This may significantly impact search result relevancy, but speeds up the indexing process.

New API route: /batches

The new /batches endpoint allow you to query information about task batches.

GET /batches returns a list of batch objects:

curl  -X GET 'http://localhost:7700/batches'

This endpoint accepts the same parameters as GET /tasks route, allowing you to narrow down which batches you want to see. Parameters used with GET /batches apply to the tasks, not the batches themselves. For example, GET /batches?uid=0 returns batches containing tasks with a taskUid of 0 , not batches with a batchUid of 0.

You may also query GET /batches/:uid to retrieve information about a single batch object:

curl  -X GET 'http://localhost:7700/batches/BATCH_UID'

/batches/:uid does not accept any parameters.

Batch objects contain the following fields:

{
  "uid": 160,
  "progress": {
    "steps": [
      {
        "currentStep": "processing tasks",
        "finished": 0,
        "total": 2
      },
      {
        "currentStep": "indexing",
        "finished": 2,
        "total": 3
      },
      {
        "currentStep": "extracting words",
        "finished": 3,
        "total": 13
      },
      {
        "currentStep": "document",
        "finished": 12300,
        "total": 19546
      }
    ],
    "percentage": 37.986263
  },
  "details": {
    "receivedDocuments": 19547,
    "indexedDocuments": null
  },
  "stats": {
    "totalNbTasks": 1,
    "status": {
      "processing": 1
    },
    "types": {
      "documentAdditionOrUpdate": 1
    },
    "indexUids": {
      "mieli": 1
    }
  },
  "duration": null,
  "startedAt": "2024-12-12T09:44:34.124726733Z",
  "finishedAt": null
}

Additionally, task objects now include a new field, batchUid. Use this field together with /batches/:uid to retrieve data on a specific batch.

{
  "uid": 154,
  "batchUid": 142,
  "indexUid": "movies_test2",
  "status": "succeeded",
  "type": "documentAdditionOrUpdate",
  "canceledBy": null,
  "details": {
    "receivedDocuments": 1,
    "indexedDocuments": 1
  },
  "error": null,
  "duration": "PT0.027766819S",
  "enqueuedAt": "2024-12-02T14:07:34.974430765Z",
  "startedAt": "2024-12-02T14:07:34.99021667Z",
  "finishedAt": "2024-12-02T14:07:35.017983489Z"
}

Done by @irevoire in #5060, #5070, #5080

Other improvements

• New query parameter for GET /tasks: reverse. If reverse is set to true, tasks will be returned in reversed order, from oldest to newest tasks. Done by @irevoire in #5048
• Phrase searches withshowMatchesPosition set to true give a single location for the whole phrase @flevi29 in #4928
• New Prometheus metrics by @PedroTurik in #5044
• When a query finds matching terms in document fields with array values, Meilisearch now includes an indices field to _matchesPosition specifying which array elements contain the matches by @LukasKalbertodt in #5005
• ⚠️ Breaking vectorStore change: field distribution no longer contains _vectors. Its value used to be incorrect, and there is no current use case for the fixed, most likely empty, value. Done as part of #4900
• Improve error message by adding index name in #5056 by @airycanon

Fixes 🐞

• Return appropriate error when primary key is greater than 512 bytes, by @flevi29 in #4930
• Fix issue where numbers were segmented in different ways depending on tokenizer, by @dqkqd in meilisearch/charabia#311
• Fix pagination when embedding fails by @dureuill in #5063
• Fix issue causing Meilisearch to ignore stop words in some cases by @ManyTheFish in #5062
• Fix phrase search with attributesToSearchOn in #5062 by @ManyTheFish

Misc

• Dependencies updates
  • Update benchmarks to match the new crates subfolder by @Kerollmops in #5021
  • Fix the benchmarks by @irevoire in #5037
  • Bump Swatinem/rust-cache from 2.7.1 to 2.7.5 in #5030
  • Update charabia v0.9.2 by @ManyTheFish in #5098
  • Update mini-dashboard to v0.2.16 version by @curquiza in #5102
• CIs and tests
  • Improve performance of delete_index.rs by @DerTimonius in #4963
  • Improve performance of create_index.rs by @DerTimonius in #4962
  • Improve performance of get_documents.rs by @PedroTurik in #5025
  • Improve performance of formatted.rs by @PedroTurik in #5043
  • Fix the path used in the flaky tests CI by @Kerollmops in #5049
• Misc
  • Rollback the Meilisearch Kawaii logo by @Kerollmops in #5017
  • Add image source label to Dockerfile by @wuast94 in #4990
  • Hide code complexity into a subfolder by @Kerollmops in #5016
  • Internal tool: implement offline upgrade from v1.10 to v1.11 by @irevoire in #5034
  • Internal tool: implement offline upgrade from v1.11 to v1.12 by @ManyTheFish in #5146
  • Meilisearch is now able to retrieve Katakana words from a Hiragana query by @tats-u in meilisearch/charabia#312
  • Improve error handling when writing into LMDB by @Kerollmops in #5089

❤️ Thanks again to our external contributors:

• Meilisearch: @airycanon, @DerTimonius, @flevi29, @LukasKalbertodt, @PedroTurik, @wuast94
• Charabia: @dqkqd @tats-u