Merge branch 'main' into use-path-restricted-const-es

docs/changelog/104711.yaml

@@ -0,0 +1,5 @@
+pr: 104711
+summary: "Fixing NPE when requesting [_none_] for `stored_fields`"
+area: Search
+type: bug
+issues: []

docs/changelog/105373.yaml

@@ -0,0 +1,5 @@
+pr: 105373
+summary: "Fix parsing of flattened fields within subobjects: false"
+area: Mapping
+type: bug
+issues: []

docs/changelog/105617.yaml

@@ -0,0 +1,5 @@
+pr: 105617
+summary: Fix HTTP corner-case response leaks
+area: Network
+type: bug
+issues: []

docs/changelog/105622.yaml

@@ -0,0 +1,5 @@
+pr: 105622
+summary: Distinguish different snapshot failures by log level
+area: Snapshot/Restore
+type: enhancement
+issues: []

docs/changelog/105629.yaml

@@ -0,0 +1,5 @@
+pr: 105629
+summary: Show owner `realm_type` for returned API keys
+area: Security
+type: enhancement
+issues: []

docs/changelog/105633.yaml

@@ -0,0 +1,6 @@
+pr: 105633
+summary: "[Connector API] Bugfix: support list type in filtering advenced snippet\
+  \ value"
+area: Application
+type: bug
+issues: []

docs/changelog/105636.yaml

@@ -0,0 +1,5 @@
+pr: 105636
+summary: Flip dynamic mapping condition when create tsid
+area: TSDB
+type: bug
+issues: []

docs/changelog/105670.yaml

@@ -0,0 +1,5 @@
+pr: 105670
+summary: "Painless: Apply true regex limit factor with FIND and MATCH operation"
+area: Infra/Scripting
+type: bug
+issues: []

docs/changelog/105689.yaml

@@ -0,0 +1,6 @@
+pr: 105689
+summary: Fix `uri_parts` processor behaviour for missing extensions
+area: Ingest Node
+type: bug
+issues:
+ - 105612

docs/changelog/105693.yaml

@@ -0,0 +1,6 @@
+pr: 105693
+summary: Fix error 500 on invalid `ParentIdQuery`
+area: Search
+type: bug
+issues:
+ - 105366

docs/reference/esql/esql-get-started.asciidoc

@@ -37,7 +37,7 @@ image::images/esql/source-command.svg[A source command producing a table from {e
 
 The <<esql-from>> source command returns a table with documents from a data
 stream, index, or alias. Each row in the resulting table represents a document.
-This query returns up to 500 documents from the `sample_data` index:
+This query returns up to 1000 documents from the `sample_data` index:
 
 [source,esql]
 ----
@@ -237,7 +237,7 @@ include::{esql-specs}/eval.csv-spec[tag=gs-eval-stats-backticks]
 To track statistics over time, {esql} enables you to create histograms using the
 <<esql-auto_bucket>> function. `AUTO_BUCKET` creates human-friendly bucket sizes
 and returns a value for each row that corresponds to the resulting bucket the
-row falls into. 
+row falls into.
 
 For example, to create hourly buckets for the data on October 23rd:
 
@@ -272,7 +272,7 @@ image::images/esql/esql-enrich.png[align="center"]
 
 Before you can use `ENRICH`, you first need to
 <<esql-create-enrich-policy,create>> and <<esql-execute-enrich-policy,execute>>
-an <<esql-enrich-policy,enrich policy>>. 
+an <<esql-enrich-policy,enrich policy>>.
 
 include::{es-repo-dir}/tab-widgets/esql/esql-getting-started-widget-enrich-policy.asciidoc[]
 
@@ -344,4 +344,4 @@ For more about data processing with {esql}, refer to
 [[esql-getting-learn-more]]
 === Learn more
 
-To learn more about {esql}, refer to <<esql-language>> and <<esql-using>>.
+To learn more about {esql}, refer to <<esql-language>> and <<esql-using>>.

docs/reference/esql/esql-kibana.asciidoc

@@ -103,7 +103,7 @@ detailed warning, expand the query bar, and click *warnings*.
 === The results table
 
 For the example query, the results table shows 10 rows. Omitting the `LIMIT`
-command, the results table defaults to up to 500 rows. Using `LIMIT`, you can
+command, the results table defaults to up to 1000 rows. Using `LIMIT`, you can
 increase the limit to up to 10,000 rows.
 
 NOTE: the 10,000 row limit only applies to the number of rows that are retrieved

docs/reference/esql/esql-limitations.asciidoc

@@ -9,7 +9,7 @@
 [[esql-max-rows]]
 === Result set size limit
 
-By default, an {esql} query returns up to 500 rows. You can increase the number
+By default, an {esql} query returns up to 1000 rows. You can increase the number
 of rows up to 10,000 using the <<esql-limit>> command.
 include::processing-commands/limit.asciidoc[tag=limitation]
 

docs/reference/esql/source-commands/from.asciidoc

@@ -26,7 +26,7 @@ corresponds to a field, and can be accessed by the name of that field.
 [NOTE]
 ====
 By default, an {esql} query without an explicit <<esql-limit>> uses an implicit
-limit of 500. This applies to `FROM` too. A `FROM` command without `LIMIT`:
+limit of 1000. This applies to `FROM` too. A `FROM` command without `LIMIT`:
 
 [source,esql]
 ----
@@ -38,7 +38,7 @@ is executed as:
 [source,esql]
 ----
 FROM employees
-| LIMIT 500
+| LIMIT 1000
 ----
 ====
 

docs/reference/how-to/size-your-shards.asciidoc

@@ -221,6 +221,15 @@ GET _cat/shards?v=true
 ----
 // TEST[setup:my_index]
 
+[discrete]
+[[shard-count-per-node-recommendation]]
+==== Add enough nodes to stay within the cluster shard limits
+
+The <<cluster-shard-limit,cluster shard limits>> prevent creation of more than
+1000 non-frozen shards per node, and 3000 frozen shards per dedicated frozen
+node. Make sure you have enough nodes of each type in your cluster to handle
+the number of shards you need.
+
 [discrete]
 [[field-count-recommendation]]
 ==== Allow enough heap for field mappers and overheads

docs/reference/modules/cluster/misc.asciidoc

@@ -24,35 +24,46 @@ API can make the cluster read-write again.
 
 [discrete]
 [[cluster-shard-limit]]
-==== Cluster shard limit
+==== Cluster shard limits
 
-There is a soft limit on the number of shards in a cluster, based on the number
-of nodes in the cluster. This is intended to prevent operations which may
-unintentionally destabilize the cluster.
+There is a limit on the number of shards in a cluster, based on the number of
+nodes in the cluster. This is intended to prevent a runaway process from
+creating too many shards which can harm performance and in extreme cases may
+destabilize your cluster.
 
-IMPORTANT: This limit is intended as a safety net, not a sizing recommendation. The
-exact number of shards your cluster can safely support depends on your hardware
-configuration and workload, but should remain well below this limit in almost
-all cases, as the default limit is set quite high.
+[IMPORTANT]
+====
 
-If an operation, such as creating a new index, restoring a snapshot of an index,
-or opening a closed index would lead to the number of shards in the cluster
-going over this limit, the operation will fail with an error indicating the
-shard limit.
+These limits are intended as a safety net to protect against runaway shard
+creation and are not a sizing recommendation. The exact number of shards your
+cluster can safely support depends on your hardware configuration and workload,
+and may be smaller than the default limits.
 
-If the cluster is already over the limit, due to changes in node membership or
-setting changes, all operations that create or open indices will fail until
-either the limit is increased as described below, or some indices are
-<<indices-open-close,closed>> or <<indices-delete-index,deleted>> to bring the
-number of shards below the limit.
+We do not recommend increasing these limits beyond the defaults. Clusters with
+more shards may appear to run well in normal operation, but may take a very
+long time to recover from temporary disruptions such as a network partition or
+an unexpected node restart, and may encounter problems when performing
+maintenance activities such as a rolling restart or upgrade.
 
-The cluster shard limit defaults to 1,000 shards per non-frozen data node for
+====
+
+If an operation, such as creating a new index, restoring a snapshot of an
+index, or opening a closed index would lead to the number of shards in the
+cluster going over this limit, the operation will fail with an error indicating
+the shard limit. To resolve this, either scale out your cluster by adding
+nodes, or <<indices-delete-index,delete some indices>> to bring the number of
+shards below the limit.
+
+If a cluster is already over the limit, perhaps due to changes in node
+membership or setting changes, all operations that create or open indices will
+fail.
+
+The cluster shard limit defaults to 1000 shards per non-frozen data node for
 normal (non-frozen) indices and 3000 shards per frozen data node for frozen
-indices.
-Both primary and replica shards of all open indices count toward the limit,
-including unassigned shards.
-For example, an open index with 5 primary shards and 2 replicas counts as 15 shards.
-Closed indices do not contribute to the shard count.
+indices. Both primary and replica shards of all open indices count toward the
+limit, including unassigned shards. For example, an open index with 5 primary
+shards and 2 replicas counts as 15 shards. Closed indices do not contribute to
+the shard count.
 
 You can dynamically adjust the cluster shard limit with the following setting:
 
@@ -99,12 +110,13 @@ For example, a cluster with a `cluster.max_shards_per_node.frozen` setting of
 `100` and three frozen data nodes has a frozen shard limit of 300. If the
 cluster already contains 296 shards, {es} rejects any request that adds five or
 more frozen shards to the cluster.
+--
 
-NOTE: These setting do not limit shards for individual nodes. To limit the
-number of shards for each node, use the
+NOTE: These limits only apply to actions which create shards and do not limit
+the number of shards assigned to each node. To limit the number of shards
+assigned to each node, use the
 <<cluster-total-shards-per-node,`cluster.routing.allocation.total_shards_per_node`>>
 setting.
---
 
 [discrete]
 [[user-defined-data]]

docs/reference/rest-api/security.asciidoc

@@ -12,6 +12,8 @@ Use the following APIs to perform security activities.
 * <<security-api-has-privileges>>
 * <<security-api-ssl>>
 * <<security-api-get-builtin-privileges>>
+* <<security-api-get-settings>>
+* <<security-api-update-settings>>
 * <<security-api-get-user-privileges>>
 
 [discrete]

docs/reference/rest-api/security/get-api-keys.asciidoc

@@ -134,6 +134,7 @@ A successful call returns a JSON structure that contains the information of the
       "invalidated": false, <6>
       "username": "myuser", <7>
       "realm": "native1", <8>
+      "realm_type": "native",
       "metadata": { <9>
         "application": "myapp"
       },
@@ -289,6 +290,7 @@ A successful call returns a JSON structure that contains the information of one
       "invalidated": false,
       "username": "myuser",
       "realm": "native1",
+      "realm_type": "native",
       "metadata": {
         "application": "myapp"
       },

docs/reference/rest-api/security/get-settings.asciidoc

@@ -5,6 +5,8 @@
 <titleabbrev>Get Security settings</titleabbrev>
 ++++
 
+Retrieves settings for the security internal indices.
+
 [[security-api-get-settings-prereqs]]
 ==== {api-prereq-title}
 
@@ -14,11 +16,16 @@
 ==== {api-description-title}
 This API allows a user to retrieve the user-configurable settings for the 
 Security internal index (`.security` and associated indices). Only a subset of 
-the index settings — those that are user-configurable—will be shown. This includes:
+the index settings — those that are user-configurable—will be shown. This
+includes:
 
 - `index.auto_expand_replicas`
 - `index.number_of_replicas`
 
+
+[[security-api-get-settings-example]]
+==== {api-examples-title}
+
 An example of retrieving the security settings:
 
 [source,console]

docs/reference/rest-api/security/query-api-key.asciidoc

@@ -299,6 +299,7 @@ retrieved from one or more API keys:
       "invalidated": false,
       "username": "elastic",
       "realm": "reserved",
+      "realm_type": "reserved",
       "metadata": {
         "letter": "a"
       },
@@ -411,6 +412,7 @@ A successful call returns a JSON structure for API key information including its
       "invalidated": false,
       "username": "myuser",
       "realm": "native1",
+      "realm_type": "native",
       "metadata": {
         "application": "my-application"
       },

docs/reference/rest-api/security/update-settings.asciidoc

@@ -5,11 +5,31 @@
 <titleabbrev>Update Security settings</titleabbrev>
 ++++
 
+Updates the settings of the security internal indices.
+
+
 [[security-api-update-settings-prereqs]]
 ==== {api-prereq-title}
 
 * To use this API, you must have at least the `manage_security` cluster privilege.
 
+
+[[security-api-update-settings-request-body]]
+==== {api-request-body-title}
+
+`security`::
+(Optional, object) Settings to be used for the index used for most security 
+configuration, including Native realm users and roles configured via the API.
+
+`security-tokens`::
+(Optional, object) Settings to be used for the index used to store 
+<<security-api-get-token,tokens>>.
+
+`security`::
+(Optional, object) Settings to be used for the index used to store 
+<<security-api-activate-user-profile, profile>> information.
+
+
 [[security-api-update-settings-desc]]
 ==== {api-description-title}
 This API allows a user to modify the settings for the Security internal indices 
@@ -19,6 +39,10 @@ be modified. This includes:
 - `index.auto_expand_replicas`
 - `index.number_of_replicas`
 
+
+[[security-api-update-settings-example]]
+==== {api-examples-title}
+
 An example of modifying the Security settings:
 
 [source,console]
@@ -43,18 +67,3 @@ The configured settings can be retrieved using the
 is not in use on the system, but settings are provided for it, the request will 
 be rejected - this API does not yet support configuring the settings for these 
 indices before they are in use.
-
-
-==== {api-request-body-title}
-
-`security`::
-(Optional, object) Settings to be used for the index used for most security 
-configuration, including Native realm users and roles configured via the API.
-
-`security-tokens`::
-(Optional, object) Settings to be used for the index used to store 
-<<security-api-get-token,tokens>>.
-
-`security`::
-(Optional, object) Settings to be used for the index used to store 
-<<security-api-activate-user-profile, profile>> information.

docs/reference/setup/logging-config.asciidoc

@@ -150,7 +150,9 @@ update settings API>> to change the related logger's log level. Each logger
 accepts Log4j 2's built-in log levels, from least to most verbose: `OFF`,
 `FATAL`, `ERROR`, `WARN`, `INFO`, `DEBUG`, and `TRACE`. The default log level is
 `INFO`. Messages logged at higher verbosity levels (`DEBUG` and `TRACE`) are
-only intended for expert use.
+only intended for expert use. To prevent leaking sensitive information in logs,
+{es} will reject setting certain loggers to higher verbosity levels unless
+<<http-rest-request-tracer,insecure network trace logging>> is enabled.
 
 [source,console]
 ----
@@ -227,7 +229,7 @@ to `OFF` in `log4j2.properties` :
 ----
 logger.deprecation.level = OFF
 ----
-Alternatively, you can change the logging level dynamically: 
+Alternatively, you can change the logging level dynamically:
 
 [source,console]
 ----