New "Context" field in Opentelemtry Log Model

[kiranpatel11]

What are you trying to achieve?

Consider introducing Context field to the Log Model. Please see below Additional Context for more details.

What did you expect to see?

Below field to be part of the Log Model.

Field Name	Description
Context	Contextual information about the execution

Additional context.

AFAIU, there are two root-level fields Resource and Attributes that contains information as per two different lifecycle.

Resource : It contains relatively static information associated with lifecycle of the service (service_name), lifecycle of the particular version or k8s pod, etc. We expect to see the same details for Resource for all LogEvents logged during that lifecycle.

Attributes : It contains dynamic information associated with individual LogEvent. e.g. http.method, http.response_status or http.url. Considering this characteristic, Attributes are usually not defined as part of the static log patterns in most libraries (log4j or logback), otherwise we would see empty Attributes for most of the LogEvent which are not associated with http request/response.

Proposal is to introduce new Context field which can serve below usecase:

As TraceId and SpanId are associated with distributed REQUEST call-tree - single request flowing through A-->B-->C. It does not seem to fit the usecases like user sessions/conversationId in saga pattern. Typical example is Order Fulfillment workflow, during the order workflow, same distributed service may get called but with most likely with different TraceId depending on design and implementation. In such case, these different calls can be correlated using something like OrderNumber. Another example is associating all traces with same UserId or UserSession.

Typical implementation uses Context pattern (e.g. MDC in slf4j) to log such fields without explicitly being part of the Log statements and associated with the lifecycle of the request - which is somewhere middle-ground between the purpose served by Resource and Attributes fields, Hence the need for the new Context field in the otep log model.

[pmm-sumo]

I think this is a related issue: #1613

[kiranpatel11]

@pmm-sumo , yes it is partially related not entirely.

What I am proposing here is to differentiate according to the lifecycle of the data accociated with the LogEvent.

What is mentioned in #1613 for Attributes make sense - it can be used for arbitrary key/value pairs, but from implementation perspective its quite challenging to populate MDC fields as well as event specific fields under the same Attributes bucket.

I have created a demo app using logback, I will submit it to contrib/examples to help the discussion.

[reyang]

One possible way is to explore if attributes can be used (with some semantic convention) instead of having to introduce a top level concept. Will be discussed in the upcoming logging SIG.

[tigrannajaryan]

I do not see the need for a top-level field in the log data model. These can be recorded as LogRecord Attributes. The log data model design specifically reserves top-level fields for data that is present almost always, which is not the case for the "Context" proposed here, it appears to be a special use case. Log data model encourages using Attributes for such cases.
I think this proposal should be submitted as a semantic convention suggestion. It should also apply to both logs and traces, this is not unique to logs.

Another example is associating all traces with same UserId or UserSession.

There is already a semantic convention for user id: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/semantic_conventions/span-general.md#general-identity-attributes

[tigrannajaryan]

Marking as candidate for closing with "won't fix" unless we see more arguments in favour.

[tigrannajaryan]

Closing since no further arguments provided.