Trace support for Apollo client SDK on iOS.

[dineshiOSDev]

Question

Unable to trace network calls with distributed trace from our iOS mobile application since we rely on Apollo client SDK for networking.

Currently we are using individual span on network manager but that doesn't works for distributed traces across the lambda level.

[ncreated]

Hey @dineshiOSDev 👋.

Currently we are using individual span on network manager but that doesn't works for distributed traces across the lambda level.

Not sure what do you exactly mean by "doesn't work" 🙂, so I'll do some assumptions about your setup:

• using DatadogTrace module;
• sending URLRequests to GraphQL service;
• the service is instrumented with Datadog tracer;
• for each URLRequest you create span manually using iOS tracer (Tracer.shared().startSpan(...)).

To enable distributed tracing, you must propagate span context in URLRequest headers. As we explain in this doc:

To manually propagate the trace, inject the span context into URLRequest headers:

var request: URLRequest = ... // the request to your API

let span = tracer.startSpan(operationName: "network request")

let headersWriter = HTTPHeadersWriter(samplingRate: 20)
tracer.inject(spanContext: span.context, writer: headersWriter)

for (headerField, value) in headersWriter.tracePropagationHTTPHeaders {
    request.addValue(value, forHTTPHeaderField: headerField)
}

The above snippet explains how to inject trace headers into given request. Assuming this request is sent to a service instrumented with Datadog, you will see distributed trace created for 20% of requests (samplingRate: 20). Next to HTTPHeadersWriter we also provide W3CHTTPHeadersWriter and B3HTTPHeadersWriter.

Let us know if this was helpful or provide more context on what doesn't work.

[dineshiOSDev]

@ncreated Thank you so much for getting back so quickly.
Yes currently we are creating individual span like you mentioned.

Since we completely rely on Apollo client SDK
I have added above logic at Apollo Interceptor were we get the URL Request.

func interceptAsync<Operation: GraphQLOperation>(
       chain: RequestChain,
       request: HTTPRequest<Operation>,
       response: HTTPResponse<Operation>?,
       completion: @escaping (Result<GraphQLResult<Operation.Data>, Error>) -> Void) {
          if var localrequest = try? request.toURLRequest(), let url = localrequest.url?.absoluteString {
               let spanTracer = Tracer.shared().startSpan(operationName: url)
               let headersWriter = HTTPHeadersWriter.init(sampleRate: 20)
               Tracer.shared().inject(spanContext: spanTracer.context, writer: headersWriter)
               for (headerField, value) in headersWriter.traceHeaderFields {
                   localrequest.addValue(value, forHTTPHeaderField: headerField)
               }
               
               SpanCollector.default.startSpan(id:  Operation.operationName, span: spanTracer)
           }
      }

The flame graph just shows only the front end trace and it couldn't map/co-related to lambda level.

Is there any docs or links that you can help us for datadog support for apollo sdk please.
Things works as expected with android part.
(Additionally this trace id format looks different for android and ios )

[ncreated]

@dineshiOSDev Thanks for more context. The way you use Datadog APIs looks fine 👌, but I think that your problem might be in the way you're using Apollo interceptor. Could you confirm that requests which are sent from your app do not include x-datadog-* headers, whereas your code is actually setting them on the localrequest instance?

I had a glance look at Apollo's example of UserManagementInterceptor and I believe that after adding headers you should process the request back through the chain as in their snippet:

request.addHeader(name: "Authorization", value: "Bearer \(token.value)")
chain.proceedAsync(
   request: request,
   response: response,
   interceptor: self,
   completion: completion
)

Without calling the chain back, you might be mutating the request in-place, without sending the result back into system. I'm not an expert in Apollo, hence this can be wrong theory.

[dineshiOSDev]

Hi @ncreated
Actually we do process the chain back but then too we get only the trace from iOS.request were the distributed trace is not showing on the flame graph.

I tried two way one suggested here
#416 (comment)
and the above one like you suggested.

But both has the same result.
When trying the second way with individual span like

for (headerField, value) in headersWriter.tracePropagationHTTPHeaders {
    request.addValue(value, forHTTPHeaderField: headerField)
}

below are the values that are added

1. x-datadog-sampling-priority
2. x-datadog-parent-id
3. x-datadog-tags
4. x-datadog-trace-id

Also i see all the traceid sent from ios has a constant id("_dd.p.tid=\(traceID.idHiHex)") appended to each trace.
where its not on the android traces. Is there a way that we can eliminate this ?

[ganeshnj]

_dd.p.tid tag is by design to support 128 bit trace id in spans. This shouldn't have any impact as it is a backward compatible change if your backend is not updated to handle 128 bit ids.

The format of the trace id follows https://github.com/DataDog/dd-sdk-ios/blob/develop/DatadogInternal/Sources/NetworkInstrumentation/TraceID.swift#L219

I did a quick test and it seems to work as expected with

        Datadog.initialize(
            with: Datadog.Configuration(
                clientToken: "<your id>",
                env: "tests",
                service: "apollo",
                uploadFrequency: .frequent
            ),
            trackingConsent: .granted
        )

        Trace.enable(with: .init(urlSessionTracking: Trace.Configuration.URLSessionTracking(
            firstPartyHostsTracing: .traceWithHeaders(hostsWithHeaders: ["http://127.0.0.1:3000": [.datadog]], sampleRate: 100)
        )))
        URLSessionInstrumentation.enable(with: .init(delegateClass: URLSessionClient.self))

let apolloClient = ApolloClient(url: URL(string: "http://127.0.0.1:3000/graphql")!)
 apolloClient.fetch(query: Graphing.GetAllMessagesQuery()) { result in
                    switch result {
                    case .success(let data):
                        print(data)
                    case .failure(let error):
                        print(error)
                    }
                }

I'm curious if you can share?

• if you have any logs from your lambda?
• version of the trace agent used?
• lambda language?

[dineshiOSDev]

@ganeshnj
language of the lambdas is JavaScript, and the tracer versions are as follows:

• "datadog-cdk-constructs-v2": "1.13.0"
• nodeLayerVersion: 108
• extensionLayerVersion: 56

Also the above setup logs all the traces without the help of individual span.
But the traces still has the information only from iOS mobile app. The distributed trace through the lambda level is not mapped.

we can create a support ticket just in case if you need see through the logs.

[ganeshnj]

Yes, please open a support ticket.

[dineshiOSDev]

@ganeshnj @ncreated
Thanks for the fix
We see the distributed traces getting co related after the fix
#1807

[ganeshnj]

Great. Closing it now.