[BUG]: Python int too large to convert to C long #12240

lucas03 · 2025-02-06T10:37:41Z

Tracer Version(s)

doesnt work on:

2.20.0
2.19.3
2.17.5
2.15.4
2.14.7
2.13.2
2.12.4
2.11.8
2.10.7

Python Version(s)

Python 3.11

Pip Version(s)

pip 24.0

Bug Report

Our pytest job fail randomly.

Reproduction Code

No response

Error Logs

FAILED test/integration/mambo/status_banners/test_status_banners.py::test_banners__schedule_change_major[web-True] - RuntimeError: failed to pack span: <Span(id=4307602835879970050,trace_id=137759318570304310240605327865437761465,parent_id=17664599921369065393,name=flask.request)>. Exception: Python int too large to convert to C long

Libraries in Use

pytest==8.3.4
datadog==0.51.0
ddtrace==2.20.0

Operating System

Debian GNU/Linux 11 (bullseye)

mabdinur · 2025-02-07T20:03:46Z

Hey @lucas03,

Can you open a support ticket here: https://help.datadoghq.com/hc/en-us/requests/new.

The ddtrace-py library serializes trace data into msgpack. One of your spans likely contains an integer that is too large to encode as a C long (likely by this method).

To debug further you can configure a trace filter to capture span attributes (Span.duration_ns, Span.start_ns, Span._metrics, etc.) that contain integers that can not be converted to a C long. Once we identify the source of this value we can take steps to correct it.

lucas03 · 2025-02-10T14:55:58Z

Can you open a support ticket here

Your request (#2028629) has been received and will be reviewed by our team.

The ddtrace-py library serializes trace data into msgpack. One of your spans likely contains an integer that is too large to encode as a C long (likely by this method).

Could the problem be with datadog lib or something else? I have tried downgrading to the original version of ddtrace, but failing test is still there (flaky, not failing all the time)

To debug further you can configure a trace filter to capture span attributes (Span.duration_ns, Span.start_ns, Span._metrics, etc.) that contain integers that can not be converted to a C long. Once we identify the source of this value we can take steps to correct it.

Could you give me code snippet you have in mind? BTW I have ddtrace disabled in tests, but that doesn't seem to work.

lucas03 · 2025-02-10T15:00:37Z

requirements.txt

amqp==5.2.0
amqp==5.3.1

anyio==4.4.0
anyio==4.8.0

async-timeout==4.0.3
async-timeout==5.0.1

attrs==23.2.0
attrs==25.1.0

babel==2.15.0
babel==2.16.0

billiard==4.2.0
billiard==4.2.1

bleach==6.1.0
bleach==6.2.0

bottle==0.12.25
bottle==0.13.2

bytecode==0.15.1
bytecode==0.16.1

cachetools==5.4.0
cachetools==5.5.1

certifi==2024.7.4
certifi==2024.12.14

charset-normalizer==3.3.2
charset-normalizer==3.4.1

click==8.1.7
click==8.1.8

cryptography==43.0.3
cryptography==44.0.0

datadog==0.49.1
datadog==0.51.0

ddtrace==2.10.1
ddtrace==2.20.0

deprecated==1.2.14
deprecated==1.2.18

dnspython==2.6.1
dnspython==2.7.0

envier==0.5.2
envier==0.6.1

gevent==24.2.1
gevent==24.11.1

google-api-core==2.19.1
google-api-core==2.24.1

google-auth==2.32.0
google-auth==2.38.0

google-cloud-pubsub==2.23.0
google-cloud-pubsub==2.27.3

google-cloud-storage==2.18.0
google-cloud-storage==2.19.0

google-crc32c==1.5.0
google-crc32c==1.6.0

google-resumable-media==2.7.1
google-resumable-media==2.7.2

googleapis-common-protos==1.63.2
googleapis-common-protos==1.66.0

greenlet==3.0.3
greenlet==3.1.1

grpc-google-iam-v1==0.13.1
grpc-google-iam-v1==0.14.0

grpcio==1.65.1
grpcio==1.70.0

grpcio-status==1.65.1
grpcio-status==1.70.0

gunicorn==22.0.0
gunicorn==23.0.0

hiredis==3.0.0
hiredis==3.1.0

httpcore==1.0.5
httpcore==1.0.7

httpx==0.27.0
httpx==0.28.1

idna==3.7
idna==3.10

importlib-metadata==8.0.0
importlib-metadata==8.5.0

importlib-resources==6.4.0
importlib-resources==6.5.2

isodate==0.6.1
isodate==0.7.2

jinja2==3.1.4
jinja2==3.1.5

joserfc==1.0.0
joserfc==1.0.2

kiwi-cache==1.0.1
kiwi-cache==1.0.2

kiwi-google-metadata-auth==0.1.0
kiwi-google-metadata-auth==0.2.0

kiwi-json==0.10.0
kiwi-json==0.10.1

kiwi-structlog-config==0.3.3
kiwi-structlog-config==0.3.4

kombu==5.3.7
kombu==5.4.2

lxml==5.2.2
lxml==5.3.0

lz4==4.3.3
lz4==4.4.3

markupsafe==2.1.5
markupsafe==3.0.2

marshmallow==3.21.3
marshmallow==3.26.0

pathable==0.4.3
pathable==0.4.4

phonenumbers==8.13.40
phonenumbers==8.13.53

pillow==10.4.0
pillow==11.1.0

platformdirs==4.2.2
platformdirs==4.3.6

prompt-toolkit==3.0.47
prompt-toolkit==3.0.50

proto-plus==1.24.0
proto-plus==1.26.0

protobuf==5.27.2
protobuf==5.29.3

psycopg==3.1.19
psycopg==3.2.4

psycopg-binary==3.1.19
psycopg-binary==3.2.4

pyasn1==0.6.0
pyasn1==0.6.1

pyasn1-modules==0.4.0
pyasn1-modules==0.4.1

pydantic==2.8.2
pydantic==2.10.6

pydantic-core==2.20.1
pydantic-core==2.27.2

pyjwt==2.8.0
pyjwt==2.10.1

pyparsing==3.1.2
pyparsing==3.2.1

pytz==2024.1
pytz==2024.2

pyyaml==6.0.1
pyyaml==6.0.2

redis==5.0.7
redis==5.2.1

request-session==0.15.0
request-session==0.16.2

ruamel-yaml==0.18.6
ruamel-yaml==0.18.10

ruamel-yaml-clib==0.2.8
ruamel-yaml-clib==0.2.12

schwifty==2024.6.1
schwifty==2025.1.0

sentry-sdk==2.11.0
sentry-sdk==2.12.0

simplejson==3.19.2
simplejson==3.19.3

six==1.16.0
six==1.17.0

sqlalchemy==2.0.31
sqlalchemy==2.0.37

structlog==24.4.0
structlog==25.1.0

types-python-dateutil==2.9.0.20240316
types-python-dateutil==2.9.0.20241206

tzdata==2024.1
tzdata==2025.1

urllib3==2.2.2
urllib3==2.3.0

webob==1.8.7
webob==1.8.9

wrapt==1.16.0
wrapt==1.17.2

xlsxwriter==3.2.0
xlsxwriter==3.2.1

xmltodict==0.13.0
xmltodict==0.14.2

zeep==4.2.1
zeep==4.3.1

zipp==3.19.2
zipp==3.21.0

zope-interface==6.4.post2
zope-interface==7.2

and test-requirements.txt

alembic==1.13.2
alembic==1.14.1

certifi==2024.7.4
certifi==2024.12.14

charset-normalizer==3.3.2
charset-normalizer==3.4.1

coverage==7.6.0
coverage==7.6.10

cryptography==43.0.3
cryptography==44.0.0

faker==35.0.0
faker==26.0.0

fakeredis==2.23.3
fakeredis==2.26.2

idna==3.7
idna==3.10

kiwi-json==0.10.0
kiwi-json==0.10.1

mako==1.3.5
mako==1.3.8

markupsafe==2.1.5
markupsafe==3.0.2

mypy==1.11.0
mypy==1.14.1

polyfactory==2.16.2
polyfactory==2.18.1

pyaml==24.7.0
pyaml==25.1.0

pyparsing==3.1.2
pyparsing==3.2.1

pytest==8.3.2
pytest==8.3.4

pytest-cov==5.0.0
pytest-cov==6.0.0

pytest-randomly==3.15.0
pytest-randomly==3.16.0

pyyaml==6.0.1
pyyaml==6.0.2

redis==5.0.7
redis==5.2.1

six==1.16.0
six==1.17.0

soupsieve==2.5
soupsieve==2.6

sql-metadata==2.12.0
sql-metadata==2.15.0

sqlalchemy==2.0.31
sqlalchemy==2.0.37

sqlparse==0.5.1
sqlparse==0.5.3

types-bleach==6.1.0.20240331
types-bleach==6.2.0.20241123

types-cachetools==5.4.0.20240717
types-cachetools==5.5.0.20240820

types-cffi==1.16.0.20240331
types-cffi==1.16.0.20241221

types-html5lib==1.1.11.20240228
types-html5lib==1.1.11.20241018

types-python-dateutil==2.9.0.20240316
types-python-dateutil==2.9.0.20241206

types-pytz==2024.1.0.20240417
types-pytz==2024.2.0.20241221

types-pyyaml==6.0.12.20240724
types-pyyaml==6.0.12.20241230

types-redis==4.6.0.20240726
types-redis==4.6.0.20241004

types-requests==2.32.0.20240712
types-requests==2.32.0.20241016

types-setuptools==71.1.0.20240726
types-setuptools==75.8.0.20250110

types-simplejson==3.19.0.20240310
types-simplejson==3.19.0.20241221

urllib3==2.2.2
urllib3==2.3.0

waitress==3.0.0
waitress==3.0.2

webob==1.8.7
webob==1.8.9

webtest==3.0.0
webtest==3.0.3

lucas03 · 2025-02-11T14:22:44Z

this is the traceback from tests:

_________________________ test_pending_flight_changes __________________________
self = <_TracedIterable at 0x7c1c724aee80 for ClosingIterator at 0x7c1c72fc23d0>
    def __next__(self):
        try:
            if self._self_wrapped_is_iterator:
                return next(self._wrapped_iterator)
            else:
>               return next(self.__wrapped__)
/usr/local/lib/python3.11/site-packages/ddtrace/_trace/trace_handlers.py:61: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <werkzeug.wsgi.ClosingIterator object at 0x7c1c72fc23d0>
    def __next__(self) -> bytes:
>       return self._next()
E       StopIteration
/usr/local/lib/python3.11/site-packages/werkzeug/wsgi.py:500: StopIteration
During handling of the above exception, another exception occurred:
>   ???
ddtrace/internal/_encoding.pyx:501: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
>   ???
E   OverflowError: Python int too large to convert to C long
ddtrace/internal/_encoding.pyx:892: OverflowError
During handling of the above exception, another exception occurred:
app = <test.integration.mambo.conftest.MamboClient object at 0x7c1c72e3f1d0>
scenario_return_trip = Scenario(booking=<kw.naomi.booking.bookings.Booking object at 0x7c1c7290e250>, user=<kw.naomi.booking.users.User objec...cGxlLm9yZyIsImJyYW5kIjoia2l3aWNvbSIsImV4cCI6MTczOTMwNDM5Mn0.J7VAvQgWnYWMV_9HWHmbZDaJq_Igx5Vy2bdRU2TlP6s'}, payments=[])
mock_flight_changes = (<kw.naomi.booking.alternatives.PendingFlightChange object at 0x7c1c859d20d0>, <kw.naomi.booking.alternatives.PendingFlightChange object at 0x7c1c724ad250>)
    def test_pending_flight_changes(app, scenario_return_trip, mock_flight_changes):
        response = app.get(f"/bookings/{scenario_return_trip.booking.id}/itinerary_changes/pending")
        assert response.status_code == 200
>       assert response.json == {
            "pending_flight_changes": [{"id": mock_flight_changes[1].id}, {"id": mock_flight_changes[0].id}]
        }
test/integration/mambo/itinerary_changes/test_pending_flight_changes.py:89: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/local/lib/python3.11/site-packages/werkzeug/wrappers/response.py:646: in json
    return self.get_json()
/usr/local/lib/python3.11/site-packages/werkzeug/wrappers/response.py:672: in get_json
    data = self.get_data()
/usr/local/lib/python3.11/site-packages/werkzeug/wrappers/response.py:314: in get_data
    self._ensure_sequence()
/usr/local/lib/python3.11/site-packages/werkzeug/wrappers/response.py:376: in _ensure_sequence
    self.make_sequence()
/usr/local/lib/python3.11/site-packages/werkzeug/wrappers/response.py:391: in make_sequence
    self.response = list(self.iter_encoded())
/usr/local/lib/python3.11/site-packages/werkzeug/wrappers/response.py:50: in _iter_encoded
    for item in iterable:
/usr/local/lib/python3.11/site-packages/werkzeug/wsgi.py:500: in __next__
    return self._next()
/usr/local/lib/python3.11/site-packages/ddtrace/_trace/trace_handlers.py:63: in __next__
    self._finish_spans()
/usr/local/lib/python3.11/site-packages/ddtrace/_trace/trace_handlers.py:80: in _finish_spans
    self._self_span.finish()
/usr/local/lib/python3.11/site-packages/ddtrace/_trace/span.py:313: in finish
    self._finish_ns(time_ns())
/usr/local/lib/python3.11/site-packages/ddtrace/_trace/span.py:326: in _finish_ns
    cb(self)
/usr/local/lib/python3.11/site-packages/ddtrace/_trace/tracer.py:869: in _on_span_finish
    p.on_span_finish(span)
/usr/local/lib/python3.11/site-packages/ddtrace/_trace/processor/__init__.py:370: in on_span_finish
    self._writer.write(spans)
/usr/local/lib/python3.11/site-packages/ddtrace/internal/writer/writer.py:324: in write
    self._write_with_client(client, spans=spans)
/usr/local/lib/python3.11/site-packages/ddtrace/internal/writer/writer.py:347: in _write_with_client
    client.encoder.put(spans)
ddtrace/internal/_encoding.pyx:844: in ddtrace.internal._encoding.MsgpackEncoderV05.put
    ???
ddtrace/internal/_encoding.pyx:845: in ddtrace.internal._encoding.MsgpackEncoderV05.put
    ???
ddtrace/internal/_encoding.pyx:851: in ddtrace.internal._encoding.MsgpackEncoderV05.put
    ???
ddtrace/internal/_encoding.pyx:848: in ddtrace.internal._encoding.MsgpackEncoderV05.put
    ???
ddtrace/internal/_encoding.pyx:511: in ddtrace.internal._encoding.MsgpackEncoderBase.put
    ???
ddtrace/internal/_encoding.pyx:515: in ddtrace.internal._encoding.MsgpackEncoderBase.put
    ???
ddtrace/internal/_encoding.pyx:538: in ddtrace.internal._encoding.MsgpackEncoderBase.put
    ???
ddtrace/internal/_encoding.pyx:519: in ddtrace.internal._encoding.MsgpackEncoderBase.put
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
>   ???
E   RuntimeError: failed to pack span: <Span(id=9557175821412020941,trace_id=137800164174119564133353386103302162856,parent_id=17168043924156404373,name=flask.request)>. Exception: Python int too large to convert to C long
ddtrace/internal/_encoding.pyx:503: RuntimeError

lucas03 · 2025-02-18T12:04:14Z

I found how I introduced this issue. As a part of requirements upgrade, I squashed a small change in:

from this:

    from ddtrace import config, tracer
    from ddtrace.sampler import RateSampler
    tracer.configure(enabled=False)

to this:

    if not test:
        from ddtrace import tracer
        tracer.configure(enabled=False)

so this no longer runs in our pytest jobs. This causes the span to be constructed with that big trace_id.

I don't know why, as I'd expect spans not to be created if not explicitly configured. And we have this in pytest-ini file:

addopts = --strict-markers -p no:ddtrace -p no:ddtrace.pytest_bdd -p no:ddtrace.pytest_benchmark -p no:ddtrace.internal._unpatched

but anyway, fixed this issue by adding this to our conftest.py file:

tracer.configure(enabled=False)

lucas03 added the bug label Feb 6, 2025

wantsui mentioned this issue Feb 14, 2025

fix(encoding): fix encoding errors when the span started a very long time ago #12368

Draft

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG]: Python int too large to convert to C long #12240

[BUG]: Python int too large to convert to C long #12240

lucas03 commented Feb 6, 2025 •

edited

Loading

mabdinur commented Feb 7, 2025

lucas03 commented Feb 10, 2025

lucas03 commented Feb 10, 2025

lucas03 commented Feb 11, 2025

lucas03 commented Feb 18, 2025

[BUG]: Python int too large to convert to C long #12240

[BUG]: Python int too large to convert to C long #12240

Comments

lucas03 commented Feb 6, 2025 • edited Loading

Tracer Version(s)

Python Version(s)

Pip Version(s)

Bug Report

Reproduction Code

Error Logs

Libraries in Use

Operating System

mabdinur commented Feb 7, 2025

lucas03 commented Feb 10, 2025

lucas03 commented Feb 10, 2025

lucas03 commented Feb 11, 2025

lucas03 commented Feb 18, 2025

lucas03 commented Feb 6, 2025 •

edited

Loading