Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

taos.new_bind_params的timestamp绑定有很多隐式操作不容易理解造成bug #249

Open
lonelyleaf opened this issue May 16, 2024 · 0 comments

Comments

@lonelyleaf
Copy link

  1. 参数中如果传入的是List[float],会自动根据precision再做乘法,但int又不会,我批量写入的时候debug很久都没找到timestamp out of range的问题就是隐式做了乘法,既然我都指定精度了,为什么sdk内部还要乘法?

图片

如果设计如此可不可以完善文档说明这些问题?

比如这样一个用例

timestamp_list  = (series.index.astype('int64') / 10 ** 6).tolist()
params[0].timestamp(timestamp_list)

会报错 Timestamp data out of range

stmt = c_void_p(2809582648032)

    def taos_stmt_execute(stmt):
        # type: (ctypes.c_void_p) -> None
        """Execute a statement query
        @stmt: TAOS_STMT*
        """
        res = _libtaos.taos_stmt_execute(stmt)
        if res != 0:
>           raise StatementError(msg=taos_stmt_errstr(stmt), errno=res)
E           taos.error.StatementError: [0x060b]: Timestamp data out of range
  1. 所有的bind都不支持直接传入numpy的array,这点不太方便

  2. 如果我传入一个List[datetime]但带时区的,绑定timestamp也会报错

from datetime import timedelta, datetime

import pandas as pd
import numpy as np
import taos
from iso8601 import iso8601
from sqlalchemy import create_engine, text
from taos import TaosMultiBind

conn = taos.TaosConnection = (
    taos.connect(host="127.0.0.1",
                 user="root",
                 port=6030,
                 password="taosdata",
                 timezone="Asia/Shanghai")
)

start_time = iso8601.parse_date('2024-04-28T00:00:00+08:00')
end_time = start_time + timedelta(hours=24)
time_range = pd.date_range(start_time, end_time, freq='min')
random_values = np.random.uniform(0, 45, len(time_range))
random_series = pd.Series(random_values, index=time_range)

stmt = conn.statement("INSERT INTO ? USING abc.metric TAGS(?) VALUES(?, ?, ?)")
tags = taos.new_bind_params(1)
tags[0].binary(['test_tag'])

params: TaosMultiBind = taos.new_bind_params(3)

time_list = random_series.index.to_pydatetime().tolist()
value_list = random_series.values.tolist()
gateway_id_list = np.full(random_series.size, 'test_id').tolist()
# datetime with timezone will throw exception
# TypeError: can't subtract offset-naive and offset-aware datetimes
params[0].timestamp(time_list)
params[1].float(value_list)
params[2].nchar(gateway_id_list)

conn.close()

# print index
print('batch write success')

error:

Traceback (most recent call last):
  File "E:\projects\python\py-wi-water-model-edge\venv\lib\site-packages\taos\bind.py", line 428, in timestamp
    buffer = cast(values, c_void_p)
  File "C:\Python310\lib\ctypes\__init__.py", line 510, in cast
    return _cast(obj, obj, typ)
ctypes.ArgumentError: argument 1: <class 'TypeError'>: wrong type

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "E:\projects\python\py-wi-water-model-edge\py_wi_water_model_edge\ts\tdengine_write_test.py", line 34, in <module>
    params[0].timestamp(time_list)
  File "E:\projects\python\py-wi-water-model-edge\venv\lib\site-packages\taos\bind.py", line 431, in timestamp
    buffer = buffer_type(*[_datetime_to_timestamp(value, precision) for value in values])
  File "E:\projects\python\py-wi-water-model-edge\venv\lib\site-packages\taos\bind.py", line 431, in <listcomp>
    buffer = buffer_type(*[_datetime_to_timestamp(value, precision) for value in values])
  File "E:\projects\python\py-wi-water-model-edge\venv\lib\site-packages\taos\bind.py", line 215, in _datetime_to_timestamp
    return int(round((value - _datetime_epoch).total_seconds() * 1000))
TypeError: can't subtract offset-naive and offset-aware datetimes
Exception ignored in: <function TaosConnection.__del__ at 0x000002C9AEB44280>
Traceback (most recent call last):
  File "E:\projects\python\py-wi-water-model-edge\venv\lib\site-packages\taos\connection.py", line 341, in __del__
  File "E:\projects\python\py-wi-water-model-edge\venv\lib\site-packages\taos\connection.py", line 62, in close
TypeError: 'NoneType' object is not callable
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant