Smart energy meter _TZE200_nslr42tt reports inadequate power values

[pznamenskii]

What happened?

HA via z2m on Sonoff ZBDongle-P; Tuya _TZE200_nslr42tt pairs OK and all values are reported; however power readings are well off the adequate line. Especially power readings go crazy when usage exceed 1.5kW.

"ac_frequency": 50,
"current": 9.43,
"current_a": 0.996,
"current_b": 6.668,
"current_c": 1.774,
"device_status": 0,
"energy": 274.64,
"energy_a": 34.827,
"energy_b": 16.109,
"energy_c": 223.713,
"linkquality": 171,
"power": 429496420,
"power_a": 25,
"power_b": 38606,
"power_c": 381,
"power_factor_a": 9,
"power_factor_b": 100,
"power_factor_c": 94,
"temperature": 39,
"voltage_a": 221.8,
"voltage_b": 226.6,
"voltage_c": 227.4

What did you expect to happen?

power readings correlate with voltage and current from this universe

How to reproduce it (minimal and precise)

No response

Zigbee2MQTT version

1.32.2-1

Adapter firmware version

20210708

Adapter

SONOFF Zigbee 3.0 USB Dongle Plus ZBDongle-P

Debug log

No response

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days

[pznamenskii]

any clues?

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days

[scarycky]

I'll bring up the topic, have you decided in any way? I have the same problem, the same power 2 phase (I think I need to update)

[eokgnah]

The crazy readings are "negative" values.

I measured a single line with two clamps. one reversed of the other.

{
    "ac_frequency": 50,
    "current": 0.29,
    "current_a": 0.143,
    "current_b": 0,
    "current_c": 0.146,
    "device_status": 0,
    "energy": 178.68,
    "energy_a": 69.161,
    "energy_b": 32.805,
    "energy_c": 76.721,
    "last_seen": "2024-08-04T13:46:06+02:00",
    "linkquality": 163,
    "power": 429496730,
    "power_a": 19,
    "power_b": 0,
    "power_c": 39302,
    "power_factor_a": 58,
    "power_factor_b": 0,
    "power_factor_c": 100,
    "temperature": 37.6,
    "voltage_a": 234.1,
    "voltage_b": 234.1,
    "voltage_c": 234.2
}

{"data":[9,37,0,0,143,0,0,19],"type":"Buffer"},"datatype":0,"dp":6}],"seq":28416}' from endpoint 1 with groupID 0
{"data":[9,38,0,0,146,153,153,134],"type":"Buffer"},"datatype":0,"dp":8}],"seq":28928}' from endpoint 1 with groupID 0

dt0, dp6 = 0,0,19

• which is power_A (measured forward) 19W

dt0, dp8 = 153,153,134

• which is power_B (measured reverse) and should be about -19W, but it is shown: 39302

I assume the conversion is wrong and doesn't mind the negative value.

The Meter i use has the Fingerprint: _TZE200_nslr42tt

[eokgnah]

I'll bring up the topic, have you decided in any way? I have the same problem, the same power 2 phase (I think I need to update)

could it be that either your L2 clamp is clipped arroud the cable in reverse (mind the direction arrow printed on the clamp) or L2 is producing and not consuming. Maybe a balcony solar on L2 or something similar. Because the crazy numbers seem to be negative power values (production).

[eokgnah]

FFFF = 39321

so if the value is bigger then 30000 then the real value is ((39321-VLAUE) * -1)

in my example this would be:
"power_c": 39302,
((39321 - 39302) * -1) = -19

which is correct, since the reversed clamp on the same phase measured:
"power_a": 19,

i have no clue how to implement this into the tuya.js

[eokgnah]

@Koenkk
pls consider reopening the issue because of the new findings regarding the "inadequate values".
i think the reason is a wrong conversion.

[Darkangeel-hd]

Hi everyone, I'm having this issue too, and following what @eokgnah it makes total sense that those values are just negative numbers
But the value that he mentions (39322) is different in my end
In my test, with a constant 5W draw, tested on every phase meter, when in correct position I get a reading of 5W, but also on every phase meter when reversed i get 39317W
which suggest a calculation of ((39322−VALUE) * -1), which in my test would give me the -5 watts
39321 is 0x9999
39322 is 0x999A
i dunno which one is it

What I'm unsure about is the global energy number format
with one clamp at 39317W (-5W) it says 429496725W
with one at 39317W (-5W) and other at 5W, it says 429496730W

quick edit
would this mean that 429496730 is -0 and 429496725 is -5W
something like along the lines of ((429496730 − VALUE) * -1)
??

as im very new to the whole zigbee and Z2M stuff i'm not sure where or how to "debug" the values or see the data transfer
ill try read the docs tomorrow and see if i can figure something out
it would be lovely if we could get the reversed readings on this thing too

[Darkangeel-hd]

I was too tired yesterday, @eokgnah was right, and my load may have changed power during measurements
It makes more sense than normal positive measurements go till either 6553W (0x1999, which seems a bit too low or 32767W (0x7FFF)
and that negative measurements start at 39321W (0x9999), which would be -0W, and go till 32768W (0x8000), which would be −6553W

those arbitrary numbers seem a bit off to me tho

What happens to the whole 0x999A to 0xFFFF range??
cause to make it would have make more sense to use numbers in 2's complement and call it a day...

anyway, I would read the docs today and try to get measurements

[Darkangeel-hd]

after running some more test
with all three clamps on the same line with the same load on different directions

Total positive:

Screenshots when total is positive value

By the pictures, we can deduct that 39125W is -197W ,39124W is -198W and 39123W is -199W

So we should be using ((39322-VALUE) * -1)
or 0x999A - VALUE from values greater than 7FFF

correct me if I'm wrong, please

regarding the global power value

Screenshot

if the above calculations are okay, then here we find: -199 (39123) + 198 -198 (39124), which would result in 199W (429496531) leaving the global value to the formula `((429496730−VALUE)* -1)` if value is above 0x0FFFFFFF (268435455W).

More images here:

Screenshots when total is negative value

I sadly notice that even if we have the negative values for the inmediate power, the energy usage when values are negative rises as if they were positive
so i dont think we could have net usage anyway...

@Koenkk in a bit familliar with js and ts, but i dont think im up to the task, nevertheless, where can i lear a bit more on how the converters works and how to test them?
i read the Support new devices and Support new Tuya devices but i still have doubt on how to do some stuff
sadly I don't have a Tuya ZB gateway to try and use de auto discovery method for data points

[Koenkk]

@Darkangeel-hd

Could you check if the issue is fixed with the following external converter:

• save this as file next to configuration.yaml as ext_converter.js
• add it to configuration.yaml:

external_converters:
  - ext_converter.js

• start z2m,
• in the frontend, check if the description of this device now ends with CUSTOM (this indicates the external converter has been loaded correctly)
• check if issue is fixed

[Darkangeel-hd]

Hi @Koenkk, thank you for replying
I'll try the new converter as soon as I can, cause I'm away from home for a few days and i left the meter disconnected.
Thank you so much

[Darkangeel-hd]

Hello again @Koenkk
I tried the external converter

it shows the same measurements as the original

Spoiler

And i have this error repeated in the logs

error: z2m: Exception while calling fromZigbee converter: Buffer is not defined}

Im using version zigbee2mqtt-1.37.1-r0 from Alpine repos, should i try it on a more up to date version?
using docker for example?

[Darkangeel-hd]

Okay so i added const {Buffer} = require('buffer'); to the definition as per your comment in here

now i have these readings

and with a greater load

Those readings seem very accurate to me (taking into account the A*V = W formula)
So i guess it's working fine

Now what would be left to do is "fix" the global reading

Which i guess would be almost the same code, but substracting 0x1999999A instead of 0x999A

[Darkangeel-hd]

I came up with some rudimentary edit of your functions

    globalVariant2Custom: () => {
        return {
            from: (v) => {
                let power = v;
                if (power > 0x0FFFFFFF) {
                    power = (0x1999999a - power) * -1;
                }
                return power;
            },
        };
    },

Which of course ain't how it should be, but i think it allows some testing on my end

and got the following results

Spoiler

Which seem a bit inaccurate

Edit:

Welp, after a bunch of minutes analyzing the results with mqtt explorer

i cannot tell if the inaccuracies are because of the sensor data of because of the converter value
as the difference between the total power and the per phase power of the only connected phase keeps on changing

It seems to me that the global value is updated on another interval and it just remains on the old value till the next update

Edit 2:
Yeah my hypothesis seems right
as the same behavior is displayed when using normal positive values

difference ranging from 1 to 10W

[Darkangeel-hd]

So we at last got both functions to get accurate negative readings on this device

Regarding the accumulated energy consumption, a Riemann integral can be used in home assistant to get a pretty accurate value, I've already tested that on other devices.

Here's the external converter code i am using right now, in case there's any doubts

ext_converter.js

const fz = require('zigbee-herdsman-converters/converters/fromZigbee');
const tz = require('zigbee-herdsman-converters/converters/toZigbee');
const exposes = require('zigbee-herdsman-converters/lib/exposes');
const reporting = require('zigbee-herdsman-converters/lib/reporting');
const tuya = require('zigbee-herdsman-converters/lib/tuya');
const e = exposes.presets;
const ea = exposes.access;
const legacy = require('zigbee-herdsman-converters/lib/legacy');
const lumi = require('zigbee-herdsman-converters/lib/lumi');
const utils = require('zigbee-herdsman-converters/lib/utils');
const {Buffer} = require('buffer');

const valueConverter = {
    phaseVariant2WithPhaseCustom: (phase) => {
        return {
            from: (v) => {
                // https://github.com/Koenkk/zigbee2mqtt/issues/18603#issuecomment-2277697295
                const buf = Buffer.from(v, 'base64');
                let power = buf[7] | (buf[6] << 8);
                if (power > 0x7fff) {
                    power = (0x999a - power) * -1;
                }
                return {
                    [`voltage_${phase}`]: (buf[1] | (buf[0] << 8)) / 10,
                    [`current_${phase}`]: (buf[4] | (buf[3] << 8)) / 1000,
                    [`power_${phase}`]: power,
                };
            },
        };
    },
    globalVariant2Custom: () => {
        return {
            from: (v) => {
                let power = v;
                if (power > 0x0FFFFFFF) {
                    power = (0x1999999c - power) * -1;
                }
                return power;
            },
        };
    },
};

const definition = {
    fingerprint: tuya.fingerprint('TS0601', ['_TZE200_nslr42tt']),
    model: 'TS0601_3_phase_clamp_meter',
    vendor: 'Tuya',
    description: '3-phase clamp power meter CUSTOM',
    fromZigbee: [tuya.fz.datapoints],
    toZigbee: [tuya.tz.datapoints],
    configure: tuya.configureMagicPacket,
    whiteLabel: [
        {vendor: 'MatSee Plus', model: 'PC321-Z-TY'},
        {vendor: 'Owon', model: 'PC321-Z-TY'},
    ],
    exposes: [
        e.ac_frequency(),
        e.temperature(),
        e.current(),
        e.power(),
        e.energy(),
        tuya.exposes.energyWithPhase('a'),
        tuya.exposes.energyWithPhase('b'),
        tuya.exposes.energyWithPhase('c'),
        tuya.exposes.voltageWithPhase('a'),
        tuya.exposes.voltageWithPhase('b'),
        tuya.exposes.voltageWithPhase('c'),
        tuya.exposes.powerWithPhase('a'),
        tuya.exposes.powerWithPhase('b'),
        tuya.exposes.powerWithPhase('c'),
        tuya.exposes.currentWithPhase('a'),
        tuya.exposes.currentWithPhase('b'),
        tuya.exposes.currentWithPhase('c'),
        tuya.exposes.powerFactorWithPhase('a'),
        tuya.exposes.powerFactorWithPhase('b'),
        tuya.exposes.powerFactorWithPhase('c'),
    ],
    meta: {
        multiEndpointSkip: ['power_factor', 'power_factor_phase_b', 'power_factor_phase_c', 'energy'],
        tuyaDatapoints: [
            [132, 'ac_frequency', tuya.valueConverter.raw],
            [133, 'temperature', tuya.valueConverter.divideBy10],
            [1, 'energy', tuya.valueConverter.divideBy100],
            [101, 'energy_a', tuya.valueConverter.divideBy1000],
            [111, 'energy_b', tuya.valueConverter.divideBy1000],
            [121, 'energy_c', tuya.valueConverter.divideBy1000],
            [131, 'current', tuya.valueConverter.divideBy1000],
//            [9, 'power', tuya.valueConverter.raw],
            [9, 'power', valueConverter.globalVariant2Custom()],
            [102, 'power_factor_a', tuya.valueConverter.raw],
            [112, 'power_factor_b', tuya.valueConverter.raw],
            [122, 'power_factor_c', tuya.valueConverter.raw],
            [6, null, valueConverter.phaseVariant2WithPhaseCustom('a')],
            [7, null, valueConverter.phaseVariant2WithPhaseCustom('b')],
            [8, null, valueConverter.phaseVariant2WithPhaseCustom('c')],
            [134, 'device_status', tuya.valueConverter.raw],
        ],
    },
};

module.exports = definition;

of course, it should be refined, as i got little idea of how it does actually work

[Koenkk]

Thanks, I've integrated the code.

Assuming this can be closed now.

Changes will be available in the dev branch in a few hours from now.

[Darkangeel-hd]

Hi @Koenkk Thank you so much
yes I will assume we can close this issue now, I don't know if @eokgnah have any other ideas?

the accumulated energy consumption being wrong when we have negative readings (they are counted as positive) is something that is device dependent and (imo) out of the scope of Z2M.
We should deal with that elsewhere.

[btchans]

Hi @Koenkk Thank you so much yes I will assume we can close this issue now, I don't know if @eokgnah have any other ideas?

the accumulated energy consumption being wrong when we have negative readings (they are counted as positive) is something that is device dependent and (imo) out of the scope of Z2M. We should deal with that elsewhere.

after updating tothe latest z2m power_[x] is now correctly displayed/calculated.
the combined value is something over 4xx.xxx.xxx.

would love to see that fixed too.
please link the solution, if you find one.

and thanks for the affords so far!

[Koenkk]

@Darkangeel-hd does the power show the correct value for you?

[eokgnah]

I tested with a fresh install of z2m.
Looks fine to me now! :-)
Thanks!

[btchans]

also the overall power?
did you remove the ext_converter.js?

[eokgnah]

also the overall power? did you remove the ext_converter.js?

Sorry, i only looked at the negative values on the L1-3 with a quick test setup.
But now that i know, i will look into it further this evening.
ext_converter.js? no... did not alter anything.
just a plain new install on a new machine for testing.
ubuntu, mqtt, z2m, nodered

[Darkangeel-hd]

@Darkangeel-hd does the power show the correct value for you?

Yes, but i'm still using the external converter, as i did not update z2m to the latest version (i'm using alpine repos one, not docker)
I can try using the functions you added to the repo if you want me to, as i saw that the global power one is different.

Pardon my very late reply, I was very busy all this time

[Darkangeel-hd]

On another note...
I'm pretty sure that the current usage reported by the device is the apparent energy, not the active one
here a screenshot

case if you do the calculations, the active current should be lower.
V*A != W
V*A*PF ~= W

@Koenkk should we deal with this in z2m or in the receiving end?

EDIT:
I should also mention that when the power is negative (producing) the power factor stays at 100% even though the V*A doesn't look similar to W
so I don't really know how to deal with those edge cases

[Koenkk]

So probably we are using the wrong data points, to figure those out: https://www.zigbee2mqtt.io/advanced/support-new-devices/03_find_tuya_data_points.html