Missing BT device when scanning stopped.

[krystianpe]

First of all amazing project.

Some feedback after testing it. I had to recompile bodycomposition because I'm using RPi with arm64. I can provide the 64-bit version if you would wish to add it. I also had some problems with BLE scanning. Turns out when the scanning was manually stopped (instead of the timeout) it wouldn't add the last device detected (which in that case would be the MiScale). For now I just removed that line from the code and reduced scanning time to 20sec. Not sure why that happens...

[RobertWojtowicz]

First of all amazing project.

Thanks

Some feedback after testing it. I had to recompile bodycomposition because I'm using RPi with arm64. I can provide the 64-bit version if you would wish to add it.

Orignal project is here https://github.com/davidkroell/bodycomposition and includes the arm64 package.

I also had some problems with BLE scanning. Turns out when the scanning was manually stopped (instead of the timeout) it wouldn't add the last device detected (which in that case would be the MiScale). For now I just removed that line from the code and reduced scanning time to 20sec. Not sure why that happens...

I've been using my solution for over a year and I haven't noticed such things. Are you using Arduino ESP32 version 1.4 library? or maybe you are using RPI for BLE synchronization? (unfortunately I cannot help with that).

[krystianpe]

Orignal project is here https://github.com/davidkroell/bodycomposition and includes the arm64 package.

It does but the latest version is missing the fix for the EOF issue.

I've been using my solution for over a year and I haven't noticed such things. Are you using Arduino ESP32 version 1.4 library? or maybe you are using RPI for BLE synchronization? (unfortunately I cannot help with that).

Sorry, I wasn't specific enough here. I'm using LOLIN32 board but with Arduino ESP32 2.0.0. I couldn't find version 1.4 (all I see is version 2.0.0 and then 1.0.6 and below). Btw. my logs look like this (no code changes, except for the extra log added):

Mi Body Composition Scale 2 Garmin Connect v2.2

* Starting BLE scan:
* BLE device found with address: 8c:79:f5:16:11:49, non-target device
* BLE device found with address: 5f:21:cc:a5:65:18, non-target device
* BLE device found with address: 0c:95:41:ca:76:8b <= target device
* foundDevices.getCount(): 2
* Waiting for next scan, going to sleep

[RobertWojtowicz]

It does but the latest version is missing the fix for the EOF issue.

Wait a while, new version will come out, working on extending functionality:
davidkroell/bodycomposition#9
tormoder/fit#61

Sorry, I wasn't specific enough here. I'm using LOLIN32 board but with Arduino ESP32 2.0.0. I couldn't find version 1.4 (all

Download from here https://github.com/espressif/arduino-esp32
Change repository and version: https://www.hackster.io/abdularbi17/how-to-install-esp32-board-in-arduino-ide-1cd571
Maybe it's version 1.0.4, I might have made a mistake, I'll check it and let you know

[krystianpe]

Wait a while, new version will come out, working on extending functionality: davidkroell/bodycomposition#9 tormoder/fit#61

Thanks. For now I compiled one by myself and works fine. But I'll keep an eye on that repo.

Sorry, I wasn't specific enough here. I'm using LOLIN32 board but with Arduino ESP32 2.0.0. I couldn't find version 1.4 (all

Download from here https://github.com/espressif/arduino-esp32 Change repository and version: https://www.hackster.io/abdularbi17/how-to-install-esp32-board-in-arduino-ide-1cd571 Maybe it's version 1.0.4, I might have made a mistake, I'll check it and let you know

If it's 1.0.4 then I'll try it out.

[krystianpe]

I couldn't compile it against 1.0.4, some errors around WiFiUdp.h. Had more luck with 1.0.5 and 1.0.6 but with both had weird BT behaviour. It would scan the same devices multiple times and in the end it would still show 0 devices found. So for me the only working version is 2.0.0 but luckily I found out that adding 200ms delay between ::getScan() and ::stop() solves the problem of that last detected device not being on the list.

if (advertisedDevice.getAddress().toString() == scale_mac_addr) {
        Serial.println(" <= target device");
        BLEScan *pBLEScan = BLEDevice::getScan(); // found what we want, stop now
        delay(200);
        pBLEScan->stop();
      }

[RobertWojtowicz]

I had the same, and that's why the older library (1.0.4) solved the problem. I am adding your fix to the code.

[RobertWojtowicz]

@krystianpe
I tested your fix and on the 2.0.0 library and I see that it doesn't work quite right, it looks like this:

* Starting BLE scan:
* BLE device found with address: 24:fc:e5:8f:ce:bf, non-target device
* BLE device found with address: 3e:e4:ce:ed:d2:d3, non-target device
* BLE device found with address: 79:96:d6:d1:29:85, non-target device
* BLE device found with address: 68:27:19:1b:31:83, non-target device
* BLE device found with address: 4d:73:47:6e:c9:37, non-target device
* BLE device found with address: c8:b2:1e:c9:66:83 <= target device
* Waiting for next scan, going to sleep

On the old 1.0.4 library, it looks like this:

* Starting BLE scan:
* BLE device found with address: 30:c2:88:c7:82:85, non-target device
* BLE device found with address: 24:fc:e5:8f:ce:bf, non-target device
* BLE device found with address: c8:b2:1e:c9:66:83 <= target device
* Reading BLE data complete, finished BLE scan
* Connecting to WiFi: connected
* IP Address: 192.168.4.18
* Connecting to MQTT: connected
* Publishing MQTT data: 62.65;525;kg;7;1636193299;2021-11-6 11:8:19;4.14;95
* Waiting for next scan, going to sleep

Did you correct anything else ?

[krystianpe]

Yup, turns out it wasn't always working. For now I changed it to:

class MyAdvertisedDeviceCallbacks : public BLEAdvertisedDeviceCallbacks {
    bool deviceFound = false;

    void onResult(BLEAdvertisedDevice advertisedDevice) {
      Serial.print("* BLE device found with address: ");
      Serial.print(advertisedDevice.getAddress().toString().c_str());
      if (advertisedDevice.getAddress().toString() == scale_mac_addr) {
        Serial.println(" <= target device");
        deviceFound = true;
      }
      else {
        Serial.println(", non-target device");
        if (deviceFound) {
          BLEScan *pBLEScan = BLEDevice::getScan(); // found what we want, stop now
          pBLEScan->stop();
        }
      }
    }
}

it's not perfect but seems to work. Now it stops scanning on the next device after MiScale is detected.

[RobertWojtowicz]

@krystianpe
Thanks for the info, it definitely works better on the old 1.0.4 library, I really recommend it, it works faster.

[krystianpe]

Yup, fixed my problem with 1.0.4 and BigSur (espressif/arduino-esp32#4408) and was able to compile against it. It works... but I wouldn't be myself if I left it as it is, so I dig some more and found something that looks like a probable cause of it.
https://github.com/espressif/arduino-esp32/blob/674cf812e7feb42aafd644649c1889d59af447f2/libraries/BLE/src/BLEScan.cpp#L127-L133

Here it passes the advertised device to the callback but at this point it's not yet stored in the vector, so when you stop scanning it ignores the last result. It seems like that's how they want it to be, though maybe it would make sense to report it...
Workaround for that is to store locally serviceData during that callback and then just parse that data instead of going through the BLEScanResults in search for the scale. Like:

std::string miScaleData = "";
...
        Serial.println(" <= target device");
        miScaleData = advertisedDevice.getServiceData();
        BLEScan *pBLEScan = advertisedDevice.getScan(); // found what we want, stop now
        pBLEScan->stop();
...
  BLEScanResults foundDevices = pBLEScan->start(30);
  if (miScaleData.length() > 0) {
    parseData(miScaleData);
    miScaleData = "";
  }
...
  void parseData(std::string d) {
  String hex;
  uint8_t* mdp = (uint8_t*)d.data();
  char *pHex = BLEUtils::buildHexData(nullptr, mdp, d.length());
  hex = pHex;
  free(pHex);
...

Previously I would say this works but since I had so many failed attempts, I'm not promising anything now :D (though it works for me)

[RobertWojtowicz]

@krystianpe
I am not a good programmer (as an admin I use scripts), can you send me the file with the changes or pull reuqest ? (changes are not compiling correctly, probably I corrected it wrong)

[krystianpe]

Sure, here are the changes in better context. I'm sending it here instead of PR, as I made few other changes and I'm not sure you'd want them, so pick whatever you need from this. I also tested it against 2.0.1-RC1, as it seems to have quite a few bug fixes, including some in the BLE library.

...
// Battery voltage measurement, for LOLIN32 D32 PRO is pin 35
Battery18650Stats battery(35);

std::string miScaleData = "";

// MQTT details
...
class MyAdvertisedDeviceCallbacks : public BLEAdvertisedDeviceCallbacks {

    void onResult(BLEAdvertisedDevice advertisedDevice) {
      Serial.print("* BLE device found with address: ");
      Serial.print(advertisedDevice.getAddress().toString().c_str());
      if (advertisedDevice.getAddress().toString() == scale_mac_addr) {
        Serial.println(" <= target device");
        miScaleData = advertisedDevice.getServiceData();
        BLEScan *pBLEScan = advertisedDevice.getScan(); // found what we want, stop now
        pBLEScan->stop();
      }
      else {
        Serial.println(", non-target device");
      }
    }
};
...
void ScanBLE() {
  Serial.println("* Starting BLE scan:");
  BLEDevice::init("");
  BLEScan *pBLEScan = BLEDevice::getScan(); //Create new scan.
  pBLEScan->setAdvertisedDeviceCallbacks(new MyAdvertisedDeviceCallbacks(), false, true);
  pBLEScan->setActiveScan(false); //Active scan uses more power.
  pBLEScan->setInterval(0x50);
  pBLEScan->setWindow(0x30);

  // Scan for 30 seconds
  BLEScanResults ignored = pBLEScan->start(30, false);
  if (miScaleData.length() > 0) {
    parseData(miScaleData);
    miScaleData = "";
  }
}

void parseData(std::string d) {
  String hex;
  uint8_t* mdp = (uint8_t*)d.data();
  char *pHex = BLEUtils::buildHexData(nullptr, mdp, d.length());
  hex = pHex;
  free(pHex);
  float weight = stoi2(hex, 22) * 0.005;
  float impedance = stoi2(hex, 18);
  if (unNoImpedanceCount < 3 && impedance <= 0) {
    errorLED_scan();
  }
  ...

[RobertWojtowicz]

@krystianpe

I tested your fix on version 2.0.0, this is how it solves the problem with BLE, unfortunately I noticed a very unstable connection to WiFi (every second connection is successful, this problem does not exist in version 1.0.4).
Can you check if you have a similar problem ?

[krystianpe]

I haven't noticed any issues but will have a look. Have you tried 2.0.1-RC1, maybe that one has something that would help. Either way I'll check and get back to you if I find anything helpful.

[RobertWojtowicz]

Same problem on version 2.0.1-rc1 :(, back to stable version 1.0.4