Update platformio/espressif32 to the latest 6.7.0

[mskvortsov]

The issue #2793 of bluetooth pairing failure seems to be no longer present as has been tested on tlora_t3s3_v1. (Who knows what new issues the update might introduce.)

[caveman99]

Please use the same version for all esp32 targets and not necessarily the latest one.

[thebentern]

Cool. We'll do some testing and probably wait on another release before merging into a new alpha.

[mskvortsov]

There is some improvement in the final binary size.
Master against this branch for tlora-t3s3-v1:

Memory region         Used Size  Region Size  %age Used
     iram0_0_seg:       87776 B     362240 B     24.23%
     iram0_2_seg:     1471460 B    8388576 B     17.54%
     dram0_0_seg:      176520 B     345856 B     51.04%
     drom0_0_seg:     2032013 B    8388576 B     24.22%
    rtc_iram_seg:          33 B       8176 B      0.40%
    rtc_data_seg:          44 B       8176 B      0.54%
    rtc_slow_seg:          24 B         8 KB      0.29%
RAM:   [===       ]  32.1% (used 105128 bytes from 327680 bytes)
Flash: [========= ]  86.8% (used 2104449 bytes from 2424832 bytes)

Memory region         Used Size  Region Size  %age Used
     iram0_0_seg:       88472 B     362240 B     24.42%
     iram0_2_seg:     1368404 B    8388576 B     16.31%
     dram0_0_seg:      176776 B     345856 B     51.11%
     drom0_0_seg:     1857877 B   33554400 B      5.54%
    rtc_iram_seg:          33 B       8176 B      0.40%
    rtc_data_seg:          44 B       8176 B      0.54%
    rtc_slow_seg:          24 B         8 KB      0.29%
RAM:   [===       ]  31.9% (used 104680 bytes from 327680 bytes)
Flash: [========  ]  80.7% (used 1956141 bytes from 2424832 bytes)

rak11200:

Memory region         Used Size  Region Size  %age Used
     iram0_0_seg:      129664 B       128 KB     98.93%
     iram0_2_seg:     1443743 B    3342304 B     43.20%
     dram0_0_seg:       96172 B     124580 B     77.20%
     drom0_0_seg:      457867 B    4194272 B     10.92%
    rtc_iram_seg:         103 B       8176 B      1.26%
    rtc_data_seg:         108 B       8176 B      1.32%
    rtc_slow_seg:          24 B       7680 B      0.31%
  extern_ram_seg:          0 GB         4 MB      0.00%
RAM:   [===       ]  29.3% (used 96128 bytes from 327680 bytes)
Flash: [========  ]  84.5% (used 2048833 bytes from 2424832 bytes)

Memory region         Used Size  Region Size  %age Used
     iram0_0_seg:      117336 B       128 KB     89.52%
     iram0_2_seg:     1353739 B    3342304 B     40.50%
     dram0_0_seg:       94844 B     124580 B     76.13%
     drom0_0_seg:      412643 B    4194272 B      9.84%
    rtc_iram_seg:         103 B       8176 B      1.26%
    rtc_data_seg:         108 B       8176 B      1.32%
    rtc_slow_seg:          24 B       7680 B      0.31%
  extern_ram_seg:          0 GB         4 MB      0.00%
RAM:   [===       ]  28.9% (used 94836 bytes from 327680 bytes)
Flash: [========  ]  78.2% (used 1896877 bytes from 2424832 bytes)

[thebentern]

@mskvortsov can you re-test pairing BLE from scratch? I was initially surprised to see that it worked on my T-Beam Supreme PR with a previously cached BLE auth. However I was subsequently disappointed today when I found that the immediate auth failure does in fact still occur for me on brand new pairings with the 6 digit pin.

[mskvortsov]

I tested for the Android app only, as the failure in the mentioned #2793 is for Android specifically. A re-test is passing right now as well:

INFO  | ??:??:?? 489 Using random passkey
INFO  | ??:??:?? 489 *** Enter passkey 396662 on the peer side ***
DEBUG | ??:??:?? 489 [Screen] showing bluetooth screen
INFO  | ??:??:?? 505 BLE authentication complete

The iOS app is unavailable in my region, so I can't test that. However, after building the macOS app, I'm also disappointed to see now that the authentication fails:

INFO  | ??:??:?? 259 Using random passkey
INFO  | ??:??:?? 259 *** Enter passkey 438672 on the peer side ***
DEBUG | ??:??:?? 259 [Screen] showing bluetooth screen
INFO  | ??:??:?? 269 BLE disconnect

@thebentern Do you use the iPhone app to test?

In any case, since there are still auth issues, the PR is no good. Maybe I'll try to investigate this deeply in the coming days (sorry, without any obligation). In the meantime, I can extract the LTO and newlib nano parts into a separate pull request if these changes are welcome at all.

[thebentern]

The iOS app is unavailable in my region, so I can't test that. However, after building the macOS app, I'm also disappointed to see now that the authentication fails:

On the T-Beam Supreme I had failures in both iOS and Android. This was the case in that original issue as well, though I don't believe it was specifically reported

[mskvortsov]

The only thing that works stably for me is disabling BLE_SM_PAIR_AUTHREQ_SC. Far from ideal.

Sniffing the packets exchanged between phone and device revealed nothing. The only difference between Android and iPhone I spotted was that the latter additionally tried to negotiate an MTU size of 512 before starting the pairing procedure which looked the same right until an auth rejection packet.

Partial success was obtained by overwriting framework-arduinoespressif32's libbtbb.a and libphy.a with the ones from espressif/esp-phy-libs#release/v4.4. S3 connected to iPhone, C3 did not.

Porting to IDF v5.1.2 changed nothing compared to v4.4.7: Android connects, but iPhone does not (to a C3 device, S3 requires more effort to port). The same for Nimble update.

[thebentern]

A hunch, but it almost seems like the issue is happening with the persistence of the pairing that is preventing it from succeeding on new pairings.

Porting to IDF v5.1.2 changed nothing compared to v4.4.7: Android connects, but iPhone does not (to a C3 device, S3 requires more effort to port). The same for Nimble update.

[mskvortsov]

persistence of the pairing that is preventing it from succeeding on new pairings

I'm not entirely sure what you mean, but at least setting CONFIG_BT_NIMBLE_MAX_BONDS to 0 has no effect.

Adding some tracing to Nimble and comparing a successful trace against a failing one shows divergence at the very end of the auth procedure. BLE_HCI_EVCODE_ENCRYPT_CHG event is not fired ("ble_hs_event_rx_hci_ev 08" below).

 XXX ble_sm_ltk_req_reply_tx
 ble_hs_hci_cmd_send: ogf=0x08 ocf=0x001a len=18
-0x1a 0x20 0x12 0x01 0x00 0xb1 0x70 0x59 0xb7 0xba 0xf5 0xf2 0x8d 0x62 0x7b 0x67 0xd6 0x39 0xac 0x11 0x14
+0x1a 0x20 0x12 0x01 0x00 0xdb 0x3b 0x3d 0xbb 0x89 0x38 0x66 0xb2 0x00 0xbd 0xcf 0xc3 0x94 0x02 0x2a 0xce
 XXX ble_hs_hci_rx_evt, ev->opcode 0e cmd_complete->opcode 0020 enqueue 0
 XXX ble_sm_proc_set_timer
 XXX ble_sm_timer
 XXX ble_sm_extract_expired
 XXX ble_sm_num_procs
 XXX ble_hs_hci_rx_evt, ev->opcode 08 cmd_complete->opcode 0100 enqueue 1
-XXX ble_hs_event_rx_hci_ev 08
-XXX ble_hs_hci_evt_process
-XXX ble_hs_hci_evt_dispatch_find, found, event code 08
-XXX ble_hs_hci_evt_encrypt_change
-XXX ble_sm_enc_change_rx
-XXX ble_sm_enc_event_rx
-XXX ble_sm_proc_find
-XXX ble_sm_proc_matches
-YYY proc state 05
-XXX ble_sm_update_sec_state encrypted 1 authenticated 1 bonded 0 key_size 16

And right after that ble_sm_pair_fail_tx effectively terminates the pairing:

 XXX ble_sm_proc_find
 XXX ble_sm_proc_matches
-XXX ble_sm_key_rxed
-rx_key_flags=0x08
 XXX ble_sm_process_result
 XXX ble_sm_proc_find
 XXX ble_sm_proc_matches
-XXX ble_sm_proc_set_timer
+XXX ble_sm_proc_remove
+XXX ble_sm_num_procs
+XXX ble_sm_pair_fail_tx
+XXX ble_sm_cmd_get
+XXX ble_sm_tx

nimble-arduino-1.4.1-trace.patch
ble-debug-android-pass.txt
ble-debug-ios-fail.txt

[mskvortsov]

Enabling IRK distribution from both ends solved the issue for my setup, and now both android and apple apps connect successfully to both the esp32c3 and esp32s3 devices I have. So there is no need to disable BLE SC. Supposedly, BLE RPA was involved somehow.

[mskvortsov]

@thebentern Could you check whether the last fix works for you?

[thebentern]

Works perfectly now, @mskvortsov. Stellar work! That one had been stumping us for a while. This is a big deal to be able to move forward with the newer framework features.