Getting lots of "Peripheral Disconnected" on iOS 15 and 16

[QuentinFarizon]

DFU Bootloader version (please complete the following information):

• SDK version: 16.0.0
• Bonding used: no
• Library version: 4.9.0

Device information (please complete the following information):

• Device: all versions
• OS: iOS 15.6 to 16.1

Describe the bug
We are automatically logging DFU errors when they occur on customer phones, and we have an uptick of "Peripheral Disconnected" errors for customers on recent iOS versions (15.6 to 16.1) and especially on 16.0 and 16.1.
It happens uniformly across all device versions, iPhone 8 to 15,3

Here is the number of errors per device version :

  version   | count 
------------+-------
 iOS 16.0.2 |    48
 iOS 15.6.1 |    44
 iOS 16.0   |    24
 iOS 16.1   |    24
 iOS 15.7   |    14
 iOS 16.0.3 |     8
 iOS 15.5   |     5
 iOS 16.1.1 |     4
 iOS 14.4   |     1

Customer phones that have this error can never update their devices, but their device is updatable from an other phone (be it Android or another iPhone).

We have two iPhone at office, but we do not reproduce.

Logs
Message : Peripheral Disconnected
Description : The specified device has disconnected from us.

[dinesharjani]

There was a change in iOS 16, not official, but BLE documentation regarding minimum connection parameters is enforced much more strictly than it was before, hence the disconnections. Please check whether the minimum connection parameters of the device firmware you're trying to DFU to are valid, Apple's Documentation regarding this can be found here: https://developer.apple.com/library/archive/qa/qa1931/_index.html

[philips77]

The numbers may be correlated with distribution of iOS version used. iOS 16 and 15.6.1 are the most popular. Did you try to get a percentage of dfu ending with failures per dfu with any results on those iOS versions?

[QuentinFarizon]

There was a change in iOS 16, not official, but BLE documentation regarding minimum connection parameters is enforced much more strictly than it was before, hence the disconnections. Please check whether the minimum connection parameters of the device firmware you're trying to DFU to are valid, Apple's Documentation regarding this can be found here: https://developer.apple.com/library/archive/qa/qa1931/_index.html

Thanks for the reference, we will try changing our advertising period to one recommended. However, our customers dont have any issue scanning our devices for anything other than DFU, si I'm not sure why this would be an issue specific to DFU.

[QuentinFarizon]

The numbers may be correlated with distribution of iOS version used. iOS 16 and 15.6.1 are the most popular. Did you try to get a percentage of dfu ending with failures per dfu with any results on those iOS versions?

This is a projet that has been up for 3+ years, we started getting this issue with the rise of iOS 15.6, and even more now with iOS 16 becoming popular

[dinesharjani]

@QuentinFarizon scanning does not 'connect to a device'. It's just scanning. DFU is a problem if the connection parameters are wrong becuase the iPhone will drop the connection if it can't negotiate for parameters it likes. We noticed this about 1.5 years ago when DFU'ing between the Mac and a firmware device, with the Mac proving to be a lot more strict regarding holding the connection. So if you're able to, try to test if you can run your software from a Mac and try to upgrade. Apple allows iPhone(s) to be used as BLE Sniffers so you can catch the traffic and get more information as to why the connection is being dropped. Anyway - the intervals are important because for DFU the connection needs to be held for a while, and likely, will be negotiated a couple of times from one side or the other.

[QuentinFarizon]

@QuentinFarizon scanning does not 'connect to a device'. It's just scanning. DFU is a problem if the connection parameters are wrong becuase the iPhone will drop the connection if it can't negotiate for parameters it likes. We noticed this about 1.5 years ago when DFU'ing between the Mac and a firmware device, with the Mac proving to be a lot more strict regarding holding the connection. So if you're able to, try to test if you can run your software from a Mac and try to upgrade. Apple allows iPhone(s) to be used as BLE Sniffers so you can catch the traffic and get more information as to why the connection is being dropped. Anyway - the intervals are important because for DFU the connection needs to be held for a while, and likely, will be negotiated a couple of times from one side or the other.

Yes sorry, but our customers also dont have any issue connecting our devices for anything other than DFU, even for long periods.
But, granted, DFU could be a special case.

[dinesharjani]

Can you reproduce the issue? Or give us some form of example or sample firmware we can flash to a devkit to replicate the issue?

[philips77]

This is a projet that has been up for 3+ years, we started getting this issue with the rise of iOS 15.6, and even more now with iOS 16 becoming popular

The data are taken over what time period?

[QuentinFarizon]

Well, 3 years Le mer. 23 nov. 2022, 10:07, Aleksander Nowakowski ***@***.***> a écrit :

…

This is a projet that has been up for 3+ years, we started getting this issue with the rise of iOS 15.6, and even more now with iOS 16 becoming popular The data are taken over what time period? — Reply to this email directly, view it on GitHub <#492 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIATUIGQ2C2EYM2FHE4ZU4TWJXNEVANCNFSM6AAAAAASEKXMHQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[wiesener]

Any progress here?

[dinesharjani]

@wiesener this is outside of our control as well - we can't change the BLE API behaviour on iOS. We only make our apps based on it and make our own observations about it. We've given you already all of the info that we have regarding the connection parameters as well - for the moment it looks like iOS 16.1 reverted to the old behaviour, so you should see an improvement as customers update their devices.

[wiesener]

Okay, but the thing gets more complicated. I've tested several devices and versions.
The iphone 7 does not work with iOS 15.7.
The iphone SE 2nd gen with iOS 15.7 does work
The Ipad 6 th gen with 15.7 does work sometimes
The Ipad 6 th gen with 16.1 does always work
The Iphone 12 with 16.1 does always work

Kind of strange. I guess it is not just the ios version but as well the chip set.

[philips77]

I'll have a look into this library this or next week. Sorry for the delay, but I am busy with some other tasks.

[QuentinFarizon]

On one customer, we pushed (DFU via an Android) the change of avertising period to the Apple-recommended value of 318.75 ms (instead of the 300 ms we had).
After that, the customer succeeded in doing other DFUs (it never worked before).

I feel this is now a strong requirement from Apple, maybe this should be enforced (or strongly suggested) in the NRF SDK ? At least warning that other values may cause issues with DFU on iOS.

[philips77]

I've heard that iOS 16.1 stepped back from making it a strong requirement, although didn't have a chance to confirm it yet.

[QuentinFarizon]

@philips77 It definitely got worse with iOS 16.1 and onward. No customers on iOS can now do a DFU.

Changing advertising period didn't in fact solve the issue, and couldn't, because the real issue is that we are disconnected from device, after a successful connection, just when sending the command to switch to Bootloader.

We have investigated further, and by comparing DFU sequences on iOS and Android, we can see that on iOS an ATT MTU exchange is done, but not on Android (see logs below). We suspect that this MTU request started being made from iOS 15.7 and onward, and that there is a race condition between this MTU request and the sending of command to switch to bootloader.

Contrary to the nRF Connect application, where you connect, then choose the file, then click start DFU, we start the DFU as soon as the connection succeeds

So we added a 3 seconds delay between connect and starting the DFU, and it definitely solved the issue.

I think that this lib should somehow wait for the MTU request to be done, before sending the command.

Logs from device when connecting+DFU from iOS :

00> <info> app: Phone connected
00> <debug> app: PHY update request.
00> <debug> app: Unkwown on_ble_peripheral_evt : 34
00> <debug> nrf_ble_gatt: Peer on connection 0x1 requested a data length of 251 bytes.
00> <debug> nrf_ble_gatt: Updating data length to 27 on connection 0x1.
00> <debug> app: Unkwown on_ble_peripheral_evt : 35
00> <debug> nrf_ble_gatt: Data length updated to 27 on connection 0x1.
00> <debug> nrf_ble_gatt: max_rx_octets: 27
00> <debug> nrf_ble_gatt: max_tx_octets: 27
00> <debug> nrf_ble_gatt: max_rx_time: 2120
00> <debug> nrf_ble_gatt: max_tx_time: 2120
00> <debug> app: ATT MTU exchange completed. central 0x17 peripheral 0x17
00> <debug> app: Unkwown on_ble_peripheral_evt : 36
00> <debug> nrf_ble_gatt: Peer on connection 0x1 requested an ATT MTU of 293 bytes.
00> <debug> nrf_ble_gatt: Updating ATT MTU to 23 bytes (desired: 23) on connection 0x1.
00> <info> app: Data len is set to 0x14(20)
00> <debug> app: ATT MTU exchange completed. central 0x17 peripheral 0x17
00> <debug> app: Unkwown on_ble_peripheral_evt : 85
00> <debug> app: Unkwown on_ble_peripheral_evt : 80
<from there, nothing, we are connected, but we do not receive the DFU command>

Logs from device when connecting+DFU from Android :

00> <info> app: Phone connected
00> <debug> app: Unkwown on_ble_peripheral_evt : 18
00> <debug> app: Unkwown on_ble_peripheral_evt : 18
00> <debug> app: Unkwown on_ble_peripheral_evt : 80
00> <debug> app: Unkwown on_ble_peripheral_evt : 81
00> <info> app: Device is preparing to enter bootloader mode.
00> <debug> app: Disconnected connection handle 1
00> <info> app: Disconnected 1 links.
00> <debug> app: In ble_dfu_buttonless_bootloader_start_finalize
00> 
00> <info> app: Device will enter bootloader mode.
00> <info> app: Power management wants to reset to DFU mode.
00> <info> app: Power management allowed to reset to DFU mode.

[QuentinFarizon]

Update: Adding a 3 seconds delay improved for some customers, but issue remain for others (on ~25% of our customers on iOS).