First test and a little report

[duke2421]

Hi @T-vK

Thanks, that was fast.
In general it is working. In your Hid device descriptor was a (2) instead of (1) at the report id 2.
REPORT_ID(1), 0x02, // REPORT_ID (2)
With that I get a driver error in Win10.

In the array keyReport.keys you wrote all in array 0 and for "a" the code is 0x04 not 0x61 ;)

        keyReport.keys[0] = 0x61; // "a" 
        keyReport.keys[0] = 0x00;
        keyReport.keys[0] = 0x00;
        keyReport.keys[0] = 0x00;
        keyReport.keys[0] = 0x00;
        keyReport.keys[0] = 0x00;

I changed that to

        keyReport.keys[0] = 0x04; // "a"
        keyReport.keys[1] = 0x00;
        keyReport.keys[2] = 0x00;
        keyReport.keys[3] = 0x00;
        keyReport.keys[4] = 0x00;
        keyReport.keys[5] = 0x00;

Now I am able to sent 4 keys through my rotary encoder knob ("a" with short press, "b" with long press, "c" with rotate CW and "d" with rotate CCW). but I am not able to control "play/pause" and so on. I tried it with "0xcd" but I think, these keys are not handled in the same way as normal key strokes.

[T-vK]

That sounds pretty good so far. I hope I can find some more time to work on this tomorrow.
In the meantime you could try

uint8_t media[2];
media[0] = 0xcd;
media[1] = 0;

this->inputKeyboard->setValue(media,2);
this->inputKeyboard->notify();
 
.....


bleKeyboardInstance->inputKeyboard = bleKeyboardInstance->hid->inputReport(2); // <-- input REPORTID for the media keys

inside the class..

In theory report id 1 is for the keyboard functionality and report id 2 is for the media key functionality.
If it only works when both report ids are set to 1 in the descriptor, then we have a problem.

[T-vK]

I was not able to get it to work after applying your suggested changes. Was there anything else that you had to do?
As soon as bleKeyboard.begin() gets called I get an error:

Stack smashing protect failure!

abort() was called at PC 0x4014d9af on core 0

Backtrace: 0x40091c4c:0x3ffd8650 0x40091e7d:0x3ffd8670 0x4014d9af:0x3ffd8690 0x400eaf49:0x3ffd86b0 0x40137c55:0x3ffd8710 0x401174c5:0x3ffd8730 0x4011cd82:0x3ffd8750 0x400f7c0a:0x3ffd8770 0x40090461:0x3ffd87a0

Rebooting...
ets Jun  8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:1100
load:0x40078000,len:9232
load:0x40080400,len:6400
entry 0x400806a8
Starting BLE work!

I decoded the stacktrace:

Decoding stack results
0x40091c4c: invoke_abort at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/esp32/panic.c line 155
0x40091e7d: abort at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/esp32/panic.c line 170
0x4014d9af: __stack_chk_fail at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/esp32/stack_check.c line 36
0x400eaf49: BTM_BleWriteScanRsp at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/bt/bluedroid/stack/btm/btm_ble_gap.c line 1628
0x40137c55: bta_dm_ble_set_scan_rsp at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/bt/bluedroid/bta/dm/bta_dm_act.c line 4958
0x401174c5: bta_dm_sm_execute at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/bt/bluedroid/bta/dm/bta_dm_main.c line 394
0x4011cd82: bta_sys_event at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/bt/bluedroid/bta/sys/bta_sys_main.c line 496
0x400f7c0a: btu_task_thread_handler at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/bt/bluedroid/stack/btu/btu_task.c line 233
0x40090461: vPortTaskWrapper at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/freertos/port.c line 143

But I have no idea what it means. I don't even know what /Users/ficeto is supposed to be. It's certainly nothing that exists on my computer.

Anyway, I applied your changes and modified the code so that it can send the media keys (hopefully). (See example)

[duke2421]

Hi,

In the device desrciptor I had to make another change, the last REPORT_COUNT was on "0x07", with that I get a driver error in Win10. I changed it to "0x08" and then under Win10 I can send the "normal" keyboard keys.

About the media buttons, a little succes.
First I tried your solution, hardcoded with that change:
bleKeyboardInstance->inputKeyboard = bleKeyboardInstance->hid->inputReport(2); // <-- input REPORTID for the media keys
I didn't change back the second REPORT_ID to (2) in the device descriptor, but it seems to work because now, if I press a buttom, the media controls in Win10 are showing up, but I can't controll anything.

Then I removed the hardcoded stuff again, only the reportid stays 2 and I changed the struct "KeyReport" to only have the keys array (removed "modifiers" and "reserved").

This is, what I send through my arduino code then:

        keyReport.keys[0] = 0xCD; // Play/Pause
        keyReport.keys[1] = 0x00; 
    
        bleKeyboard.sendReport(&keyReport); 

Also with that, the media controlls in Win10 are showing up, but I can't control anything.

[T-vK]

I'd love to test it, but I can't even get to the point where I would connect to the device. I'll try compiling and uploading from a different computer later today.

[duke2421]

Hi, with your new code I can't get it to work, too. I will revert back to the working code and send it to you.

But I saw, what you changed, so I will try to adapt the changes to my "working" code. I am on Holiday, so I have some time to test ;)

[duke2421]

Here you can download the working code to sending the "normal" keyboard key strokes. In the examples you can find my code.

https://duke24.badberg-online.de/s/ESP32-Media-Controller

[duke2421]

Hi,
I am not able to get the consumer controls to work The dialog on Win10 is showing, if I press a button, but nothing more happens. With android the volume slides in an uncontrolled way, no matter which button I send.

Here you can download my last state, maybe it helps you a bit.
https://duke24.badberg-online.de/s/ESP32-BLE-Consumer

[T-vK]

Thanks, my crashes were caused by the deviceName being too long. Took me forever to figure that out.
I got it to work on Android. My Linux machine only recognizes the normal keyboard events though...

[duke2421]

Nice work, I uploaded your example and with android it was working out of the box, but not with Win10. I then changed the REPORT_COUNT from 0x0f to 0x10 and it is also working with Win10.

[T-vK]

Awesome, now it works on Linux too! I'll see if I can make the interface a bit nicer so that it can be used like the usb keyboard library.

[T-vK]

I got it to the point where bleKeyboard.print("Hello") etc would work, but for the media keys I couldn't figure it out yet. https://github.com/T-vK/ESP32-BLE-Keyboard/tree/test

Ideally, it would work just like that bleKeyboard.write(KEY_MEDIA_PLAY_PAUSE); ...

The main issue is definitely that the original USB keyboard lib is not designed to support media keys.

[duke2421]

Hm, with Win10 I had to change the report count from 0x0A to 0x10 again, but then it is working as expected. I am able to send media keys with bleKeyboard.write(KEY_MEDIA_PLAY_PAUSE); and words with bleKeyboard.print("Hello"); also single Letters are working with bleKeyboard.write

So I would say, your library is nearly ready ;) 👍

[duke2421]

Only problem now is, if the ESP32 is powered off and on again, I have to reconnect it, no matter if Win10 or android.

[T-vK]

I just made an important discovery. Apparently the device or device descriptor gets cached to some degree. At least on Android and Linux. After disconnecting the device, unpairing it and then disabling bluetooth the code from yesterday suddenly just worked for me on Android and Linux.
Nevertheless, 0x10 should be the correct report size. (For some reason my brain was telling me that 0x0F was 16.) You are definitely right about 0x10. (And it works fine with that)

Automatic reconnecting works fine for me on Android and Linux most of the time. But sometimes I have to disable and enable Bluetooth again on the device or manually connect again. I'm not really sure what is going wrong there.

[duke2421]

You are right, 0x0f is 16. I don't know, why the report count has to be n of usage +1, but with that it works ;)

About the reconnect problem I found this: nkolban/esp32-snippets#632 (comment)
But I am not sure, if I changed the code at the right place. I thought, it has to be in BleConnectionStatus.cpp but the problem isn't gone, could you have a look at it?

Everytime I reboot the ESP or just plug it out and back in, I have to disable and enable Bluetooth in Android, otherwise I can't control Android. With Windows I have to remove it and then add it new. Simply disable and enable BT is not enough.

Btw, thanks for including me in your credits.

[T-vK]

No, you are the one who is right:
0x09 = 9
0x0A = 10
0x0B = 11
0x0C = 12
0x0D = 13
0x0E = 14
0x0F = 15
0x10 = 16

I'll take a look at it.

I assume you've already tried:

uint8_t val[] = {0x01, 0x00};
desc->setValue(val, 2);

in BleConnectionStatus.cpp

[duke2421]

Of course, 0x10 is 16, (I had the same thinking mistake as you😁 )

I tried in variuos ways, last thing I tried:

#include "BleConnectionStatus.h"

BleConnectionStatus::BleConnectionStatus(void) {
}

void BleConnectionStatus::onConnect(BLEServer* pServer)
{
  this->connected = true;
  BLE2902* desc = (BLE2902*)this->inputKeyboard->getDescriptorByUUID(BLEUUID((uint16_t)0x2902));
  //desc->setNotifications(true);
  uint8_t val[] = {0x01, 0x00};
  desc->setValue(val, 2);
  Serial.println("Device connected!"); 
}

void BleConnectionStatus::onDisconnect(BLEServer* pServer)
{
  this->connected = false;
 // BLE2902* desc = (BLE2902*)this->inputKeyboard->getDescriptorByUUID(BLEUUID((uint16_t)0x2902));
 // desc->setNotifications(false);
  Serial.println("Device disconnected!");
}

Also with `setNotifications(true);´ but no luck.

[T-vK]

I've had no luck either. Maybe @nkolban has an idea, as he's the author of all the low level BLE HID libraries.