extmod/modbluetooth: Fixes for bluekitchen/btstack. #11239

jimmo · 2023-04-11T14:55:58Z

This PR brings btstack functionality up to match NimBLE (excluding L2CAP channels). This needed to be done anyway, but is high priority for BLE on the Pico W (see #10739).

The three main fixes are:

Make indicate/notify behave identically, and match NimBLE.
Include the value handle in the read/write complete events for GATT client. (Previously btstack always gave 65535 as the handle).
Make central-initiated MTU exchange work on btstack.

Also

Update tests to pass on both. ble_characteristic.py, ble_descriptor.py,ble_gap_advertise.py, ble_gap_connect.py, ble_gap_device_name.py, ble_gattc_discover_services.py, ble_gatt_data_transfer.py, ble_mtu.py,ble_subscribe.py now run identically on both.
Updates documentation (mainly to remove btstack-specific limitations).
Some minor NimBLE fixes, e.g. for modbluetooth: Descriptor flag mapping #6864

TODO: More testing of btstack pairing/bonding.

This work was funded through GitHub Sponsors.

github-actions · 2023-04-11T15:09:43Z

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:    +0 +0.000% standard
      stm32:    +0 +0.000% PYBV10
        rp2:    +0 +0.000% PICO

jimmo · 2023-04-11T22:42:11Z

Include the value handle in the read/write complete events for GATT client. (Previously btstack always gave 65535 as the handle).

This is important to support aioble, which requires this to work (which is why aioble only previously worked on NimBLE).

See also micropython/micropython-lib#643 for a corresponding update to aioble related to other fixes.

extmod/nimble/hal/hal_uart.c

dpgeorge · 2023-04-22T12:17:58Z

I reverted the commit that changed run-multitests.py to poll stdin during wait(), because it breaks the CI. I'd prefer to fix that by making the unix port run scheduled code during mp_hal_stdin_rx_chr().

docs/library/bluetooth.rst

codecov-commenter · 2023-04-22T12:31:57Z

Codecov Report

Merging #11239 (4478c0d) into master (b525f1c) will not change coverage.
The diff coverage is n/a.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@           Coverage Diff           @@
##           master   #11239   +/-   ##
=======================================
  Coverage   98.50%   98.50%           
=======================================
  Files         155      155           
  Lines       20549    20549           
=======================================
  Hits        20241    20241           
  Misses        308      308

Impacted Files	Coverage Δ
ports/unix/mpconfigport.h	`100.00% <ø> (ø)`
py/gc.c	`99.18% <ø> (ø)`

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

extmod/btstack/modbluetooth_btstack.c

tests/multi_bluetooth/ble_gap_advertise.py

ports/unix/mpbthciport.c

dpgeorge · 2023-04-22T13:07:20Z

I also pushed a commit to tidy up a comment.

tools/hci_trace_to_pcap.py

Makes gatts_notify and gatts_indicate work in the same way: by default they send the DB value, but you can manually override the payload. In other words, makes gatts_indicate work the same as gatts_notify. Note: This removes support for queuing notifications and indications on btstack when the ACL buffer is full. This functionality will be reimplemented in a future commit. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

This adds a mechanism to track a pending notify/indicate operation that is deferred due to the send buffer being full. This uses a tracked alloc that is passed as the content arg to the callback. This replaces the previous mechanism that did this via the global pending op queue, shared with client read/write ops. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

This replaces the previous pending operation queue (that used to also be shared with pending server notify/indicate ops) with a single pending operation per connection. This allows the value handle to be correctly passed to the Python-level events. Also re-structure GATT client event handling to simplify the packet handler functions. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

On unix, time.sleep is implemented as select(timeout=<time>) which means that it does not run the poll hook during sleeping. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

btstack only supports central-initiated, so this allows us to have a test that works on both (ble_mtu.py), and then another one for just the NimBLE supported behavior (ble_mtu_peripheral.py). Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

There was no event handler for central-initiated MTU exchange. Fix truncation of notify/indicate to match NimBLE. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

Obtaining the end_handle was added in cacc96d. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

This allows gatts_write(..., send_update=True) to work, which will send notifications/indications to subscribed clients. btstack already created the CCCD but writes to it were ignored. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

The att_flags for descriptors does not use the same bitfield as for characteristics. This was leading to NimBLE descriptors getting the wrong flags set. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

This is unused in the client, only needed in the server. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

- Use HCI_TRACE macro consistently. - Use the same colour formatting. - Add a tool to convert to .pcap for Wireshark. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

Even if the user doesn't call ble.gatts_register_services, always provide the default services. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

jimmo · 2023-04-26T01:39:57Z

I reverted the commit that changed run-multitests.py to poll stdin during wait(), because it breaks the CI. I'd prefer to fix that by making the unix port run scheduled code during mp_hal_stdin_rx_chr().

OK. I've squashed out the commit and the revert.

Also rebased the other changes.

dpgeorge · 2023-04-26T02:04:17Z

OK, thanks for updating, looks good now!

dpgeorge · 2023-04-26T02:27:23Z

Merged!

In micropython/micropython#11239 we added support for passing data to gatts_indicate (to make it match gatts_notify). This adds the same to aioble. Also update the documentation to mention this (and fix some mistakes and add a few more examples). This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

jimmo mentioned this pull request Apr 11, 2023

aioble: Fix descriptor flag handling. micropython/micropython-lib#643

Merged

jimmo force-pushed the btstack-fixes branch from d920c73 to 5e986bd Compare April 11, 2023 15:00

dpgeorge added the extmod Relates to extmod/ directory in source label Apr 12, 2023

jimmo force-pushed the btstack-fixes branch 2 times, most recently from 514950c to 8bfc9a8 Compare April 13, 2023 04:27

dpgeorge reviewed Apr 14, 2023

View reviewed changes

extmod/nimble/hal/hal_uart.c Outdated Show resolved Hide resolved

jimmo force-pushed the btstack-fixes branch from 8bfc9a8 to f739741 Compare April 21, 2023 02:08