Introduction

ThingsBoard is an open-source server-side platform that allows you to monitor and control IoT devices. It is free for both personal and commercial usage and you can deploy it anywhere. If this is your first experience with the platform we recommend to review what-is-thingsboard page and getting-started guide.
This sample demo performs collection of temperature and humidity values produced by BLE broadcasting devices and further visualization on the real-time web dashboard. In this example we use HTU21D connected to ESP32 and Xiaomi Smart Temperature & Humidity Sensor. The purpose of this application is to demonstrate ThingsBoard data collection API , visualization capabilities, gateway API and the capabilities of Bluetooth Low Energy protocol, which you can use with both custom and mass-produce devices.
Data is collected by a python script that is running on Raspberry Pi. It pushes data to ThingsBoard server via MQTT protocol by using ThingsBoard MQTT client Python SDK library. Data is visualized using built-in customizable dashboard. Demo that is running on Raspberry Pi is written in Python which is quite simple and easy to understand.
ThingsBoard BLE Demo with Raspberry PI, ESP32, Xiaomi MI and HTU21D sensors - 图1

List of hardware

Hardware setup

ESP32 Wiring scheme

ESP32 HTU21D
3V3 VIN
GND GND
18 SDA
19 SCL

The following picture summarizes the connection of ESP32 and HTU21D:
ThingsBoard BLE Demo with Raspberry PI, ESP32, Xiaomi MI and HTU21D sensors - 图5

Raspberry Pi

In this setup we use Raspberry Pi 2 Model B V1.1 with Raspbian GNU/Linux 9.8 (stretch) installed on it. Raspbian is a free operating system based on Debian optimized for the Raspberry Pi hardware. This model does’t support Bluetooth, so it needs a usb dongle. We use Grand-X BT40G. Make sure your dongle is recognized by system. You can check it with the following command:

  1. $> lsusb
  2. ...
  3. Cambridge Silicon Radio, Ltd Bluetooth Dongle (HCI mode)
  4. ...

Whole setup:
ThingsBoard BLE Demo with Raspberry PI, ESP32, Xiaomi MI and HTU21D sensors - 图6

Preparing ESP32

Application capabilities

In this project ESP is a GATT server, which creates BLE service with two characteristics, one is read-only, and the second is for both reading and writing. Also it reads temperature and humidity data from connected HTU21D sensor and broadcast it as notifications to all nearby BLE devices. You can find the source code here. It is based on Espressif GATT demo server example and uses this library to interact with HTU21D.

Flashing ESP32 BT firmware

You need ESP-IDF (Espressif IoT Development Framework) to program your ESP32. It is the official development framework for the ESP32 chip. Install it with the following instructions:

  1. mkdir ~/esp
  2. cd ~/esp
  3. git clone --recursive https://github.com/espressif/esp-idf.git
  4. git checkout 3bf56cdd1

Set up IDF_PATH before running any other command:

  1. export IDF_PATH=~/esp/esp-idf

The python packages required by ESP-IDF are located in IDF_PATH/requirements.txt. You can install them by running:

  1. python -m pip install --user -r $IDF_PATH/requirements.txt

Note

Please check the version of the Python interpreter that you will be using with ESP-IDF. For this, run the command python --version and depending on the result, you might want to use python2, python2.7 or similar instead of just python, e.g.
See this guide for more info.
Now you can flash your ESP:

  1. git clone https://github.com/thingsboard/gatts_demo
  2. cd gatts_demo
  3. make flash

Run make monitor if you want to see logs of ESP and check if everything is okay. Example of output:

  1. I (30) boot: ESP-IDF v4.0-dev-584-g3bf56cdd1 2nd stage bootloader
  2. I (30) boot: compile time 14:00:10
  3. I (30) boot: Enabling RNG early entropy source...
  4. I (36) boot: SPI Speed      : 40MHz
  5. I (40) boot: SPI Mode       : DIO
  6. I (44) boot: SPI Flash Size : 4MB
  7. I (48) boot: Partition Table:
  8. I (52) boot: ## Label            Usage          Type ST Offset   Length
  9. I (59) boot:  0 nvs              WiFi data        01 02 00009000 00006000
  10. I (66) boot:  1 phy_init         RF data          01 01 0000f000 00001000
  11. I (74) boot:  2 factory          factory app      00 00 00010000 00100000
  12. I (81) boot: End of partition table
  13. I (85) esp_image: segment 0: paddr=0x00010020 vaddr=0x3f400020 size=0x23384 (144260) map
  14. I (145) esp_image: segment 1: paddr=0x000333ac vaddr=0x3ffbdb60 size=0x02c5c ( 11356) load
  15. I (150) esp_image: segment 2: paddr=0x00036010 vaddr=0x40080000 size=0x00400 (  1024) load
  16. 0x40080000: _WindowOverflow4 at /home/yulia/esp/esp-idf/components/freertos/xtensa_vectors.S:1779
  17. I (152) esp_image: segment 3: paddr=0x00036418 vaddr=0x40080400 size=0x09bf8 ( 39928) load
  18. I (177) esp_image: segment 4: paddr=0x00040018 vaddr=0x400d0018 size=0x71ea0 (466592) map
  19. 0x400d0018: _flash_cache_start at ??:?
  20. I (341) esp_image: segment 5: paddr=0x000b1ec0 vaddr=0x40089ff8 size=0x08d50 ( 36176) load
  21. 0x40089ff8: r_lld_pdu_rx_handler at ??:?
  22. I (367) boot: Loaded app from partition at offset 0x10000
  23. I (367) boot: Disabling RNG early entropy source...
  24. I (368) cpu_start: Pro cpu up.
  25. I (371) cpu_start: Application information:
  26. I (376) cpu_start: Project name:     gatt_server_demos
  27. I (382) cpu_start: App version:      b2732e4-dirty
  28. I (387) cpu_start: Compile time:     Jun  3 2019 17:06:46
  29. I (393) cpu_start: ELF file SHA256:  2d3d593eed804970...
  30. I (399) cpu_start: ESP-IDF:          v4.0-dev-584-g3bf56cdd1
  31. I (406) cpu_start: Starting app cpu, entry point is 0x400810e0
  32. 0x400810e0: call_start_cpu1 at /home/yulia/esp/esp-idf/components/esp32/cpu_start.c:267
  33. I (0) cpu_start: App cpu up.
  34. I (416) heap_init: Initializing. RAM available for dynamic allocation:
  35. I (423) heap_init: At 3FFAFF10 len 000000F0 (0 KiB): DRAM
  36. I (429) heap_init: At 3FFB6388 len 00001C78 (7 KiB): DRAM
  37. I (435) heap_init: At 3FFB9A20 len 00004108 (16 KiB): DRAM
  38. I (441) heap_init: At 3FFBDB5C len 00000004 (0 KiB): DRAM
  39. I (447) heap_init: At 3FFC8EA0 len 00017160 (92 KiB): DRAM
  40. I (453) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
  41. I (460) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
  42. I (466) heap_init: At 40092D48 len 0000D2B8 (52 KiB): IRAM
  43. I (472) cpu_start: Pro cpu start user code
  44. I (155) cpu_start: Starting scheduler on PRO CPU.
  45. I (0) cpu_start: Starting scheduler on APP CPU.
  46. I (157) GATTS_DEMO: starting ESP gatt demo

Preparing MI Sensor

Install the battery in sensor.

Preparing Raspberry Pi

Demo script scans for available BLE devices and connects to them. It can read a direct value of BLE characteristic from ESP and receive notifications from both ESP and Mi. If needed, a re-scan command can be issues from the dashboard to connect to new devices.

Implementation details

If you are not interested in modifying or extending a list of supported devices, you can skip this sub-section. The demo script consists of few modules: the main module and couple of extension modules.
The main module is responsible for receiving commands from ThingsBoard and sending telemetry data. It also scans nearby devices and connects to them.
Extensions are modules that responsible for extracting application-specific data from the BLE device. If you want to implement support for a new device, you can create additional extension class and inherit it from provided intarface. Use this as an example.

Installing gateway scripts on Raspberry Pi

Login to your Raspberry PI using SSH and install demo. It requires at least python3.4. Also script requires superuser rights for Bluetooth management, so you need to install it system-wide.

  1. sudo apt-get install libglib2.0-dev
  2. sudo pip3 install tb-ble-adapter

Device provisioning

Run python script before importing dashboard. For this you have to create gateway device and specify it in command arguments:

  1. sudo tb_ble_adapter -t $GATEWAY_ACCESS_TOKEN

Example of logs with device scanning, discovery of known devices and device connection:

  1. Loading MiTempHumidityExtension extension...
  2. Loading EspGattDemoExtension extension...
  3. Scanning BLE devices...
  4. Discovered BT device: 10:73:48:30:84:49
  5. Discovered BT device: d8:a0:1d:40:93:fa
  6. Discovered BT device: 3d:0b:d8:a9:ab:9d
  7. Discovered BT device: 4e:8e:10:d8:c4:a1
  8. Discovered BT device: 1b:83:77:79:2e:29
  9. Discovered BT device: 00:1e:c0:75:d0:a3
  10. Discovered BT device: 4c:65:a8:df:8e:3f
  11. Discovered BT device: 4c:65:a8:df:85:c0
  12. Discovered BT device: 4c:65:a8:df:94:d1
  13. Received new data from: 4c:65:a8:df:8e:3f
  14. Received new data from: 4c:65:a8:df:85:c0
  15. Received new data from: 4c:65:a8:df:94:d1
  16. Received new data from: 4c:65:a8:df:8e:3f
  17. Received new data from: 4c:65:a8:df:8e:3f
  18. Received new data from: 4c:65:a8:df:85:c0
  19. Received new data from: 4c:65:a8:df:94:d1
  20. Received new data from: 4c:65:a8:df:8e:3f
  21. Received new data from: 4c:65:a8:df:85:c0
  22. Received new data from: 4c:65:a8:df:94:d1
  23. Discovered BT device: 2e:ff:58:11:60:71
  24. Discovered BT device: 49:91:8f:6c:7a:17
  25. Received new data from: 4c:65:a8:df:8e:3f
  26. Received new data from: 4c:65:a8:df:8e:3f
  27. Received new data from: 4c:65:a8:df:85:c0
  28. Received new data from: 4c:65:a8:df:8e:3f
  29. Received new data from: 4c:65:a8:df:85:c0
  30. Received new data from: 4c:65:a8:df:94:d1
  31. Received new data from: 4c:65:a8:df:8e:3f
  32. Device 10:73:48:30:84:49 (random), RSSI=-52 dB
  33.   Manufacturer = 06000109200298bf2a039d57aa7fda15c7cc1772001b3cd9881bf3e8e0
  34. Device d8:a0:1d:40:93:fa (public), RSSI=-43 dB
  35.   Flags = 06
  36.   Complete Local Name = ESP_GATTS_DEMO
  37.     [!] Known device found: ESP_GATTS_DEMO
  38.   Tx Power = 03
  39.   Complete 16b Services = 000000ee-0000-1000-8000-00805f9b34fb,000000ff-0000-1000-8000-00805f9b34fb
  40. Device 3d:0b:d8:a9:ab:9d (random), RSSI=-76 dB
  41.   Manufacturer = 0600010920029944e816d9764a15972cd9df7db619bf8dc0875737d50a
  42. Device 4e:8e:10:d8:c4:a1 (random), RSSI=-72 dB
  43.   Flags = 1a
  44.   Manufacturer = 4c001005131c9659d5
  45. Device 1b:83:77:79:2e:29 (random), RSSI=-91 dB
  46.   Manufacturer = 060001092002fd5da974543017b16d0518788003bed6946bb155378f84
  47. Device 00:1e:c0:75:d0:a3 (public), RSSI=-90 dB
  48.   Flags = 06
  49.   Complete 128b Services = 00035b03-58e6-07dd-021a-08123a000300
  50.   Complete Local Name = D1900924
  51. Device 4c:65:a8:df:8e:3f (public), RSSI=-59 dB
  52.   Flags = 06
  53.   16b Service Data = ffffe0845fe76916
  54.   Complete Local Name = MJ_HT_V1
  55.     [!] Known device found: MJ_HT_V1
  56.   Complete 16b Services = 0000180f-0000-1000-8000-00805f9b34fb,0000180a-0000-1000-8000-00805f9b34fb
  57. Device 4c:65:a8:df:85:c0 (public), RSSI=-51 dB
  58.   Flags = 06
  59.   16b Service Data = ffffca4e2ef743aa
  60.   Complete Local Name = MJ_HT_V1
  61.     [!] Known device found: MJ_HT_V1
  62.   Complete 16b Services = 0000180f-0000-1000-8000-00805f9b34fb,0000180a-0000-1000-8000-00805f9b34fb
  63. Device 4c:65:a8:df:94:d1 (public), RSSI=-57 dB
  64.   Flags = 06
  65.   16b Service Data = fffff61d60016488
  66.   Complete Local Name = MJ_HT_V1
  67.     [!] Known device found: MJ_HT_V1
  68.   Complete 16b Services = 0000180f-0000-1000-8000-00805f9b34fb,0000180a-0000-1000-8000-00805f9b34fb
  69. Device 2e:ff:58:11:60:71 (random), RSSI=-96 dB
  70.   Manufacturer = 06000109200232a466069323755cf0a3f6333f3575276dd81e6a2940ba
  71. Device 49:91:8f:6c:7a:17 (random), RSSI=-71 dB
  72.   Flags = 1a
  73.   Manufacturer = 4c001005131cc5c74c
  74. Connecting to device: MJ_HT_V1_4C65A8DF8E3F
  75. Polling data from: MJ_HT_V1_4C65A8DF8E3F
  76. Received data: b'T=23.9 H=43.5\x00'
  77. Data received: {'temperature': 23.9, 'humidity': 43.5}
  78. Sending data to TB: {'ts': 1559827797240, 'values': {'temperature': 23.9, 'humidity': 43.5}}
  79. Disconnecting from device

It creates devices: ThingsBoard BLE Demo with Raspberry PI, ESP32, Xiaomi MI and HTU21D sensors - 图7

Provision your dashboard

Download the dashboard file using this link. Now you have to import it. To do this, configure aliases in such way (use gateway device from Device provisioning section):
ThingsBoard BLE Demo with Raspberry PI, ESP32, Xiaomi MI and HTU21D sensors - 图8
ThingsBoard BLE Demo with Raspberry PI, ESP32, Xiaomi MI and HTU21D sensors - 图9
ThingsBoard BLE Demo with Raspberry PI, ESP32, Xiaomi MI and HTU21D sensors - 图10
Also this dashboard needs a custom widget bundle. Download it with this link and import using import/export instructions.

Data visualization

Finally, open ThingsBoard Web UI. You can access this dashboard by logging in as a tenant administrator.
In case of local installation:

  • login: tenant@thingsboard.org
  • password: tenant

In case of live-demo server:

  • login: your live-demo username (email)
  • password: your live-demo password

See live-demo page for more details how to get your account.
Go to “Devices” section, open created devices details and switch to “Latest telemetry” tab. If all is configured correctly you should be able to see latest telemetry values sent via BLE in the table.
ThingsBoard BLE Demo with Raspberry PI, ESP32, Xiaomi MI and HTU21D sensors - 图11
After, open “Dashboards” section and find previously imported dashboard “Device GW” and observe the following picture:
ThingsBoard BLE Demo with Raspberry PI, ESP32, Xiaomi MI and HTU21D sensors - 图12

Troubleshooting

In order to perform troubleshooting, you must check demo script output. It displays connection status with every device, as well as received and sent data.

See also

Browse other samples or explore guides related to main ThingsBoard features: