Support for btmon in Mynewt

Btmon is very useful tool for HCI messaging monitoring on Linux systems. It helps to understand how Bluetooth host talks to controller, debug issues, implement new features in the host etc. When we have started work with Mynewt we knew that we need tool like this. Of course Mynewt has a way to print all the HCI logs in hex, but decoding it on fly is not the best thing to do. So here it is, Mynewt can be configured in the way that btmon can be used. Detailed information you can find here. Below we show an example how to use it.

For the demo purpose we will use btshell application running on nRF52840 PDK. We will configure it to most convenient option, which is: keep bi-directional console on UART and configure Mynewt to send HCI over RTT to btmon. Let’s start from creating target:

newt target create btshell
newt target set btshell app=@apache-mynewt-core/apps/btshell bsp=@apache-mynewt-core/hw/bsp/nordic_pca10056 build_profile=optimized

We should not forget about boot image:

newt target create nrf52_boot 
newt target set nrf52_boot app=@apache-mynewt-core/apps/boot bsp=@apache-mynewt-core/hw/bsp/nordic_pca10056 build_profile=optimized

Now we need to add additional syscfg.yml file into target/btshell/ folder in order to enable HCI over RTT. That file should look like this:

cat targets/btshell/syscfg.yml 

Once configuration is done, let’s build btshell and flash it onto nRF52840pdk

newt build nrf52_boot
newt load nrf52_boot
newt build btshell
newt create-image btshell 1
newt load btshell

Board is ready to play with. Shell is available under /dev/ttyACMx . Test it by typing help

Now we came to most interesting part, HCI tracing. Since btmon can only read data from tty, we need small tool to redirect RTT data to PTY. It is called rtt2pty and is ready for download from our GitHub here. It is delivered as an Eclipse project, so just import it, build and run. Here is how the outcome should look like:

> ./rtt2pty -b btmonitor
Connected to:
  J-Link OB-SAM3U128-V2-NordicSemi compiled Jun  9 2017 14:57:23
  S/N: 683958770
Searching for RTT control block...
Found 3 up-buffers.
Using buffer #1 (size=256)
PTY name is /dev/pts/3

Run btmon:

btmon --tty=/dev/pts/3

And that is it. Now for test let us try to scan for devices around with btshell (don’t forget to set address of device for btshell) e.g.

btshell> set addr=11:11:11:11:11:11
btshell> scan

Happy tracing!

Łukasz Rymanowski

Automated Bluetooth testing and qualification of Mynewt NimBLE with auto-pts

We are happy to announce that auto-pts now supports Mynewt NimBLE stack!

To test and qualify your Bluetooth product you have to use a tool called Profile Tuning Suite (PTS) from Bluetooth SIG organization. It is a Windows program that you can use to manually run test cases using a graphical user interface. The process of testing manually requires a tester to operate both the device and the PTS program. This is a tedious job and can take one man-month. We described this process in one of our blog posts before. There is a tool that can speed that up to a few hours by fully automating the testing process.

auto-pts is a Bluetooth PTS automation framework that coordinates test execution between PTS and the Implementation Under Test (IUT). It automates over 400 test cases from GAP, GATT, L2CAP, SM, and MESH profiles. Auto-pts supports already BlueZ and Zephyr and it is supporting Mynewt NimBLE now!

On the IUT side, you will need to use the bttester app. It communicates with the auto-pts via UART. Bluetooth Testing Protocol (BTP) is used as a communication protocol. You can find the specification in the auto-pts repository. Check out the README on how to configure and run the auto-pts tool.

Happy testing! 🙂

Developing Apache Mynewt in Clion with quick and accurate indexing

It is not always possible to use a modern IDE for embedded software development. Sometimes your project uses a custom build system or a workflow that is hard to integrate with your favorite IDE. Apache Mynewt project has a modular design with features split into packages, dependencies and configuration files. It simplifies the architecture of the whole system and gives a lot of power to the user but also complicates the build process. To manage all of that a Newt tool was created. It is a smart build and package management system, that can handle multiple source repositories, decide what to include in a build and manage dependencies between components of the system. And one of it’s newer features is an integration with CMake.

Clion is a powerful cross-platform IDE for C and C++. It is easy to use, very responsive and supports tons of integrations. And it comes with vi emulation too! Most importantly, it uses CMake as a build system.

So now you can use Clion to develop Apache Mynewt applications!

We already described in this blog how to configure and use Eclipse with Mynewt. It is possible but requires some configuration. On the other hand, working on Mynewt in Clion requires almost no configuration.

How to work on Mynewt project in Clion

newt target cmake <target> command generates a CMakeLists.txt file for a specific target inside project root directory. This file contains all the CMake configuration options and a list of files to include in a build. You can then open the project inside Clion and the IDE should load the file correctly. Indexing in Clion is lightning fast! It takes just a few seconds and you are ready to start working on your project. You almost don’t notice it. If you need to change the target you are using, just run the command again with the new target and Newt will overwrite the CMakeLists.txt file. Clion will detect a change in this file and reindex your project. Just like that!

Indexing of a Mynewt project in Eclipse takes over a minute! So this is a huge improvement to your workflow. Another very important factor is the quality of indexing. Clion recognizes all the definitions and detects which files are included in the build and even which parts of the file are excluded. So you can jump to definitions, search by file or by symbol, refactor code, find usage and much more.

Building your project in Clion also works but you will still need to build and flash using Newt tool since both build processes are not yet equivalent.

Debugging Mynewt in Clion

You can also debug Mynewt in Clion but it requires a bit more configuration. You will need to create a GDB Remote Debug configuration inside Run/Debug Configurations window. Set the following fields:

Unfortunately, you will need to create a separate configuration for each target you want to debug.

Inside terminal run newt debug <target> -n – this will run gdb in server mode and will wait for the client to connect. Then in Clion select your debug configuration and click Debug button. That’s it!

Now you can develop your Apache Mynewt project inside a modern, powerful IDE with fast and accurate indexing.

How to do software update of a Bluetooth Mesh device running Apache Mynewt

As you might know, there is not yet specified way for a software update over the Bluetooth Mesh. This is high demand feature and for sure it will come. However, if this feature is needed just right now and your product uses Apache Mynewt, there is a way to mitigate the issue.

Apache Mynewt supports Bluetooth 5 features and in particular the one called Advertising Extensions. It gives a way to support multi advertising and this gives an idea to have advertising instance dedicated to Mesh operations and in the same time an instance which will allow to connect for the software update purpose.

This tutorial will guide you through configuration and simple update of firmware over BLE on Mynewt of Mesh node supporting Generic On/Off model.

This example scenario use:

Configuration for targets

Device A system configuration modification:

Device B system configuration modification:

Additional packages requirement by Device B

Since Device B will be updated via BLE it requires some additional packages.

Packages required:

Building images

Both devices running Mynewt have to be flashed with bootloader first as presented below.

Example configuration:

$ newt target show nrf52840pdk_boot

Then devices have to be flashed with target application.

Example configuration (Dev B):

$ newt target show nrf52840pdk_btshell

Build and flash devices example. Start with bootloader:

$ newt clean nrf52840pdk_boot
$ newt build nrf52840pdk_boot
$ newt load nrf52840pdk_boot
$ newt clean nrf52840pdk_btshell

Then application (use 1.0.1 as version for second image of Dev B):

$ newt build nrf52840pdk_btshell
$ newt create-image nrf52840pdk_btshell 1.0.0
$ newt load nrf52840pdk_btshell

Afterwards, devices are ready to interact via shell.

Default serial setup for communication with boards

Configure simple Mesh network

As soon as devices are set up and terminal is connected, it’s time to configure mesh and make some data traffic.
Mynewt Mesh provides simple configuration client over shell which will be used to configure Generic On/Off client/server in our scenario.

Initialize mesh and application in mesh module

First we need to provision devices. Since Apache Mynewt does not support provisioner yet, let us use hardcored credentials. Run below commands on both devices changing <addr> to 0x0001 on Device A and 0x0002 on Device B.

btshell> select mesh
mesh> init
mesh> provision 0 <addr>

Configure Device A as a Client and Device B as a Server. Configuration will be done from Device A.

mesh> app-key-add 0 0 1
mesh> mod-app-bind 1 0 0x1001

Now we are ready to configure Device B to be a server. For this set destination address of Device B on Device A, then continue configuration on Device A of test applications.

mesh> dst 0x0002
mesh> app-key-add 0 0 1
mesh> mod-app-bind 2 0 0x1000

To verify mesh configuration is successfully, send some generic model operations e.g. set on/off from Device A

mesh> gen-onoff-set 5

If changing of value was successfully you will get indication on Device B.

Light state: onoff=5 lvl=0x0000

Prepare stack for firmware update over air of Device B

To update firmware over air, Device B has to configure advertising instance first. Pay attention on device name, since it is required by Newtmgr to connect.

Example setting:

mesh> select btshell
btshell> advertise-configure connectable=1 scannable=1 legacy=1 own_addr_type=random
btshell> advertise-set-adv-data name=deviceB
btshell> advertise-set-addr addr=ff:de:cb:11:22:33
btshell> security-set-data our_key_dist=7
btshell> security-set-data their_key_dist=7
btshell> advertise-start

PC (Newtmgr)

Configure connection and upload image to slot 2 on device via Bluetooth:
Note: To access Bluetooth controller on host, newtmgr may require to be run as root thus you may need to run all commands via sudo.

$ newtmgr conn add myble type=ble connstring="peer_name=deviceB"
$ newtmgr -c myble image upload deviceB2.img

Upload progress will be indicated with progress bar and current upload data to total data indicator. Successful upload of image will be indicated with “done” information on Newtmgr.

After successful upload a test of image is required. By test it means that image will be swapped to slot 0 after reboot and this image will be run. If something goes wrong with tested image, system will be rebooted automatically (program asserts, corrupted image). The Mynewt has also feature which backs up in case of wrong firmware – not confirmed image will be swapped back to slot 1 and previous image will be run from slot 0 after any system reboot.

Current status of device slots can be checked with command:

$ newtmgr -c myble image list
    version: 1.0.0
    bootable: true
    flags: active confirmed
    hash: a8e9fbba9c690da87faf714496e55453967fcfe05bb10f9863c983a5b4788cd0
    version: 1.0.1
    bootable: true
    hash: 31c58b0a207ecf96d62bf6ea8acc9073375d92a8b0cf76fa29cb8f48c29b5250

Select image for testing after reboot of system

$ newtmgr -c myble image test 31c58b0a207ecf96d62bf6ea8acc9073375d92a8b0cf76fa29cb8f48c29b5250

And image will be marked with pending flag

$ newtmgr -c myble image list
    version: 1.0.0
    bootable: true
    flags: active confirmed
    hash: a8e9fbba9c690da87faf714496e55453967fcfe05bb10f9863c983a5b4788cd0
    version: 1.0.1
    bootable: true
    flags: pending
    hash: 31c58b0a207ecf96d62bf6ea8acc9073375d92a8b0cf76fa29cb8f48c29b5250

System can be easily rebooted with Newtmgr using reset command

newtmgr -c myble reset

After system reboot a new image should be run from slot 0 with active flag set

$ newtmgr -c myble image list
    version: 1.0.1
    bootable: true
    flags: active
    hash: 31c58b0a207ecf96d62bf6ea8acc9073375d92a8b0cf76fa29cb8f48c29b5250
    version: 1.0.0
    bootable: true
    flags: confirmed
    hash: a8e9fbba9c690da87faf714496e55453967fcfe05bb10f9863c983a5b4788cd0

Image can be confirmed to be permanently set as default running image after reboot

$ newtmgr -c myble image confirm 31c58b0a207ecf96d62bf6ea8acc9073375d92a8b0cf76fa29cb8f48c29b5250
$ newtmgr -c myble image list
    version: 1.0.1
    bootable: true
    flags: active confirmed
    hash: 31c58b0a207ecf96d62bf6ea8acc9073375d92a8b0cf76fa29cb8f48c29b5250
    version: 1.0.0
    bootable: true
    hash: a8e9fbba9c690da87faf714496e55453967fcfe05bb10f9863c983a5b4788cd0

Not confirmed but active image after reboot will be swapped back to slot 1 and confirmed image will be default again.

Swap back to confirmed image prevents system from being deadlocked on bad image.

Uploaded image verification

btshell> select mesh
mesh> init
mesh> provision 0 <addr> (suggested address on A = 0x0001, B = 0x0002)
mesh> dst 0x0002

Final mesh verification

To check if everything goes right it’s recommended to do verification of  some data exchange. Data exchange operation and mesh setup was previously described in “Initialize mesh and application in mesh module” part.

Grzegorz Kołodziejczyk

Bluetooth Mesh on Apache Mynewt

Bluetooth Mesh implementation is the latest addition to Apache Mynewt Nimble stack. This tutorial will should you how to use it. If you want to learn more about Bluetooth Mesh visit this site.

This tutorial will provide you with a simple yet fully functional mesh structure.

For this demo, we will use Bluetooth Mesh by Silicon Labs application for Android, which will act as a provisioner.  You can download it for Android using Google Play. Furthermore, we need to use the blemesh application running on multiple nRF52840 PDK devices acting as nodes.

Bluetooth Mesh provides two bearers in form of GATT and advertising. Unfortunately, Silicon Labs application only supports PB-GATT (GATT Provisioning Bearer), so we have to enable PB-GATT for all nodes – this can be done by setting BLE_MESH_PB_GATT configuration option. One of our targets should also contain GATT Proxy Service so it can relay messages between application and other nodes which do not use GATT bearer. This is done by setting BLE_MESH_GATT_PROXY flag.

Building targets

Let’s start by creating target for a proxy device:

$ newt target create blemesh
$ newt target set blemesh app=@apache-mynewt-core/apps/blemesh
$ newt target set blemesh bsp=@apache-mynewt-core/hw/bsp/nordic_pca10056
$ newt target set blemesh build_profile=optimized
$ newt target set blemesh syscfg=BLE_MESH_PB_GATT=1: \
> BLE_MESH_DEV_UUID='(uint8_t[16]){0x22, 0x20, 0}'

Note that we do not set device public address since Bluetooth Mesh implementation uses non-resolvable private address (NRPA) as they are not really important in mesh. What is important though is a device UUID as set above.

We also need target for bootloader:

$ newt target create nrf52840_boot
$ newt target set nrf52840_boot app=@apache-mynewt-core/apps/boot
$ newt target set nrf52840_boot bsp=@apache-mynewt-core/hw/bsp/nordic_pca10056
$ newt target set nrf52840_boot build_profile=optimized

Now you can build and flash bootloader and application onto device:

$ newt build nrf52840_boot
$ newt load nrf52840_boot
$ newt build blemesh
$ newt create-image blemesh 0
$ newt load blemesh

Afterwards, our devices is all set and ready for action. But one device is not enough to form an interesting mesh network, therefore we will prepare more.  This process is pretty similar to the previous one, so you can just amend your previous target or create another one. The only significant difference is that each other device should get different device UUID. Also they should only have GATT Provisioning Service enabled (i.e. no GATT Proxy Service):

$ newt target set blemesh syscfg=BLE_MESH_PB_GATT=1: \
> BLE_MESH_DEV_UUID='(uint8_t[16]){0x22, 0x21, 0}'

Now you can flash bootloader and application onto other device(s).

Configuring Bluetooth Mesh Silicon Labs application

As soon as your devices are ready, it is time to create your own mesh network using the Android app. We’ve recorded a video that should help you with the whole process:

Note that you also need some serial terminal program to monitor the output of every device, as they will show the authorization code that is required to provision your device.

Then, if you follow the rest of the instructions from the video, everything should work just fine.

Happy meshing!

Szymon Czapracki

Interoperability testing is important!

As we all know, each Bluetooth product shall work according to Bluetooth specification. It is fairly big specification, and that’s why before releasing your Bluetooth product it is very important that it can talk to other Bluetooth devices. Basically, we should check if we understand specification in the same way as others. That is why UnPlugFest (UPF) is organized 3 times per year by Bluetooth SIG, so developers can test their solutions with other products to verify they work together. Codecoup understands that and encourages its customers to be there. Speaking of which, we just came back from UPF56 in Atlanta where we had two teams testing two different IoT platforms.

Łukasz Rymanowski

Bluetooth device qualifiaction

Whenever you create a Bluetooth product you have to go through a process of qualification in order to get Bluetooth SIG stamp 🙂 This process is usually painful for developers and testers, especially when you are doing it for the first time. Fortunately, some of the Open Source stacks provide instructions on how to do this, which makes things bit easier. In this post, we will show you how to start a Bluetooth device qualification testing using an nRF52 board running Mynewt OS.

Mynewt is a Real-time operating system build with IoT in mind and provides open source networking stack including Bluetooth Low Energy. More details can be found on their website.

Mynewt allows to develop and run apps that perform specific tasks. bletiny is one of such applications. It allows controlling the Bluetooth stack from the command line. The application includes a simple shell functionality and allows e.g. start advertising or discovery.

A tutorial describing the necessary steps to run the application is available on their documentation website. To communicate with the application we will need to set up a serial connection. There is a tutorial for that too. To check if everything works properly you can run b help command, which lists all available commands in the bletiny app.

[12:08:39:333] b help
[12:08:39:336] 831277:Available commands:
[12:08:39:345] 831277: adv
[12:08:39:345] 831278: conn
[12:08:39:345] 831278: chrup
[12:08:39:345] 831278: datalen
[12:08:39:355] 831279: disc
[12:08:39:355] 831279: find
[12:08:39:355] 831280: help
[12:08:39:364] 831280: l2cap
[12:08:39:364] 831280: mtu
[12:08:39:364] 831281: passkey
[12:08:39:374] 831281: read
[12:08:39:374] 831281: rssi
[12:08:39:374] 831282: scan
[12:08:39:374] 831282: show
[12:08:39:374] 831282: sec
[12:08:39:382] 831283: set
[12:08:39:382] 831283: store
[12:08:39:390] 831283: term
[12:08:39:390] 831284: update
[12:08:39:390] 831284: tx
[12:08:39:400] 831285: wl
[12:08:39:400] 831285: write
[12:08:39:400] 831285: svcchg

The instructions for the tests can be found in the net/nimble/host/pts directory in the apache-mynewt-core project tree.

tree net/nimble/host/pts/

├── pts-gap.txt
├── pts-gatt.txt
├── pts-l2cap.txt
├── pts-sm.txt
├── README.txt
└── tpg
    ├── 90359-20161220-172100175.tpg
    └── 90359-20161220-172113981.pts

1 directory, 7 files

The contents of the directory are described in the README.txt file:

This folder contains qualification tests results against BT SIG Profile Test

pts-FOO.txt files contain result for specific profiles or protocols. This
includes PTS version, test date, enabled tests, results etc.

In addition to tests results 'tpg' folder contains Test Plan Generator
configuration files that can be imported by PTS for tests configuration.

Fragment of the test instructions file:

PTS test results for GAP

PTS version: 7.0
Tested: 10-Jan-2017

PASS test passed
FAIL test failed
INC test is inconclusive
N/A test is disabled due to PICS setup

Test Name Result Notes

TC_BROB_BCST_BV_01_C PASS b adv conn=non
TC_BROB_BCST_BV_02_C PASS b adv conn=non
TC_BROB_BCST_BV_03_C PASS b set irk= e.g: 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:11


This file contains a list of test descriptions. Each description consists of a test name, it’s result and list of instructions needed to execute the test.

After configuring the device, we can move on to setting up the PTS.


Setting-up PTS environment

Install PTS drivers for the dongle

In device manager select the PTS dongle device, open Properties and update driver. Browse for driver in C:\Program Files (x86)\Bluetooth SIG\Bluetooth PTS\PTS Driver\win32 and select csrbc.sys. If you are using a 64-bit system, then change win32 to win64.

Create new workspace and search for device

Launch PTS and select Create new workspace. Set your device in advertising mode and search for it in PTS. It should appear in the device list. Select it, so that the device address field is filled and move to the next step.

Import Test Plan

Fill in the name and location of the workspace and import the Test Plan. The Test Plan can be found in the net/nimble/host/pts directory in the apache-mynewt-core project tree.

Select test suite(s)

Select the Test Suites you are interested in and move them to the list on the right.

Use Bluetooth Protocol Viewer

It’s a great debugging tool. It allows you to see exactly what was transmitted and received by the PTS in great detail. Each frame is formatted into a very readable tree-like structure, and the frames are categorized by the protocol.

Important definitions

Before we move on to the testing itself, some important definitions that are used in PTS:

For more definitions read Test Strategy and Terminology Overview document.

Example #1

As a fist example we will run test TC_BROB_BCST_BV_01_C. In the PTS right-click on the test in Workspace Tool Window and select Run from context menu. PTS starts the test and displays an information window saying, that a user action is required:

Following the instructions, we execute b adv conn=non in the bletiny app which starts the advertising procedure in non-connectable mode. Here is the output of the test execution in the PTS:

Test case : TC_BROB_BCST_BV_01_C started
- MTC: Receive Advertising report successfully.
- MTC: Flags type is not present in any scan response data received.
- MTC: Test case ended
-Final Verdict: PASS
TC_BROB_BCST_BV_01_C finished

Example #2

The second example will be a little more complex. We will run TC_SEC_AUT_BV_13_C test. It verifies that the IUT properly rejects the service request where there is insufficient authentication and then completes service correctly with Lower Tester as a peripheral role.

The instruction for this test is a bit longer:

TC_SEC_AUT_BV_13_C PASS Note: in PTS confirm that IUT supports GATT Server
                        b set sm_data mitm_flag=1
                        b set sm_data io_capabilities=1
                        b conn peer_addr=
                        b disc svc conn=
                        Note: in PTS enter handle for characteristics
                        value which requires authenticated
                        pairing for read
                        b passkey conn= action=3 key=123456
                        Note: enter '123456' passkey in PTS

First, we run the test and confirm in the PTS that our device supports GATT Server. We then execute b set commands that set some Security Manager related data. Then PTS requires us to send a connection request to the PTS.

We send the request by executing b conn peer_addr= command, where is the address of the PTS dongle. Then we start service discovery by executing the `b disc svc conn=`, where is the connection handle. You can list connections with the b show conn command.

After that, the PTS requires us to enter the handle for characteristic value, which requires authentication for reading. You should check the GATT Server database for this value.

PTS then requires us to enter Secure-ID (aka passkey). Before we do that, we need to set a passkey in the bletiny app, so that the device knows what passkey to use, to respond to the PTS. To do that, we type in b passkey conn= action=3 key=123456. After that, we enter the same passkey in the PTS window.

After following these steps, the test should pass. The output of the test is shown below:

Test case : TC_SEC_AUT_BV_13_C started
- MTC: ATT connection established successfully.
- MTC: SEC_LE_PAIRING_COMPLETE_IND received successfully.
- MTC: SEC_LE_ACCESS_CFM received successfully.
- MTC: IUT respond as requested.
- MTC: SEC_LE_PAIR_REQUEST_IND received successfully.
- MTC: SEC_LE_PAIR_REQUEST_IND received successfully.
- MTC: SEC_LE_PAIRING_COMPLETE_IND received successfully.
- MTC: SEC_LE_ACCESS_CFM received successfully.
- MTC: ATT_READ_CFM received successfully.
- MTC: ATT connection disconnected successfully.
- MTC: Test case ended
-Final Verdict: PASS
TC_SEC_AUT_BV_13_C finished


ELCE and OpenIoT Summit Europe 2016 videos are online

Video from Szymon’s presentation on OpenIoT is available now

There were a lot of interesting sessions this year on both Embedded Linux Conference Europe and OpenIoT Summit Europe. All videos are available here.