Whenever you create a Bluetooth product you have to go through a process of qualification in order to get Bluetooth SIG stamp 🙂 This process is usually painful for developers and testers, especially when you are doing it for the first time. Fortunately, some of the Open Source stacks provide instructions on how to do this, which makes things bit easier. In this post, we will show you how to start a Bluetooth device qualification testing using an nRF52 board running Mynewt OS.
Mynewt is a Real-time operating system build with IoT in mind and provides open source networking stack including Bluetooth Low Energy. More details can be found on their website.
Mynewt allows to develop and run apps that perform specific tasks. bletiny is one of such applications. It allows controlling the Bluetooth stack from the command line. The application includes a simple shell functionality and allows e.g. start advertising or discovery.
A tutorial describing the necessary steps to run the application is available on their documentation website. To communicate with the application we will need to set up a serial connection. There is a tutorial for that too. To check if everything works properly you can run b help command, which lists all available commands in the bletiny app.
[12:08:39:333] b help
[12:08:39:336]
[12:08:39:336] 831277:Available commands:
[12:08:39:345] 831277: adv
[12:08:39:345] 831278: conn
[12:08:39:345] 831278: chrup
[12:08:39:345] 831278: datalen
[12:08:39:355] 831279: disc
[12:08:39:355] 831279: find
[12:08:39:355] 831280: help
[12:08:39:364] 831280: l2cap
[12:08:39:364] 831280: mtu
[12:08:39:364] 831281: passkey
[12:08:39:374] 831281: read
[12:08:39:374] 831281: rssi
[12:08:39:374] 831282: scan
[12:08:39:374] 831282: show
[12:08:39:374] 831282: sec
[12:08:39:382] 831283: set
[12:08:39:382] 831283: store
[12:08:39:390] 831283: term
[12:08:39:390] 831284: update
[12:08:39:390] 831284: tx
[12:08:39:400] 831285: wl
[12:08:39:400] 831285: write
[12:08:39:400] 831285: svcchgThe instructions for the tests can be found in the net/nimble/host/pts directory in the apache-mynewt-core project tree.
tree net/nimble/host/pts/
net/nimble/host/pts/
├── pts-gap.txt
├── pts-gatt.txt
├── pts-l2cap.txt
├── pts-sm.txt
├── README.txt
└── tpg
├── 90359-20161220-172100175.tpg
└── 90359-20161220-172113981.pts
1 directory, 7 filesThe contents of the directory are described in the README.txt file:
This folder contains qualification tests results against BT SIG Profile Test
Suite.
pts-FOO.txt files contain result for specific profiles or protocols. This
includes PTS version, test date, enabled tests, results etc.
In addition to tests results 'tpg' folder contains Test Plan Generator
configuration files that can be imported by PTS for tests configuration.Fragment of the test instructions file:
PTS test results for GAP
PTS version: 7.0
Tested: 10-Jan-2017
Results:
PASS test passed
FAIL test failed
INC test is inconclusive
N/A test is disabled due to PICS setup
-------------------------------------------------------------------------------
Test Name Result Notes
-------------------------------------------------------------------------------
TC_MOD_NDIS_BV_01_C N/A
TC_MOD_LDIS_BV_01_C N/A
TC_MOD_LDIS_BV_02_C N/A
TC_MOD_LDIS_BV_03_C N/A
TC_MOD_GDIS_BV_01_C N/A
TC_MOD_GDIS_BV_02_C N/A
TC_MOD_NCON_BV_01_C N/A
TC_MOD_CON_BV_01_C N/A
-------------------------------------------------------------------------------
TC_BROB_BCST_BV_01_C PASS b adv conn=non
TC_BROB_BCST_BV_02_C PASS b adv conn=non
TC_BROB_BCST_BV_03_C PASS b set irk= e.g: 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:11
........This file contains a list of test descriptions. Each description consists of a test name, it’s result and list of instructions needed to execute the test.
After configuring the device, we can move on to setting up the PTS.
In device manager select the PTS dongle device, open Properties and update driver. Browse for driver in C:\Program Files (x86)\Bluetooth SIG\Bluetooth PTS\PTS Driver\win32 and select csrbc.sys. If you are using a 64-bit system, then change win32 to win64.
Launch PTS and select Create new workspace. Set your device in advertising mode and search for it in PTS. It should appear in the device list. Select it, so that the device address field is filled and move to the next step.
Fill in the name and location of the workspace and import the Test Plan. The Test Plan can be found in the net/nimble/host/pts directory in the apache-mynewt-core project tree.
Select the Test Suites you are interested in and move them to the list on the right.
It’s a great debugging tool. It allows you to see exactly what was transmitted and received by the PTS in great detail. Each frame is formatted into a very readable tree-like structure, and the frames are categorized by the protocol.
Before we move on to the testing itself, some important definitions that are used in PTS:
For more definitions read Test Strategy and Terminology Overview document.
As a fist example we will run test TC_BROB_BCST_BV_01_C. In the PTS right-click on the test in Workspace Tool Window and select Run from context menu. PTS starts the test and displays an information window saying, that a user action is required:
Following the instructions, we execute b adv conn=non in the bletiny app which starts the advertising procedure in non-connectable mode. Here is the output of the test execution in the PTS:
Test case : TC_BROB_BCST_BV_01_C started
- MTC: Successfully received HCI_LE_CLEAR_WHITE_LIST_COMPLETE_EVENT.
- MTC: Successfully received HCI_LE_ADD_DEVICE_TO_WHITE_LIST_COMPLETE_EVENT.
- MTC: Successfully received HCI_LE_SET_SCAN_PARAMETERS_COMPLETE_EVENT.
- MTC: Successfully received HCI_LE_SET_SCAN_ENABLE_COMPLETE_EVENT.
- MTC: Receive Advertising report successfully.
- MTC: Flags type is not present in any scan response data received.
- MTC: Successfully received HCI_LE_SET_SCAN_ENABLE_COMPLETE_EVENT.
- CM_EXIT
- CM_EXIT
- CM_EXIT
- CM_EXIT
- MTC: Test case ended
-Final Verdict: PASS
TC_BROB_BCST_BV_01_C finishedThe second example will be a little more complex. We will run TC_SEC_AUT_BV_13_C test. It verifies that the IUT properly rejects the service request where there is insufficient authentication and then completes service correctly with Lower Tester as a peripheral role.
The instruction for this test is a bit longer:
TC_SEC_AUT_BV_13_C PASS Note: in PTS confirm that IUT supports GATT Server
b set sm_data mitm_flag=1
b set sm_data io_capabilities=1
b conn peer_addr=
b disc svc conn=
Note: in PTS enter handle for characteristics
value which requires authenticated
pairing for read
b passkey conn= action=3 key=123456
Note: enter '123456' passkey in PTSFirst, we run the test and confirm in the PTS that our device supports GATT Server. We then execute b set commands that set some Security Manager related data. Then PTS requires us to send a connection request to the PTS.
We send the request by executing b conn peer_addr= command, where is the address of the PTS dongle. Then we start service discovery by executing the `b disc svc conn=`, where is the connection handle. You can list connections with the b show conn command.
After that, the PTS requires us to enter the handle for characteristic value, which requires authentication for reading. You should check the GATT Server database for this value.
PTS then requires us to enter Secure-ID (aka passkey). Before we do that, we need to set a passkey in the bletiny app, so that the device knows what passkey to use, to respond to the PTS. To do that, we type in b passkey conn= action=3 key=123456. After that, we enter the same passkey in the PTS window.
After following these steps, the test should pass. The output of the test is shown below:
Test case : TC_SEC_AUT_BV_13_C started
- MTC: Successfully received HCI_LE_CLEAR_WHITE_LIST_COMPLETE_EVENT.
- MTC: Successfully received HCI_LE_ADD_DEVICE_TO_WHITE_LIST_COMPLETE_EVENT.
- MTC: Successfully received HCI_LE_SET_ADVERTISING_PARAMETERS_COMPLETE_EVENT.
- MTC: Successfully received HCI_LE_SET_ADVERTISING_DATA_COMPLETE_EVENT.
- MTC: Successfully received HCI_LE_SET_SCAN_RESPONSE_DATA_COMPLETE_EVENT.
- MTC: Successfully received HCI_LE_SET_ADVERTISE_ENABLE_COMMAND_COMPLETE_EVENT.
- MTC: ATT connection established successfully.
- MTC: SEC_LE_PAIRING_COMPLETE_IND received successfully.
- MTC: SEC_LE_ACCESS_CFM received successfully.
- MTC: IUT respond as requested.
- MTC: SEC_LE_PAIR_REQUEST_IND received successfully.
- MTC: SEC_LE_PAIR_REQUEST_IND received successfully.
- MTC: SEC_LE_PAIRING_COMPLETE_IND received successfully.
- MTC: SEC_LE_ACCESS_CFM received successfully.
- MTC: ATT_READ_CFM received successfully.
- MTC: ATT connection disconnected successfully.
- CM_EXIT
- CM_EXIT
- CM_EXIT
- CM_EXIT
- MTC: Test case ended
-Final Verdict: PASS
TC_SEC_AUT_BV_13_C finished