Embedded Testing with PlatformIO – Part 3: Remoting

In the previous parts of this blog post series we’ve been looking into unit testing of IoT devices, using PlatformIO and its test capabilities. For web developers, unit testing has a long and solid history and is considered an integral part of software development, since a number of other things depend on it. Beyond having metrics about for example code coverage, one of the key things to have within web development is a continuous integration (and also – delivery) pipeline. Would you want to deploy a web application to a production environment if some of the tests fail? Probably not.

Things are truly different for embedded development, but at ThingForward we think that Continuous Integration (CI) should be also an integral part of a development toolchain for the IoT. Now quite some CI tools are installed on servers – as part of an inhouse DC installation or as a cloud service. But Embedded Development includes flashing of devices, which are typically connected to a host, i.e. via USB, J-Link, ST-LINK and others. You cannot really do this when using a cloud CI service, and you probably do not want to do this when your CI server resides in some locked racks in your basement.

Luckily, PlatformIO includes a form of remote control capability, called PIO Remote(tm). So this blog post is going to be about how to get Remote up and running.

Overview

In the first and second parts of testing series, the unit test cases were set up for an embedded project locally by using PlatformIO’s testing features (Remember the pio test command?). This embedded project runs on multiple platforms: native (i.e. developer’s notebook) and embedded (i.e. mcu). In this third blogpost, we will examine closely the PIO Remote function and its capabilities. Then, the simple project will be implemented to show you how things work.

This blogpost is following the first two, so it is envisaged that you already have PlatformIO and other tools, and that you know how to work with unit tests.

Here is a sketch that is exactly simplified by using PlatformIO’s main Remote schema in sake of showing the technology lying behind.

Remote agent and node

Basically, PIO Remote is taking the role of old-school SSH, tunneling and VPN methods to prevent you from spending more effort and time. The developers can use target boards remotely from all around the world.

The idea is testing the embedded projects directly on target device, if there are limited amount of boards given. Both the agent and client should be computers which are entitled to connect microcontrollers or development boards and be able to compile code for these. For this project, the PIO Agent is a Raspberry Pi 3, the target board is NodeMCUv2 and they are attached with a USB cable.

The embedded project is created and saved on client side and the request will be directly sent via network connections. It works because both agent and node connect to the PIO Remote Cloud service, and operate under the same user identity.

Here are the simple steps for starting to use Remote:

  • Login to your Platform IO account on agent and client side.
  • Start Remote agent framework on agent side.
  • List and see the remote devices from client side.
  • Process remote tests on target device.
  • Monitor the results by Remote Serial Port Monitor

If you want to try this out, you’d definitely need a PlatformIO account. Registration is free, and the free PIO Community plan includes a number of remoting invocations to try this feature. However, extended usage requires a paid plan. This tutorial should work with the community plan. So make sure to register if you want to follow the steps here.

Install PlatformIO agent and node

Make sure to have PlatformIO core installed on both agent and node. This can be a single machine, or the agent may run somewhere else. We installed it on a Raspberry Pi 3, so this will be our PlatformIO agent.

On the agent side, connect your target device. Use ´platformio device list´ to find out, which USB device port your OS connected it to. In our case, it’s ´/dev/TTYS0´, but it may be different on your hardware. Remember the port.

Get the example code

Let’s start to code. First, please download our code from second blogpost:

$ git clone --branch blogpost2 https://github.com/thingforward/unit-testing-with-platformio.git 

Please update the platform.ini file of the project by adding the upload ports (from above), to decide on which board belongs to which port:

[env:nodemcuv2]
platform = espressif8266
board = nodemcuv2
framework = arduino
test_filter = nodemcuv2
;add
upload_port = /dev/ttyUSB0

[env:native]
platform = native
test_filter = native

As a vital step, you have to login to your Platform IO account on both client and agent sides in order to make both sides talk to each other and run remote tests.

$ pio account login

To see which accesses/plans you have, run:

$ pio account show

It should at least show permissions for Remote Unit Testing, Remote Serial Port monitor and Remote Agent List.

Agent

The next step is initiating and naming the agent. On the agent side, run:

$ pio remote agent start -n rpi1

That command starts the remote agent, giving this agent the name of rpi1. This command does not daemonize, so make sure to leave the shell open or daemonize it appropriately.

Then you will see that the agent initiation is successful:

PlatformIO Plus (https://pioplus.com) v0.10.1
2017-08-09 09:17:06 [info] Name: rpi1
2017-08-09 09:17:06 [info] Connecting to PIO Remote Cloud
2017-08-09 09:17:07 [info] Successfully connected
2017-08-09 09:17:07 [info] Authenticating
2017-08-09 09:17:07 [info] Successfully authorized

The remote agent start command has some additional options: You may want to run the agent in another working directory, using --working-dir. An interesting option is --share where you allow other colleagues with PlatformIO accounts access to your agent. Please use pio remote agent start –h for help on this.

Let’s keep going further at the client side.

Client

Make sure to pio account login on the client side as well. The initiated agent can be seen on client side by:

$ pio remote agent list

In our case it shows:

PlatformIO Plus (https://pioplus.com) v0.10.2
rpi1
----
ID: 7129b985088ef75da7daf4a873802e306ee490ed
Started: 2017-08-17 14:05:30

After seeing the remote agent on client side, let’s upload the code to the target device. Make sure you cd to what you git-cloned above and run pio remote:

$ cd unit-testing-with-platformio
$ pio remote -a rpi1 run -t upload -e nodemcuv2

The remote command knows some subcommands, such as run. Essentially, this runs the “run” command with a -t upload option, but instead of doing this locally, it redirects the command and all I/O to the named remote agent (-a rpi above). We need to add -e nodemcuv2 to indicate what environment to run under.

Here is the output from the client side:

PlatformIO Plus (https://pioplus.com) v0.10.7
Building project locally
[Tue Sep  5 09:41:19 2017] Processing nodemcuv2 (platform: espressif8266; upload_port: /dev/ttyUSB0; board: nodemcuv2; test_ignore: native; framework: arduino)
--------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
Collected 24 compatible libraries
Looking for dependencies...
Library Dependency Graph
|-- <mod1>
Checking program size
text       data     bss     dec     hex filename
221030      888   29408  251326   3d5be .pioenvs/nodemcuv2/firmware.elf
========================= [SUCCESS] Took 0.99 seconds =========================

================================== [SUMMARY] ==================================
Environment nodemcuv2   [SUCCESS]
Environment native      [SKIP]
========================= [SUCCESS] Took 0.99 seconds =========================
Uploading firmware remotely
[Tue Sep  5 07:41:28 2017] Processing nodemcuv2 (platform: espressif8266; upload_port: /dev/ttyUSB0; board: nodemcuv2; test_ignore: native; framework: arduino)
--------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
Looking for upload port...
Use manually specified: /dev/ttyUSB0
Uploading .pioenvs/nodemcuv2/firmware.bin
Uploading 226064 bytes from .pioenvs/nodemcuv2/firmware.bin to flash at 0x00000000
................................................................................ [ 36% ]
................................................................................ [ 72% ]
.............................................................                    [ 100% ]
========================= [SUCCESS] Took 33.53 seconds =========================

================================== [SUMMARY] ==================================
Environment nodemcuv2   [SUCCESS]
Environment native      [SKIP]
========================= [SUCCESS] Took 33.53 seconds =========================

Here is the update that can be seen on agent side. Please observe the last two sentences:

PlatformIO Plus (https://pioplus.com) v0.10.7
2017-09-05 07:39:06 [info] Name: rpi1
2017-09-05 07:39:06 [info] Connecting to PIO Remote Cloud
2017-09-05 07:39:07 [info] Successfully connected
2017-09-05 07:39:07 [info] Authenticating
2017-09-05 07:39:07 [info] Successfully authorized
2017-09-05 07:41:16 [info] Remote command received: psync
2017-09-05 07:41:28 [info] Remote command received: run

Now, it is time to run unit tests on target device. In sake of demonstrating PIO Remote features, the same test cases are used from blogpost 2. Here is the test remote command:

$ pio remote -a rpi1 test -e nodemcuv2

PlatformIO Plus (https://pioplus.com) v0.10.7
Building project locally
Verbose mode can be enabled via `-v, --verbose` option
Collected 2 items

===================== [test::nodemcuv2] Building... (1/3) =====================
Please wait...
Testing project remotely
PlatformIO Plus (https://pioplus.com) v0.10.7
Verbose mode can be enabled via `-v, --verbose` option
Collected 2 items

===================== [test::nodemcuv2] Uploading... (2/3) =====================
Please wait...

====================== [test::nodemcuv2] Testing... (3/3) ======================
If you don't see any output for the first 10 secs, please reset board (press reset button)

test/nodemcuv2/test_main.cpp:24:test_mod1   [PASSED]
-----------------------
1 Tests 0 Failures 0 Ignored

================================ [TEST SUMMARY] ================================
test:native/env:nodemcuv2   [IGNORED]
test:nodemcuv2/env:nodemcuv2    [PASSED]
========================= [PASSED] Took 36.57 seconds =========================

On the agent side we find the execution in the log output:

(…)
2017-09-05 07:44:51 [info] Remote command received: psync
2017-09-05 07:44:56 [info] Remote command received: test

Results of the test cases are identical to second blog post results, it’s just executed on the remote side. The pio remote command allows for a number of subcommands, such as run, test, or the monitoring of the serial port, quite useful, too:

$ pio remote -a rpi1 device monitor

We’ve just added a Hello World output to the loop function, here is the output on client side:

PlatformIO Plus (https://pioplus.com) v0.10.7
Starting Serial Monitor on rpi1:/dev/ttyUSB0
--- Miniterm on socket://localhost:52553  9600,8,N,1 ---
--- Quit: Ctrl+C | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
0_~?_4_Ҷ␖__␔OCAR__Hello World
Hello World
(...)

Putting it all together, we now have another jigsaw piece towards our IoT Continous Integration pipeline. Development environment, CI server and a Debugging/Monitoring/Flashing station may reside at totally different locations.

Connectivity between agent and node relies on the PlatformIO account where both sides are equipped (and logged in) with. On remote servers you typically do not want to share your personal login data. Luckily, pio account command offers a token mechanism where one can create a personal authentication token, to be configured into other services.

That concludes the third part of our testing series. Have fun!

Eren

Andreas

Series