Embedded Testing with PlatformIO – Part 4: Continuous Integration

Welcome to the fourth part of our “embedded testing” series. From our former blogposts, you are now familiar with PlatformIO and its powerful features like unit testing and remote controlling. Lately we’ve been diving into PIO Remote – a feature to flash firmware and monitor serial output from remote network locations (! Yes, make sure to check it out) - and you are now ready to plant your microcontrollers in your digital farm, share them with your colleagues and build-test-upload-test your local projects directly on these target devices from literally everywhere.

A lot of iteration speed within web development comes from Continuous Integration & Testing pipelines – Services that completely automate build and testing processes, as well as deployment of software artifacts to servers. It would be nice to have something comparable for the world of IoT as well.

With the help of the this little post, you will be able to do that. What are we waiting for? Let’s start by cloning the repository that includes the basic unit testing operations used last time:

Recall the physical setup, we’ve been using a RaspberryPi and ran the PlatformIO Remote Agent there. Next we had our board (a nodemcuv2) directly connected to the RPi via USB. The local project will be uploaded and tested remotely from our local computer. Don’t forget to start you’re your remote agent – on the RPi -:

$ pio account login
<your login data>
$ pio remote agent start -n rpi1

Continuous Integration, as a Service

What we’d like to achieve here is to build, test and flash from a remote location – that is, from a Continuous Integration Service on the web. We chose TravisCI, a well-known CI service that’s around for quite a while. TravisCI works by cloning your code from your Github repository and operating on it.

Please visit our sample code project page at GitHub, a fork this repo into your own GitHub Account. Then clone your repo to your local machine:

$ git clone https://github.com/<YOUR _GITHUB_ACCOUNT>/ci-with-platformio.git

Good. Now it is time to meet to Travis CI!

CI as a service

Travis CI is a hosted, distributed continuous integration service used to build and test software projects hosted at GitHub. First of all, we are starting with registration on Travis CI. Please navigate to travis-ci.org and register, with your GitHub Account.

We are assuming that you already have a registered GitHub account, which makes everything way easier. Hit the button “Sign in with GitHub”. It takes a little time to synchronize your repository names from GitHub to Travis and it is a good time gap to refill your coffee mug.

Synchronization is done automatically by Travis CI. The public repositories can be seen directly there. Nevertheless, it is possible that sometimes your public repositories become invisible, so do not hesitate to hit the “Sync account” button on right-top until you see them.

Here is the first sight of your account in Travis. TravisLogin

You can now turn on/off your projects that you want to use in Travis CI.

Important Note: If you want to use PIO Remote or PIO Unit Testing, then the following step is vital! Please click on the little gear-icon like settings button adjacent to the project name:

TravisProject

Under general settings, you can set them as you see in the screenshot or specify them in another way according to your requirements. You can limit the number of concurrent jobs, build branch and pull request updates, or basically not. Do not forget to activate “Build only if .travis.yml is present”.

Here is the reason for “important note”: In sake of using PIO Remote and Unit Testing functions, the account token should have been set as a variable under Environment Variables settings. To do so, please login to your PIO Account on command line:

$ pio account login
$ pio account token

Personal Authentication Token: 111222333444555xxxyyyaaabbbcccdddeeefff

After giving your account password, you can obtain your account token. Copy the token from command line window, go to Travis CI, under the Environment Variables settings, type with the flag name of PLATFORMIO_AUTH_TOKEN, paste your token next:

TravisProject

Hit “Add” button and be sure that the Travis CI is successfully authorized.

The next step is configuring Travis CI. PlatformIO is written in Python and it is recommended that to be run within Travis CI Python isolated environment.

When initializing a new PlatformIO project in a location, an initial .travis.yml is placed there. In case you imported a project from somewhere else or set it up manually, create a .travis.yml right under your local project file.

To do that, type following under your local project path;

$ touch .travis.yml

Here is how your project looks like:

$ tree
├── README.md
├── lib
│   └── mod1
│       └── src
│           ├── mod1.cpp
│           └── mod1.h
├── src 
│   └── main.cpp
├── platformio.ini
├── .travis.yml
├──test
│    └── native
│        └── test_first.cpp
│    └── nodemcuv2
└        └── test_main.cpp

Let’s configure .travis.yml. It’s a YAML file that tells TravisCI what to do with your embedded project.

$ nano .travis.yml

As a good starting point, you can use following sketch directly, please copy and paste it to your .travis.yml file.

language: python
python:
    - "2.7"
sudo: false
cache:
    directories:
        - "~/.platformio"
env:
    - PLATFORMIO_CI_SRC=src/main.cpp
install:
    - pip install -U platformio
script:
    - pio remote -a rpi1 run -t upload -e nodemcuv2
    - pio remote -a rpi1 test -e nodemcuv2

Short speech on the configuration file:

  • Python 2.7 is selected as the language. You also have other possibilities to do that.
  • sudo and cache will be left as they are.
  • env stands for the project location. You can add another projects to your CI process.
  • install is responsible for library dependencies. If you have any dependencies, you may need to update this part.
  • script just automatizes what you did on command line until today, manually. Whatever you write here will be executed by Travis automatically.

After saving and closing .travis.yml, let’s add-commit-push everything what we did. To do so:

$ git add .
$ git commit -m “commit .travis.yml v1”
$ git push

Whenever you execute git push, Travis CI is triggered. The following actions can be traced under Travis website:

Before push:

BeforePush

And afterwards:

AfterPush

Travis got notified of our changes and starts to run the build process. You can always track the log by sliding down in order to see what is going on!

When the build is successful, it is reported as “passed”:

BuildPassed

When we look on the RPi, the remote agent side, here is the output:

PlatformIO Plus (https://pioplus.com) v0.10.7
2017-09-07 13:42:33 [info] Name: rpi1
2017-09-07 13:42:33 [info] Connecting to PIO Remote Cloud
2017-09-07 13:42:34 [info] Successfully connected
2017-09-07 13:42:34 [info] Authenticating
2017-09-07 13:42:34 [info] Successfully authorized
2017-09-07 13:46:48 [info] Remote command received: psync
2017-09-07 13:46:49 [info] Remote command received: run
2017-09-07 13:47:29 [info] Remote command received: psync
2017-09-07 13:47:31 [info] Remote command received: test

So all the CI triggering logic happened within the cloud service, but the result has been flashed onto the device on our desk. Nice!

Maybe you realized that you are keep getting your build results from Travis CI as per email too. Here are some tips to set them as you wish:

Under your .travis.yml, under scripts, please type:

To add your friends to the feed:

notifications:
  email:
    - one@example.com
    - other@example.com

Or to close notifications:

notifications:
  email: false

To change notification settings:

notifications:
  email:
    recipients:
      - one@example.com
      - other@example.com
    on_success: never # default: change
    on_failure: always # default: always

To change your default notification eMail address:

$ git config user.email "mynewemail@example.com"

To change your global notification eMail address for every project:

$ git config user.email "mynewemail@example.com"

And there are other types of notifications and their usage

Finalizing…

So what did we actually achieve? Using PlatformIO’s remoting capabilities, we’re able to transfer the build & testing process, including the installation of necessary libraries and tool chains, to a cloud service. However, devices can still be connected locally, i.e. a notebook or as in our setup, a RPi simulation our IoT integration farm.

The added value lies in the automation of these processes. Every time a changes is committed and pushed, build processes are started, tests can be run. Enter the world of Agile Embedded :-)

This concludes the fourth part of this series, and we’re evolving from testing into running a complete IoT Toolchain. Stay tuned for the upcoming parts, where we’ll be looking under the hood of PlatformIO’s build system and see how to integrate special actions.

Eren

Andreas

Series