Platformio for the Win

A few months back I posted on this blog about the many ways to run code on an Arduino. While there are use cases for which Circuit Python or Simulink make sense, most of the uses this author has encountered involve installing code on an Arduino, and rarely if ever reprogramming the Arduino. For this use case, especially when cost and energy efficiency (battery life) are taken into account, compiled code i.e. C++ using the Arduino Framework, makes sense. Yet many projects at Bucknell, encounter problems with the compiled code approach because the default tool, the Arduino IDE, has a number of short comings. Fortunately, there is Platformio.

Platformio is at heart a front end for the same open source tools that power the Arduino IDE. The difference is in the intended audience. While the Arduino IDE attempts to make microcontroller programming accessible to those who have never programmed before, Platformio attempts to provide programmers with a no nonsense build system that plays well with the programmer’s choice of editor, version control system, and development operating system. Platformio also isolates projects from one another and allows programmers to target multiple microcontroller models with a common code base. This power comes with a small caveat, most documentation available online when one searches for “Arduino” only covers usage of the Arduino IDE. It is left to the programmer to know how to translate the documentation to Platformio.

Getting Platformio Installed

Installing Platformio is platform dependent and depends on whether you want to make use of the VS Code or Atom.io integration. Most programmers will find that the command line interface (CLI) is easy to use and the integration options to be harmless but unnecessary. Assuming you will be installing the command line version of Platformio, Platformio Core, in the simplest manner possible…

macOS

Apple ships the macOS with an out of date version of most of the bundled command line tools making development on the stock macOS less than ideal. Most Mac users therefore install the brew package manager to obtain a stable but up to date command line environment. With the brew package manager in place, installing Platformio Core is as simple as opening a Terminal.app window and while connected to the internet issuing the command:

brew platformio

Linux

Linux users understand that Linux is more a family of operating systems than a consistent product of a dedicated and focused product team. As such your mileage will vary but most recent releases by major distributions (Debian, Fedora Core, and variants e.g. Ubuntu, RHEL, CentOS) include udev and python 3.5+ allowing you install Platformio Core simply by opening a Terminal window and while connected to the internet issuing the following commands:

pip install -U platformio
curl -fsSL https://raw.githubusercontent.com/platformio/platformio-core/master/scripts/99-platformio-udev.rules | sudo tee /etc/udev/rules.d/99-platformio-udev.rules
sudo udevadm control --reload-rules
sudo udevadm trigger

System administrators may want to prefix the first command with sudo and drop the -U to install Platformio for all users. If you are using third party arduino boards you may also want to check out how to configure udev for additional Arduino compatible boards on this blog.

Windows

As is often the case, installing Platformio on Windows is much harder than on Mac or Linux but it basically boils down to installing python, installing the device drivers for your Arduino board(s), and finally using a python script to install Platformio. It may be easier to enable the Windows Subsytem for Linux, install Ubuntu from the Windows store, and finally follow the instructions above for Linux.

Using Platformio – Part 1

The Platformio CLI is similar to CLI tools such as git, pip, composer, yarn etc in that is invoked using the name of the command e.g. platformio or the shorter alias pio ((hereafter I will use pio when referring to the command line tool) followed by subcommand and optionally by a sub-subcommand, “flags” aka “switches” prefixed by a “-” when using a single letter abbreviation or “–” when using a long flag / parameter name, and finally by any required parameters. E.g.

pio subcommand sub-subcommand --flag --parameter parameter_value

pio and all of its subcommands recognize the -h and --help flag as a request to print an informative help message.

pio -h

Starting a Project

The first difference between the Arduino IDE and Platformio is that the Arduino IDE forces you into placing your projects in a particular folder (~/Documents/Arduino/${PROJECT_NAME}) but imposes no structure on your project while Platformio forces you to adopt a standard structure for your project but allows you to choose the location for your project. This author organizes his projects using the ~/Documents/${CLIENT_NAME}/${PROJECT_NAME} convention. To create the folder structure for a new project, Platformio provides the “init” subcommand. Therefore, the first step when starting a new project is to create a directory for it and initialize the Platformio folder structure inside that directory. Typically, you will have an idea which microcontroller board you will be using when you start a project; thus the microcontroller can be provided as a parameter though it is optional and you can change or add more target boards later.

mkdir ~/Documents/${CLIENT_NAME}/${PROJECT_NAME}
cd ~/Documents/${CLIENT_NAME}/${PROJECT_NAME}
pio init -b ${BOARD_IDENTIFIER}

Board identifiers are perhaps the most cryptic part of using Platformio. This can in part be attributed to the popularity and diversity of the Arduino ecosystem. There are many manufacturers producing Arduino compatible microcontroller boards, some of which use similar names for similar boards; however not all boards with similar names are equal when it comes to compiling code, and a programmer must take care to select the correct board identifier. Fortunately, Platformio has a “boards” subcommand which can help you find the correct identifier.

pio boards ${QUERY}

Issue the command:

pio boards uno

for example to see a list of boards containing “uno” in the identifier or name and arranged by support package (roughly processor family). The classic Arduino brand Uno is selected with identifier uno while the ESP32vn IoT Uno is selected with the esp32vn-iot-uno identifier. Identifiers generally reflect the name of the device but there is no consistency in how special characters and white space characters are treated.

Once you have created a project, you should take the time to become familiar with the project structure. This structure should seem familiar to anyone who has worked on an open source project as it follows the same organizational pattern.

Briefly; the “.gitignore” file is provided to exclude from version control any libraries installed and managed by Platformio as well as build intermediates. The “include” directory is where header files should be placed if you are developing a library for distribution using Platformio. The “lib” directory is where you should place libraries you have developed and have not shared so they can be downloaded using Platformio. The “platformio.ini” file is the configuration file for the project akin to a build.xml or meson.build file, more on this file later. The “src” directory is where you should place your source code files. There is no naming convention enforced for files in this folder e.g. the main program must be in main.cpp; other than files containing C++ code should be named with .cpp as the file name extension rather than .ino as used by the Arduino IDE. This author typically names the file containing the setup() and loop() functions firmware.cpp, declares class interfaces in files named ${CLASS}.h and class definitions in ${CLASS}.cpp. For example, a class for a sensor embedded in a floor might be declared in “FloorSensor.h” and defined in “FloorSensor.cpp”. Many if not most projects will only have a single .cpp file in the src directory. The “test” directory should contain testing code to be run in a continuous integration (CI) environment such as Travis CI. The “.travis.yml” provides templates for configuring CI testing using Travis CI.

Missing from the Platformio project structure are two important files for project management; the “LICENSE” and “README” files. The “LICENSE” file tells others what license your code is released under. All projects should have a “LICENSE” file; even internal only projects, though they may simply state the code is for internal, commercial use only and not licensed for distribution as source code. Further, all projects should have a “README” file (or “README.md” if you prefer) telling others, or more often your future self what the project does, what hardware it works with, how to compile and install it (see Building and Uploading a.k.a. Running a Project below) and who is responsible for the code. In this author’s experience, your future self will thank you. Bottom line, the second step to creating a new project is to add the missing LICENSE and README files. (See also this README.md how to) An optional but highly recommended third step to starting a new project is to put the project under revision control. Using git makes this as easy as issuing the commands:

git init
git add platformio.ini

Should the project turn into a keeper, remember to add your source files to version control, provision a cloud git repo, add an origin to your local repository, and push the project to the cloud. For more see Pro Git by Scott Chacon and Ben Straub.

While this may seem like a considerable amount of work for a simple project all before creating your first source code file, this modicum of self discipline will, with a bit of practice, take only seconds, help you structure your thoughts (who is this program intended for leads to selection of LICENSE file, what does it do and how does it work should be in the README), and result in better programming.

Programming with Platformio

There really is no such thing as programming with Platformio. While the Arduino IDE provides a code editor which it strongly encourages you to use, Platformio is designed to manage and build projects, not to be a source code editor. This is a great design choice as there are already a number of good and a few great source code editors available for free.

This author uses Microsoft's Visual Studio Code these days but was long a user of TextMate and years ago, BBEdit.  Along the way I have used more feature packed (bloated) products such as  Microsoft's Visual Studio, Eclipse, Dreamweaver, Apple's XCode, and CodeWarrior but have found that a simple editor which does little more than syntax coloring and code folding is more comfortable for me. Code completion and popups/popovers only get in the way. Recently, I have found that linting especially for interpreted languages has come a long way and been a help. I do however miss the power of the Find/Replace found in TextMate and CodeWarrior (for BeOS). 

There are two considerations when writing code for a project managed with Platformio as opposed to the Arduino IDE. The Arduino IDE in its efforts to make programming easier for novices has added a tweak which renders its .ino files invalid .cpp files; namely an undocumented, nonstandard preprocessor hack adding:

#include <Arduino.h>

at the beginning of .ino files before handing them to the compiler toolchain. The other consideration is less specific to the Arduino IDE and more an artifact of the low level nature of C and C++. When writing code in a language such as Java or Python you can add a method or function by simply adding it in your source code file, beginning or end does not matter so long as it is in the intended scope. You can then call this method or function from anywhere in your code before or after the declaration. In C the rules are more complicated; in particular, a function must be defined before any code where it is invoked. Therefore,

#include <stdlib>

int main(void) {
add(2, 3);
}

int add(int a, int b) {
return a + b;
}

is not valid code but

#include <stdlib>

int add(int a, int b) {
return a + b;
}


int main(void) {
add(2, 3);
}

and

#include <stdlib>

int add(int, int);

int main(void) {
add(2, 3);
}


int add(int a, int b) {
return a + b;
}

are both valid. The later example being a “forward declaration“. In any case Platformio as of version 4.1.0 in its quest to make no assumptions about you or how you code, does not provide a template file that one would get if using the Arduino IDE. You could be using a framework other than Arduino or even writing your code in Lua. Assuming you are using the Arduino framework and writing your code in C/C++, here is a simple template to get you started:

/**
* [Description of Program here]
*
* We are using the Arduino Framework thus the functions `setup` and `loop`

* are special. If this were a plain C/C++ project, one could imagine main
* being written:
*
* int main() {
* Arduino.init();
* setup();
* while(true) {
* loop();
* }
* }
*
* Author [Your name and email here]
* License [Indicate license here]
*/
#include <Arduino.h> // includes stdlib.h and string.h

/**
* setup is a special function defined by the Arduino platform
* that is called once after the core firmware initialization
* happens but before loop is called for the first time
*/
void setup(void) {
// put statements to run once on startup / after reset here e.g:
// Serial.begin(19200);
}

/**
* loop is a special function defined by the Arduino platform that is
* called continuously in an infinite loop once platform specific
* initialization and setup() have run.
*/
void loop(void) {
// put statements to run continuously here
}

Using Platformio – Part 2

Standing on the Shoulders of Giants a.k.a. Library Management

One of the major factors propelling the adoption of the Arduino ecosystem is the large number of peripherals, especially peripherals with high quality prewritten drivers. Without the efforts of vendors like Adafruit the time needed to integrate a sensor like the LSM303 into a microcontroller project is on the order of days for an experienced programmer and out of reach for novices. While the Arduino IDE does have a library manager, which allows one to download and install libraries into the IDE, Platformio takes library management to the next level by isolating code libraries within a project similar to Python‘s virtual environments but with less fuss. New in version 4, the platformio.ini file can track the libraries required for a project so they can automatically be downloaded when a project is checked out from version control on a new machine; similar to composer for PHP, node or yarn for Javascript, or cargo for Rust.

Library management has until recently long been maddening for software engineers. How does one find a useful library, integrate it into one’s project, keep it up to date, and keep the library from breaking other projects? Platformio solves each of these problems letting you focus on your code. To work with libraries, Platformio provides the “lib” subcommand. Unlike the Platformio subcommands introduced thus far, the lib subcommand is followed by a sub-subcommand.

pio lib search ${QUERY}
pio lib install ${LIBRARY_IDENTIFIER}
pio lib update ${LIBRARY_IDENTIFIER}

Finding a library requires using the “search” sub-subcommand followed by a query, typically a keyword e.g. “Neopixel” or “ws2812” which is the model of the controller used in Adafruit’s NeoPixel LED arrays. As the query is on the command line, remember to quote queries including whitespace or special characters. Ex.

pio lib search "LSM303 Magnetometer"

One of more powerful ways to search libraries is to search for header files provided by a library. For instance if you were looking at some example code from Adafruit for the LSM 303 Acceleromter. You might see that it begins with:

#include <Adafruit_LSM303_Accel.h>
#include <Adafruit_Sensor.h>
#include <Wire.h>

Now Wire.h is part of the Arduino framework but what library provides the other two includes? pio lib search can usually tell:

tkegan@brick:~$ pio lib search Adafruit_LSM303_Accel.h
Found 1 libraries:

Adafruit LSM303 Accel
========================
ID: 6775

Unified Accelerometer sensor driver for Adafruit's LSM303 Breakout

Keywords: sensors
Compatible frameworks: Arduino
Compatible platforms: Atmel AVR, Atmel megaAVR, Atmel SAM, Espressif 32, Espressif 8266, GigaDevice GD32V, Infineon XMC, Intel ARC32, Kendryte K210, Microchip PIC32, Nordic nRF51, Nordic nRF52, ST STM32, ST STM8, Teensy, TI MSP430
Authors: Adafruit

You can also search for libraries online.

Once you have found a library that should meet your needs you can install it into your project using pio lib install specifying the library by ID:

pio lib install 6775

or by name (the blue text in the example above), remember to quote library names containing special characters or whitespace:

pio lib install "Adafruit LSM303 Accel"

This author recommends always using the library name however for reasons which will be covered below. Either way, Platformio will install the library into you project specifically making a copy in the hidden directory withing you project named .pio. The library will automatically be included when you build your project. If the library does not work out you can remove it from your project with:

pio lib uninstall ${LIBRARY_IDENTIFIER}

where LIBRARY_IDENTIFIER is the name or ID number of the library to remove. Remember that libraries take up memory on your device whether you use them or not and they can interfere with one another, so it is best to remove unused libraries. In case you have forgotten the libraries you have installed, you can list them with pio lib list.

Should you have found a library that is useful in your project you will want to document the dependency. Historically, programmers would include a list of dependencies in their README and that is still considered a good practice. Platformio can additionally keep a record of your dependencies in the platformio.ini file. Should you track dependencies in this way, you and your collaborators can easily get up to speed on a IoT project by cloning or checking out the project from hosted version control, changing into the project directory and issuing pio lib install. To add a library as a dependency, use the install sub-subcommand with the --save flag; don’t forget to commit your change to the platformio.ini file and push it to you hosted version control system.

pio lib install --save ${LIBRARY_IDENTIFIER}
git add platformio.ini
git commit -m "Added ${LIBRARY_NAME} as a dependency
git push

One word of warning: as of Platformio version 4.0, the ${LIBRARY_IDENTIFIER} you use on the command line is recorded in the platformio.ini file. If you use the libraries ID number, your platformio.ini can become quite cryptic. Using the library’s name makes for a more readable platformio.ini file which can be helpful when reviewing code commits.

An important benefit of how platformio installs libraries into projects is that you can update a library in one project without updating it across all of your projects. As developers improve their libraries, sometimes they introduce changes which can break your code. With the ability to selectively update libraries on a project by project basis you can install the latest version in one project, and take the time to adapt your code to the changes without having to update all of your other projects at the same time. This author has found this especially helpful when managing both development and production projects on the same PC.

To update all libraries installed in a project simply issue pio lib update. To update a single library you can specify it as a parameter.

pio lib update ${LIBRARY_IDENTIFIER}

Building and Uploading a.k.a. “Running” a Project

Today we take it for granted that a microcontroller should be programmable by connecting a USB cable to it and clicking a button. Twenty years ago, getting a microcontroller to run your code was hard. Often industrial processes had to be duplicated by pulling chips out of sockets and placing them in special devices known as “programmers”. Some microcontrollers even required code to be sent to a special company known as a “fabricator” to have it fashioned into a read only memory module. Arduino compatible boards are in no small part responsible for this shift, so it is understandable that from one perspective the Platformio CLI appears to be a step back. A better perspective is that the Platformio CLI makes building and running code on your microcontroller relatively straightforward for all supported microcontrollers while providing capabilities beyond that of a button push interface such as customizing build flags and scriptability.

To build your project managed with Platformio issue pio run. For projects that support multiple microcontrollers, board loadable packages will be created for all targeted microcontrollers by default. You can limit the build process to a select microcontroller using the -e flag.

pio run -e ${BOARD_IDENTIFIER}

To push your compiled code to your microcontroller, connect the microcontroller to your PC using the proper cabling and issue pio run -t upload. If your project is set up with multiple targets add the -e flag.

pio run -e ${BOARD_IDENTIFIER} -t upload

Learning More

Platformio is a powerful tool for managing and building projects targeting common microcontroller packages. The powerful CLI takes guess work out of the process and isolates your projects from one another preventing problems commonly experienced by those using the Arduino IDE. While this post covers the basics and a bit more, Platformio has a number of features e.g. a serial monitor, CI build automation that have not. To learn more the definitive source is the Platformio Docs.


Posted

in

by

Tags: