We’ve been working on a new Faraday project called faradayio. It’s goal is clear, to simplify everything. The way faradayio simplifies the project is by making Faraday appear as if it’s a network interface by using a TUN adapter. This is similar to how VPN software works on your computer. We collect all network traffic destined for the “Faraday” network device and send it over a tunnel in software. At the end of this tunnel the IP packets are sent over the serial port to Faraday using SLIP framing provided by sliplib. We’ll talk about TUN devices in a future blog post but for now lets look at the serial port.
Unit Tests and Serial Ports
Unit testing is an important part of programming. By thoroughly testing the code in an automatic way we can ensure code quality and performance as new features are developed. Serial ports present a challenge however, because data is sent over a physical wire (USB, RS232, RS422, etc) while a unit test lives completely in software and is often implemented on remote servers (such as Travis-CI). It is incredibly inconvenient to require a real serial port for automated unit testing. This is where pyserial url handlers come in!
Pyserial Loopback URL Handler
Pyserial offers the function serial_for_url() which accepts several URL handlers providing various software-based connections such as loopbacks and sockets! loop:// is perfect for unit testing simple serial connections. It will provide a pyserial instance that simply loops the data written to the serial port back and can be read by any pyserial read function.
The GitHub Gist below shows a test class used in our unit testing.
Now let’s look at how to use this test class on a continuous integration service, Travis-CI. The Gist below is the entire unit test for faradayio serial port code. Skim through the test and then read the description of each test case to get an idea for why and how it’s used.
The socketOne() test is a simple check that a loopback serial port was setup correctly. “Hello world!” is written into the test serial port and immediately read back. Some UTF-8 encoding/decoding is necessary but otherwise this test is boilerplate pyserial code.
This test is a bit more complicated. test_input is a variable that is replaced with each item in the list directly prior to the definition of the test using pytest parameterizing. The items are created from the string and sliplip libraries which provide ASCII test values as well as SLIP constants (escape bytes, stop bytes, etc). By pragmatically testing these inputs we can be sure all ASCII and SLIP bytes are sent correctly over the serial port.
The general flow of this test includes setting up a test loopback serial port and then a faradayio Faraday instance using it. We then generate a known good SLIP message called slipMsg. This will be used to validate the faradayio code. After slipMsg is created, the faradayio send() code is used to send the same message over the serial port after which the data is read back using pyserial. Reading back with pyserial eliminates the use of faradayio receive code and compartmentalizes the test to only send-based code. The string received from the serial port loopback is then asserted against the known good slipMsg. The two variables should be exactly the same.
As you can guess, this is the exact same test as used for sending except it is used to test the faradayio receive() code. The only difference is that we use pyserial to directly write a manually SLIP encoded message into the test serial port and then use faradayio code to receive it. Due to how faradayio and sliplib work, the receive function returns a list of messages received (there could be more than one SLIP encoded frame) and therefore the receive function is iterated through with a for loop. However, since we only expect one response it’s safe to just check the expected answer on all items in the loop.
Build #7 of the Faraday/faradayio repository shows a successful unit test of serial code. Below is an excerpt showing just the pytest unit testing output. The test_serial.py file was run by pytest and created 33 iterations of tests using the parameterization of pytest input values. These 33 tests were completed in just 760ms!
The Value of Unit Testing Serial Ports
This may seem trivial but it’s incredibly useful. We define an expected interface and that should never change. Each test is a check for proper input/output relationships. These unit tests run on our local development computers as well as on remote servers. Doing so helps us ensure our code maintains compatibility and high quality. Now that we perform automated unit testing the code is checked every time a new pull request is performed on GitHub and we can be confident that nothing unexpected will get through.
Subscribe to Blog via Email
Author: Bryce Salmi
Licensed radio amateur KB1LQC and Co-Founder of FaradayRF. Professional Electrical Engineer designing and building avionics for rockets and spacecraft during the day and developing the future of digital amateur radio experimentation by night. All opinions are my own.