The Sony Spresense development board is ideally suited for IoT and tinyML applications, now supported by Edge Impulse.
The Spresense is more of a hardware/software ecosystem than just a development board as it comprises several components and is designed for expansion via either Arduino shields or custom add-on hardware. The main board itself is compact enough to be dropped into many production-grade systems without much fuss, and from a software standpoint, you have several options — whether using the lower level C SDK based on the NuttX RTOS to the more high-level and familiar Arduino IDE and Arduino API. There is even support for Python and Java if C/C++ is not your thing.
Let’s take a deeper look at the main board (CXD5602PWBMAIN1), which comes in three regional variants (US/Japan, China and Europe) that pack quite a punch for its size.
A high-level block diagram shows the main features at a glance.
The main board is centered around Sony’s CXD5602 system on a chip (SoC), which is what differentiates this board and makes it a superior choice for embedded ML. The SoC is jam-packed with six Arm Cortex-M4F CPU cores each with an integrated floating point unit and clocked up to 156MHz to help with the heavy lifting required for running inference. This is combined with 1.5MB of shared SRAM, meaning that this chip will have no trouble at all with the more demanding embedded ML applications. While the main board is advertised as having six CPU cores, it actually has eight — the half dozen high-performance DSP-capable cores as well as an additional Cortex-M0 + and -M4 but more on that in part two.
The CXD5602 is supported on the main board by Sony’s CXD5247 power and audio management IC that provides advanced power management functionality for the CXD5602, including power path control, battery management and audio interfacing specifically developed for the CXD5602.
8MB of flash memory is included on the main board to provide ample storage for applications, again this should be provide more than enough capacity for demanding tinyML applications.
In addition to the CXD5602 and CXD5247, the main board also has a USB to serial convertor to cater for serial programming and allows for serial console monitoring while keeping the main USB port free. An onboard chip antenna for the GNSS built into the CXD5602 ensures that you don’t need to add any hardware to make use of the GPS, however due to the nature of GPS and varying performance in different environments, external antennas are sometimes required and this is handled by means of an external U.FL connector.
Power options include using USB or supplying power directly to the headers, something familiar to most Arduino users. There is additional support for battery power via power pins next to the USB connector. If you’re wanting to develop low-power applications, you can make use of the power management API in the low-level C Spresense SDK or the higher-level Low-Power Arduino API. This is possible due to the energy efficient features of the CXD5602 such as the ability to power down individual cores and run the cores at 32.736MHz as part of the low-power mode. The CXD5247 also provides the power management support for enabling low-power operation.
Beyond that, the main board has a compact Hirose board-to-board connector on its underside that allows it to be attached to the Spresense extension board and a pair of headers for basic interfacing and attaching the recommended third-party accessory boards. It is worth noting that the onboard headers offer limited I/O and everything is accessible via the board to board connector, so should you wish to design the main board into your own application and forgo the Spresense extension board detailed in the following section, you could achieve this by interfacing to your own system via this board-to-board connector.
Accessories
The small size of the main board and the limited available I/O means that for any serious tinyML applications you will need the official accessories
Extension board: The Spresense extension (CXD5602PWBEXT1) breakout board has the familiar Arduino Uno pinout and header arrangement for mounting Arduino shields, the pins also map directly to the equivalent Arduino pin names when developing using the Arduino IDE. The extension board itself includes a microSD card socket, which is required for storing captured data such as samples and image frames that are the typical in ML applications.
Direct access is available to the CXD5247’s Class D’s stereo output by means of a speaker interface should you wish to directly interface two small speakers. A small low-power audio stereo headphone amplifier is included on the expansion board as well, and this amplifier interfaces to the 3.5mm headphone jack to ensure that the audio output is suitably matched to headphones for wearable applications. For audio capture, the analog microphone interface is directly exposed via jumper pins and it supports various types of microphones (dynamic, electret and MEMS). The same applies to the digital microphone interface, which is accessible via pins on the extension board. By leveraging these audio features with ML using Edge Impulse as part of your development work flow, you could build your own custom smart speaker type applications, for example. The high-performance cores combined with the high-resolution audio codec means you can build high-end audio applications.
Yet another handy feature is the configurable digital I/O levels that can be set to 5V or 3.3V with a jumper. This provides the flexibility for interfacing to systems that run at different levels with out needing to add level shifting in between, less messy wiring on a bread board when interfacing to your preferred hardware.
In addition to the standard Arduino headers providing easy mapping via the Arduino IDE for the analog and digital interfaces, there is even a Micro-USB Type B connector that is directly connected to the main USB port on the CXD5602.
Finally, there is a CoreSight Debug header for direct Serial Wire Debugging if you wish to directly debug the CXD5602.
Camera board: Sony produces their own image sensors and have incorporated their five-megapixel ISX012 sensor in the officially supported camera board (CXD5602PWBCAM1) designed to interface to the camera connector. The ISX012 is a full-color sensor that supports onboard JPEG encoding and can also output RAW and Y/C and RGB raster data. Shutter speeds as fast 1/42000 of a second are supported with 15 frames per second at full resolution means that this camera is quite capable for many common computer vision applications. The sensor was originally meant for mobile phones, but we have seen some of the amazing things done with the host of cameras that have found their way into other developer kits, and while the lens configuration is not as flexible as those found in industrial-grade hardware, I am sure we will see some amazing things done as this camera is pushed to its limits backed by the powerful CXD5206.
Wi-Fi: Neither the main board nor the extension have any Wi-Fi hardware and you could opt for using standard Arduino shields for this or you could add the officially recommended IDY iS110B Wi-Fi board, which attaches directly to the headers on the main board or you could integrate your own. The iS110B is has a 2.4GHz IEEE802.11b/g/n radio. There are also some nice features, like built in MQTT, TCP and HTTP clients as well as many other useful ones that can be controlled using the Arduino environment that you can use to add IoT functionality to your ML project.
Rohm add-on boards: It’s worth mentioning that there are a number of other add-on boards available made by Rohm, such as the SPRESENSE-SENSOR-EVK-701 modules that measure acceleration, pressure and geomagnetism and support a range of stackable add on sensors that can measure light, heart rate and color, all of which are applicable to embedded ML applications.
At this point, we have gone over the main board and the accessories and you will no doubt realize that the Spresense system has a wealth of features that make it ideal for a diverse range of ML applications. Where you may have previously used two or three separate boards for a more complex application, you can now use one single board.
The good news for embedded developers is that that the Spresense is natively supported by Edge Impulse and you can head here to get started.
That’s it for part one. In part two, I will go over the CXD5602 and CXD5247 in a bit more detail to highlight features that make this piece of hardware a game changer.