Sonata system

Sonata is a system for evaluating the usage of CHERIoT Ibex core as a microcontroller for embedded, IoT and Operational Technology applications. The system contain a number of peripherals (I2C, SPI, GPIO, USB, and UART) and the CHERIoT Ibex core itself.

It is designed for use on FPGA and specifically targets the Sonata FPGA board, but as the entire design (from example PCB to software) is open-source it can be run on any similar system.

This project is designed to look like a normal microcontroller in terms of usability, including SDK, examples, and normal capabilities such as debuggers. But underneath that the CHERIoT capabilities provides a high level of "default security" that simplifies designing embedded systems in a secure manner. You can see the complete documentation for the project, but note it is under active development so substantial improvements are to be made.

Sonata is part of the Sunburst Project funded by UKRI / DSbD under grant number 107540.

The sonata developement system

Current status

We are delighted to announce the release of the Sonata system! The latest release is available now from GitHub here.

This new release provides a base FPGA image with I2C/SPI/UART peripherals that are suitable for use with the expansion headers as well as the on-board hardware.

On the software side a full build flow for Microsoft's CHERIoT RTOS is available along with driver support for the Sonata peripherals in the sonata-software repository. A flash bootloader enables easy 'drag and drop' programming where a generated UF2 is copied to Sonata’s virtual USB drive (in a similar manner to the RP2040 and arm Mbed platforms). Environment setup and build instructions are available for Windows, macOS and Linux.

If you find any issues with the Sonata system, or have features you would like to propose, please create an issue on GitHub. For support with the board itself please use the forum on the NewAE website, create an issue on the sonata-pcb repository, or reach out to NewAE directly.

Getting started

If you have a Sonata board, you can jump to the Getting Started guide. This will walk you through plugging in the board, building example software, and programming the software. For more advanced usage, you can see a Reference Manual similar to what a normal microcontroller reference manual (peripherals, features, etc.) and then see the FPGA development flow if you wish to modify the soft-core itself.

You can also work with a simulated environment, and there is a nice guide on this in the sonata-software documentation. This simulates the entire in Verilator, allow you to develop both hardware (the FPGA) and software (running code) programs.

Documentation introduction

This documentation is built using mdBook. If you are reading this file in GitHub, you should instead see the pre-built documentation on the lowRISC Website which includes the full documentation.

If you'd like to build a copy of the documentation locally, see the Building Documentation page.

License

Unless otherwise noted, everything in the repository is covered by the Apache License, Version 2.0. See the LICENSE file for more information on licensing.

Getting started with the Sonata board

This guide helps you to get started with the Sonata board by building code with CHERIoT technology enabled. One more advanced feature of Sonata is you can adjust the number and type of peripherals included. This is described in the section FPGA Development, for this getting started guide you will use one of the default setups.

The Sonata board is a prototype board and is under active development. This documentation is in the process of being updated. Some parts of the documentation may be out of date or otherwise confusing because of this. This getting started material and the documentation in the sonata-software repository provides the best material for new users to focus on.

Getting started steps

To get started with your Sonata board, there are three steps you'll need to do. First, head to the Sonata System Release Page where you'll find the latest releases.

  1. Program the RP2040 With the latest firmware to get any bug fixes by entering bootloader mode & dragging the rpi_rp2_v0.XX.uf2.
  2. Get the latest FPGA image that corresponds with the software you are building. This requires you to just drag the new sonata_vXX.bit.slot1.uf2 file into the 'SONATA' drive that comes up when you plug in the board to your computer.
  3. Build the example code and download to the soft-core you loaded in step 1.

Follow along each of the following sections to complete these tasks.

Reloading the RP2040 USB Controller

Before plugging in your Sonata board, hold down the SW9 labelled "RP2040 Boot", and while holding this button plug your board into your laptop using the Main USB.

A drive called RPI-RP2 should pop up on your computer and drag rpi_rp2_v0.X.uf2 into it. This drive should automatically dismount once the file is transferred and remount as SONATA.

Downloading the rpi_rp2_v.X.uf2 file

Currently the RP2040 firmware is available from the Sonata Systems releases, which ensures your RP2040 firmware matches the Sonata FPGA and firmware expectations.

The source & latest release for the RP2040 are also found on the Sonata RP2040 repo.

Reloading the FPGA Image

The first thing you should do before building the firmware is to get the latest version of the FPGA image, called the "bitstream". This contains the configuration for the microcontroller core & peripherals. The "release version" of the bitstream must match the configuration you use to build the software, as if the bitstream is a different version than what the software is expecting, you are not going to have fun!

When you download a release from the Sonata System page, you'll have a matching bitstream and software setup.

While you can build your own bitstream as described in FPGA Development, we recommend starting with our prebuilt bitstream first. Building the bitstream requires installing Vivado which takes a large amount of hard drive space and requires a separate manual installation process (as well as the build process is much slower than a software compile, so adds delay until you can play with CHERIoT).

Selecting a Bitstream

When the Sonata board is plugged in, it loads one of three bitstreams. This is selected by the switch below the USB port labeled Bitstream:

The LEDs besides the switch show the current image selected as well for confirmation.

The slot used by a bitstream is selected by the UF2 file. The bitstream UF2 provided in the release uses Slot 1.

Drag & Drop Programming

To program the Sonata bitstream:

  1. Download the bitstream from our releases
  2. Make sure that you have the bitstream switch (SW3) set to 1.
  3. Plug in your Sonata board using the main USB. You should see a 'SONATA' drive (see troubleshooting section if unsure).
  4. Copy the updated FPGA sonata-vX.Y.bit.slot1.uf2 file to the drive and wait for the copy to complete (on Linux note the copy command may return immediately, so you need to wait until it's done.)
  5. The board should automatically restart once the image is copied over. You should see the 'FPGA Config' LED come on:

This indicates the FPGA configuration succeeded. This LED should stay on. You should also see the Ibex Boot LED come on indicating the processor core has booted.

The FPGA Config LED reflects the state of the FPGA DONE pin. If this LED is not on your board will not work, as there is no logic (core) loaded, or it has become corrupted. This is true even if you are not building Sonata designs but using the board as a general-purpose FPGA board. See troubleshooting below if this LED does not come on, or appears to only come on briefly.

Programming on Power Cycle

Once the copy completes (it can take from 15-120 seconds), you should see the device reboot and the Ibex boot LED come on as mentioned. If you unplug & replug the USB cable, it will also reprogram the FPGA. The bitstream is stored on FLASH memory on the Sonata board.

Troubleshooting

Sonata Mass Storage Drive Issues

The Sonata board when plugged in should show a mass storage drive with these files:

LOG.TXT
OPTIONS.TXT
README.TXT

If the board has a RPI loader, visible because you'll see the file INFO_UF2.txt with the contents Model: Raspberry Pi RP2, you may need to reload the RP2040 as described in the Sonata Board Updating Firmware section. This could be because the RPI Boot button was held down when plugging in the board.

The Sonata board will print status and messages to the LOG.TXT which can be helpful for debugging. It should show the status of valid-looking bitstreams:

TEST CRC Test PASS
CRIT: FW_VER 0.4.0
INFO: Bitstream found in slot 0
INFO: Bitstream found in slot 1
INFO: Bitstream found in slot 2
INFO: Firmware found in slot 0
INFO: No firmware in slot 1
INFO: No firmware in slot 2
INFO: Using slot 0
INFO: Bitstream in flash @ 0, programming 18AC70 bytes...
INFO: Finished programming CRC=98E5AFD6
INFO: Bitstream prog success

FPGA Config LED not coming on

If the FPGA Config LED is not coming on, this could indicate the bitstream was designed for a different FPGA, or some other hardware issue. This should be troubleshooted with the OpenFPGALoader utility as described further down this page.

Ibex Boot LED not coming on

If the FPGA Config LED is on but the Ibex Boot LED is not, it may be that you have programmed (or selected) a different bitstream than one that runs the CHERIoT demo. Try reloading the bitstream, and try power cycling the device.

Device Rebooting During/After Programming (No Serial Activity)

The Sonata board takes a fair amount of power (>500mA) from the USB interface, and should be connected via USB-C. Typically it is close enough to the USB 2.0 limits that it will work with the adapter most of the time, but if you are having reliability issues we recommend trying a different computer, ideally one with a USB-C port.

If you use a recent openFPGALoader build from at least May 25, 2024 you can print the min/max VCCINT ranges. To do this, simply run:

openFPGALoader -c ft4232 -X

This will print several XADC values, pay careful attention to minvccint:

{"temp": 39.9061,
    "maxtemp": 40.3194,
    "mintemp": 25.9852,
"raw":  {"0": 40684, "1": 21949, "2": 39270, "3": 0, "4": 0, "5": 0, "6": 21948, "7": 0},
"vccint": 1.00415,
   "maxvccint": 1.00635,
   "minvccint": 1.00195,
"vccaux": 1.79663,
   "maxvccaux": 1.79883,
   "minvccaux": 1.79517,
}

The minvccint and maxvccint should be fairly close as shown here. Larger ranges indicate possible ringing, or a minvccint near or below 0.95V indicates a brown-out. You can compare the results of loading a different bitstream to see what is normal for your board.

See note below about if you get permissions error running openFPGALoader.

FPGA Programming via USB/JTAG

If for some reason the mass storage programming isn't working, you can also use the built-in FTDI JTAG programming. This requires the setup described in FPGA Programming to build openFPGALoader. Once built, you simply run:

openFPGALoader -c ft4232 sonata_top.bit

Note this requires the udev setup described in FPGA Programming. If you are lazy you can just run the command as root instead (not recommended, but can be helpful for troubleshooting on VMs):

sudo openFPGALoader -c ft4232 sonata_top.bit

You can also check if the flag --read-register STAT is available (newer than 0.12.2 is needed) which will print detailed information about the configuration status with a recent version of openFPGALoader. This is especially helpful if you are trying to understand why the DONE LED is not coming on:

CRC Error      No CRC error
Part Secured   0
MMCM lock      1
DCI match      1
EOS            0
GTS CFG B      0
GWE            0
GHIGH B        0
MODE           7
INIT Complete  1
INIT B         0
Release Done   0
Done           0
ID Error       ID error
DEC Error      0
XADC Over temp 0
STARTUP State  0
Reserved       0
BUS Width      x1
Reserved       8

Building Examples

The following contains some simple examples you can build for the Sonata board. Once you've got these builds working, you can easily add more features to the example code.

Using our template

Please go to the Sonata software repository using the branch appropriate to your release and build a full application from there. Inside your setup you should simply be able to build it like this:

git clone --recurse-submodule \
    https://github.com/lowRISC/sonata-software.git
cd sonata-software
nix develop .
xmake -P examples/

After running this you should see the build run to completion and report success, the critical lines indicating a successful build are (note output size may differ):

Converted to uf2, output size: 74752, start address: 0x101000
Wrote 74752 bytes to build/cheriot/cheriot/release/sonata_simple_demo.uf2
[100%]: build ok, spent 6.827s

You can drag and drop this UF2 file into the SONATA drive to program the firmware.

UART output

On Linux use the following command to check you can receive serial output:

screen /dev/ttyUSB2 921600

On Mac this is similar

screen /dev/tty.usbserial-LN100302 921600

On Windows, connecting to serial ports directly from within WSL2 (default) is not possible. Connecting from WSL1 is possible, but we recommend to use PuTTY to connect to serial ports. Alternatively you can use Termite.

Select "Serial" as "Connection type", put the COM port in the "Serial line" text field, and set "Speed" to 921600. To find out what serial ports are available, you can open Device Manager and all connected serial ports are listed under "Ports (COM & LPT)" section.

Baremetal examples

This is only for advanced users. If you want to build the baremetal examples in the Sonata repo you can follow these instructions.

First setup a toolchain. Note none of the current Nix environments have exactly the correct set of dependencies to build the baremetal examples. This will change but for the time being you either need to alter a Nix environment (either add cmake to the sonata-software environment or the CHERIoT toolchain to the sonata-system environment) or setup a toolchain outside Nix.

Additional Toolchain Setup

Besides the compiler, there are a few more features the example code depends on.

SRecord Tools

The build environment uses srecord tools, which you can install with:

sudo apt install srecord

CHERIoT LLVM

IMPORTANT: Set this environmental variable to the llvm build you did in the previous part:

export CHERIOT_LLVM_BIN=/path/to/cheriot-llvm/bin

WARNING: The path to /path/to/cheriot-llvm/bin should point to the build directory you created, not just the root checked out cheriot-llvm directory. The path will look something like: ~/llvm-tools/cheriot-llvm/builds/cheriot-llvm/bin

Please make sure you unset the CHERIOT_RTOS_SDK environment variable as cmake will automatically pull in the correct version.

The following assumes you have run the source .venv/bin/activate command in the terminal you are using, and you are currently at the root directory of your local sonata-system git repository.

Building Baremetal Examples

To build, run the following from the root of the directory which will build the examples:

cmake -B sw/cheri/build -S sw/cheri 
cmake --build sw/cheri/build

The build output is put in the sw/cheri/build directory. Two files of interest are created for each target: an ELF file which has no extension and a *.vmem file. The *.vmem file can be used to load directly into the FPGA bitstream, described in more detail on the Programming the Sonata Software page.

If you get an error that CMake will not be able to correctly generate this project., check back in the list to see if you see an error within the output similar to clang: error: unknown argument: '-mxcheri-rvc'. If this happens it means the wrong (non-CHERIoT) compiler was used. Check back to see what compiler is being used.

Loading software onto the FPGA

You can load software onto the FPGA over USB (JTAG) using:

./util/mem_helper.sh load_program -e sw/cheri/build/tests/spi_test

There are actually four different ways of loading the program - we normally use JTAG for development, but you can also program it into the serial flash device on the board. See the page Programming the Sonata Software.

Developer Guide

The Sonata architecture comprises a number of components:

  • Physical board architecture: This is the board that hosts the FPGA (field programmable gate array), all other components, headers and interfaces.
  • FPGA configuration architecture: It defines everything we are using a hardware description language for. This includes hardware IP blocks and bus architecture.
  • Software architecture: This includes toolchain, operating system and applications.

Before we go into the architecture of the system, it is good to understand the use cases that we are envisioning, so that we can derive our architectural requirements from that.

Use cases and requirements

The Sonata is meant to be used by academics and industry users who are interested in experimenting with CHERIoT in embedded and IoT applications. This is the main reason why we are building a custom FPGA board so that we can make the platform easily usable. We also need the board to be as affordable as possible while still being performant and usable.

This ease of use comes in handy for classroom and demonstration use cases, for which we think this board will come in quite handy. In the classroom, it is also pertinent that we have a debuggable system.

Because we are focussing on embedded and IoT applications, we need to ensure connectivity. This includes being connectable to standard peripherals as well as being extendable with functionality required for niche use-cases.

The other major benefit of creating a custom board is that we can highlight CHERIoT specific features. We envision users of this board to want to show off and experiment with the new CHERIoT technology. If they are using the board as a demonstrator they will want to show off the prowess of CHERI, and if they are experimenting with the board they will want CHERI to be visible.

Like most physical boards, it is nice for it to be as interactive as possible. This means being able to accept user input and respond to them visibly and in an interactable way.

Beyond the physical Sonata board, we expect the soft design that is used to configure the FPGA to also be integrable into a bigger design. For example, we envision it being connected to OpenTitan Earl Grey through a bridge interface.

In short, these are our general requirements:

  • Usable
  • Affordable
  • Debuggable
  • Connectable
  • Extendable
  • CHERI visible
  • Interactive
  • Integrable

Detailed specifications

Toolchain

The toolchain will build on top of the work already done on CHERIoT, which uses a fork of LLVM. Through Sonata, we are not proposing any changes to the CHERIoT instruction set. We may need some changes to allow code to be stored in memory that does not have associated tags.

The toolchain for software development is described in the Software toolchain setup section. If these instructions ever go out of date, you should be able to find the up to date instructions to build the toolchain from the CI YAML file.

Applications

Especially for the usable and CHERI-visible requirements, it is important that we have a set of demonstration applications. One demonstration application is cycling through each of the CHERIoT exception types with code snippets showing what went wrong. We can show what happens when CHERI is enabled and when it is disabled. We will provide at least some of these applications in bare-metal mode where they do not need an operating system.

Operating system

The CHERIoT RTOS work will need some reworking in terms of memory layout and drivers to work on the Sonata system.

FPGA (Soft-Core) architecture

The toolchain for FPGA (hardware/gateware) development is described in the FPGA Development section.

PCB architecture

This is described in the Board documentation.

Software Programming

There are four ways of programming the software:

  1. You can use the flash storage on the Sonata board. This does not require special tools and allows an image to come online automatically at boot.

  2. You can use OpenOCD to program the image onto RAM and then run this image. This is typically used during development.

  3. You can use the CHERIoT serial bootloader. This loads the image into RAM on the CHERIoT system and then runs the image.

  4. You can 'splice' the software into the FPGA image. This provides a single 'bitstream' including both hardware definition and software. This can be helpful for making system images that come alive as soon as possible after boot.

Flash Programming

OpenOCD/JTAG Programming

Serial Bootloader

Will this still be supported?

Splicing into FPGA Image

Setting up the Toolchain for Software Development

This is only for advanced users who wish to build their own toolchain.

All the special CHERIoT goodness comes with its own compiler that understands how to use it. For this reason you'll need to build a special toolchain from source. Luckily, it should be easy if you follow our simple instructions.

If building on Windows, the following instructions have also been confirmed to work with WSL2 with the exception of edalize and fusesoc, which are not required for software development.

Sonata Setup

git clone https://github.com/lowRISC/sonata-system
cd sonata-system

# Setup python venv
python3 -m venv .venv
source .venv/bin/activate

# Install python requirements
pip3 install -r python-requirements.txt

This installs requirements for both software and FPGA development. You may get errors on edalize and fusesoc -- don't panic, you don't need those for software. Just ignore the errors, you should see the rest of the packages installed successfully.

In the future, if you dare close this terminal, you'll need to do this before building Sonata examples:

cd sonata-system
source .venv/bin/activate

Building Toolchain

To build the toolchain, you'll need:

  • clang
  • ninja-build
  • lld (llvm linker)
  • cmake

On Ubuntu (including WSL), you can install them with:

sudo apt get install clang ninja-build lld cmake

HINT: You can see all the commands used to setup the test running in the CI YAML file. This provides a set of commands that is tested on each commit, in case you are having trouble building anything and want to see the expected output.

Build the toolchain with (again be sure this is in the .venv terminal):

git clone --depth 1 https://github.com/CHERIoT-Platform/llvm-project cheriot-llvm
cd cheriot-llvm
git checkout cheriot
# Create build directory
export LLVM_PATH=$(pwd)
mkdir -p builds/cheriot-llvm
cd builds/cheriot-llvm
# Build the toolchain
cmake ${LLVM_PATH}/llvm -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;lld" -DCMAKE_INSTALL_PREFIX=install -DLLVM_ENABLE_UNWIND_TABLES=NO -DLLVM_TARGETS_TO_BUILD=RISCV -DLLVM_DISTRIBUTION_COMPONENTS="clang;clangd;lld;llvm-objdump;llvm-objcopy" -G Ninja
export NINJA_STATUS='%p [%f:%s/%t] %o/s, %es '
ninja install-distribution

NOTE: Currently the WSL2 build is broken and requires a patch applied as follows:

wget https://github.com/llvm/llvm-project/commit/4ad9ec8a328ccb3b836c993bba954366f05b2fd4.patch
git am < 4ad9ec8a328ccb3b836c993bba954366f05b2fd4.patch

Note the checkout and build will take some time, and the build process may have limited output during some steps.

This should put the binaries in a bin subdirectory of your build folder (the builds/cheriot-llvm folder you made).

With the CHERIoT/LLVM toolchain built, you can now continue to setup the examples and build them.

FPGA development

This page is only for if you want to make changes to the RTL of the bitstream. In most cases you can just use the standard bitstream published in the releases.

Dependencies

FPGA Build

The Sonata bitstream is generated using Vivado.

Bitstream

To build the bitstream, make sure to build the baremetal software to create the correct SRAM image. Then run this command:

fusesoc --cores-root=. run --target=synth --setup --build lowrisc:sonata:system

You can also manually set the initial value of the SRAM, for example:

fusesoc --cores-root=. run --target=synth --setup --build lowrisc:sonata:system --SRAMInitFile=$PWD/sw/cheri/build/tests/uart_check.vmem

FPGA Programming

Drag & Drop Programming

The easiest way to program the FPGA is to use the built-in programming feature. This reads a .bit file you drag onto the USB drive that comes up when the USB is plugged in, and will program it into the FPGA. This will also save it to SPI flash in one of the three "Slots", and will automatically reprogram on board power-on.

Using this drag & drop programming is the suggested way of updating the board with "normal" Sonata core images. For development it's suggested to use the JTAG programmer, as it will be much faster and avoids lots of writes to the SPI flash.

JTAG Programming Using On-Board FTDI Chip

The Sonata board includes a FTDI chip that can program the FPGA. This also allows usage inside of Vivado to access features such as Integrated Logic Anlayzer (ILA) blocks.

On the backside of the board, confirm switches SW1 are all set to ON as shown in this photo (this was the state the board is shipped in, so if you haven't touched the switches it should still be in that state):

The switches in the "off" state will isolate the FTDI and allow you to use an external JTAG probe (such as Xilinx Platform Cable USB II).

USB rules for Linux

To allow openFPGAloader to program our device, add the following rules:

sudo su
cat <<EOF > /etc/udev/rules.d/99-openfpgaloader.rules
# Copy this file to /etc/udev/rules.d/

ACTION!="add|change", GOTO="openfpgaloader_rules_end"

# gpiochip subsystem
SUBSYSTEM=="gpio", MODE="0664", GROUP="plugdev", TAG+="uaccess"

SUBSYSTEM!="usb|tty|hidraw", GOTO="openfpgaloader_rules_end"

# Original FT232/FT245 VID:PID
ATTRS{idVendor}=="0403", ATTRS{idProduct}=="6001", MODE="664", GROUP="plugdev", TAG+="uaccess"

# Original FT2232 VID:PID
ATTRS{idVendor}=="0403", ATTRS{idProduct}=="6010", MODE="664", GROUP="plugdev", TAG+="uaccess"

# Original FT4232 VID:PID
ATTRS{idVendor}=="0403", ATTRS{idProduct}=="6011", MODE="664", GROUP="plugdev", TAG+="uaccess"

# Original FT232H VID:PID
ATTRS{idVendor}=="0403", ATTRS{idProduct}=="6014", MODE="664", GROUP="plugdev", TAG+="uaccess"

# Original FT231X VID:PID
ATTRS{idVendor}=="0403", ATTRS{idProduct}=="6015", MODE="664", GROUP="plugdev", TAG+="uaccess"

# anlogic cable
ATTRS{idVendor}=="0547", ATTRS{idProduct}=="1002", MODE="664", GROUP="plugdev", TAG+="uaccess"

# altera usb-blaster
ATTRS{idVendor}=="09fb", ATTRS{idProduct}=="6001", MODE="664", GROUP="plugdev", TAG+="uaccess"
ATTRS{idVendor}=="09fb", ATTRS{idProduct}=="6002", MODE="664", GROUP="plugdev", TAG+="uaccess"
ATTRS{idVendor}=="09fb", ATTRS{idProduct}=="6003", MODE="664", GROUP="plugdev", TAG+="uaccess"

# altera usb-blasterII - uninitialized
ATTRS{idVendor}=="09fb", ATTRS{idProduct}=="6810", MODE="664", GROUP="plugdev", TAG+="uaccess"
# altera usb-blasterII - initialized
ATTRS{idVendor}=="09fb", ATTRS{idProduct}=="6010", MODE="664", GROUP="plugdev", TAG+="uaccess"

# dirtyJTAG
ATTRS{idVendor}=="1209", ATTRS{idProduct}=="c0ca", MODE="664", GROUP="plugdev", TAG+="uaccess"

# Jlink
ATTRS{idVendor}=="1366", ATTRS{idProduct}=="0105", MODE="664", GROUP="plugdev", TAG+="uaccess"

# NXP LPC-Link2
ATTRS{idVendor}=="1fc9", ATTRS{idProduct}=="0090", MODE="664", GROUP="plugdev", TAG+="uaccess"

# NXP ARM mbed
ATTRS{idVendor}=="0d28", ATTRS{idProduct}=="0204", MODE="664", GROUP="plugdev", TAG+="uaccess"

# icebreaker bitsy
ATTRS{idVendor}=="1d50", ATTRS{idProduct}=="6146", MODE="664", GROUP="plugdev", TAG+="uaccess"

# orbtrace-mini dfu
ATTRS{idVendor}=="1209", ATTRS{idProduct}=="3442", MODE="664", GROUP="plugdev", TAG+="uaccess"

LABEL="openfpgaloader_rules_end"

EOF

exit

Run the following to reload the rules:

sudo udevadm control --reload-rules
sudo udevadm trigger

Add user to plugdev group:

sudo usermod -a $USER -G plugdev

Programming with openFPGALoader (Linux/MacOS)

Programming the FPGA:

openFPGALoader -c ft4232 build/lowrisc_sonata_system_0/synth-vivado/lowrisc_sonata_system_0.bit

You can also check if the flag --read-register STAT is available (newer than 0.12.2 is needed) which will print detailed information about the configuration status with a recent version of openFPGALoader. This is especially helpful if you are trying to understand why the DONE LED is not coming on:

$ openFPGALoader -c ft4232 build/lowrisc_sonata_system_0/synth-vivado/lowrisc_sonata_system_0.bit --read-register STAT

CRC Error      No CRC error
Part Secured   0
MMCM lock      1
DCI match      1
EOS            0
GTS CFG B      0
GWE            0
GHIGH B        0
MODE           7
INIT Complete  1
INIT B         0
Release Done   0
Done           0
ID Error       ID error
DEC Error      0
XADC Over temp 0
STARTUP State  0
Reserved       0
BUS Width      x1
Reserved       8

In this example there is an ID error that means the wrong bitstream was used (e.g., built for an A35, A75, or A100 and not for the A50).

Vivado (Windows)

The FTDI on the board is setup to work with Vivado, and should be detected on a recent version of Vivado (tested with 2023.2), older versions will not work. It also doesn't seem to currently work on Linux, although it "should" be supported according to Xilinx.

Reloading the FTDI Chip

The FT4232H on the board needs to be programmed by Vivado to work within Vivado. This is done from the TCL Console within Vivado.

With a blank FTDI chip, you would run the command:

program_ftdi -write -ftdi=FT4232H -serial LNXXXX -board "Sonata"

Where LNXXXX is the serial number on the sticker of the board (the serial number is optional, but with a serial number set the related COM ports will always come up on Windows with the same COM port number, which can be helpful).

This will fail if the FTDI chip was programmed before. If you need to erase an FTDI chip, you would run the command:

program_ftdi -erase

The program_ftdi command can also be run from the system command prompt with the Vivado path setup.

JTAG Programming Using External Probe

If using an external probe, you need to connect to P10, labeled FPGA JTAG. This is normally done with flying wire leads.

Be sure you set all of the 4-position DIP switches within SW1 to OFF on the bottom of the board, otherwise your external JTAG probe will be fighting with the FTDI lines. This will cause unreliable operation.

Simulation

The Sonata simulation environment uses Verilator.

Building

Use the following command to build the simulator binary. The resulting executable will be available at ./result/bin/sonata-simulator.

nix build .#sonata-simulator

If you'd like more control over the build, you can manually invoke FuseSoC with the following. The resulting executable can be found at ./build/lowrisc_sonata_system_0/sim-verilator/Vtop_verilator.

NUM_CORES=4
fusesoc --cores-root=. run \
  --target=sim --tool=verilator --setup \
  --build lowrisc:sonata:system \
  --verilator_options="-j $NUM_CORES" \
  --make_options="-j $NUM_CORES"

To enable tracing append, --verilator_options='+define+RVFI' to the command above.

Running baremetal

Running the simulator can be accomplished with the following command, where you can change the meminit argument to a different program if you wish:

./build/lowrisc_sonata_system_0/sim-verilator/Vtop_verilator -t --meminit=ram,./sw/cheri/cheri_sanity/boot.elf

I recommend that you make the following change to the sanity check to see quicker changes in simulation:

diff --git a/sw/cheri/cheri_sanity/boot.cc b/sw/cheri/cheri_sanity/boot.cc
index 547abb3..7f7781d 100644
--- a/sw/cheri/cheri_sanity/boot.cc
+++ b/sw/cheri/cheri_sanity/boot.cc
@@ -32,7 +32,7 @@ extern "C" uint32_t rom_loader_entry(void *rwRoot)
        uint32_t switchValue = 0;
        while (true) {
                gpioValue ^= GPIO_VALUE;
-               for (int i = 0; i < 5000000; i++) {
+               for (int i = 0; i < 5; i++) {
                        switchValue = *((volatile uint32_t *) gpi);
                        switchValue <<= 4; // shift input onto LEDs and skipping LCD pins
                        *((volatile uint32_t *) gpo) = gpioValue ^ switchValue;

Running with boot stub

The Sonata software repository assumes you have a boot loader that jumps to 0x00101000. To help with this we have a sim_boot_stub, which you can build as follows assuming you have access to CHERIoT LLVM:

make -C sw/cheri/sim_boot_stub

You can then run your example code through the simulator as follows:

./build/lowrisc_sonata_system_0/sim-verilator/Vtop_verilator -t -E sw/cheri/sim_boot_stub/sim_boot_stub -E /path/to/sonata-software/build/cheriot/cheriot/release/sonata_simple_demo

Debugging

If you want to look at the internal design in more details, you can explore the waveforms produced by the simulation using GTKWave:

gtkwave sim.fst data/pc_and_gpo.gtkw

Building Documentation

The documentation uses mdBook see the installation guide for further details on installation.

Once mdBook is installed the documentation can be built and viewed with:

mdbook serve --open
# Avoid FuseSoC using copied files in the book directory
touch book/FUSESOC_IGNORE

The second line can be ignored if you aren't building the FPGA bitstream (which uses fusesoc).

Windows Quick-Start

On Windows the easiest installation method is to copy the precompiled mdbook.exe available as a release on the previous link to the sonata-system root directory.

./mdbook.exe serve --open

Reference Manual for Sonata Core

In a classical microcontroller, you would have a core along with the peripherals around the core. On the Sonata system this is all part of an open-source FPGA design, which allows you to modify the core to add new features (and for us to add updates to your core without needing you to desolder your main IC!).

This also means you can customize your design. You may want to have a different number of UARTs or SPI blocks for example. This document describes the base configuration.

The FPGA image is parameterizable to enable custom setups. It should be easy, for example, to change the number of UART, SPI and I2C instances. We will provide pre-built images for common configurations.

Interoperate

For the interoperable requirement, we need to make sure our hardware design can interact with that of OpenTitan Earl Grey. Since OpenTitan Earl Grey uses a TileLink Uncached Lightweight (TL-UL) bus, we use the same in the Sonata system to ease designing a bridge interface.

Hardware IP blocks

To support all the peripherals that are on the FPGA boards, we need corresponding hardware IP blocks for Ibex to be able to interact with them:

  • I2C for QWIIC
  • SPI for the LCD screen and ethernet
  • GPIO for buttons and LEDs
  • HyperRAM controller

There might be other IP blocks necessary for interacting with headers such as an analogue to digital converter. We also need some modifications to CHERIoT Ibex, which are detailed in its own page.

Wherever possible, we reuse existing, high-quality, open-source hardware IP blocks that are fit for purpose.

Memory layout

For all registers in this section, the functionality is mapped onto the least significant bits of registers and each register is 32 bits wide.

Base addressSizeFunctionality
0x0010_0000256 KiBInternal SRAM
0x3000_00004 KiBRevocation tags
0x4000_00001 MiBTagged RAM
0x4010_00007 MiBUntagged RAM
0x8000_00004 KiBGPIO
0x8000_10004 KiBPWM
0x8000_20004 KiBReserved for DMA
0x8000_30004 KiBHyperRAM
0x8000_40004 KiBReserved
0x8000_50004 KiBPinmux
0x8000_60004 KiBR-Pi Header GPIO
0x8000_70004 KiBArduino GPIO
0x8000_80004 KiBPMOD GPIO
0x8000_90004 KiBRGB LED controller
0x8000_A0004 KiBHardware revoker
0x8000_B0004 KiBADC
0x8004_000064 KiBTimer
0x8010_00001 MiBUART
0x8020_00001 MiBI2C host
0x8030_00001 MiBSPI host
0x8040_00001 MiBUSB device
0x8800_000064 MiBPLIC
0xB000_00004 KiBDebug module

Clocking infrastructure

The whole system is driven by the same clock with the exception of the HyperRAM controller. Optionally the HyperRAM controller can be clocked higher than the rest of the chip. To accommodate this, we introduce a synchronization interface with primitive FIFOs.

Memory architecture

We have a few different types of memory in the Sonata system: FPGA SRAM, HyperRAM and flash. With CHERI we need to think about capability tags and revocation tags. Any memory that needs to contain capabilities must have one capability tag per 32 bits. Any memory that needs to be revocable must have one revocation tag per 32 bits.

Capability tags

All capability tags live in SRAM. All SRAM that is allocated for code and data will have corresponding capability tags. Any data stored to HyperRAM and flash are not expected to be tagged. Since capability tags are out of band information and do not need to be memory mapped, we can store these within the error correction bits that are available on the FPGA's memory.

We envision that code can live in HyperRAM with an instruction cache for improved performance. However, this does require code to be able to live in untagged memory. This should be fine as CHERI capabilities are derived and manipulated at runtime, but does require toolchain changes to LLVM and the corresponding RTOS.

Revocation tags

Revocation tags are essential in providing temporal memory safety in CHERI. This only covers a subset of memory that is likely to be used by the heap. Setting the revocation bit effectively stops any capability with that base address from being loaded from memory. This is a temporary step as the revocation engine scans through memory to invalidate all capabilities to this address. Once the complete memory is scanned, the revocation bit can be unset and the memory can be reused.

In Sonata, the revocation tags only cover a subset of mapped memory. They should apply to memory regions that are most likely to be used as heap, it is likely this will cover all of internal SRAM and some of HyperRAM. Unlike capability tags, revocation tags need to be memory mapped so the memory allocator can manipulate them.

In CHERIoT Ibex the size of memory allocated for this is defined by TSMapSize which indicates how many 32-bit words can be used for revocation bits. The default value for this is 1,024, which corresponds to 8 KiB. In CHERIoT Safe the size of the revocation tag memory is 16 KiB.

List of SRAM blocks

Here's a list of blocks by size that we need to allocate in SRAM. The XC7A35T has 100 blocks of 18 kilobit block RAM, see datasheet. In total that gives use 225 KiB of block RAM, but we may not efficiently map onto 18 kilobit blocks and thus lose some memory. The block RAM usage in the table below was calculated using Vivado 2023.2's block memory generator.

TypeSizeWidthDepthRAM Blocks
Internal memory128 KiB3332,76860
Revocation tags16 KiB324,0968
RAM capability tags32 KiB328,19215
Instruction cache data4 KiB645122
Instruction cache tags1 KiB225121
Total181 KiB86
Available225 KiB100

Ibex

For details on how Ibex works please look at the vendored in documentation. This page highlights the changes that we made to Ibex for the Sonata system.

The Sonata board has a CHERI enabled LED which is hard wired to the value of cheri_pmode_i of the Ibex core module.

Capability exception LEDs

In the CHERIoT specification there are number of capability exception codes. The Sonata board has dedicated LEDs to indicate any of these errors. When one of these exceptions is seen, it gets latched and displayed on the LED. The LEDs do not clear once the exception is handled, instead software needs to clear these LEDs through a special CSR. Custom M-mode CSR 0xBC0 is used for this purpose. If bit 0 is set to 1 then hardware no longer drives the LEDs and software is in full control over the output. The other bits positions correspond to the value of the exception code.

Bit offsetDescription
24Permit access system registers violation
22Permit store local capability violation
21Permit store capability violation
19Permit store violation
18Permit load violation
17Permit execute violation
3Seal violation
2Tag violation
1Bounds violation
0Disable

GPIO

General purpose input and output is used for Ibex to interact with the buttons and switches on the board. It is also used to drive the LEDs on the board. There are also the GPIO pins of the various headers.

For the input this module provides a raw value as well as a debounced value. Debouncing is useful to avoid counting a single button press multiple times. For more information on contact bounce, see this Wikipedia page.

OffsetRegister
0x0000Output
0x0004Input
0x0008Debounced input
0x000CDebounce threshold
0x6000R-Pi output
0x6004R-Pi input
0x6008R-pi debounced input
0x600CR-pi debounce threshold
0x7000Arduino output
0x7004Arduino input
0x7008Arduino debounced input
0x700CArduino debounce threshold
0x8000PMOD output
0x8004PMOD input
0x8008PMOD debounced input
0x800CPMOD debounce threshold

Output

The output register displays the specified value onto the boards output.

Bit offsetDescription
22mikroBUS reset
21mikroBUS SPI chip select
20Arduino SPI chip select
19R-Pi SPI1 chip select 0
18R-Pi SPI1 chip select 1
17R-Pi SPI1 chip select 2
16R-Pi SPI0 chip select 0
15R-Pi SPI0 chip select 1
14Ethernet reset
13Ethernet SPI chip select
12Flash SPI chip select
11-4LEDs
3LCD backlight
2LCD DC
1LCD reset
0LCD chip select

In this case writing a one will turn an LED on and a zero will turn the LED off.

Input

Both input registers have the same bit mapping. The only difference between the registers is that the latter has debounced signals.

Bit offsetDescription
16-14Software select switches (1, 2, 3)
13mikroBUS interrupt
12-5DIP switches
4-0Joystick (left, down, up, right, press)

The input registers are used to interact with the joystick, the button and the DIP switches that are available on the Sonata board.

Debounce threshold

Not implemented yet.

This register indicates how many clock cycles an input needs to be stable before it shows up on the output. The same threshold applies to all of the inputs.

Bit offsetDescription
31-0Threshold

Raspberry Pi HAT

The Raspberry Pi HAT header has 11 GPIO pins. When writing, it only writes the bits for which the output is set to enable. The input and output registers have the same bit mapping.

Bit offsetDescription
10GPIO 27
9GPIO 26
8GPIO 25
7GPIO 24
6GPIO 23
5GPIO 22
4GPIO 13
3GPIO 12
2GPIO 6
1GPIO 5
0GPIO 4

For the output register it is teh same order as above:

Bit offsetDescription
26-16Output enable
10-0Output value

When the output enable is set to zero it instead acts as an input pin.

Arduino Shield

The Arduino Shield header has 10 GPIO pins. When writing, it only writes the bits for which the output is set to enable. The input and output registers have the same bit mapping.

Bit offsetDescription
9GPIO 9
8GPIO 8
7GPIO 7
6GPIO 6
5GPIO 5
4GPIO 4
3GPIO 3
2GPIO 2
1GPIO 1
0GPIO 0

For the output register it is teh same order as above:

Bit offsetDescription
25-16Output enable
9-0Output value

When the output enable is set to zero it instead acts as an input pin.

PMOD

The two PMOD headers has 16 GPIO pins. When writing, it only writes the bits for which the output is set to enable. The input and output registers have the same bit mapping.

Bit offsetDescription
15-8PMOD 1
7-0PMOD 0

For the output register it is teh same order as above:

Bit offsetDescription
31-16Output enable
15-0Output value

When the output enable is set to zero it instead acts as an input pin.

Pulse width modulation (PWM)

Pulse width modulation allows you to create a block wave with a certain duty cycle. It is useful for use cases like dimming LEDs.

OffsetRegister
0x00Enable
0x10Config 0
0x14Config 1
0x18Config 2
0x1CConfig 3
0x20Config 4
0x24Config 5
0x28Config 6
0x2CConfig 7

Enable

This has a bit per PWM on whether to enable it or not.

Bit offsetDescription
7Enable for PWM 7
......
1Enable for PWM 1
0Enable for PWM 0

Config

For each output, there is a register defining the pulse width and how long the complete wave is (indicated by the counter value).

Bit offsetDescription
31-16Counter
15-0Pulse width

HyperRAM controller

HyperRAM is used as an alternative to flash. Compared to flash, HyperRAM has similar performance but it avoids the need for a quad-speed SPI controller to interact with the flash. The HyperRAM controller is the interface between the Sonata system and the actual chip.

OffsetRegister
0x00Configuration 0
0x04Configuration 1

For details of what these configuration registers do please consult Section 9.4 and 9.5 of the datasheet.

Because the latency of accessing data memory through the HyperRAM will be quite slow, we introduce a fully-associated cache of a few words to improve performance. It is anticipated main data storage will be in SRAM with the HyperRAM storing small amounts of data interleaved with code so more significant caching is unnecessary.

Capability enabled RAM

The HyperRAM controller is also where the capability tags live for the tagged part of the RAM. In Sonata we allocate 32 KiB for capability tags for RAM which translates in to 1 MiB for tagged RAM.

Analogue to digital converter (ADC)

The FPGA provides an analogue to digital converter called XADC, and this hardware IP block provides a memory mapped way of interacting with that. Please see the specification for XADC for a more detailed description of how it works.

Analogue Input Channels

The six Arduino shield analogue input pins (A0-A5) are connected to six of the XADC Vaux channels.

PinChannel
A0Vaux[4]
A1Vaux[12]
A2Vaux[5]
A3Vaux[13]
A4Vaux[6]
A5Vaux[14]

The input range at the shield pin is 0-5V. Conversion from a single-ended 0-5V signal at the shield pin to a 0-1V differential signal at the XADC inputs is handled by on-board buffering and balancing circuit. Note that impedance drops above 3.3V due to diode clamps on a resistor-gated branch that also connects each pin to a digital FPGA input (currently unused).

Sensors

A set of on-chip sensors can also be read by the XADC. The most interesting of these is a temperature sensor. The rest are various supply/reference voltages.

Registers

The XADC is accessed via its Dynamic Reconfiguration Port (DRP), which give access to various 16-bit control and status registers. These registers have been memory mapped at a 32-bit intervals to simplify accesses.

DRP AddressRegister
0x00 0x01 .. 0x3FStatus
0x40 0x41 .. 0x7FControl
Memory-Mapped OffsetRegister
0x000 0x004 .. 0x0FCStatus
0x100 0x104 .. 0x1FCControl

By default, the XADC will continuously sample all six routed analogue inputs and all on-chip sensors at a low sample rate, with conversion results ready to be read by the host from the status registers. This default configuration is achieved by specifying initial values for the configuration registers in the RTL. The sample rate or other configuration can be changed by writing to the control registers.

For a more detailed description of all of these registers please see Chapter 3 of the XADC specification.

Pin multiplexer

This allows software to dynamically switch FPGA pins between input and output as well as reassign them for SPI, I2C, UART, etc. The block also allows pad control. An example use-case is changing whether an LED is driven by the GPIO or by the PWM.

We will use the pin multiplexer from OpenTitan, but then parameterized for Sonata. More details on how this changes the register map will be provided in the future.

In general, we expect each pin can be connected to a limited subset of blocks. The output of GPIO and PWM should only be able to control one pin, while blocks like UART, SPI and I2C should have multiple pins that they can connect to. The exact mapping of which pin can be driven by which hardware IP block will be provided in the future.

CHERIoT hardware revocation engine

This block is integrated from the CHERIoT Safe repository. It controls the hardware revoker inside CHERIoT Ibex which is the thing that does the revocation, it is looking for capabilities to freed memory. When a piece of memory is freed its revocation bits are set. The hardware revoker sweeps from the start until the end address for capabilities that point to any revoked region. For more information on how this works please refer to Section 3.3.3 (background pipelined revoker) of this paper.

These are the register offsets:

OffsetRegister
0x0000Start address to begin sweeping.
0x0004End address to stop sweeping.
0x0008Most significatn byte is set to constant 0x55. least significant bit is asserted for one cycle when sweeping starts.
0x000CEpoch, where the last bit means a sweep is currently happening.
0x0010Read interrupt status, but is only high for one cycle.
0x0014Enable bit for raising an interrupt when sweeping is done.
0x0040Debug: FIFO empty and read data values.
0x0044Debug FIFO full, empty and depth values.

Timer

The timer that we use in Sonata is based on the timer present in the Ibex repository. It is a simple memory mapped timer that sends an interrupt to Ibex after a specified amount of time. The time values in this block are 64 bits, which is why it has a high and a low register for each value.

OffsetRegister
0x00Time low
0x04Time high
0x08Time compare low
0x0CTime compare high

The processor can set the time by writing to the time low and high registers and then set a time compare value to compare to. Internally, the block increments the time by one each clock cycle. Once the internal counter has exceeded the compare value, it raises an interrupt with the Ibex.

Universal asynchronous receiver/transmitter (UART)

The Sonata system uses the OpenTitan UART. You can find the register definitions here.

There are multiple UART instances in Sonata to connect to any of the following targets:

  • USB
  • RS-232
  • mikroBUS
  • Arduino shield
  • Raspberry Pi hat

By default, Sonata includes 3 UART blocks, each with an offset of 0x1000 from each other.

Inter-integrated circuit (I2C) host

For the I2C block in Sonata, we use the OpenTitan IP. We hardcode this block to be in host mode, so you can ignore the target functionality, including register ovrd, val, target_id, acqdata and txdata. Other than those you can find the register definitions here. The registers 0x00 - 0x10 are also not accessible. The control register is just hardwired to be in host mode.

For Sonata, we include two I2C blocks. The registers of the second I2C block can be accessed with and additional 0x1000 offset. These can be connected to any of these I2C targets:

  • Two for the QWIIC connectors.
  • One for the mikroBUS.
  • One for the Raspberry Pi hat.
  • One for the Arduino shield.

Serial peripheral interface (SPI) host

The SPI in Sonata only has the capability to be a host. This is a simple hardware IP block that can transmit and receive bytes over SPI.

In Sonata, there are multiple uses for SPI:

  • LCD screen
  • Ethernet
  • Flash
  • Raspberry Pi hat
  • Arduino shield
  • mikroBUS

The current Sonata system configuration has two SPI instantiations, one for the LCD screen and one for Flash. The Sonata top-level will need modification to add more SPI blocks for other uses.

The offset for each of the blocks is shown below, with each additional block having a 0x1000 offset from the previous.

SPI InstanceOffset (from SPI base)
Flash0x0
LCD Screen0x1000
Ethernet MAC0x2000
Raspberry Pi HAT SPI00x3000
Raspberry Pi HAT SPI10x4000
Arduino Shield0x5000
mikroBUS Click0x6000

Overview

Each SPI block has a two 64 entry FIFOs one for transmit and one for receive. To begin an SPI transaction write to the START register. Bytes do not need to be immediately available in the transmit FIFO nor space available in the receive FIFO to begin the transaction. The SPI block will only run the clock when its able to proceed. Note that the CS pin is not handled by the SPI block and must be dealt with via GPIO and controlled with software.

Note Interrupts are not yet implemented

Register Table

NameOffsetLengthDescription
spi.INTR_STATE0x04Interrupt State Register
spi.INTR_ENABLE0x44Interrupt Enable Register
spi.INTR_TEST0x84Interrupt Test Register
spi.CFG0xc4Configuration register. Controls how the SPI block transmits
spi.CONTROL0x104Controls the operation of the SPI block. This register can
spi.STATUS0x144Status information about the SPI block
spi.START0x184When written begins an SPI operation. Writes are ignored when the
spi.RX_FIFO0x1c4Data from the receive FIFO. When read the data is popped from the
spi.TX_FIFO0x204Bytes written here are pushed to the transmit FIFO. If the FIFO

INTR_STATE

Interrupt State Register

  • Offset: 0x0
  • Reset default: 0x0
  • Reset mask: 0x1f

Fields

[{"name": "rx_full", "bits": 1, "attr": ["ro"], "rotate": -90}, {"name": "rx_watermark", "bits": 1, "attr": ["ro"], "rotate": -90}, {"name": "tx_empty", "bits": 1, "attr": ["ro"], "rotate": -90}, {"name": "tx_watermark", "bits": 1, "attr": ["ro"], "rotate": -90}, {"name": "complete", "bits": 1, "attr": ["rw1c"], "rotate": -90}, {"bits": 27}]
BitsTypeResetNameDescription
31:5Reserved
4rw1c0x0completeOn-going SPI operation has completed and the block is now idle
3ro0x0tx_watermarkTransmit FIFO level is at or below watermark
2ro0x0tx_emptyTransmit FIFO is empty
1ro0x0rx_watermarkReceive FIFO level is at or above watermark
0ro0x0rx_fullReceive FIFO is full

INTR_ENABLE

Interrupt Enable Register

  • Offset: 0x4
  • Reset default: 0x0
  • Reset mask: 0x1f

Fields

[{"name": "rx_full", "bits": 1, "attr": ["rw"], "rotate": -90}, {"name": "rx_watermark", "bits": 1, "attr": ["rw"], "rotate": -90}, {"name": "tx_empty", "bits": 1, "attr": ["rw"], "rotate": -90}, {"name": "tx_watermark", "bits": 1, "attr": ["rw"], "rotate": -90}, {"name": "complete", "bits": 1, "attr": ["rw"], "rotate": -90}, {"bits": 27}]
BitsTypeResetNameDescription
31:5Reserved
4rw0x0completeEnable interrupt when INTR_STATE.complete is set.
3rw0x0tx_watermarkEnable interrupt when INTR_STATE.tx_watermark is set.
2rw0x0tx_emptyEnable interrupt when INTR_STATE.tx_empty is set.
1rw0x0rx_watermarkEnable interrupt when INTR_STATE.rx_watermark is set.
0rw0x0rx_fullEnable interrupt when INTR_STATE.rx_full is set.

INTR_TEST

Interrupt Test Register

  • Offset: 0x8
  • Reset default: 0x0
  • Reset mask: 0x1f

Fields

[{"name": "rx_full", "bits": 1, "attr": ["wo"], "rotate": -90}, {"name": "rx_watermark", "bits": 1, "attr": ["wo"], "rotate": -90}, {"name": "tx_empty", "bits": 1, "attr": ["wo"], "rotate": -90}, {"name": "tx_watermark", "bits": 1, "attr": ["wo"], "rotate": -90}, {"name": "complete", "bits": 1, "attr": ["wo"], "rotate": -90}, {"bits": 27}]
BitsTypeResetNameDescription
31:5Reserved
4wo0x0completeWrite 1 to force INTR_STATE.complete to 1.
3wo0x0tx_watermarkWrite 1 to force INTR_STATE.tx_watermark to 1.
2wo0x0tx_emptyWrite 1 to force INTR_STATE.tx_empty to 1.
1wo0x0rx_watermarkWrite 1 to force INTR_STATE.rx_watermark to 1.
0wo0x0rx_fullWrite 1 to force INTR_STATE.rx_full to 1.

CFG

Configuration register. Controls how the SPI block transmits and receives data. This register can only be modified whilst the SPI block is idle.

  • Offset: 0xc
  • Reset default: 0x20000000
  • Reset mask: 0xe000ffff

Fields

[{"name": "HALF_CLK_PERIOD", "bits": 16, "attr": ["rw"], "rotate": 0}, {"bits": 13}, {"name": "MSB_FIRST", "bits": 1, "attr": ["rw"], "rotate": -90}, {"name": "CPHA", "bits": 1, "attr": ["rw"], "rotate": -90}, {"name": "CPOL", "bits": 1, "attr": ["rw"], "rotate": -90}]
BitsTypeResetName
31rw0x0CPOL
30rw0x0CPHA
29rw0x1MSB_FIRST
28:16Reserved
15:0rw0x0HALF_CLK_PERIOD

CFG . CPOL

The polarity of the spi_clk signal. When CPOL is 0 clock is low when idle and the leading edge is positive. When CPOL is 1 clock is high when idle and the leading edge is negative

CFG . CPHA

The phase of the spi_clk signal. When CPHA is 0 data is sampled on the leading edge and changes on the trailing edge. The first data bit is immediately available before the first leading edge of the clock when transmission begins. When CPHA is 1 data is sampled on the trailing edge and change on the leading edge.

CFG . MSB_FIRST

When set the most significant bit (MSB) is the first bit sent and received with each byte

CFG . HALF_CLK_PERIOD

The length of a half period (i.e. positive edge to negative edge) of the SPI clock, measured in system clock cycles reduced by 1. At the standard Sonata 50 MHz system clock a value of 0 gives a 25 MHz SPI clock, a value of 1 gives a 12.5 MHz SPI clock, a value of 2 gives a 8.33 MHz SPI clock and so on.

CONTROL

Controls the operation of the SPI block. This register can only be modified whilst the SPI block is idle.

  • Offset: 0x10
  • Reset default: 0x0
  • Reset mask: 0xfff

Fields

[{"name": "TX_CLEAR", "bits": 1, "attr": ["wo"], "rotate": -90}, {"name": "RX_CLEAR", "bits": 1, "attr": ["wo"], "rotate": -90}, {"name": "TX_ENABLE", "bits": 1, "attr": ["rw"], "rotate": -90}, {"name": "RX_ENABLE", "bits": 1, "attr": ["rw"], "rotate": -90}, {"name": "TX_WATERMARK", "bits": 4, "attr": ["rw"], "rotate": -90}, {"name": "RX_WATERMARK", "bits": 4, "attr": ["rw"], "rotate": -90}, {"bits": 20}]
BitsTypeResetName
31:12Reserved
11:8rw0x0RX_WATERMARK
7:4rw0x0TX_WATERMARK
3rw0x0RX_ENABLE
2rw0x0TX_ENABLE
1wo0x0RX_CLEAR
0wo0x0TX_CLEAR

CONTROL . RX_WATERMARK

The watermark level for the receive FIFO, depending on the value the interrupt will trigger at different points:

  • 0: 1 or more items in the FIFO
  • 1: 2 or more items in the FIFO
  • 2: 4 or more items in the FIFO
  • 3: 8 or more items in the FIFO
  • 4: 16 or more items in the FIFO
  • 5: 32 or more items in the FIFO
  • 6: 56 or more items in the FIFO

CONTROL . TX_WATERMARK

The watermark level for the transmit FIFO, depending on the value the interrupt will trigger at different points:

  • 0: 1 or fewer items in the FIFO
  • 1: 2 or fewer items in the FIFO
  • 2: 4 or fewer items in the FIFO
  • 3: 8 or fewer items in the FIFO
  • 4: 16 or fewer items in the FIFO

CONTROL . RX_ENABLE

When set incoming bits are written to the receive FIFO. When clear incoming bits are ignored.

CONTROL . TX_ENABLE

When set bytes from the transmit FIFO are sent. When clear the state of the outgoing spi_cipo is undefined whilst the SPI clock is running.

CONTROL . RX_CLEAR

Write 1 to clear the receive FIFO.

CONTROL . TX_CLEAR

Write 1 to clear the transmit FIFO.

STATUS

Status information about the SPI block

  • Offset: 0x14
  • Reset default: 0x0
  • Reset mask: 0x7ffff

Fields

[{"name": "TX_FIFO_LEVEL", "bits": 8, "attr": ["ro"], "rotate": 0}, {"name": "RX_FIFO_LEVEL", "bits": 8, "attr": ["ro"], "rotate": 0}, {"name": "TX_FIFO_FULL", "bits": 1, "attr": ["ro"], "rotate": -90}, {"name": "RX_FIFO_EMPTY", "bits": 1, "attr": ["ro"], "rotate": -90}, {"name": "IDLE", "bits": 1, "attr": ["ro"], "rotate": -90}, {"bits": 13}]
BitsTypeResetNameDescription
31:19Reserved
18roxIDLEWhen set the SPI block is idle and can accept a new start command.
17roxRX_FIFO_EMPTYWhen set the receive FIFO is empty and any data read from it will be undefined.
16roxTX_FIFO_FULLWhen set the transmit FIFO is full and any data written to it will be ignored.
15:8roxRX_FIFO_LEVELNumber of items in the receive FIFO
7:0roxTX_FIFO_LEVELNumber of items in the transmit FIFO

START

When written begins an SPI operation. Writes are ignored when the SPI block is active.

  • Offset: 0x18
  • Reset default: 0x0
  • Reset mask: 0x7ff

Fields

[{"name": "BYTE_COUNT", "bits": 11, "attr": ["wo"], "rotate": 0}, {"bits": 21}]
BitsTypeResetNameDescription
31:11Reserved
10:0wo0x0BYTE_COUNTNumber of bytes to receive/transmit in the SPI operation

RX_FIFO

Data from the receive FIFO. When read the data is popped from the FIFO. If the FIFO is empty data read is undefined.

  • Offset: 0x1c
  • Reset default: 0x0
  • Reset mask: 0xff

Fields

[{"name": "DATA", "bits": 8, "attr": ["ro"], "rotate": 0}, {"bits": 24}]
BitsTypeResetNameDescription
31:8Reserved
7:0roxDATAByte popped from the FIFO

TX_FIFO

Bytes written here are pushed to the transmit FIFO. If the FIFO is full writes are ignored.

  • Offset: 0x20
  • Reset default: 0x0
  • Reset mask: 0xff

Fields

[{"name": "DATA", "bits": 8, "attr": ["wo"], "rotate": 0}, {"bits": 24}]
BitsTypeResetNameDescription
31:8Reserved
7:0wo0x0DATAByte to push to the FIFO

USB 2.0 device

The USB (Universal Serial Bus) device hardware IP block in Sonata is taken from the OpenTitan project. You can find the full documentation here.

Multiple USB devices are allowed to be instantiated at a 0x1000 offset from each other up to a maximum of 256 instantiations.

Platform-level interrupt controller (PLIC)

The PLIC is a RISC-V interrupt controller and specifically we are using OpenTitan's interrupt controller. Please find more details in the official specification. Part of these details are is the memory map, which shows all registers from the base until base + 0x3FFFFFC. Not all of these registers are mapped, most importantly we only have one core, so only one context (context 0).

The following table shows the current list of interrupts that are fed through the PLIC. For more detailed descriptions of what each interrupt means, please refer to the documentation of each individual hardware IP block. More interrupts may be added to this table in the future.

NumberBlockInterrupt description
0NoneTied to zero
1, 9UART 0, 1Transmit watermark
2, 10UART 0, 1Receive watermark
3, 11UART 0, 1Transmit empty
4, 12UART 0, 1Receive overflow
5, 13UART 0, 1Receive frame error
6, 14UART 0, 1Receive break error
7, 15UART 0, 1Receive timeout
8, 16UART 0, 1Receive parity error
17, 32I2C 0, 1Format FIFO threshold
18, 33I2C 0, 1Receive FIFO threshold
19, 34I2C 0, 1Acquire FIFO threshold
20, 35I2C 0, 1Receive FIFO overflow
21, 36I2C 0, 1Received NACK
22, 37I2C 0, 1SCL interference
23, 38I2C 0, 1SDA interference
24, 39I2C 0, 1Stretch timeout
25, 40I2C 0, 1SDA unstable
26, 41I2C 0, 1Command complete
27, 42I2C 0, 1Transmit stretch
28, 43I2C 0, 1Transmit threshold
29, 44I2C 0, 1Acquire FIFO full
30, 45I2C 0, 1Unexpected stop
31, 46I2C 0, 1Host timeout
47EthernetInterrupt from external SPI ethernet chip (KSZ8851SNLI-TR). Check the interrupt status register for details.

Debug module

The debug module we use in Sonata supports the RISC-V debug specification. It also has support for CHERIoT so that capabilities can be viewed and manipulated.

Sonata Board Reference

This section focusses on what needs to be physically present on the board and explicitly leaves the configuration of the FPGA and the software for other sections. The Sonata board has the features shown here:

The Sonata Board

Configuration

The 'usable' requirement makes it worth thinking well about configuration. We want to provide multiple bitstreams and software images that can be switched between on the board without having to reprogram it. This is why we have a switch for bitstream and a switch for software images.

For example, we could have a CHERI and non-CHERI bitstream both available on the board. For software, we can switch between demo applications, for example CHERI compartmentalization versus CHERI exceptions. Introducing these physical switches also fulfills the 'interactive' requirement.

To make multiple bitstreams available, we introduce a USB connector that looks like mass storage to a user, where multiple bitstreams can be stored and changed without hassle. There is an RP2040 on the board to manage these configurations.

Also part of this 'usable' requirement is to have enough memory to store the bitstreams and software. We introduce two separate flash chips for these purposes and a HyperRAM chip.

Peripherals

The 'connectable' requirement means that we need to introduce common peripherals on the board. After consultation with the community, we settled on the following list:

  • Ethernet
  • RS-232
  • RS-485
  • MicroSD
  • ADC

Headers

The Sonata Board

For both the 'connectable' and 'extendable' requirements, we provide a number of headers so that custom functionality can be added:

  • Raspberry Pi header
  • Arduino shield (only 3.3V versions supported)
  • microBUS Click
  • Sparkfun QWIIC
  • PMOD
  • 30-pin R/A header

Although, the FPGA pins can be used independently, due to the physical layout of the headers on the FPGA board, it is not possible to plug in Raspberry Pi, Arduino shield and MicroBus click headers simultaneously. You can still use them simultaneously with modified wiring.

The 2 PMODs and the R/A header cannot be used at the same time since they use the same pins. We don't expect this to be a problem as most applications should only need to use one expansion board.

Debug

For ease of use, we have one USB connector that can power the board as well as debug using JTAG and two virtual UARTs. This means that users only have to connect one cable. Besides the JTAG over USB, we provide external JTAG and UART headers to enable users to use different setups if they need to.

User interface

The user interface is where we address the 'CHERI-visible' and 'interactive' requirements. In terms of user input, we have:

  • DIP switches
  • Button
  • Joystick

In terms of output, we have:

  • LEDs
  • LCD screen
  • CHERI-specific capability exception LEDs

Affordable

In order to meet the 'affordable' requirement, we choose a low-end FPGA to reduce the costs. We choose a Xilinx Artix 7 FPGA because it has a typical amount of memory for embedded use-cases while being able to clock the design higher than similarly priced alternatives and being supported by many tools.

Technical details

If you want to know more details on what the actual design looks like, please have a look at NewAE's repository of the Sonata PCB.

This includes the entire design sources, available currently in Altium, with a KiCad version being released shortly.

You can find a direct link to the schematics for your reading pleasure.

Default DIP switch settings

We recommend the following default state for the dipswitches:

  • On the front:
    • SW3 "Bitstream": 2
    • SW7 "SW App": 1
    • SW4 "US[0-7]": all set to off
    • SW6 : all set to off
  • On the back:
    • SW1 : all set to on
    • SW2 : 1-6 set to on and 7-8 set to off

RP2040 firmware

You can build new RP2040 firmware (or find release UF2 files) from the Sonata RP2040 repository.

With that UF2 file you can now update your firmware by holding down SW9, labelled "RP2040 Boot". While holding this button plug your Sonata board in using the main USB. A drive called RPI-RP2 should now appear which you can drag the UF2 file onto.