ETL Test Stand Firmware

Table of Contents

• Build Instructions
• Directories
• Block Diagram
• Hardware Setup
• XDMA

Build Instructions

make init
make all

Directories

├── README.org     -- This readme
├── Makefile       -- GNU Makefile
├── readout_board  -- Readout Board Firmware
├── etroc          -- ETROC Firmware
├── wrappers       -- Wrappers around IP/Submodules
├── xdc            -- XDC Constraints Files
├── doc            -- Documentation & Images
├── ip             -- Xilinx IP
├── lpgbt-fpga     -- Submodule: LPGBT-FPGA Core
├── gbt-sc         -- Submodule: CERN GBT-SC Core
├── ipbus-firmware -- Submodule: IPBus Firmware
├── Top            -- Hog Projects
└── Hog            -- Hog Build System

Block Diagram

Hardware Setup

Details of the hardware setup and a shopping list can be read at shopping.

XDMA

We are using the Xilinx PCIE XDMA driver for register access to the KCU105 board. It is configured for 8 GT/s, 1 lane PCIE (we can expand the number of lanes in the future if needed).

The open source XDMA driver must be installed on a Linux PC in order to use the board.

Installation

1. Install the XDMA Kernel Driver

Based on instructions from https://www.xilinx.com/support/answers/65444.html

XDMA driver should be compiled from source. Code is at: https://xilinx.github.io/dma_ip_drivers/2019.2/linux-kernel/html/build.html

A working recipe tested on Ubuntu 16.04 is:

sudo apt-get install libaio1 libaio-dev
git clone https://github.com/Xilinx/dma_ip_drivers
cd dma_ip_drivers/QDMA/linux-kernel
make clean && make
sudo make install
sudo insmod bin/qdma-pf.ko

2. Set up XDMA uDEV Rules

For non-root access to the XDMA driver you should add a udev rule:

sudo wget https://github.com/RHSResearchLLC/XilinxAR65444/raw/master/Linux/Xilinx_Answer_65444_Linux_Files/etc/udev/rules.d/xdma-udev-command.sh https://github.com/RHSResearchLLC/XilinxAR65444/raw/master/Linux/Xilinx_Answer_65444_Linux_Files/etc/udev/rules.d/60-xdma.rules -P /etc/udev/rules.d/
sudo udevadm control --reload-rules
sudo udevadm trigger

3. Reboot

n.b you may need to reboot after installation before this works

• try rebooting with a bitstream loaded already

4. Check for the XDMA Device

The lspci command in Unix can be used to list and display information about devices connected to the system’s PCI(e) buses. After running /sys/bus/pci/rescan, you should run lspci to check that the Xilinx device has been detected correctly - if it is, the output of lspci will contain a line like:

01:00.0 Serial controller: Xilinx Corporation Device 8031

You should also now see xdma devices in /dev/, e.g.

$ ls /dev/x*
/dev/xdma0_c2h_0     /dev/xdma0_events_10  /dev/xdma0_events_14  /dev/xdma0_events_4  /dev/xdma0_events_8
/dev/xdma0_control   /dev/xdma0_events_11  /dev/xdma0_events_15  /dev/xdma0_events_5  /dev/xdma0_events_9
/dev/xdma0_events_0  /dev/xdma0_events_12  /dev/xdma0_events_2   /dev/xdma0_events_6  /dev/xdma0_h2c_0
/dev/xdma0_events_1  /dev/xdma0_events_13  /dev/xdma0_events_3   /dev/xdma0_events_7