From 1e601cff565b53461e3835be464b8f4b235c63dc Mon Sep 17 00:00:00 2001 From: Alex Forencich Date: Sun, 13 Mar 2022 23:32:01 -0700 Subject: [PATCH] Initial commit of sphinx documentation --- docs/Makefile | 20 + docs/make.bat | 35 + docs/source/conf.py | 80 ++ docs/source/contents.rst | 21 + docs/source/debugging.rst | 46 + docs/source/devicelist.rst | 122 ++ docs/source/diagrams/svg/corundum_block.odg | Bin 0 -> 18822 bytes docs/source/diagrams/svg/corundum_block.svg | 983 +++++++++++++++ .../svg/corundum_queue_manager_block.odg | Bin 0 -> 17360 bytes .../svg/corundum_queue_manager_block.svg | 471 +++++++ .../diagrams/svg/corundum_tdma_ctrl_block.odg | Bin 0 -> 16119 bytes .../diagrams/svg/corundum_tdma_ctrl_block.svg | 430 +++++++ .../svg/corundum_tx_scheduler_block.odg | Bin 0 -> 17562 bytes .../svg/corundum_tx_scheduler_block.svg | 530 ++++++++ docs/source/diagrams/svg/queue_pointers.odg | Bin 0 -> 11630 bytes docs/source/diagrams/svg/queue_pointers.svg | 148 +++ docs/source/gettingstarted.rst | 626 ++++++++++ docs/source/glossary.rst | 46 + docs/source/index.rst | 67 + docs/source/macaddr.rst | 42 + docs/source/modules/cpl_queue_manager.rst | 7 + docs/source/modules/cpl_write.rst | 7 + docs/source/modules/desc_fetch.rst | 7 + docs/source/modules/index.rst | 15 + docs/source/modules/mqnic_app_block.rst | 1099 +++++++++++++++++ docs/source/modules/mqnic_core.rst | 870 +++++++++++++ docs/source/modules/mqnic_core_axi.rst | 128 ++ docs/source/modules/mqnic_core_pcie.rst | 289 +++++ docs/source/modules/mqnic_core_pcie_s10.rst | 135 ++ docs/source/modules/mqnic_core_pcie_us.rst | 224 ++++ docs/source/modules/mqnic_egress.rst | 13 + docs/source/modules/mqnic_ingress.rst | 15 + docs/source/modules/mqnic_interface.rst | 17 + docs/source/modules/mqnic_interface_rx.rst | 13 + docs/source/modules/mqnic_interface_tx.rst | 13 + docs/source/modules/mqnic_l2_egress.rst | 105 ++ docs/source/modules/mqnic_l2_ingress.rst | 109 ++ docs/source/modules/mqnic_ptp.rst | 129 ++ docs/source/modules/mqnic_ptp_clock.rst | 122 ++ docs/source/modules/mqnic_ptp_perout.rst | 95 ++ .../modules/mqnic_tx_scheduler_block.rst | 242 ++++ docs/source/modules/overview.rst | 29 + docs/source/modules/queue_manager.rst | 40 + docs/source/modules/rx_checksum.rst | 7 + docs/source/modules/rx_engine.rst | 7 + docs/source/modules/rx_hash.rst | 7 + docs/source/modules/tx_checksum.rst | 7 + docs/source/modules/tx_engine.rst | 7 + docs/source/modules/tx_scheduler_rr.rst | 196 +++ docs/source/operations.rst | 246 ++++ docs/source/porting.rst | 62 + docs/source/rb/bmc_alveo.rst | 49 + docs/source/rb/bmc_gecko.rst | 73 ++ docs/source/rb/cqm_event.rst | 86 ++ docs/source/rb/cqm_rx.rst | 86 ++ docs/source/rb/cqm_tx.rst | 86 ++ docs/source/rb/drp.rst | 105 ++ docs/source/rb/flash_bpi.rst | 102 ++ docs/source/rb/flash_spi.rst | 93 ++ docs/source/rb/fw_id.rst | 133 ++ docs/source/rb/gpio.rst | 43 + docs/source/rb/i2c.rst | 46 + docs/source/rb/if.rst | 77 ++ docs/source/rb/if_ctrl.rst | 119 ++ docs/source/rb/index.rst | 94 ++ docs/source/rb/null.rst | 21 + docs/source/rb/phc.rst | 163 +++ docs/source/rb/phc_perout.rst | 134 ++ docs/source/rb/qm_rx.rst | 86 ++ docs/source/rb/qm_tx.rst | 86 ++ docs/source/rb/sched_block.rst | 35 + docs/source/rb/sched_ctrl_tdma.rst | 127 ++ docs/source/rb/sched_rr.rst | 136 ++ docs/source/rb/tdma_sch.rst | 210 ++++ docs/source/tuning.rst | 101 ++ 75 files changed, 10220 insertions(+) create mode 100644 docs/Makefile create mode 100644 docs/make.bat create mode 100644 docs/source/conf.py create mode 100644 docs/source/contents.rst create mode 100644 docs/source/debugging.rst create mode 100644 docs/source/devicelist.rst create mode 100644 docs/source/diagrams/svg/corundum_block.odg create mode 100644 docs/source/diagrams/svg/corundum_block.svg create mode 100644 docs/source/diagrams/svg/corundum_queue_manager_block.odg create mode 100644 docs/source/diagrams/svg/corundum_queue_manager_block.svg create mode 100644 docs/source/diagrams/svg/corundum_tdma_ctrl_block.odg create mode 100644 docs/source/diagrams/svg/corundum_tdma_ctrl_block.svg create mode 100644 docs/source/diagrams/svg/corundum_tx_scheduler_block.odg create mode 100644 docs/source/diagrams/svg/corundum_tx_scheduler_block.svg create mode 100644 docs/source/diagrams/svg/queue_pointers.odg create mode 100644 docs/source/diagrams/svg/queue_pointers.svg create mode 100644 docs/source/gettingstarted.rst create mode 100644 docs/source/glossary.rst create mode 100644 docs/source/index.rst create mode 100644 docs/source/macaddr.rst create mode 100644 docs/source/modules/cpl_queue_manager.rst create mode 100644 docs/source/modules/cpl_write.rst create mode 100644 docs/source/modules/desc_fetch.rst create mode 100644 docs/source/modules/index.rst create mode 100644 docs/source/modules/mqnic_app_block.rst create mode 100644 docs/source/modules/mqnic_core.rst create mode 100644 docs/source/modules/mqnic_core_axi.rst create mode 100644 docs/source/modules/mqnic_core_pcie.rst create mode 100644 docs/source/modules/mqnic_core_pcie_s10.rst create mode 100644 docs/source/modules/mqnic_core_pcie_us.rst create mode 100644 docs/source/modules/mqnic_egress.rst create mode 100644 docs/source/modules/mqnic_ingress.rst create mode 100644 docs/source/modules/mqnic_interface.rst create mode 100644 docs/source/modules/mqnic_interface_rx.rst create mode 100644 docs/source/modules/mqnic_interface_tx.rst create mode 100644 docs/source/modules/mqnic_l2_egress.rst create mode 100644 docs/source/modules/mqnic_l2_ingress.rst create mode 100644 docs/source/modules/mqnic_ptp.rst create mode 100644 docs/source/modules/mqnic_ptp_clock.rst create mode 100644 docs/source/modules/mqnic_ptp_perout.rst create mode 100644 docs/source/modules/mqnic_tx_scheduler_block.rst create mode 100644 docs/source/modules/overview.rst create mode 100644 docs/source/modules/queue_manager.rst create mode 100644 docs/source/modules/rx_checksum.rst create mode 100644 docs/source/modules/rx_engine.rst create mode 100644 docs/source/modules/rx_hash.rst create mode 100644 docs/source/modules/tx_checksum.rst create mode 100644 docs/source/modules/tx_engine.rst create mode 100644 docs/source/modules/tx_scheduler_rr.rst create mode 100644 docs/source/operations.rst create mode 100644 docs/source/porting.rst create mode 100644 docs/source/rb/bmc_alveo.rst create mode 100644 docs/source/rb/bmc_gecko.rst create mode 100644 docs/source/rb/cqm_event.rst create mode 100644 docs/source/rb/cqm_rx.rst create mode 100644 docs/source/rb/cqm_tx.rst create mode 100644 docs/source/rb/drp.rst create mode 100644 docs/source/rb/flash_bpi.rst create mode 100644 docs/source/rb/flash_spi.rst create mode 100644 docs/source/rb/fw_id.rst create mode 100644 docs/source/rb/gpio.rst create mode 100644 docs/source/rb/i2c.rst create mode 100644 docs/source/rb/if.rst create mode 100644 docs/source/rb/if_ctrl.rst create mode 100644 docs/source/rb/index.rst create mode 100644 docs/source/rb/null.rst create mode 100644 docs/source/rb/phc.rst create mode 100644 docs/source/rb/phc_perout.rst create mode 100644 docs/source/rb/qm_rx.rst create mode 100644 docs/source/rb/qm_tx.rst create mode 100644 docs/source/rb/sched_block.rst create mode 100644 docs/source/rb/sched_ctrl_tdma.rst create mode 100644 docs/source/rb/sched_rr.rst create mode 100644 docs/source/rb/tdma_sch.rst create mode 100644 docs/source/tuning.rst diff --git a/docs/Makefile b/docs/Makefile new file mode 100644 index 000000000..d0c3cbf10 --- /dev/null +++ b/docs/Makefile @@ -0,0 +1,20 @@ +# Minimal makefile for Sphinx documentation +# + +# You can set these variables from the command line, and also +# from the environment for the first two. +SPHINXOPTS ?= +SPHINXBUILD ?= sphinx-build +SOURCEDIR = source +BUILDDIR = build + +# Put it first so that "make" without argument is like "make help". +help: + @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) + +.PHONY: help Makefile + +# Catch-all target: route all unknown targets to Sphinx using the new +# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). +%: Makefile + @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) diff --git a/docs/make.bat b/docs/make.bat new file mode 100644 index 000000000..6fcf05b4b --- /dev/null +++ b/docs/make.bat @@ -0,0 +1,35 @@ +@ECHO OFF + +pushd %~dp0 + +REM Command file for Sphinx documentation + +if "%SPHINXBUILD%" == "" ( + set SPHINXBUILD=sphinx-build +) +set SOURCEDIR=source +set BUILDDIR=build + +if "%1" == "" goto help + +%SPHINXBUILD% >NUL 2>NUL +if errorlevel 9009 ( + echo. + echo.The 'sphinx-build' command was not found. Make sure you have Sphinx + echo.installed, then set the SPHINXBUILD environment variable to point + echo.to the full path of the 'sphinx-build' executable. Alternatively you + echo.may add the Sphinx directory to PATH. + echo. + echo.If you don't have Sphinx installed, grab it from + echo.https://www.sphinx-doc.org/ + exit /b 1 +) + +%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% +goto end + +:help +%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% + +:end +popd diff --git a/docs/source/conf.py b/docs/source/conf.py new file mode 100644 index 000000000..5b34fecef --- /dev/null +++ b/docs/source/conf.py @@ -0,0 +1,80 @@ +# Configuration file for the Sphinx documentation builder. +# +# This file only contains a selection of the most common options. For a full +# list see the documentation: +# https://www.sphinx-doc.org/en/master/usage/configuration.html + +import datetime +import sys + +# -- Path setup -------------------------------------------------------------- + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +# +# import os +# import sys +# sys.path.insert(0, os.path.abspath('.')) + + +# -- Project information ----------------------------------------------------- + +project = 'Corundum' +author = f'{project} contributors' +copyright = f'2019-{datetime.datetime.now().year}, {author}' + + +# -- General configuration --------------------------------------------------- + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +extensions = [ + "sphinxcontrib.inkscapeconverter", +] + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +# This pattern also affects html_static_path and html_extra_path. +exclude_patterns = [] + +root_doc = 'contents' + +numfig = True + +highlight_language = 'none' + + +# -- Options for HTML output ------------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +# +html_theme = 'alabaster' + +try: + import sphinx_rtd_theme + html_theme = 'sphinx_rtd_theme' +except ImportError: + sys.stderr.write('Warning: The Sphinx \'sphinx_rtd_theme\' HTML theme was '+ + 'not found. Make sure you have the theme installed to produce pretty '+ + 'HTML output. Falling back to the default theme.\n') + + html_theme = 'default' + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +html_static_path = ['_static'] + +graphviz_output_format = 'svg' + + +# -- Options for LaTeX output ------------------------------------------------ + +latex_show_pagerefs = True +latex_show_urls = 'footnote' diff --git a/docs/source/contents.rst b/docs/source/contents.rst new file mode 100644 index 000000000..be6160066 --- /dev/null +++ b/docs/source/contents.rst @@ -0,0 +1,21 @@ +.. _contents: + +======== +Contents +======== + +.. toctree:: + :numbered: + :includehidden: + + index + gettingstarted + debugging + tuning + porting + macaddr + operations + modules/index + rb/index + devicelist + glossary \ No newline at end of file diff --git a/docs/source/debugging.rst b/docs/source/debugging.rst new file mode 100644 index 000000000..185415a21 --- /dev/null +++ b/docs/source/debugging.rst @@ -0,0 +1,46 @@ +.. _debugging: + +========= +Debugging +========= + +The server rebooted when configuring the FPGA +============================================= + +This is a common problem caused by the server management subsystem (IPMI, iLO, iDRAC, or whatever your server manufacturer calls it). It detects the PCIe device falling off the bus when the FPGA is reset, and does something in response. In some machines it just complains bitterly by logging an event and turning on an angry red LED. In other machines, it does that, and then immediately reboots the machine. Highly annoying. However, there is a simple solution for this that involves some poking around in PCIe configuration registers to disable PCIe fatal error reporting on the port that the device is connected to. + +Run the ``pcie_disable_fatal_err.sh`` script before you configure the FPGA. Specify the PCIe device ID of the FPGA board as the argument (``xy:00.0``, as shown in ``lspci``). You'll have to do a warm reboot after loading the configuration if the PCIe BAR configuration has changed. This will most likely be the case when going from a stock flash image that brings up the PCIe link to Corundum, but not from Corundum to Corundum unless something was changed in the PCIe IP core configuration. If the BAR configuration has not changed, then using the ``pcie_hot_reset.sh`` script to perform a hot reset of the device may be sufficient. The firmware update utility ``mqnic-fw`` includes the same functionality as ``pcie_disable_fatal_err.sh`` and ``pcie_hot_reset.sh``, so if the card is running a Corundum design, then you can use ``mqnic-fw -t`` to disable fatal error reporting and reset the card before connecting to it via JTAG. + +The link is down +================ + +Things to check, in no particular order: + +- Try a hot reset of the card +- Try an unmodified "known good" design for your board, either corundum or verilog-ethernet +- Try using a direct attach copper cable to rule out issues with optical transceivers. +- Try a different link partner---try a NIC instead of a switch, or a different model NIC. +- txdisable/lpmode/reset pins---only applies if you're using optical transceivers or active optical cables. If these pins are pulled the wrong way, the lasers in the transceiver will not turn on, and the link will not come up. (No, I have never wasted an hour waiting for Vivado to do its thing after pulling lpmode the wrong way...on several different boards...) +- Optical module CDR settings---if you're trying to run a 25G or 100G optical transceiver or active optical cable at 10G or 40G, you may need to disable the module CDRs via I2C by writing 0x00 to MSA register 98 on I2C address 0x50 (CDR control). (No, I have not wasted several days trying to figure out why the electrical loopback works fine at 10G, but the optical transceiver only works at 25G...) +- Check settings at link partner---some devices are better about figuring out the proper configuration than others and need to have the correct settings applied manually (e.g. Mellanox NICs are quite good, but most packet switches can be rather bad about this and may only look at the line rate reported in the transceiver EEPROM instead of what's actually going on on the link). Also check to make sure the link partner doesn't have some sort of disagreement with the cable/transceiver - some devices, usually switches, are very picky about what the EEPROM says about who manufactured the cable. +- Check FEC settings---in general, 100G devices seem to require the use of RS-FEC, 10G and 25G usually run fine without FEC, but it may need to be manually disabled on the link partner. +- Serdes configuration---ensure the correct line rate, gearbox settings, etc. are correct. Some boards also have p/n swapped (e.g. ExaNIC X10/X25), so check tx/rx invert settings. (Yes, I managed to figure *that* one out after some head-scratching despite not having access to the schematic) +- Serdes site locations---make sure you're using the correct pins. +- Serdes reference clock configuration---make sure the reference clock matches the serdes configuration, and on some boards the reference clock needs to be configured in some way before use. + +Ping and iperf don't work +========================= + +Things to check, in no particular order: + +- Check that the interface is up (``ip link set dev up``) +- Check that the interface has an IP address assigned (``ip addr``, ``ip -c a``, ``ip -c -br``, to check, ``ip addr add 192.168.1.1/24 dev `` to set) +- The corundum driver does not currently report the link status to the OS, so check for a link light (not all design variants implement this) and check the link partner for the link status (``ip link``, NO-CARRIER means the link is down at the PHY layer) +- Try hot resetting the card with the link partner connected (clear up possible RX DFE problem) +- Check tcpdump for inbound traffic on both ends of the link ``tcpdump -i -Q in`` to see what is actually traversing the link. If the TX direction works but the RX direction does not, there is a high probability it is a transceiver DFE issue that may be fixable with a hot reset. +- Check with ``mqnic-dump`` to see if there is anything stuck in transmit queues, transmit or receive completion queues, or event queues. + +The device loses its IP address +=============================== + +This is not a corundum issue, this is NetworkManager or a similar application causing trouble by attempting to run DHCP or similar on the interface. There are basically four options here: disable NetworkManager, configure NetworkManager to ignore the interface, use NetworkManager to configure the interface and assign the IP address you want, or use network namespaces to isolate the interface from NetworkManager. Unfortunately, if you have a board that doesn't support persistent MAC addresses, it may not be possible to configure NetworkManager to deal with the interface correctly. diff --git a/docs/source/devicelist.rst b/docs/source/devicelist.rst new file mode 100644 index 000000000..31509abb4 --- /dev/null +++ b/docs/source/devicelist.rst @@ -0,0 +1,122 @@ +.. _device_list: + +=========== +Device list +=========== + +This section includes a summary of the various devices supported by Corundum, including a summary of board-specific features. + +PCIe +==== + +This section details PCIe form-factor targets, which interface with a separate host system via PCI express as a PCIe endpoint. + +.. table:: Summary of the various devices supported by Corundum. + + ============ ================= ==================== ========== + Manufacturer Board FPGA Board ID + ============ ================= ==================== ========== + Alpha Data ADM-PCIE-9V3 XCVU3P-2FFVC1517I 0x41449003 + Exablaze ExaNIC X10 XCKU035-2FBVA676E 0x1ce40003 + Exablaze ExaNIC X25 XCKU3P-2FFVB676E 0x1ce40009 + Silicom fb2CG\@KU15P XCKU15P-2FFVE1760E 0x1c2ca00e + Digilent NetFPGA SUME XC7V690T-3FFG1761 0x10ee7028 + Intel DK-DEV-1SMC-H-A 1SM21CHU1F53E1VG 0x11720001 + Xilinx Alveo U50 XCU50-2FSVH2104E 0x10ee9032 + Xilinx Alveo U200 XCU200-2FSGD2104E 0x10ee90c8 + Xilinx Alveo U250 XCU250-2FIGD2104E 0x10ee90fa + Xilinx Alveo U280 XCU280-L2FSVH2892E 0x10ee9118 + Xilinx VCU108 XCVU095-2FFVA2104E 0x10ee806c + Xilinx VCU118 XCVU9P-L2FLGA2104E 0x10ee9076 + Xilinx VCU1525 XCVU9P-L2FSGD2014E 0x10ee95f5 + Xilinx ZCU106 XCZU7EV-2FFVC1156E 0x10ee906a + ============ ================= ==================== ========== + +.. table:: Summary of available interfaces and on-board memory. + + ================= ========= ========== =============================== ===== + Board PCIe IF Network IF DDR HBM + ================= ========= ========== =============================== ===== + ADM-PCIE-9V3 Gen 3 x16 2x QSFP28 16 GB DDR4 2400 (2x 1G x72) \- + ExaNIC X10 Gen 3 x8 2x SFP+ \- \- + ExaNIC X25 Gen 3 x8 2x SFP28 \- \- + fb2CG\@KU15P Gen 3 x16 2x QSFP28 16 GB DDR4 2400 (4x 512M x72) \- + NetFPGA SUME Gen 3 x8 4x SFP+ 8 GB DDR3 1866 (2x 512M x64) \- + DK-DEV-1SMC-H-A Gen 3 x8 2x QSFP28 16 GB DDR4 2666 (2x 512M x72) 16 GB + Alveo U50 Gen 3 x16 1x QSFP28 \- 8 GB + Alveo U200 Gen 3 x16 2x QSFP28 64 GB DDR4 2400 (4x 2G x72) \- + Alveo U250 Gen 3 x16 2x QSFP28 64 GB DDR4 2400 (4x 2G x72) \- + Alveo U280 Gen 3 x16 2x QSFP28 32 GB DDR4 2400 (2x 2G x72) 8 GB + VCU108 Gen 3 x8 1x QSFP28 4 GB DDR4 2400 (2x 256M x80) \- + VCU118 Gen 3 x16 2x QSFP28 4 GB DDR4 2400 (2x 256M x80) \- + VCU1525 Gen 3 x16 2x QSFP28 64 GB DDR4 2400 (4x 2G x72) \- + ZCU106 Gen 3 x4 2x SFP+ 2 GB DDR4 2400 (256M x64) \- + ================= ========= ========== =============================== ===== + +.. table:: Summary of support for various ancillary features. + + ================= ============ ============ ========== + Board I2C :sup:`1` MAC :sup:`2` FW update + ================= ============ ============ ========== + ADM-PCIE-9V3 N :sup:`3` Y :sup:`5` Y + ExaNIC X10 N :sup:`3` Y Y + ExaNIC X25 N :sup:`3` Y Y + fb2CG\@KU15P Y Y Y + NetFPGA SUME Y N :sup:`7` N :sup:`8` + DK-DEV-1SMC-H-A N N N + Alveo U50 N :sup:`4` Y Y + Alveo U200 Y Y Y + Alveo U250 Y Y Y + Alveo U280 N :sup:`4` Y Y + VCU108 Y Y :sup:`5` Y + VCU118 Y Y :sup:`5` Y + VCU1525 Y Y :sup:`5` Y + ZCU106 Y Y :sup:`5` Y + ================= ============ ============ ========== + +- :sup:`1` I2C access to optical modules +- :sup:`2` Persistent MAC address storage +- :sup:`3` Supported in hardware, driver support in progress +- :sup:`4` Limited read/write access via BMC pending driver support, full read/write access requires support in BMC firmware +- :sup:`5` Can read MAC from I2C EEPROM, but EEPROM is blank from factory +- :sup:`6` MAC available from BMC, but accessing BMC is not yet implemented +- :sup:`7` No on-board EEPROM +- :sup:`8` Flash sits behind CPLD, not currently exposed via PCIe + +.. table:: Summary of the board-specific design variants and some important configuration parameters. + + ================= ========================= ==== ======= ==== ===== + Board Design IFxP RXQ/TXQ MAC Sched + ================= ========================= ==== ======= ==== ===== + ADM-PCIE-9V3 mqnic/fpga_10g/fpga 2x1 256/8K 10G RR + ADM-PCIE-9V3 mqnic/fpga_10g/fpga_tdma 2x1 256/256 10G TDMA + ADM-PCIE-9V3 mqnic/fpga_25g/fpga 2x1 256/8K 25G RR + ADM-PCIE-9V3 mqnic/fpga_25g/fpga_tdma 2x1 256/256 25G TDMA + ADM-PCIE-9V3 mqnic/fpga_100g/fpga 2x1 256/8K 100G RR + ADM-PCIE-9V3 mqnic/fpga_100g/fpga_tdma 2x1 256/256 100G TDMA + ExaNIC X10 mqnic/fpga/fpga 2x1 256/1K 10G RR + ExaNIC X25 mqnic/fpga_10g/fpga 2x1 256/8K 10G RR + ExaNIC X25 mqnic/fpga_25g/fpga 2x1 256/8K 25G RR + fb2CG\@KU15P mqnic/fpga_10g/fpga 2x1 256/8K 10G RR + fb2CG\@KU15P mqnic/fpga_10g/fpga_tdma 2x1 256/256 10G TDMA + fb2CG\@KU15P mqnic/fpga_25g/fpga 2x1 256/8K 25G RR + fb2CG\@KU15P mqnic/fpga_25g/fpga_tdma 2x1 256/256 25G TDMA + fb2CG\@KU15P mqnic/fpga_100g/fpga 2x1 256/8K 100G RR + fb2CG\@KU15P mqnic/fpga_100g/fpga_tdma 2x1 256/256 100G TDMA + NetFPGA SUME mqnic/fpga/fpga 1x1 256/512 10G RR + DK-DEV-1SMC-H-A mqnic/fpga_10g/fpga 2x1 256/1K 10G RR + Alveo U50 mqnic/fpga_10g/fpga 1x1 256/8K 10G RR + Alveo U50 mqnic/fpga_100g/fpga 1x1 256/8K 100G RR + Alveo U200 mqnic/fpga_10g/fpga 2x1 256/8K 10G RR + Alveo U200 mqnic/fpga_100g/fpga 2x1 256/8K 100G RR + Alveo U250 mqnic/fpga_10g/fpga 2x1 256/8K 10G RR + Alveo U250 mqnic/fpga_100g/fpga 2x1 256/8K 100G RR + Alveo U280 mqnic/fpga_10g/fpga 2x1 256/8K 10G RR + Alveo U280 mqnic/fpga_100g/fpga 2x1 256/8K 100G RR + VCU108 mqnic/fpga_10g/fpga 1x1 256/2K 10G RR + VCU118 mqnic/fpga_10g/fpga 2x1 256/8K 10G RR + VCU118 mqnic/fpga_100g/fpga 2x1 256/8K 100G RR + VCU1525 mqnic/fpga_10g/fpga 2x1 256/8K 10G RR + VCU1525 mqnic/fpga_100g/fpga 2x1 256/8K 100G RR + ZCU106 mqnic/fpga_10g/fpga 2x1 256/8K 10G RR + ================= ========================= ==== ======= ==== ===== diff --git a/docs/source/diagrams/svg/corundum_block.odg b/docs/source/diagrams/svg/corundum_block.odg new file mode 100644 index 0000000000000000000000000000000000000000..24845d2c73ec38113302bffecd107196e1877f8d GIT binary patch literal 18822 zcmdS9b983U)-L)cZ_JLmV|VPNlXPs`>Daby+qP}nwr$%^PXG3I_PO_rvCntM`SaE@ zYE`YZ#;jRYtF-1bWyFDCr~m*I0D#8Jq>u%q>8At$0RP0lj{p`X76$e%)&@G(*5)R9 zI`$@3mb8wR`ZQKLb|!W-R@MfV`c`@l76z8~G)A^M*2X4!b~684fq%>RUjhAZj^E1C z(8S2W_FqNWG0+;CTj|&v*wX&rN&QvmpsVxWNn!p6UDj6C4%YvO{$Fadx3V(-zt{GU zl(mkff%$)P)W62kUrqlj!2HeG*}Iq<*wHv!m>;OG*{-o7du&x{ z+n7E@kd%a3nCwYR7?3h5+89zu;Xazc5E6vw#cKOx8@!Po+I(4c>0>JUB8$q_>Kb*3 zDVpbE8Q-7421D$#Y;78q*aexSM8A*1Pk}D=xCrbLYiYD(aewSF@&>T^v-c|k?Z$A`-BVqk<7Z+iWn9mQn-L5H63o4|Vzl>X$-3%r~|31BQ&h@Aw zs~VZoqg}U>ZZXLBC5`7Hb%)+*)OlD4e%51lDzE-+l*tJj<1jb3L_3TKuP9l%{JyVl zIvl$K$$&orX&herO3BZrL(QQZQ@$R94dw{ksjkf~JJ9i{-BIhhuNZPU1TTk)Vbqoz zx17dk6S?68tsw`xPod5zIVY!26riE7CN|a4v8E-1C z!Z}fzAH2+d);igSV_j#^rcZ5Jw;2h%ZiBt57QE2UfGE)ML;JV7%`ppu4@{Ub?$|q_aNiSnAv9@@f_)l`&@!){7Q=T5$LdE%C}7r|@h4iuUj4)mo#Prr7ihZM zOXT(!p(l|MBvT5~8X;p=xDmF(Z?D}uJM5W1BqfMh7d{Q@&CtYxlM*yz`PXB0Egu>K z`fTejp$5gjFoSoy&Ym1ZW_!wTjx6@~f=LFp&B$w1cvLIz5Ph-AwGaDbi=7A4?T>8XFAzY}IVWxGG+FmFr&FzVTo4aJM54 zn0_`wRsnW&P{cR4Ir^+-)u_;6fLC?_l$~DBFY`^(Z)?4M=~(ee8F8z-Sad964Dnf} zrgZzZ)9b8zb`qDE2bA%R5+mO_^B1_j3B6tzux0|ovDyijU*5%!%$fS?n(yID$e2FD z^UK&+xb3IRf*w>2pP?&2ZWC28dgTk=G*Uzx#x6$ISlkzU4`2qM^+kj^M#w+o^T{JM zR`NALcW_{cOjoxX&^N>7K&v3Os0M^%@Q*P0h`A!T+)swb{CzrgzP)dp&P(}t@6HbK zhAWqv@EVV6iFyxhmGxicZ9iSj!IX8{$bBbwy*B?olzXN8sLP}#WHB09G&l2dK9Jxc zOh5Tn-+%#8>{b&zSwwvrm=|_8b3Cx{Vfhr+8b&8#He#uujr!iEXe+GV4Jf^Pr`hvu~?%wEGcfve(U4JeNcV ze-s4&s0&oHhHjGl#$Jkjk;jD4WOd~GeADYF9&ipRRaj`g=gU{3qcf_JB2>OBHLCKS z04=2IDTtO~5VN$&U?n}0&+o8+&TNk}OVaQ3$>ES2=2MX3_*9oJtZ`u|O578GisbLvzjX$68z^rhKN6nRP*TfeoFZmO-Z)EU$qfbqRlQ z^i%!HEbcO$gT{386NpK;9LE5&=h?Y}vE+C0V-$iTXty0XlkhAiaV92aL9@ja{enATAR#PV%v`=7;Vt!6#oJCIH{TO5uvtr*+jzCHqORRq4wZ85g&XiF584YKzyLeSc zLr-^|mXlMAj2hb=D>J6OaAUSLt?;)$xWq;KG1p>1Fr!+~vM<>LDpfA1ElQbj*ehIr z#4It^)Y}mie{h}fCC*i2a!rPuZe|~nvIfBz2$ejQd73!Z%B=)}Jjko*pN5+rC=nz# zISIuMU2xCb*Pya#l5w>u5>4*IhT=CkQa|gOinT|KB8hJuRg%?T$V73gG_+d2KBlq; zcglIGvQRu~iJ#=^0iiVyzsa_p3aDiC!L4v>!Rgz!HmMfIS>j4ms6V(Q5xaHn z!La)&cBrpoWy*P-J)k0_17*lQ-WLd%eiQH|RUzk`jC{k)kktIicSOo>{7!Mx!ivLH z#U!nj370cMIWbpunZj zpTTS58lfxvBHU|&A~;WBiUJ9niyLT2U;OsX5fG-dSnXJ9#@dR;Ooc;%<=QlhL)BYn ztyCUkKa+u;%PM(YJAgByk;c+UUtBI@1XYIISOhBc-D^A80=Y5Jy$6d=`iW^j}6HSoKGrE)%_U4*T<-1ec9FiU0 zpOZMDY8azj9lG_bZp$KC=0@D?^O7^2n-k3%$J%y>lpM~ktaq9|I+3n4Y)eO-Ko4h@ zi?yywwXrE{EfJN_maZjCrVb4{)}doa4+Ps|pI1qepyMY-p?1r)q8#0qRC{ckd?KfO zO%%|Nmio6Y=%j2Gb>FJN1jD1DK=GhJ67^1899kja0P!Sf!Dfyq$3DaNO7hp~Uv5)` z14LRw8O5iRhNe^A`QN}l{-V+Z?YrkfrS z1o4)hG74U6y`4tnxM6$Y05TFZc(NEmwx6sAc_XszS3^kQHz_f|8hb=)jf1 z#GVQdjsH+k=`0Pm_mALO)LHU?y%9XWLmHypsedKZMI@0-Gzt`i_F~reOi-MoO-g^# z!eE1?>5DYu_j%^X`OIE8o7zb|8>X_PH9N!eq&9SsV zV;97LS4m*F)L=#sQ)9InQ}bfPO-`C*6C@ zIsS&BdDMpHX-mTNqh`xVDeXY;{7D!s@h8vQeZ7@q>eyk>I`n|c`fAl4g>efBR@Ep< z;baMd@R_heWm~OZf7ytQX@9*@KfZy}>*wNA;XF~7!sOR2-@b`&<^YTvotIoqU}KjC zq$RBMn`j*?7s6UpR8#6fj{UMLS%q2vhe7Miz>2wU(;3RKtTS_l;9+@LHng{>vD_DU zFzSH~Mh{j9cx-`KQ$f`p;W~5X>lg$sUN=CGT!uCfAP=_gZ2-7Gd!pxCmaxm)4;gVtsNx5?l&-(0 zT1`O!;J^M|{?oA+?w@0?fxW$nrP03*z8ilm>~@*IeQs*m^wl$7*t2TJj-eg$4Yi{; z13>iB{HHsR}UEU)J*JJ1#Dq)yYDzT^$$IGzu5$*P>7YI!o}cev&B73SWLbc=#(fZ z&?Y$huNdH=?@WB@YlqgL&peW50r$@3^AxE}6U3s!IUK991bPdue z|29q8FqKkcL@YDrvAu8?Cg%8zOA>bAdGljo>S~ZSkc0MQp*UMv68Bk}4m){)v^ieyA_1MG}yCctWQ!h0QUdUyR7)itw0<4zViV5?i61I;q3G708`8PLo2frabpwr_z5vurK8KPuH{>pe$xjjH8xS` z@oJ1peGFgGR$-cePfjBO#}Fp=G41))(qxU9{U^?+MCfx}yz(PtPReg*%sdm%yT5e}v z({w*!4bbLEBcl2FHYhJ1&=59yNAKuUv?L zQV0GK;_*Pr?Wl;4=NzVwCx`XPJzzrUYmc=;Hi<)*&2`+!ELchD)}0B;Q1x;Gd0Yqi zbN9@1Gvh7qge0Y%KDatA5$~4yN}5PxsPe6A*>-Slx7p%*lRU#sge;gDtuyNY#U^J{ zx_LnLGVE8=51E<#C$kc48Cv@#V`hc8P@@=miWtiyGmuBXj@x!USmR0p2Bzre4f&hn+fAp}IdDj%kQob&E}me=Gy? zAiKm%zKmXEy&hYm#13IC883F)^C;Rt~_iv-2 zl#VU^)eivRpZJfBPr=y1Lf2Bq#N3Y7{@+;|YfGaL87X0S7|efcLGYp?0&)NV$lokr z5DNJB3S(L+3IG5BG7<`c6ciM!tgQU}{9aj)glD7k`(m{bk$0~n$_7`b^aunOJwT_v>R)6xm)CVi1t{j_qlnb2bo8Nxk-h3%48Y4wTM`?NZ7O}IF*>WSGpNho9gyp z$PNBf?`1U_#DNp ztF@W#ay!j%UhMPm^YaS}3-gQ03`h$PiOov)4oM1%&j^c2NeqdLi%Uq!$c#%$OG^ui z%gYKXNz2SjE6faRtIRCS$t<2}x@ANhpXYD#*xg^R4O#sA*5g z?MiOS%TCD4%PS};EGjN8{+kvRmz9+j=d_kp)>hV3Rn=5imshlu)z#Hi)z;RwSJ$_; zG`6+11?LT9Ru2`_3>GwuR<@4Tc24?Eri9L9h0SCqjTgnu6(-G>q%BmWP1jV6HRdeV z-v~2t3?Zs6dq}1Is z_-vQ9Pm~OdwGAyb^vpGk&vgtev`j9xPM=ov?lupu4}0%S`kyQWJxzw*ZiIgx$MyI3 zkM#DB^^A-R4~~tE^^MMUkFSkR&JNFw4Nh;&_l?g^&aCv#ua3`bPOQ$&&CSfOEYB{k zEi5lDFRibyk1ZX|EblICo%e2!4V}+U?9ERfEG{0dtsigAUvF&g9&YU(Z=CEL^i3U( z%pJ|GUJotY&21j8@7*u$?C$RF>>ux*?w{-)o*o=uTwLs(++N*1Kkc5}-Q7LDzP#K# ze13ku8B`bm0H~#+0=x>&s~2hT@frgf-8tS9okfdRuYzHC+f5CB%24Y^W%&?qk@h!S z%~SOMEH>l|KdGckI<8L`(qc5s&nd~6-lsR>625yw%VRLnh4T1tjj0AJlmfx-1<)X%Ufmn+0+3>U74Xyp{o6)V>bD1FCE)4}fZU}u- zlndShyZ-YhJQ@lh-)6qS#1NO{bq0b0nrBln444f5U~y+C^|Zi7{!qzmLf*1h)*deAZIEfy zpYSR=idu2T{=+Ab*EQ_MFY0ibB@iObwm3dNS|GO13%K2UJ{~6}VBrB8`x3|l0fi;_ zp)joDEozo(d|d726P&hVXIfro!d$l+TYMQ3-;z>wdeb%N0U~Y`y{dq2edmNrk6M;# zL)Bh6sq0B1agCm}#0D2=_~t!;18A>2P-i#1PQQQzU=$SGD8>%En=Zjm7;pHpkcrp1 zFaOlFhBQVER0I$T)hV2!09emKuo)Nar#>YrT-t>_kK(f4_xDH%@VutjxxsAjl2OKO zDouLl5`f-gW;Noc#InURBK~nzB}577zqWn6x!yNFq`5-Wf>S`I9jmznpryWE5u&%) zb`Qen67Zt1UzVYNZI9u3<)~SH+;(wSXgM3qELCyelfEHd6nzeBGb{NZs(=F!F#K7o zZrB8~LCj7=%+uq@pxOZ1rjI*WYu-Yb$n=B34%=YYiCxcF(N~=BGi!)XC|(~)5ZBIG z6H^=>wItWPG93?%-}I=Zl|cZK74s1DV%|Gro$F2OLXa|bSbIEXC2#=L*|wQBV*6`MJH^S2 zcS=e@g3l?6Nq3IXDlPK^=PofZI=}NL6F?;wQ-5?9lm~AH9H81d3hlD8kI?P7KySno z`FvCbe6qfLGEl=0wptaKh3lT1s$T#BR9g(<{b?C5N_#ctF-!P6-XW^_fZLKuAjXa! z0`W}Qu|&%{c|#|}`^9@Go*7FJuI9VxI&93h7x9rDcbrTC6KoUb@C-$LTaITfUh0e9 z``}EWs&bSoBn2rDU;w+9khh_i1YmaUY037zEeigxRFLk{yGE@%WWXstbi7h8sYj@O zbqx+_MBUYMP&7I=unS)P);EE}OvL7n;VbfD>#Ge?3}C8|_X080fobhw?OQRspal*; z{HWKE)v+(@O%B;+|2}l(ax2DI>LO&(j8-l;o{VN&U%`?eU_Fucw&1#@-=CF(1|$b) zqi~XX(Iy){9GoTFthKm&L(ft57WQGh!_2%clySiQ4M>F|Xf#F8>7XLWa2)}h)JK^d z|FG=3HhcgEU^?=%ds4{|G)OBS*Bgw7k??_p2iv#3mN_Nc#}$%igN^A=7C$P1NWF>a zKCaft?`dxafSoRY-=lYX{S;6gOm}Us2q+IEKx#d@%=)wtVN33gSJ{PXj|o+E(7FV& zxgBlSL+m(sV2R4bR7zWp0#szo_{eavnNek7zV*P5VhmxH$gYh69(576n*4pg&d2NX zKvED@4(OIM4xknh{8;^aR;KKg{q(khj{5*~^4J6<))qf&P|R?no)bZ!i=E-}hVbz( zZ+>XP^i;M*i0E0MVefQRp+-u|bqmFbarb*87~#ydOftmHxmX!*=>x zG|L2OG^Z2>i?tyengafTJNpsUIR3Dicc8Ag$M#OvdlLQ+PIn*xT?74-RK^802u9NC}3jh$iN!`;0bP)`=#Ua zJM9k+eD)^{uQ8MF$~d&8h!C9+@&rl}+;C1dm}HBVn7iZY4K#VgZ!j)^d|4yzM}!xw z?Z!+5KzsMNdade!&%mEf^B|X3pE+cKx$uvd7OOsAggT@O1k`p zlaeV!KuHX4m6A9$ui#SeOSIkr(qs48f*hYYDj%d0!X)<>^`3wLrWHGe4H32z?>gwJX?{WISQ8upRRDa%WiIiW<7aPhDHSthLZROvFf-th zknY{`PTbZz0Z;8unQlD5?x!7nDAH)PmAXkx);hxke=+|h_i)Y1ze~(g`q#4JS!IXm z4~~w;4a9C};^yQFq;+%zv2H}NzU`i2KH#CIKP3&-lc&$3ej&(#1RaoWbwi%X@Vh&J zFSlhQu{RrRsR^2op-^pzvFP+@z7V(jZpr{YM*5n5NX=LJHwm;|bSp||sFy7w`}D{t$w*0ku;2$k<^;g~dfn>W!M8m97cEM6 zJ7l%l#V@n2op;#E)?6UIRj6wdC)|yUEkW}&KtL53j!|?X&^mx9n}4#%yU854IQ7Nr zEMfBjm;M9dGQAvo`FVTllMHfBhnA0i9^9KI0s0Uy-}#}(3DCUsx22F01>yoBqAE5N zA3^Rg00?ZfC@{=T{1d#m_I$im-qU}+C=(=dA6~VM7u_P^osg3jbfy_ut7hX>$Y$V_mFU@cZv9QwY9tU|Zr!)$3lx(wUr(T%8bH+_vt zkj5b`djK}ItFNhH)^d~4*7$fQXK&O|MLj|9F?#YjF?ioy-LP<=v$45;2V(H9u4xzI z4qZrfMFu2O4`Xw{z+tW#<~0ZtTFQAE5{Hwob4{(ANTvi=ewrBIdif=((xaZvCq|jU zMkD|ZTNsdy507v@@`2Jg{d2>=db{zU1D@WyR1YR=zTKRQ&7pkntD<68+fuciRFY^ zI|qC|cB#Vhy;X*5K1-KO-*!;HLV*zHw7q-Ox)h_RT>m&WqwT1#01;2?wu1%fc9(^e zb};kiLIa3H0lw*e^{8+7jeDoAv%nS-toH3Sc^O;^o-*^(3`2=FRe3GxO56m~G2eqK_ zQxy#6H4>c^>9iZbWt)|&YgA3GDkuQWSPiO9pB#k@=-mMYrTTYqkJ5Zx)v&@n#4Wr}k%_g;Mc_0Rza?^Y+kWe>3+jZl~cg9`DJJKF^?r5OPL}`MpL{ zhtiF{OEQw0i{YJ{Wgh7R{punXb3xyRQV3E_ z2M9F$#xVii-xuuqv;oz2Mo17rM)(}Bx=Y;o<3Mk?KMN@H6(Eh`K-8T^78G}IlxEcf z&FTg*AQ6bSPh1g+fp3w7naB)U4jje1z1j~z55X557a9#Fp@yWyIS3h*3oz%WJCC)m zr@Hr?PyvztK2)mz6=k|otHfsz8U2)lM7CnY1|ZNl z*$$R~0oL!$3rL-3{C;?TdMuu><+KXv^LYoZzAVU510Wev;$y!LFDOHG8#-+qn5n3% z0lQh_)6LMj)n*Z~4`WTblRE1$$QjoA9q&&}KWzSO< zzxRIN6%atK?`u7I_e<;nkbf}Q)bDw{#N~ApfVjTlD`0$ixI#OwB;`!NBq%PVr%a9% zyA6*q^-H^InbQ4tx-A}}k$%bdYc6g2FkT3pJAg=ptiPGe#dof3K_Ovg`C6B`V<-&I zrC>b92h3<+qV-qnHlqDs+q;k~t@p`u^Rf}?qQy?+U)SwIs@zpn!wO6iq;MRT`Z4Fa;MatWI^CI((>O-E*{r{Fa!I0}Z`te)}}w9Cl$~FDwUF zy}lFuR?iRKT+zfhoQjPR1hcw9f$e3iM8Hj3zwbP2a?_35flm!0H8(m_@WIR+JX*&75Wv9+IL+C6Qd49|3QExr|T95%`B{y+xZhrC!2{u zz}TeU|Jsx!Y5uQ&TTUA?H4d6ZxP*;fITpAq_HdLM>}ukB>;E8Ta8aQF7%BRM0;=ed?Ro> zOt>Az{F{_0vT1PTpKhZ$Hmq_{95l$<%byl69024hz1B8zQxhV)Lsch|i7bZLC|Gyh zhP2#C4|v`06TVtaC`Jdm0-$ z*wL8n>$fa)n?1_TJy&3}?X56k#=x(mte1;;25{uM<<1uPo3ub`Zx@!$CVi{WR z%whjB&K`U+Ve=pO^Pxb#^B<51q0@eM79lOO7q`1Rid(jQV=qZ@X zln|uZwg5J9e~PBPVlhGmOFg5dfM@#BMr*lZ$usIX&ITSpceu+bbyh~!S!)%Q72b*% zjv_#1fm_nRff#KZQ2}Z&pKJkC;Hd<{u7wuT`*vo>x~u_CW2dQu+CHG@9p9M7^)-de)zs1y`{mu2aq3Gr=mFm6s&Bs8?`@Uil*9JEk_w+8yP0ngx4wWS3mUPIttH z)<3!w?nL;h%y)JR;vTzR;pGh8CwrArNMZ&X>GI$Jn`#9(v?k&Mdd0q{AtyCI@ju`&wcoHW zeFI9!uwK*SD=K{_i{l`Gk9peCjb9vH7(ud2 zBRsl`9&esmxmk8RJwcT$bV|~6{hGG^8fx)$0sIj6_j!RYxX6|7ZW_5ehRxXOTMx9xQ!C# zCSLlw?z&+z#S3nzca(G9=Hbj#EQ&#cuc~S)_HM)RnfG{IOUlYc6cbV;v$sW_Y#G{D z*Vbt6X1`b2msF=pP^vjwaO}M>vmgM>Uqhaqn1gyLEYIUnz|ZF(TW zetXF#oCWnMhvJPJt-pNmI-{G?e(2M=3;q@S8v6PFGNt|RhPD5n-hFZ2a>H(N`>fua z#QtN?7ZsEesN~c3{Xf&<|MDz0u-Ey=7c?SnEi#SnTj13jYVd~I9x%vTP^F}_RMUJx zEKRH!(5s8KFeJlLvN#fJm-1JHjow8x|m6fNH6m?F$)b#9Y z*#n~FOU8Zp;jT{K*LI>+hzt^QN6d;Rp`ZfE5h}rQbQ(j@!K9#|)ZD)yQg#ET!S_Bk zM}FovSJKQMtB)T?e_(c?re=83V37S8SiPzw3|OnAUpue-xYQd6%JFG6pfwAQuat5X zYFi%i^B#AT^`4qY&>N1*E%ZGo*-b&_B-zuqWy0>5(lh>8vdc@u+d&9Y3vKdYYd4!l zH7Imd{r0b;@|)Y51Akw7JFFBW(M?Wrrm5?|{IXIoMHQOKH55mjtPZnyD48|%MCyBjB7gT0mNzrWI z2MX9sAy>Y^*^_1u@7g{#i|3?Hpi!{Wc???U{Vw|`)!A=k^D`dZ*dkQeg>H9EAFvLl z?|z9_a2olRWl}3mcqLJ774i8q8S8TYQ;*aPVe#_oAnh*e`(osnVTS%D>QYb)o?)51 zrFB+57nSYvpqI17QCQbM6wyMEgVXR|PKO@s-xSe*a*9CvJ4Pf06?mydrG#iLbSzB_ z4eady8449~lBRuh-vXX~2h42Bc`*z2kK|6Hy^WGcsf*zUo->Uf zyIJARI&$APvhfBV7wc5L2PpEaHi&5Sbs}m#K(1XeuW2G-28&Jxm9FPGQ)49pMT-c9 zL`#Fn;u0r;4N^zAWFuX7wsNgXu)wQx-$sPM6fCyie&J*Dr3!_#e00uGtc~#rlj!sK zKm0{ano2Aw(d%WBZmc^o=vG!&%k0gQo6pX^gVq6zQluQRZg~L zzau&5>qbOL3q!PFli0hXAf66Mw3kmY+IA3^qePvU6GUdS*OmxS4n9(t+r$qFWKKjx z%emA{vADAyeg>hglFhpBXST8vE|($nFKZRd^2;Iq*$Va2vU!62r=QuM4C#OV?dO5N z#M%FOmcjn*XFV%R`+uDC|Mu>M##!uIYs_btbdFqX#oj&?A&92Y)bhddkem8}XgZ@* zLY%5||ag+|r57ie3zS>5Q@GhA|T+cZZ95OWsu@ z9_!|-D;1FmN9_`$)n$3o(q5P63myHSmFCxw^rr64_<30i%Ub2R18a)QS@b9F%;MQL zMdz8z=EE}=9EeJc1sEY^yk;n3J9F2KsaOPuHFm!Ty-FOto1v=GUQv_Dy*2RMw4rHx* z+T2ds!0;blpaqWqwy3#WMbs5Yx9Zreyfp86aFX9gdeENf+^ff#)Kn-9VV8VLNHU`3MsG^nt^7nc``n{CsCEQ}+V;`G7tv{Ho5f%Ct(f;(V< zcQSX{MCj=lp&;vJF`L+XYRBlROIn7xqTJ-ybLF=Z`dW7%RZDAJm1Uyp8Fyl?@S!U_ zWE;dqSGb3l-do#~CG1Ks;~%V&?;`OurJJCkD)wSjRbg~vXfm?Ael2=-7^aEZoeG~j zyP4V&Vp+2yNWg7CBZq%r&8!^yGO=ji?vnF;o^$T!3`demd`v|k3%K7Vhq^!$uC?jK zz=^m@yb^%ntT;N1;_qvdqFUy4dt&j*8)JyEkdn-lL+sVR-L+)}*z8I9}TN4wb3 zL;>_Z7uvGQQX-JI-B=C8O(UKlnXzLS4*fiE&am1)A48@=x%$hoC$?0EN+xwq+n=bV zxfvyDZmMvl;R)f9*bp%jC98e9r*pBGFs#Vf8NB7_LB_26PCs?7#6cQOhDZ4m=T0i^ zy;r0hxz2~YM4*0z3x`{nmc3gwN_WMyKDQ0343yhs`mV;_zQGO^)RlJIWw>05JzCry zleY9>+pG$j*pY56pe|Jsu_lIB6KXF5hb6Zp0 zl*ekWUiWsTg1s_5D6cC6EBQkly6-v*jbV}+tG=mX$83-_9aOhD0z1XbG36X3pH5TG{>^$89Og-znzxP@z<+i|%--=Rurb zsKgTkD($d<9NonQ#!#4BSdmU592q<=hL=XND?X`6EA05F7($M=Au7<;=DNVw^+BO3 zfibJ?k@1%VUub?X1qp%tB8}t#lT3|XadJ3uM0`myfmV9VJnw)M)Bs&NE)-V4Ry z#z$Xf0HT!hl?6YuGvsu`nGGa36|?E|NJG>2_Ad$#wtSd+9I-p~z}pJxt)4EuldB$& z`qB*@?K51PojxjYZ8WK23K5aH&_2&cC0Dqf)G`a+*k=Q zCGnMa=(7Wo!Z(?-bWeu}zsMm_EDxN6K9v*#^_}4%$ldlF@-hFQr{< zu~!^xz-z)q!@uTE`=u-0TXqq>`BHPW%)d7Wihig@AoU+dCeX_vC+V!gMT9Z#lh(+) zB2Mc1nK>)bT@#`VNik2w<5Ov@C&JnONzvoKns=*-+%)RRn6fukFH^j4Q+1ld6|%)$ zI{V|!Xga`6R+@;$`jy{bTRVaqr;(}RqgPR8S(}*y>`S!PiM)i>JLWW&@cD2fb zIA>)9KVT+TOIuOlGd{ zgo31A%~H`|JgiFGBYPY^W?oGj#8S3R4jSy(11u{JukSYR^qs$SjB?eZzOJ^uMq3lz zL@NDcvA+A;R@0h>MkY`Cc6JXm!1(x0_F(S?x35lqga+3_jP zZwcC#oSiW4d%F!2ELV@NIX(8?T_`{$krFT@`|JDD%JGeQBQwy*@ttZT_lmTLK%4l9 z2CL`0-|y_(<*-`7kq{j4*{ovA@Fj)JQ9;l}2hZPuo~~c?FJ&ra7R3Ht4GyJOAelLH z7)O`51@BND?Zr6sLOkpqe{YQAdRblGC+H*5WuwF$k00 zRzy&9k+FsSRKwm+6kY*Uupes3TrRP?PilhSJD8+mK^ga^Ba<>kTt!5sY{j!d~4(gk<;Oz|0W-lvYGOLmvP+9D9A zMo;z#*P;$z$|}rin*2M67vT4G{kFvNwjd{8gIHi+v+$TF6`1ptt4{HJ!w_@UBR!?{ zBP1Lmj2|LIYv|PqKKs(ZaTl88LofKO=5ka1n184CkWR%i9SFP;a|=s!H9;)NK1;(e zz%e&BP^w;8micZ^nkdmpuyEOu)~)dVH8KdB{;Z-Q*37;@(1OKL&0l%`yUByTbiywkc$$; z(v>i6QhV#E1UOKMOy8j{RUv}^3!XmTs0Y#{8G{$+KP#Dd z{tNf8-o_2cxeY?Wi7}xGgaYCsOV$Xv5$|lfD1O!x_zzGO%tZG|Rac*&P#U4r`b$JS z2LHl=p@LFO>E34X`x&kbMl{;L@Sq-pM#RY0l3UdUdr9NX@Gsl~vL&J|DS*vw+ZUkK z{{@onQUXCU6)!W8a&Esm*kriBa1Ti?f9TcMJr$HVq!F6-CssXb*9f^syzFf9`^)`2 zxt#j?ko+arNi1?xMdW`;4xm0OF6YA!BMy5F4p%+#c%}}oCtHdULRbsb44xK~aZ-quH7im$3{h9OBF zIbJ+!;GaaFDuP_|qQOdM1*qNi z1!=R!z)~}n#+2rvo~0d<%UZsJY$3*{Vr0XulD8@GEdp z`Ci|UAf9Q)bG`2r&53es!zS0e+g8pqR5#qe;2zTLe!U$Um`b6`XAbU-_7@NNJ`aGJ z1g5wql14d+#|u=o-_2HB63pvaRz0+kcOZwr@ga=VC?T23N+0uc$apkj@(-N5rZo6$ zhJWD}BiA-To>n<&#{Bx!L0}#u$q!ZeWyIYz692XQxP#FUV#}V4bmpF_l!$j3HwWgM^+qL8K625l{}I;>TDIvG4|&rptjW2bIKEf^VR_%@ulc zJsj#ptYOP}a0*X5Q7%S-7ttH5Ib|X?8!|WfQuj0c`<~&?bmpN61p0T(S%Zd>-?2olxVb^&nExzOBeZiClX2}I=L91239N6q0Nk^WXIcIe8xr}Mc}C? zm2*cV-o(D_&4AK)H07$RU#^vbruzh}(ZVJ8I>dOJ_=UQjLWz&IfPW0`@zd0r;M4C}^0XY??)yYrjN zMwuBhzNi{aFpuuzyn-wtT!}X3m(?W!$&d+K&lD?lv_LCtMnj)b$dzyPNH-ZCea)=G0V5LDFRK-pk+6N znlJ3#BHJ{(@D$Pv$Gu+9V?G{IPNfgkoAkL^p@rkT$fhZbRckA;J1W7Z)-E()vECE8 zYzW(7;diuCbb0J$A6f>A*DFDxwQ}anS+}}03&8>DE^%}MyKk>maruo2|Q1VHm z_FcQhw#cO;S~Qo>i9Lr(wNM9t`jf7L=RM3vRDOo<!a?EC+w7VkW+ zkz3w~8xm2(Qg_MRcaB5g%Sa7W1ZEP)nkq0CV3jS5MEK=v?TfBbTm7oEQkeQy+o8-E z_wr)UrZnL z;GK?3hbRyGe6bXjg^!8wUsE-vyZtHeG@z8;mT&rQdrS+W3KLii_b<4QJBuH!(^txg zw|i(YN^)(-79Fd$fICeN?>>iOD5BEf1r>Go4sJ-DurB=M{ZmeLeua8W+|pM&`(g4xccAA-ZOItSATF=? zPk)CNCGi3MS6%GJ@P&vYE^7H=%5X1-ZC1)8&5LHy;CP5gC#+Y+FUNkRF&&aDN3=Pish}S z8zdrgn6@D~$IPf3fhl<7?~|T~OCD=k&HObgbG7607V8hnN3{#6t1_4tycJ{Lxs~AP zq>fRAbK&}AkdWHKq8LSp;(AsdHCgyE}x9?Nnl`pZ-4ciGBT5W>C#1@8LhW`tX z%;kP(iUto?88zn`F)au z?d0PXOzt#tH6Y)a$|!k9FBN1Oe6%Xde&6iWbyjwNdCecQ|4|a{+fUAG0wX78v(KOH zpSK0Je_?mO=8ZgYWEKy@M8ImFslb~FLx^0NU?gVqKNjJ}TB+p`x;3hCnTWca^bwD1 zpEn*%ianoq`Lup~Vj1UV?-{X&-hARopRV3lg%2VX*QYjry6LoNu7oXPzmL0$IHQEtQJJEQ1M1B4d1^Lse?D0y`(7ox zRiHXi*3ZW)e=}dXkXnoXVV#yc2b&sL8M&FnSadx0N6NA$PoCYtm0jBUA?x(fUCtY{ zuG|xdd>dS|`1g@@tjldY&wrWmmn+n*YYmU;6~#!=$$czgzdg3L$Ghz>o9-C!FHDu) zl>c+s*MzCh7l$zZxVI&wLAXHCP)SI>^}*-TT|XympTm69!KRw;1v1gi zL7cRL+Z0sSXak4Av6%upd4*VWlChYBd=d+8b3lipAi!)a<`kfu%!1n_(Ag{q09-MS89Ov6x5Zh@Eto3BCM-U`~Ug8)82lMpAlz>+XbAMz$4(B@YJaOOqb z3JcSTYZDPdrztRB;o4FJGX-Wp@(dJcmJtD>bbzNBLFXLNbs^8EfM%x + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Host + + + + + + + + FPGA + + + + + + + + Interface + + + + + + + + Interface + + + + + + + + Port + + + + + + + + Port + + + + + + + + TXCQ + + + + + + + + RXCQ + + + + + + + + TXQ + + + + + + + + RXQ + + + + + + + + EQ + + + + + + + + Descfetch + + + + + + + + PCIe HIP + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + SFP + + + + + + + + + + + + + + + + Egress + + + + + + + + Driver + + + + + + + + SFP + + + + + + + + + + + + + + + + OS + + + + + + + + App + + + + + + + + + + + + + + MAC + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ... + + + + + + ... + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Cplwrite + + + + + + Memory + + + + + + + + + + + + + Stream + + + + + + + + + + + + + + + + + + + + DMA + + + + + + + + + + + + + PTP + + + + + + + + Ingress + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + AXI M + + + + + + + + DMA IF + + + + + + + + DRAM + + + + + + + + + + + + + + + DMA IF + + + + + + + + AXIL M + + + + + + + + PHC + + + + + + + + + + + + + + + + + + + + + AXI X-bar + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + MAC + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + FIFO + + + + + + + + App + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + TX + + + + + + + + RX + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + FIFO + + + + + + + + + + + + + + + FIFO + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Sched + + + + + + + + Sched + + + + \ No newline at end of file diff --git a/docs/source/diagrams/svg/corundum_queue_manager_block.odg b/docs/source/diagrams/svg/corundum_queue_manager_block.odg new file mode 100644 index 0000000000000000000000000000000000000000..2f0751300a1d7ab4624b3644ffffdb11f5dfa037 GIT binary patch literal 17360 zcmc({b9^P+);79gqhp&L+fF*RZQJUkqmFIcw%xJOvF)T|CpUf0yWhRf`Sv-#@7{mz zSie;@t7?vCs@B99&s=hnAfPA!03-k)q)I541wz|L1pokke?M*ktj(>Bo!sq=_3iAe z%nkLO%x!JxTx^VJZS@_^9cgXtjBSi;4V|rxZJcOL9rW$Y%ncpo{s9OH3HcAe4~+j6 ztdEeKxuKJ@gRvu>lD>hJv7Cdki@C8YBdw#0^;+trEV2N~;6|bY!6&$E3;{Sf5h!p9 zG9?44@@ffG$(o3*FLL2q{2^l7J#kPETyAl3mz(%{L~b``?RHaPY_eopsdiVL6Dt|E zuIbQlP}nAm`CfI`F8AZ@vaVgWZJwTHHzr%9LgLV9K^o%PIA=n9k)OPH_;0?HcOYt? z?PVOBThCs@zg@M5W~@UaBR59s46DG=gs4=QRa92S)={Xw7|U2q?_6j|E`jIr;R+|E z3VD{sy?CpglY8I>5QTF5B$Hy2Mp+H%HKNTJM!O3u;9w8mwTJvW1^Q3&SoD@``SwBCV?8`p7Lf>ZX``L5`J`>v{P#`ONE zxwBKJfN=(_ZVFXNxO&x=S8W1L^~?Pv_?|&l7fe$sj?L{E?Cv@t5VZ0!2G~@$Ed)}f zdATSsN~stFbhhsXJc(9>S`;XS5|{xBKS&p@SaT#GHb0QW90ny zBnHkx;=fO@w76JIbkXWUoe5)voA`Xj-+R?LWBsCD%Y&3H*Wmxp==$zWcu;c1%RLl6 z;ND!()$Jf5Jy)IoXAJ)NCB}bdg8G7=F`E#R9zy)K%~hLg-UCEEXzw4Y-%g;SZ!E>0 ztOY*u>_Y#)(G9BGRl7VF{%evtujd(x@am?c|C1US z`^zylAK$@46vOayVm>FaU)lK~Ycrsd!acm6ns7GHP&$L6PA07f&*VTbu;FSM$M1&G z#uNx47}#*CZAo6;Mi2e-NqHsdqk@Rh(UgC38oWwl>{nF6TBk#SF=+^xZ{oZ=QPj8| zh>o*cBjlS~Fq13n*h;_3)%JzaP;~&^?1}>ZrCWM5xctnyx zSVpe1m*{em5RepEiYJhu0KgdaN00IUzCU63-Jk5G$x4xkpbY3DocazUp#(hG!kqv` zVAa)C122UUb3u;)Q|$!yVfc-vB5FrF$3;ws4Of1ficZGh!K|B(BPVv1&4h*Eal6fp zb-Og+J~aNS>+H2AvNaYE0IzICF1L1TaqQfs+m(4g{j)#h==!z@xkRj$)FG=`bNh8@ znV5(VccGEw%GV`hjQ1x^)!Ure%e8A>&4EysMcz(h-rz2i^PDnT$$CoD&YHo~Qm*c9-p>Rv$`}D_aHAX@&0@L<$tg=uE}-ikAfLqnu1XcM+rA6#RaMgX&FP! zr!1u7(glTuNBhVVErk&EEta&Pi z9q-2>MM_-&)$np#kL4~xd^)4A|!-XO$LBXPkCpHO;lA>o_&eMk>V80 z*EtoRwHFRF{&`b)^ibW}ntk0~Qn$`{lpN74JsUHo?GCcv4q+BWj7zAH5?N&I%9$BS ziW}04O2#k1TtDaw3LP)@8823XS}eF%8Ta?Sps0Z5Pht|_P@RnqXY5AxhQ?jGIzjok znu^3UstP`6N7Hz`EPde13{MLF6sr>W6v2Ua?9u$UpN26JhEvp{5y3<-G_nb>mUsAp z8NQGh;>Fh3MW3FY)=!Ih19_Lx&3*YOBm|OKzO9FU6+nZS5W0yL96C3L2QQ+2I?W=V z)Ln#SDwugeY$=$yEJmY_+_4D&@%tqhjES5QlN$6n37-GREKRqspP5+{2o$rzjjUL% zU76+YG=GQ+Mu3uBTISFH6dR@K76`jBL&(BnQMd^aMzH~cM-dkC#1YGm2{u|4V#I?% z%FSjKg!$tp5&z5^Ppq!bJ9o4NapU1{=o#DBnD?zziq%$(-^Fxeo^gFG`GTbaN$en= zYMF1<7c(bzwn6*|#-4nQm(3ID6!67>6TCcTxl8m4#_oSAAwrl0D9Pn_v^f?zM7Xmp z;+!wQu@5W?&-WiS0iO`qI3|~l<*%heih=bM3p5lEMEUu}?2n=%#e}2cax8Ls)zRC##RbKNBL{DwT91kbzaied#Xm>u zQlh56^(!L)8$SMr0lL~B4=6vo>?!>5z}H~m8RdU85~y9Er2hFJQo8La&*~ksqPp~_ zGm;DYFP##r4XzJ_1@Xl#p8#sj5OpoPDF3ME}|bdWA%So9{KYrzGe&$x<48W5p8qCyK zwfD6X{3dT}&q`V+S`5{Y;}Z8=2lP7Tu=T4q#*iyyR=zly+$-GxxhjgILR!gl;Se?(S$xU>$|+~D$zHM zmoa@w=%k2*DkTO#x>`dArUQqH7WcJ$%0zFj@d^=rt1EhuD3~ zQSxS{dMqZ|t?6ksDOmFf`D@gj+93hYwVJHK!`l1*$wFDVTOYV9+=^(!6_?x7)H=WXZZGS zeFhRO8@)_M6-3eBwZ3|tt~+}=#vqJKWlyx3Utp;%J0)w*AqrfiEv%3EaB)2o6om)*QvWGB5gMFWKTDi9 zq!4-p$v4{t+4%hMJnK9qh2W^zSWAwYywQOwwHP|~x{rD36U!FmRWYzS52_A^$YhSP zsh_nFQlCd6L1pPBPtY_*k)?EAM##4Dm9S@xVL%_C9gJ`n0plbSO#dL*a%l~_!k&S! zF=A=d5x+}u+0Q=$-QjA5TR2$+4l8qi!*}^Ygt|7yhg`31Y9$+QN||B{Q@_;~12bpj zwBve*Z0i2u1b7Lw8!61I&Wvl8e?)?oU@~t$oK3Boz@m}Igwponh({+$nQc)bolcJ- z1UWvsK`eLfGS%=xGI)_hE}wMP2Y$*@m{iHi77|MUWgaJ_eFu`A`AJ*h0uLBGPAQQOisq~9F(zH$z39U=g#k5x$`#H_tm>~2y~|E7h%CN)vD#-N1{*LF_g8gsF5P1R=4Bl=@voFeId`p0I8 zwaV?g1F)JQ(@8j9qU=s&*_R)d;K^ar{%H*pZ4x{6(n9NrUR}xz^Ntf$=sWt9{-0Ck zGPa&nY{aci3G`PZ*2C<*qJ_@VSV#c)!italKkJF%FxrU4J%9Pck>>oq&^K zb~<_muEx@~@bxOKQ|KQv;>r+kU9N!RG;yZKGO0bYX|TnbvzafYqpUQ+AH03r

K zxPP3U&=&p*wj>m_m@dR@7(4~KJ_~r!dtJ--R4LRVi&^(T+buR~Ee5)IBeP@P?6edT zFY*r!!A&}+yl}z(W_nHky8ojHoTYde9F-&b~5RSnMH7Rey9!cw1GG3M1F5GSe#RcU`Ni(nWXIs;!wLrY1wG zxeHCO?OcjYr`Fu`jAn28Ghush5vq3o@9e#H#@6I3o^*Ta-A)Tp6^l*iP1k%|>&%%M z4U!3__)!6KPovx_3CLsN$*l+jswk)e&46Sd3~Q(yl01sAD_8ai$Jjp7_8bb1ys&rz zpI^<5!Bfol3h`v-fsTR_%?5R4n==t0fCs%otvS^fee7?cag*$ijSpH3NeXvC4ZSM; zR31>1R>bib{mV+S4z_(sB_7El_VL0bf(Wt-aD!`6+$+Z7c7dlJdEcP+^}_Mi5o-5* zs9E0`C*X_R8@z4GzJmR3d)mIT@BttIfbL@r z`=8gZ(7$IbPVQF5|CqDvX{|V{up@bGRO#7U+(+z-#9Ev0NR1clE1;S-Ztv=66%ruD zv*$pfiK(l28{pr+-$^!N69~u=najCYMGaPl5<7t$&K|-{f!^+~v`Jg6Dd><;x7EHf zO*WdL)lv+rsNNMt@o^;GHQB!31@_w`&4~2%o&}l|D8@g>qoFxXgnlPs9H=~XR^NwN zFyFqbV*AzJLdCbDj2}yyM&&tL2SzNSK+Ra2)1M2AzSxjyGW{zS2~-R6(a5ky zTwxX{dI4&QEJ!|)axQ3?gT_Z5)|NAUT@>H4KaWD0bMef@v`swX+jiw@aC~2j^x=70 z>)=}Js?*)CgXh zM>-(#FOIJ=ggZv2oak46=Et)lUy)K=vxGf1V%IULpxiaVgK723^^I&OLlXPynXZn# zf_SVwHq9w5`L7Zjwvw&u#8Ru%)7JOm$;F7w_hw+M1JG_(5*U81C`TRCEIPrW33YYb zh+toL0*j5-OFyrpmLS*RUG4-FHmsGf94e6-P88b3mglGM!BjNyu4<YE-h<6-1GO=Dp$V zk*!(DWfA0Jj$O+}={z8cvR2+Nvjzozw$8XU1jpnfBXlOhD(qijQ%EXZip`R5r+U@~ zh9TZ8S)K`_c>7bnz9zhM7ovIT{5Zm^18xm(NC5sY7FxT| zXA!rPzN z($wCck&LiJAp2#@5cXfn#U#2uIdCt)Ry=Fi4<4hU%yMAO6dQDMGh0~wIgzeU${&B4ip~lzDj^Jks z{w$zwM*b5a5B7nZGpZHV4ex>1%Ov}~(0s&2NKF%m%#N5d&9z%whK{!5sII$1JuV&1 z8ZteQG`vX~HS*W_NM=b(Z^v6nL=CEkQ+S2mEkQU4BwN6@Lao7j+=FI8jGLO(OCAuC z%sq8qQ)%ucc+nzD{AGo~!2#z@FHDSICSATl^hRLJF%~!QF!EL^S)iSKmlZAO%z|SR zMHFCfzIQB)U(IVZgbiz@%RE#wLP}A&cJmwH`y`!WhwSp_VWn>zRy@CzDdXtLJ?O^GQRM6 zV}FI3y2*f;av;zs&B}p0LX7h%u5LHJJEorhdtwE1y3wb8MKl#)Fscc%&w&i$Fs01E zMkH9y2sY$ssBBsPt zX1ydpaD+JZ67VNaNBd@45Ru6wme%iB;5f}D(LYor%1w&Rb>~?MQcKBRe!aN~Qgi0t zR66F7kSW1tD1JWXRkbn%*c$Rtapi`oK4IX=S933suIu-auj*=p%*V}`Hb?dIW+t}iLJ(SDu24^W;0t$TMcGkM zV&syjDRxsp8nOrvHJpe~I@}{Mm+;#4{w9i<&Khjum^(=}oE}f@n>+u618DZrFM`Kt zu-@o~7Bo?3bZ*9h6+d%h$_NoYf+xhy3EBkmK&0$ZpGmgYpxU}MH@h;!x+=)%;)IjN zmlsN^QV^;Wot3j=^-93v{KgC?tyADlizmxh#Vn)Cn;Ka7ai08$SH&dryq;HLc6R{% z6RU-@ro{`Qs;3la46Y~x6*#dd_!v^2Wu83PrHs3ve&r`nfd!heFH6F!F$a{!_kARJ z_UQ8aW2N|~7%s!F{!y;S@)9z&f;x)BYF{FlE2R4yNkfZeT(HE0bt*YlD(7XCGiHm} zO+Md00?EpX2y%;XF(*~sFJ5-nzl@gN5w2JmS$E7s^bz-Is%;cX-CyX{!$$y=bJBuIP`2+n#HEM`{|L->6~jf9lAX4UziIN z=(1Pz?pp{X38zwX*Q<>4>VqnYU?g$@vG#+AI?U#ZC5FsnS17#@DuQb)X3a(25M2lr zIMpsYq2rv=wPp{Wnc7vnCD6@RIkU4>Ra=!*Yc0|Vz7*Hyow+N-yt*+$acd)dtBalZ z?fa+HFMUdr3$))x^{H`wj3j^Wpg9bEU*O6hXfs!rrS(%GMTQ$VeuF%@dr0V?C?dH> zxh#G-l9d&e*J&f2#@)bwjSWbV=t_&!b*X{9KWgjf{2{I-$+zDa9Z!iGYqeQL7|^ z;2td<#EytvY}*Y@j$KlbfGOGKL&Rs@(D zlo4<>c37;n8B0`$E%nRs_8Q;~Z4jA?e}YF4gzoc1dNmF{mg#X9_XD-$r9fB1yRuYG zCp9~uYy7GYtC;K^OX5LVg}qCMpyQo(GR5`ie)Zt8ZxX};jHP|cP17+0!#)9Cbp0~! zK2KVuK_3^vlVIZWD*2aXWM5U+0%;$%@*Us%FU(-i`N=LshsW#nP=|re+=n@Gg?3q3 zV^Z1JiA$xPb7aS6YU4jI;f25YSv&AGLWt7X(_yM!Yulo#R;yjed|s3QTMJU`!YiS* zjA~I6uPd(8o%9+p7GRBz zfy{)y9&s&`dg0Io0s02(S`2INt`LaJY-Q z=NxaBlp_#awy{`B zNOA0C`xvwh!w#|->(rWU{McjDq7Idt$qiyMa~x@bNLiyzV7nj*;~g&>deA}hj*&n7 z8L+IN^3xX%ZY7aM=y{>R)b*E>sbmcB>p6J z6)S@B*HL$1aD=v~eV7D=SP>C12d~*8fg{)26AHdigSn)4c6J|y@O8IcF-atE9s3vX zh|iZ>kuAU3!L{{D0{VMJo?K<$;)lhVYELper7^D`rxjUaaVip2K*JUW5)P=;z z(hERvuq8ri)@z21A~tyX2on^1MR1!`jt1}GRoP|+0vZ;yaINhdT<;N_j$FR2EUvu& z!`7(tbN|sB1^}p#`eWAqhg4GoR0j6agM{cq|0_P76gPJgFor*C6y^6u&Wn|^rv1gc5Jfl3QAmu>Iy8K*(p3PoLz0Mu#8Rlcv@kW2ZUWfZ4>q-YKu*(Ej4{G9 z0TN#^v>js!9EcG)rf&0&Snfx6?%Rjg)?bd`Zn!sqRB4^|34;~ZedR;nBkfKoT1G4g z0~S`PrFY6MF$M@mB*20T4b@R9?C&y0R~BRNbsQa0v<0{>`8xGRBYmZ4f~6z0`x)B)cW#&j`d z2P-p{U*ZtZbWv#%>zG#;1#77-r7R_<`fQIhu)cM6dD@u})VuRZC~#@Sxu-O@X(n{z z3G2@Q(?!)C122Bpx17ZAgOE)YCUx2-;FICH4il`)ClzwFmeyNpRC6(*VP+k?Z;pLy z?V>UbJQZ1}fuz=z(5dP<>Yk$2-nkJkwDUQ=>8UH_JjwhX5v8XK)8;Tb`s34AuIjn zgo@oDd^$9xi~3h+B^mrCUcIe26V(K~($;w$tA0VGLC`?#J%!CH#VV0s{WCc#5s6lHz`KIdwd>laeH7JJh^yiDO~kmtz=V5Skq+W^NvuKF2`I$w->*VDCO%ulz$45*1~SlnzmgJh^k=% z$Q}C!%+QE`dI5g;et|TpjUp`KC>UAcpa-aNev!z3Z~i2J&GmE$!|x_+wYLyqEKL@n z4r#^5w&3aQ^P>h7ymK@@KF|F|S4U`!B{O$?lHlsyr%&V3WmC4s{?R%!zq6?ngg1tg z)3O?6fCEw=lTXC1XVQl+L!atsNx;v#2%<`m@3SPSJUyI$cADyLUH4vK;r?FDS?oi` z;9A_Hc&S!dayK*S^h#kXz>TsX)aIv!T%e%5@(B2To%EyH7EuKeJh-{U=^_g{0dIyJ zkV$|=TEK8o486pqRAglfaT;OS0VMNI6nP;iskP?aqpKS3Iq%Mb@!)M(y6hjn#|Bx8FXA%0QOSon6> zW&@aLGihT~#jSsiS?<_8_lDM)3Wp2B3gZ5faYColVd)>t6BMkEE||m?l6-SaBGp9? zW2_mYMWq?xr5 z^C`Q`JQH-`%g$TCMg(MzZ=XMrXo9nSD+r^H9+u*5@!w%Ed$yFZbYy z<%*M|eip~i?6Sq&aOXaaU;vDb)aLxt?nUGLgQ}2sh;t!@oq7i=$xwH$ z%d_H&SR;Edy%g08U+Wd<#XuS+H_3?+$MFM0{|Do0&A#duwqcO8$gYd z`iG<7zSJ^JI^e9x(6xbGuIwx%$#w^NtR?wy!8=6-zte>)L^SUbNSu+%!V_?0dTuTT zi^3~{J~q>Z@V5r5IBcKrwBvD zUA95p5Pkh1tD*!YY*Bi14$JbRmx;fRsfrU;ut9HaJ}WDTT|@|ul;uZF<9q3pWU_kZ zML$X0*3S`t*D1;*!+z$85%W|JO)WX!wjA}Z;(1v9$012Hr)L50>)HPtI<27LI`5Vj z@S4`oCaB1!v=el%Bv;vN4}B3^Mlx{Wj5|3VY3=)27L`-fq-^~UpxVTt5j#LK7djv^ zc@Ty=dBN=Vp$7bAOdR$Vh*H#;j_!u8SN7Fiy$c#h{@4j z^o#ouB^F2;c+k{H8R{4>B#HIf(CXULf}Kw7-SH(x6sx}npM1;o=fL>x5?~tXz>0Y2 zM{3Xe4RL&kN|TlkEPAy^U7mSpR%b&XJe?g$v^GEN@#5=9O~wR01EN@lI;+0yfly+QIvI=i;R}X6tOICFFD%t)r zwZF067S-MGc=)kU!Y$vl!W_Ornj6-I{A_TmBO}uKlKK&Ev|?mDtlBvFKY-WO?h385$rD5!O&{H>MYPAQbS(w5lBOheuR^fky~Xlx zq0lM}2f?0@vJP>viJA%ZwTj}Se#sp|J)h0x=|wW8r*rkpq4V+@UOXc)a9x3uzK^Vz zWNdi9(L=wWSgDNdXcX<-o?Zii@GXl$79=2Aa1kaV?T1o}ec{+arDXt{m6Jpo+D+D& zqAmfASs>`Imn=lO;apa2KsEGg@i@A6!F)>w5Rux+)! z&d}k}r*~&v5ml3aPKr-j&4* z@jOmXwQ+71SMtv8m(~!v*J+6oh^RZ(`qX(t*uy-fzs~xBUa=-E?b;~yPkEA5ts3eo zWO<#5j_@wkcFXz%>ha@{UcG8Mh-_L%7fNCTvn;=-6 zG);`hdtR>0N%b2~+}%pUkF7^|!oSrTNBm_s*cmtRNo*hr^pqIbs7f3umK?{+*ly`u zdOMCPQAC}Q_&p33lDwNH)M^z0{zoMT#+ty2c2JTw{K}0#8_V}&c}+t}MN}unWmnN_ zlA^p;g5ijYv8SVAvcp8>VZC8iBLp7YaR*g9w@kn3g$=N&N5-d{ySFqU1j#JVvZfJf zLB(i1pq1 zlPei(N~3_ITmfdg)Ot6N!2n<<_*+w&rP600+g6tXXDFF6la79;gP3G)K{G zy_J`BnKQ*DlWWM8b_NSHFdTxdY3?P zCwX|kIXIAcjAJ#+P24Rl++c^uiSOj$Y8kXiVUV#YE7I*~keOA!F+AUEL(GLj>lYHAM$1%(2r~T~Iq`{hAbPjB)=F+XGjZi;Puh^FAzAQ(P`>Fp?`o4?5OaY5+yW zn?}@s@-k6a$b?1!nTnB0+;0d+c+@i5bTJLRA_LO~8^Z)I&lCqE&|C6up2s%7_ zZyzR{xR|g40Px}O1ONsgK|YSq7M0>3CpS4Mr4QZ~0398joSdA7hK7ZOMLo(b3b>6HvmWSMUCtgNiAuCBAQ zvt+Ba>u9X6uWxK@Y<6~bd3ky4Y<_ol_k)JGd9#0bcY1Sk^Y-@kk>dONd!N?bEdT%; zC@w6ZHB-a1T;ZvRST%I3P4xseYz+Pxcb3`bJMh^>nRoV zC2ePRAq(NM4^WHsT`F_|p+}T|c%(Z(HI=D?WxNWun-HKL5bpQmh9R{$AzsB7I42)K zrK=4x!2h*=y7g&mP)+dHL=Z;#EO!cu4qmq!o_2=gdzmD=Bl<(Eoei8DVypsTp@HKo zg`o?OpigjIs!=jDW56^i>X)fnA2^5nXkBQZi99(hQNed32)ozMPrI zo`{iO)FRoDCT=CPoAqEl_NCO#z0(YUb@UR(m0A8Oml1_SZj+pk37adHlL*UX`Xv0>tqklFbxyH2o=F&O7K)smgO^1ROI zVkU4CIR4Kd+d`N^7>3+c-~<5WDg~(pxLOGf+S@6u3WJ27B-=%Zl0XrciDM~!$Wr7p zSWaa;UtUYv_!;>FeP>6NAUL~!4M1XM&t2p~NGy>c3-(W_DSLod3YKBTK%9xA`4JCMwQHXGdC9N_cu4 z9_B(65o8b_YYxlK=5r4r0E7?g_mU^@YgLs}iv0oFG?={_`h^3P()ukK9 zS+cW1@(GFWUVAf$EwEPfx~sDt;h}Mz-HOxnpxM^FPK~&%Ilnbyb)tbvfoZ3u&Bryq z;&$3U=BHhMa>jOe-_Qx z%jh*GppcV{I!$;qNn%i9l)6?Yn->JoR!_uQSLma>tus{!y*TAZB}3TiQV?uzu&&M;pg6B>S_6gMBBl%PZ`6R&5KZt%F}en z_T%^#dq*lN_Ci1&98OYuUJQ;cj4gTS?56|Ppi!6U$5NL^WJ!<{Rqyub=*bEoj-~2( z!T_;?1lka^rt0}0)N|q-%lX=2oLQ=Ci86q*b>As~Ld3MgCj)%*MlTW7lgM#V+_~vQCZbiv zoJRRpQ^PndRIG)noYkEr;v0q|J9ilew~gzsT8ZgKx0dk{?Q=X0pG|^rk%bHbdx$AF?zjcSEPxND zSA}iO>}fp^1(DdNKDFYe5i^s`PK%#32_oUsBn4my5_EtHMvbTx_LbMY3s8bz z7-sQ?F|m{>8ayz@0+9}VM^WL~{DseanDWfPF@lWh(7Ejp~av-v;ij8o3tSK3&xC;=wJeJMDawi}=cz$mfCIALYY2pgyGKz!r z@IYRxw7Z$k^XYxZMHx_Y>npM@*g+jUo^Jnvi(FvVU|Ygvs3c|pN}#>mmz3ti_*>$j z3#iREg63IeGrqsm_#olu9-i@rHRBitPeL$7I8{wsYjM8%=bueNXx3m1|P2N%!3R;s?q-kLyg3j%K) zpdvdu0VHVD4O|Bl=G}rvG(KH|bI@AUz{HiBo!y9zxcY@mb$xpkO@HxhHhzi-a!K*U ziPEP@iRvyUOj~WYV3vrGFnTla!zzR(NQwe9Y`!>(M*M19i0SVB#HlM zuR8s@U)}Ijl(XapT|6WiVP~^@mvkNlAc;)L0=6@8bW5zGePgfXxx!q*Klq{C>Zt*^ z1(`3bRMdYRf|P?OP*Va`fh#7Pj2r{aK7}s24A9oIeK?lQe71d5IXDr_+f*>~25i1$ zX7gl$pX$Zy(q?frycXIzcha)>->B5^09>5}Y46gt^~|#;OAkBbZ8gmMr%8tH3wsaJ zAeSnVc3%vFsDHL)Ugf~EcVW?a{h&1g^l-->UUol0Bmbu z@w=Hqiy-t2_FurVMSj6z=?SU;v#WvmT=S^s&yZrt$i)hGt$_Wa-E7~Jngj>P)Rfg& zl}(?1ciN(C^AEOdb4pVqy&HoL1?Cag$NjO$?sprp|z~Kw1 zJ6bu%KA*IL;Uo#VlDA7fg{I-}!+c?VdVaFKS*YDw(sLGYjx1%6c6Ly+Qx0WKMx$*r zq|B5AdVzBT*sKVC>d0gjms)#{E(LN4qE^sC#j<`IvUU3S$+P9VoDGrT{}Q zpU}z1~8r~tmJ%4HpYgCSW3^V@gr;B@sOLEvO^^08_rQ6sC)mxrWzn#4v6OotY6cLz<5tX zPp)+OkIoOfjr{8kn(b6NqHK?^(v0ljyTnBJvWsA~?`F2AJ7;?Mz!=v(v2P^f28?;TBol4J-QhsoOW6wVOuHUxEwVDFOJ_!LP{vo5)VSNScrj zGWKv>_ULefA5Q8`{iW;6Cl_KnqY`)o?@h-BvAqigkQxMjhP<;fgDB(*nch;20&&RCRk&cjk_X%iJWrVi@@2nkibz zP38bS2!=jL`m_lkNt5M3=jQ}&ECYM>9KpKo$>)qZ`7pcE)7R^tST}N0+J0!AvH<$p z*xsM7eIAgm0rx2>dhHi-&qz!?X(iQ*crd$*Dv|{V2|9S(0R`cDbYR^!Z#gyx$k2ng z027x<6a_kCc*l#54-b&h4JusCUNw@UENf)q^3tI+EMvMaI*Yj%8a7_3_2dA^b(r;;e^7 z6$g&|c;UhOnI|{JH*|G+q}O@-DN;6jo@_Qax$HvF&Uw>b465IHB^Q*JI9%dc`VuEc z1k9NBzNyHUl~S4W6CI0wSZ8Ju04|hq9p|Sody!mZpqecf=Q>bElnfW$Dwr|BDJs5b zJcB#rDaizMB%VjWMQET79lJVEmo^+e!9cr^?|zXQVBm$*N~{ZX92)$zL^qd#hD^a#fm%jGIo(3z%Z@hs_4-KS#Uu-O>SS(@N%G~8;!EM=F| zNzvMOpx=vMRT|??H*PT-TuK8Ey@^5bX zpJM#!|3vxY zC;V57*l(W4pMpgAFRYJ$e~16wVD_gXF#UfvpZ$UINAiE|lYaO9e~Ks1e;5noB*8wm TmjD1rAD_05?%|#H_tpOcgD$^b literal 0 HcmV?d00001 diff --git a/docs/source/diagrams/svg/corundum_queue_manager_block.svg b/docs/source/diagrams/svg/corundum_queue_manager_block.svg new file mode 100644 index 000000000..8cc844264 --- /dev/null +++ b/docs/source/diagrams/svg/corundum_queue_manager_block.svg @@ -0,0 +1,471 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Queue manager + + + + + + AXI lite + + + + + + + + + + + + + + + Descfetch + + + + + + + + + + + + + + + TXSched. + + + + + + + Ind + + Addr + + Op + + Act + + LS + + Cpl + + Tail + + Head + + 0 + + 4376 + + 2 + + 1 + + 10 + + 0 + + 5 + + 8 + + 1 + + af34 + + 0 + + 1 + + 10 + + 1 + + 9 + + 9 + + 2 + + 324c + + 0 + + 1 + + 10 + + 2 + + 3 + + 4 + + + + + + + + + + + + + + + + + + + + + + + + + + + Queue state (Block RAM) + + + + + + + + Startop + + + + + + + + + + + + + RD Addr + + + + + + + + Completeop + + + + + + + + + + + + + RD Data + + + + + + + + Delay + + + + + + + + + + + + + WR + + + + + + + Ind + + Act + + Com + + QI + + Ptr + + 0 + + 0 + + 0 + + 1 + + 9 + + 1 + + 1 + + 0 + + 0 + + 6 + + 2 + + 1 + + 1 + + 0 + + 7 + + + + + + + + + + + + + + + + + Operation table + + + + + + doorbell + + + + + + + + + + + + req + + + + + + resp + + + + + + + + + + + + + commit + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/source/diagrams/svg/corundum_tdma_ctrl_block.odg b/docs/source/diagrams/svg/corundum_tdma_ctrl_block.odg new file mode 100644 index 0000000000000000000000000000000000000000..ba96a84bfba5332cf4db59ed5227a87f5fe96896 GIT binary patch literal 16119 zcmdUWWmFx>7Vg2_-95MkcPF^J6D&A^g9UejySux)1cwmZ-Q6L$KQi~uo09~r7uf8-{6UkR4SWu>NPWlwL6D= zA|gig^&2@Gt!KfMLbe!qfKK=PcRZd2+4iKlMtba8-T-v3T4dhy??Z%90${FneHiqs z?R3k0;)x4b(*3u{hc^|Rd4Mh5bQk6>KJX3l%5K9DkQ0eMNWa5O&+e|4OD^J;E8b01 z{B$`;EJP>Q%$;C6ga|oGF!-R9DEs9mC(UCyNIXP-lby;{HqL*QH@15n5D;5qU%D?K z6r`b`tx|e6k3j%{R@gVI_`kOqmVenys}z_bGGA&hDL4#2DXLEbF@o zx)^a^UoluRej07GwZ#LZJK6;5XAl%64Gm4$qh)oc5gO7aY$zwQ)8l0x7PH;cvXO#> z5my8<fUL%a4(86+K8b#HkQfog_rSu$J9nlJCIcTvC0QVnCn z4FdVtYL3>KVR*=59X$`pW)JwgCB~@9oM~n(u2cVIG+0YwupunZW6E5VZs{N1i=8nk z^^KIbhx={H6$>;fE@Fu@0t-wglCSrDeZEA~3t-8aNa4yEV`8F2m}gr#4|*H2FoTyh zetn4eNiTCap2Ye9(^RX!mGQHRe9allPcPp0q9>RtU{mJROUizRS_;*YO7i$jTB%{n zj2nSMOEhZi(3I4y!ZaaFcvPd>aQGZO=4nLu+)zjFrs)aid#-iGr=Jhzn-4a|DIV1( z?(o^0$063RcE`$;AfnAdPEFKVcKarRtxBIevmW+1HVjtou~BuupkF{oh*OThLAAC; z+KakCe#nh|7bqet!DYkSN+(A-f&i6S^Hco{2mfDXJNr#n$e^jgv+fJTUuNSzX$*Px z|86$SzszPOxmq4w2)%!8RcM%k_y-IwLNKNiE+SrdIAz5uO!tO0bUb)2M8F4hG+J5^ z421l0i1g|S*(apqJGcsp?7AwZYOTGc^fRYqHbH_6IEvp z)#%r5`!pwbHX8CGnou(q=~>!Imms3ojqyWEkegF6Bs)j7NbawG2rKO5T; z*3hY^7NA#BTDCWl>+k`G&gTZP4@D3r$(5(`8+R?9l{ryWel9|Spn!fDnKvrarIR-w zdmG;IJioz!6;x||y`%TQyTsKCwCKk7IRcLc*$BK&4U8=p*a-4MrTbYk862aX% ztUkVuFxf((y1+zhJkpcE zI%I>w<`}NN)kGy7tV+ax(g7VjAPE)}VjW)c0oRK$B&WT&o7j^OIf4JKQ+}*^xaxNR zf2HY>%FX_B|vcOIlXtSDI6lZP6sM}&AfY#tF&DLY`+N;K?r zveZ>(Rx8K$(8XAD+2QRUE%i=sGld#!BQ6s>KG<-vaSfBcsz-GE`ed~W*zPRB0023P z|EH1u{lC2fMzA;UprKsUFF2ALS-9KB?~_h&=*Nko1|V5G8#yW>P2bnp-VgMjw~+35 zWOXPaVhGch=Y}6AbahEl@z@4WS$KHPcv!~Ti103pRV^n3@s;4Ox@g5${AS;+Gr(?3 zc92dfSS#TD7>i)J+2#BubPG<>O;r((+0SLpid8a06UK5)Vw}m_-#7ja9|=sJzVo0c zbmRotDclOm#+ZuO*dQtk%5`X1JHU!Gxxs$uYnf&eOJdA{mihJ^g?)^t>ARz6Ho@_e z%x)RRPn2HN@lrdfN}MLTpj#@1bV6iHUV0(2FEG)A@a;>{iz|!#^Sp~IjEKAJp+S=e z@w-6$l-dMcRwc_CG)yVYc;j^?)s3iiU;aAVulPx+xhMwtA~zfifmO36RhX~-%lDP8 z@(}YeXX;qWmD*iRXtCwv-)~;0Vq$F;V!pN{$UJ#Sk1jtgm}W^3F>lwxSaO+_o2d;8 zkLoy>4+o>hY>rU-D=4L|s0I0?EQ--jH8LtUB3)>{Y#rVubvHY!9xM9BWhD6dtFgYfr`G`sCFDu z!*51#l&tu2_wmu4Um$I+eC30gnVG7aGn-0Z+=-M~znU-#r39zxnkEq}fA*BFPeN|u zC5wDn!1vGNT75k<@LGEdaXTHbGsvmfT;Cb+YD~THW?gcayuM}-+pgX^d{Z!)olFo6 zB!yAqxF+pNjXTOxEuI@2Ev@|Q3Ec{XmJX#Sz(*hcj1_)GhxF8>CQuO{*a5B^Kk1#V6v2Sm|vDj&gSK?UrD=cRALXR*}Fs9gCv_higoZgC{m1vg<9%=n`mTfa;kAxDuZZgP{kzL~pwW|cu+SQgAWA0AMa17f>H*g&m zTxB`dD+q0k#_Mxm8@!g~x-0Dw<|D@$O9mqCxRVc&bXAAen`jUnnL1f0_9%5WyCVg7 zABb2o-+j~kLhA0nZyj>Dq~M%W&a4~h!TBVo1N%-dRfC_CNC|S%$b+f*nk8Xmq?;WL zRb+*E6snU6Ovt$N7Is;c9o2l`;dq@7vxaZ{{#y)G;%2D+9P@F{33 z!xL=nqZ{=VVoAMqV4}#XMLkbl^rzYVjhS%aCLxh~)Ui=;1uWKVJ zYH-KKbsWe{V8@s!@_dADi78%DNWblIK{yeyH@XH>fRZYMpJfWlB_-V&NOcE*X^!?? zJ2+3A98+Zq2>PtHo7A>NNojxb%5$EQLv6sOj#3wmeOBLxEBTx)g65JNq$gyyvGQ2m zo11{rT5L>IVj)hhg4S(^b8h*(x$=mP@Dzokpo$BHhC@KZ>{JWu_?4V$bvr*;_H_(6 z68LFS&oNDPXjP4{1;K#_C zO&9A=GEB@aH}Z8Dt)fzO>wGdK0$rN7%LVCt75PnEizxOoxzKk*4*K6z7kgr5Qyi{_ z2uar~B+1yGAc}H*a=*Vr&GeSdPRu?XS8MX7-YNLJA7(d_TYfo&Io^0DAc?97ewpxC zV8i1e0^DaL8Pj3gGK*R^9r64jl5WHIEzz=Hx{c7S!V{iTeGlvx> zgjImxQ|M~;V(k1Vw2c)eJHDQty#}J6=ibqdp9!`Sq~|khp58GB4mMhKBXG+ifp`Ke zN+OH=yy{S!m?oZSvl2Hw`I|Fu`;wrv;*i8N7W1HWE+>2W1C???40){(?!dR0%J4b? z4`W=#|IkfU8!|)IPD0`f`-Iw9Kr|ocrWuFA?zEiq8a7IA5&XtkmE;u-tYBb-snc#Y%FN4v9x2_C73DL-=yF^Ip<3L#TzE=! zrpH=6{h@Lfldy{EZLJkx1iJ3jblFtyDDBh>d~_kgr4M>@KJ7J6Uey<^9}8F7104#- zmLuXPJ8Ut6r&t5F4t!xfH&bTS$S64D;_>nEdjUNGj`by{?bDs&%hR%JAtH{STQ?|| zQ%wyhPjq$`2FMJ*J=K5=)uQaGMXW?VsK;>$afUN}rCLPkmgrUhWw&h0!9$&bUsW*n zt<{uL@VD!5DoXqqs9YsA=OPQOYhu^+PMtdL42l@FoWNmc&1u^~=3DR8^B0-s8s*~28EFX8oAiB@s%o)~}ie5(|oqW5j( z#Q^@_dj4%S4{!jwTABPd8}H~W*)MUTdaP9%m|NV2zbg*4Hs6vN$=_ANFl*S{Hq0y_ zMvdXjhQ@@}RP+2ubo+WE-GE0ds6c8i;bav#P!U1~1lyn5hZ_gK-d$>S++9)9E1>JZ z6*b%F%yk`(Vb#||;P-eM?7(+_ys@>N{}|m5OsI0HEoeTMt{(byAhg$(MonsTY*B@p zlLP@m`>8@}3|k+vhbay6@G|RAnxs7A~8x83U^5lswr&hkD2-X+HDtJ`VV6W84 z6ONXF6{#|9H_@pB)+m*clq$!8aYX<=e>+69RBn1wVSwFiB@(%E9&DsW)vj0J%9;_G`6r|1PVnV9UijE zmBzKHez%N-YFu)CTr9mR*$gT)HHWEI-=FA`Cp#}C(S>_dQbU&Hcb0Eyh0%Se#xQF zgbURg_U?{EFeUQE%x)+3I?dj!F{smR-KUCSNsnXkCwjv0$4IvYqnJJf)#XHFidY|{ zcKLJVV|L51arkiKC0a3b$nPxYG?(@4pgmpM>DO5B?_Yd-r{zDT#oMk?>86;Tg=d)`cbmg!8cd{-j1)Gm&Gf~PM!64!1vZZJVg3Bfm>!iZndNk^r^_4;SgbsgYXe|rxr}i4t+e2 z%4kMpSM%6-U57&0ZpalduQJWyXFQ{y1DN)WR5#opC4?ET0<$?_Re91BD#$(!JwfBC z_nb?Qd2_U~rLmFFW_?x3ZPW-hylf#4&QaW!?J=b9-F6RCS||FXPtudfT}lBIV%Jnv zn@lc}GOViUqX~yfX?!@wvP;R)$*YTtjq|M?tBXkAx`df9Z3M*fLpQ>)FLf*M)2f_M zn(=#91kgZJ+Sl}}#Q2+`WLkmK!ugs_ux86t$CPO#yK7th)UetXSBgLwWY@)NI#35* zQF@J1`^4&hg~dqmHH&1`f3Qr*TKlX&jEBQLf)=ueU)DFMsDP|ls~uIAC$4GN56G{{ z=|x{f7^)(x0IN%ldC*RZw)Wt1&6n)rs3TFCW58pjKQMWK*zfsFVyqqLDUmSle-DQH zk@FB5QTpXz*Fc8|(Usix7~T>RUxTG+gZ5dC<dPqUMJ{$J!nQd!tKFz7mng?v)t73_88%S9JU56Gr#PAcY!(q+wH7L9i~>!wU7W zGg7)FrzpxEh*FjELeuNoeaSU0Z@SbK;y!T-8Pav9!T3ZYD#)R?;Zx1J zInrc;5<8FH;buPVD}qzH7Bhrw(-q34G!%Tq^C3-%O&5#F3B@o=-T6@o>wJSAB=b@I z_yVXxz75vk;c)*&p8QsZwx!I1s6>{`@=%os8420eV4A0fHr^PR|6wd=*fT{mN#XfNrv8zeNMZBnOJW*F0fTER?HeiQUr5s7P3l0I{rQ zj&Fy7{nfAkV`>OSFI?GA;7mJkFy78VU}CM*34CSh>0M!04ToHSmqXy|X4oB;co7~- zY(}Lfj(U#^XJ8@=lAGH&#;dI}w~*6ZO@*}&~hY&Q|e8{`>FH{ z#GE!qpB%n#0uRqH9PyqU{GJOt@0SbDb{}piKhr)pEeX@Sr^y5lUnNnmaEa>>n&hvjfWZ6hF`P#~UrQ2>)uPAwF3Pk))%M}sV`Ymg2w>uo~%K&H@xIM^x zMBn$>#+0B{XS%0DIX$Eo$N@B3!4VICNIjJ_E<-N<>CtJA%5%+xO)%MSHPnZ0{(R@D zbn<0B7C$Sq{nu0iE{`B4AJ2ZK1tcmCcw{KN@MfuHZCnp%k2rqpW#i`k`He3H49UFt9n60|7BM^eaxerR{ynI$@Qam(owG)8TAt#eN0 z^I?qo+}*z;2(;9eO(Dj6K_~*fuH|T&)W2~4mO{BaG%m4wptpp735@E=$8;o^wh$Lx z!D_a7tEpnxA>o+2DJLbA0!B*9$qq3!&EiyVH%oTs`shcE{By$lZ#4n>6zGpBS@b57 z9`oZ>;h(SKd}lmcHUz>otSH@8<0B%cIvXUwwue z=>iA`05Hn@?yLXK@cvIW_N{>Zc8J*8n3|h8+W*GwI=p8D+S*!uG_-&7`2Wlc|5*$F zZ?x>}ZOz_TU5Ed!`FHKTamSAT_gbb_wuV5H|8|$Z$mYL$HakNb6RZF7bbqI3XKUwZ z_v=RgiyqP6v}bN@X!b^R3!4M24ecEMi}K$!>HstZI{t57=-+jPp^=e^mC0Lw+W!7} z{*H@(vkicetqt%ObN{wRKhZs~-|9zs$t)C{ko@5UdplC5W-{>Eq_l2Sja=RO=KusK z7%IEB5M-9b1I164vqZ|l;q~=(QX$kBDjaPJ#a`gp74Y_I&cSeP#U?k(4V((Wz?~%) z!o1&SK0ptd(biOe-+fHGcImdm4a!5nu4~!vlpFMOe=F1IIb~4u7BC}G7~0mflumj# z2O14|PaYyTA%|45A;~KfgI5k;l%tj{WN%#34@k&^BC(guLs@3RbIDg8YY4Na()nbF z3$ZZ{3r(NEC1pn+id;2A4xN`2r6y3voe)%R`IVQk3|O^bitRmu zMKRzMFP4s9W4%O5UoJ6~RcN94^mK8$s_XN8!;f<$&82eJ`jB9+B-kT3r!Ghj<#ID- z-cQkXB*_;lQbP>G@i2#Y5OOJ0skVed7HQ{Q9UbFMx@u$u{RJ6AUI+iGDxb)x=_hS} z%t(v|E~MgBJEs{VSC%S*GIC*UJmduia6dMgU`r{`WR|$(@tU@v58r07?@5-OQRz#Q zHhUb8cSLAx@vP9iL6x@s#e$Pr*rr=vn(&t|22AA1ysW9x z>il9eV^}9sp5;c zOQ|>#Ib<@Q!z{I_iSW&Gr}`;JU!BMQicmCP4=P?4#1*ysK z;YGJ%`H(r$i}L5{@F}Pj^&-gW>R0CN4Q%C@&0ZsCmdD)K&Z{wmmZy!dQdQagdy3#{ z&?;u+u(T%eXz=9FuHy+I5Y-A!9af!u0SR{DJN)iv0E>%;5k{Xw@a4qUM7enqY1Q#H zmlkOjWZlUJJ`Lf{=eulmEQ&6GY8Wws4%ySbt=wtJ^0WTKjzu(os6WyuY|L;yg{rEZ z`O!yxI;sqwBlsBeuR~K`N{EHL=8UgVn=Q(sYQE#$_iDb^wXS7Q40ZgPW{lCav(+nP zEH2)0&V`l?>Vo*w)TA8(F4JHbf(lMh>N>#``UUh(c0C$g6O~-mITuHJyBql<@`G^B zt0_JT-rm$cmbYCfw#@BtNCmQK0%(tlb2E{@%WP~n(|Ho5swMxkdK|J!uP*QoL$s?h z${X~|zYBHXJaf(*^RD&X2H8PQE9H=ZQWSFRg)}(F6D;@;1|-=Uj553;f7%Bj9X>X2 zO|`FeeU~|C@5KpgUD7xSlgTfe>?e6Nm2n~9l1z+4rd2UP1#-EWQ)aStylaYn!dsaH zX;qtwXo)JMRf&)HG$5g5(ic6xXK?)95jXV=$KVNR(*RS0p<~u-Z^WHqRB92FyD~g_ z&X51y2_>U^UGX)R^!W;*yFg%2%m~}h=oACOWfm!8jye!mJ>7R8pewiTRPsUul}_X! zn8fkb34Hldg#tDQqa~Mb z1)lYXDDX?o;jQjjZgkxhN-fO~pG^QahT8s{_p4v6VO?D}xFL9DYKZNHzsV&bg`*i; zmz#$XmnMv@h$q&wi4lp-7EV}N)^G%tmE4opH0&~veJ|0FC$pr*AQDki_5{^b)W;{T zEo{jm&#BKdHp(K`i1CoZ^nHN5aSb2pHxIuJo36zbC^pBsg3#~W9sm!SH)U2BFArtC z!x`RHpU7`B9~F#WdRC3DFQZUksZkxg@D&55g5{?;A#s%sbW|>ZBjAC0JXp`E=Wd6#LP#!q`0JOROPWwspOn6ckY=% ztMhOhLGmMq(D8<7h!W&N0tA&rwv_kus=r`bVxi;0}3~ zE=qfoIsr;)x)#||Zyh<%X*-1D#rud(Afchd+*Cu$71rL1UY2F#=SBj!oMz+DS?iN0 z3^n5uEp?l>b&J-^z&Dh}AtJl`9%rJC4JBuYHb-dW4|> z1Q&M{-rF+I5h0>_jz8&peWlWRscT3uIzmq= z43!rKsy|yRLMBsP=`SPY)B1WVpWu5Ues@BT0bZ=s_=3!tKQV(t`*DZHwa*F9~4Jw z|2cpY{?5p=-w9B-upm;l-(#hw4i#y?!1s~RBu@KVtYx4{@9If_*!s}xE?N@gy;~g9 z#>=d=Vv5ao0?YLo#12do=t%lbTh}mYp}KcL@oqoTt@ScKfcTQGTolq7_yI{=uzS-l zv<|}OEWFxQe;7mNEvjm)4w5ri-d`I^WwTPFqYkR78j(8AppY&Lx+}Wv%tUU58tgX8 zYPgV?LE-?ZC-OuS3x)~ya}Y!lZHSxm)P54u%46eYF@Qww0UCV(`KNBIK9k^5XjJW_ z3X`Ky6xx%T3h%*JYBUrkA$l5)vhuVC1dHY6o^`k`L_d@dYNCaH*DS<_&fxDZZ-gjk zfF}Tx;MHW;x@1&UfUjqNog{a9$%k&C+4G%9^qN@Hx!?$n86PZhHDykulNvl;Ktj$q ziZR-}uWJAf^7Vnj$)A~a$!bxS)N7=jGh#2C#%l9!8ffi+Yh8nDl}m;`P6-R@0P(fOFN!=b>cai5xAa_h2!34+}!1j^DI^ z>!adMY@CTqjK;6|7Lj1O2fRn8LHO=-_*1im^lSut0@P)P3~J7=>(LZ)i)Y{E5kGPl zhinIb+cSG&m%KQGrtO}11@bx zarIAkoTM8SqaCgWCmC%c^+F3bJ|2p~COfl%eT>_?(TK@-Q7PZ#j>sD9v+I0;nw$cO z!#-`EHLtsuGyLX#u9F^_N2V$M;W{Lilb)z?3v($CuO20IOrb+`DIVO+8YH6(+Cu7i zHOr2oKwtwgZNkz#V@(CYp%n-eCgfr8L->$9^RmO$2@Tq$tON(iia!u-GFrlN!7*!! zwH1+A?W9eJ1wGT(!Ax3GP7QBXhzpM#jDjx^I13>$=c-fr6<98oHj#y>!n2DuE-T0Z zvH}jQgxUXsIeq|F=tf$@MNDe>m}_thCyVfoRNA-PLPl`*{MKF7W9<~wX6VkH72WI0 z-84Z^pSXvgGYW&UFR6s7rEPXGxMmbxfTAxc_9TT=Q6`~)@Y6l|CQ>1?Lm_fdAu?1U zvRNUrpF_LZ)eL0X$Au6jeH(?F)OuDpWpG`N)FaZ}Xu9LCkgeAE*ACiI@F(x=2VpM%m>FWpIEv1xaP*p*q5?bu=VR|{Yi z9v7CAj`G+i8CQF!ZJRONh1!!-N~d*kJoG!?7W0=U6pJ0D7tyCpeHoYPr??>cYTuEU zi-^iH^Nb>=M_%mh$;J>5?g}~1QZu9l>BOwosu399lKApZw0tdawtAVJll@`7=Xo}l za11vqOL*E$eA9VTq80^N-owsj2pveRpYs*)p0nz+1aA&wKAL4^S!qai zYXrcQ?%f=n{=7ZCzqq`-yt;k3xqEzhd3n3X*VosP z_mQM;5%BYpqCzSzizlgYnOp2Aos|mn;Un59AmmZ7UjV52IS(&u;n7wDk+M!_TAo~I zj&+&?Mm%5lDkpfY?nR!zHX^}$gD?RgJKx?Km;nHOUF~wVC8$k?o5CBP(24M&4q#v0 zJgZdrI{Q&Sp{?vh>&Zv=U=)_vZyfMa%x}WPc4&;SpO^jtgAbBGRkmcXKfK>EXkYaS zWW`+G5~Qm+%234!e?W@HD3B_^AAo76@`(m>3j~gMUN978!qX5Sq0MIz#_nRB-<{V< zOhX-~G^!fvFN{UO1Y#||^Mu6;PB=pc<`M+>ae6XPf6D?M_kOT&zkU$gBODuFv@u8l zkOo7-RRlv#9;NNQT@!>#?5vtTIp5H9YX+jkMIeZP4PY>ekRtt5ANc}N@60LKUBZIO zN}w3|L-^W2R-PFK#v^?EVg&?1#6|%n9CL(K07(o$$9Eba7r79ES1`VYg!UFej)}q` z5`{8o2Ut_L^gDBa+XpxA#wnL^DGv_Ukc2Z_bo3I>{gDo>>nY$cXia7(|A#&Rx1Hd1SB>Ab0FKdh9GOt4)p-r6;;{B znFTBJ`9_fK12(Yl>cnP{=ait{_Bzvb@-`3vVzn;b4sa%FcYM$lm|b2XyIGR+6%H@k z6Wfk^>!01>b8rUYcPjUry&cb|~=32(sB@u*;IHlnTAQt;&Ck zOhx^pG-YlE%NMK_YMuNv(3{W@jM$ZiY&8f18Gy1ENAaP0aNo&ShEyM=q=hW*Cr?9a?3~DH|-^5R2y03~h-m^E?Amrar9H=w0r9LxLQaF7BG4hJ_1q z(vJ^ZM)a91E^GFb#G`DE>rpRswSk+TzaKp-A3Qc3OM`j1lbl)q>d`qfwzshSxYowc z__M{R0T4=RZ3#kYs0-v4#f!T6-V^D2rRNU2JcAuAEw#1}S-zI4^X!{nH zSg-|Zeb#so+tgA3x6QK>ungsl?It+GI?PXKP0<)1fJ>~s`t z)q_!HAVnZvB}g5^69i#=_VSciu|V{CsM9Kjy)POqhfYzQE8EN7`<04%s-I7D`%WsT4l%|})2d(6Xl z01n__5A`A^u2HsNgx`AL+Fqul4+S&PD6b_mX7ebjrjGH-(yzH--x6xQdv{KH775Cr z#5sC2XT;Ou0QrVDt05URuG54lbjTVSn)@1jkn@3KI^6H)euK~O+ZV&Wg7~HHYUq>o zOzZfbT0IU$B&5NhHO0XNssf%6jYhab+M++5r5E#j3Y5ul7L&TBqAPLw1~*ZWWsLbN z`=loDCIu&dM9#X*T7X{{Pl}Wl9!Unl2~aeZ4NAH%GLU*ki;JO^T}vy+@U z!il}3LhKo9<_IWa;T<&~&2@{(8L874WGghtnjg%Yf9NxA(G;H?tRIe{N^pZ%DORSO zfZGX5wmKUpuK#(lf{aFi-2E66xD_bY1mb?(K(?HSH(>upRH6l@lehI13s3`|^#G!% znV%LoW(d*;lIKfR_nrv}Bt-JL#Y|Fbo2YOL4k6}M9hAkI9cK2K&R(u=ts zGqj5^hPCDWSu~ei#gP%X9Qx>F#~9H~?AR`*8>XaZu1}N!h2Ng!dh8cFz=XQlV+qoJ zl*2wVXZte~%i|Lg%%t{7eURA_?`GbLsZ;&M^#;H*Cj8IS_+RGq{|_%Xg&v7gDUAc+ zwdUSjGyp(SOir{y*ueLnldv$}YRQVJ2+>Q*i8ERo+L)V~IQ&K%)W;3kbTc7~-MWiy zdmIb57SVx*O6k^S5gDXOVh=LjM|ZEyeT7y~jSfL~YA|&_O`iyxTZ&Z^gW}I(!_b_> zQu2;&5T8=pwtUt67JFh^J=<*nM)LDQVcYqgP9=uqsvRvTq>X=IXYTf_PjR^nuFQ7F z<$V^geFaA&8cs`-GhAowC(87^u?335jTp2XBJT}Jc9S)3T<00I-=wMHveYL9q2#IO zd($jQsd%|wBeD6sN^P_(amD^xzYyp>^dDWwM>4WOgyWBs=0^cMIq?2`n?^zJ-y!*& z^a*v1iUgfMi;EOerVV?0c`$h1>pWSoyL)3_LV~A2nS)|qT2=EuA!~@B*NTYn(k87P z1+gJC3?kJMs2yj*A;%n)kNc~vq7kBv$s2X>IAGNayz^iJRW^t}JX`Ew%3Uvb)g#T5QsYUpqITK|dj=gfe=;@G|g zp8hS)Z%HKoiSy@J)?ax9e@MaGBIeIo1^=n}=UB*Jv95mzFY;T~?7t&l|5NMFtn9B< z(jUT(`sb+0-}U~9v;9MFli=@J&VMngf3=}Mv;cr##Ni);P58f}{FTxDXQW>X!aqdm zZ;<|Ac>hmPg8v5P561VOQT}QR-~I;W7rXsC1N?uAbMrSie`bXL8Rr)#{D&~SUH&H{ z{l758{|xrmhqL?}us<`#|BUn3uFw4&oWC;2|3dnGNcrVle#kmcs8Szc>HqG4j`}^oPK5{=*!lAPoWetJ<&ORQhdbP2l=<^*;a~KE_c1 literal 0 HcmV?d00001 diff --git a/docs/source/diagrams/svg/corundum_tdma_ctrl_block.svg b/docs/source/diagrams/svg/corundum_tdma_ctrl_block.svg new file mode 100644 index 000000000..82c77ea3e --- /dev/null +++ b/docs/source/diagrams/svg/corundum_tdma_ctrl_block.svg @@ -0,0 +1,430 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + TDMA scheduler controller + + + + + + + + + + + + + + + TDMASched. + + + + + + + + + + + + + + + + + + + + + Ind + + TS0 + + TS1 + + TS2 + + 0 + + 1 + + 0 + + 1 + + 1 + + 1 + + 1 + + 0 + + 2 + + 0 + + 1 + + 1 + + + + + + + + + + + + + + + + + + 1 + + 0 + + + + + + + + + + + + + + 2 + + 0 + + + + + + + + + + + + + + 2 + + 1 + + + + + + + + + + + + + + + + + + + + + FIFO fill + + + + + + + + + + + + + + + + + + + + + TS start + + + + + + + + + + + + + + + + + + + + + + TS end + + + + + + PTP + + + + + + AXI lite + + + + + + + + + + + + Start 0 + + + + + + Start N + + + + + + Stop + + + + + + Enable bits + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + TXSched. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + TS index + + + + + + Timing + + + + \ No newline at end of file diff --git a/docs/source/diagrams/svg/corundum_tx_scheduler_block.odg b/docs/source/diagrams/svg/corundum_tx_scheduler_block.odg new file mode 100644 index 0000000000000000000000000000000000000000..5ffb3a15a13f50204be6f4b160df794d847db25f GIT binary patch literal 17562 zcmc({Wmp|a*EYI=;O-VAxVyW%ySux)y9al73GVI$2@(hp+}+*bka=d_nR&mN=bZ1? zSzNojtE<<&s%r0EU0v&5$Vq~HKmh64bIswiJ4I|hPtAxI*W6NRSa zBn3pY8=STn-I>ELb{4q27J}8hpa3R$ws%H1s37zd)^4pF8+!RLDX;bB~szMgVWXFY62^Lvp z7HMf)pT3YB4g-NV5q`@Xd$>ofnhzf9c8EZUerfYSO|Qa0O%%Mt3UDr?C|Inf?#8`s z%Ws%e{TQKU@j<7vkcVS@H{@+%$yo@iTop;;Um)z&GC(E2fGU;S!shn``^| zICWA)n3nZE#BfA4H(ULf@!vm+=Lj>DgF?#R^T3jYg}CF0=?%{keu7#H0s{R`R}BLB zrNM(pXoHjLKZg8eN#dFK;w|3oboB#6+b-+8leAs1ElEU zgr42rn2_?%anR%tV6T2@Rdh>+^|`A3r=!5gUvU4XA1+k6MF5qq`}^qlL#mup9lR2= zIjMb?G2KnE|1=x$f#iOq3}$2mFd|MZ_E#t=B1B%vHeX1%zZet!TW<;EpQ!%YzM9AO zATqB|(EsT!QZ3}^D=|j=%rgP-e+E&?Za~Zi^-5HiAkxG%=NohRc=*EZ9pj$mRefV* za1^7RCn~UiRCq4h4BGr&(NuGyGM4;1RBR1T{?43i zqDlWVf&G|2?GE)FaVHb|Zw1%lWX}=`X}E5&Wrua;x*Z*A$%KB|x<-c%a5+hEh%4IH zxAYGH04C?V5BtCOaPK@zL(!<^Dv0IQiP1>i z`BXu<&p@#FJ|B<^Y_u@8cw;*`w*=6zd;=gkS7wWDx?E4kv`Pr*_4_`O_&a(|e#>m# zJIK14xYpUnX4B+E6yxusjy>75x3m>6^`mfhP@WKj z|1ndsWQb+a#-dn}8txZmSyalRxUHgu9expaqg&cVN_LCa=CPjrS!~xTshF70Ctq|h z+p$lt`!nmm)9q=swaqOnm|&?dr8)~jZTn*j_ggU-^4P6GvgV^_5e&kHkt-zAx7uu( zp3Fveu=K7PPi=Xb5M{AgZiq<;dP{1hYxiDOeXFbT-;q_j&<1O>vXC(|ugT$jp;ZIM zIg6os_ggOq;7gXrFO{Fvj&&H$rv#oFARWmAWt&`+{Hk!6r#)&Z>vDxXu8(qA7LnyS z%^~6zxsi7hicCCb6$vXXJL5p9rweq!8%x>=EtEfF?Jhb>R<_ZU#?4**r;7LG2VK6T zXa2Nmq9p@4mG}Cb?Zo9mGS8moWYK57cIY@zIpAVl@Sm;Th(F8yT`M4u2dO+VWm$g) z2M41LSZMWYtztwdX8fP2OpZWMgKUHO2Z^7C}Tk%pxR z$OmFk8t^|s>3Rb|zfZlpG^WBw_4i8}0ROoJLg?n*1E7PxJCo}pKH!^@*CAn)AC@)0 zmsg7QlBsBDs&ZR4cf6_?l{6am=lwrK97&5FnvQhGAq4AA_y?H%c21J}-ARqHzdNaS z*T!d$7`}*&E`PuC@!c`{;E3h6{FwY(ec8KTQPTLs&S25{P~#*v z-WWTEZR^2|@c(jBw0pldX;7Gmyvx4wIfhc_e{4sJ{a9`J>or9_$$x9x%e&K}pjLbL zO$3bx)En_pS>Jd2#-N)$Rxy77jO z{O-RrhY(yFstO#0fs5O6;6ESQd^a-qz0h`OQ>On%!R^onR`-tr%b|_E*6#(m{Ue)M z_1_D*frmDgD!&)9Ta%>R?-b1p+WO(+Po37ejaEBLq^NLwVx3Ar;33`b5aQH0p79T4 zz_YV1Pt_@Eo$mQnSaG_6et97uWp84hcP~WB^xwP?%=SI{8A;&WnI=q)rxCSzRNXVMUQj2jAG6R zC}x1g%R<7CnX@9EFIfzaS(p>1$4nW%2|BsoBwm=4SYot(yL7Z*2A4ZtE%uVRf`OpA zL}?M*E=rhJoELhSujQjazRC-xobGf&$95W%4MnPY_U_-cC;IU+6(K0UI^iD4Oq$Bm z`4jg;uelxWqIdGx4#UREjogNT16dYshD+)Zc^~A%h)3vmQ7Gj~ zxrZ8=z`k*d(Mwq@$?e0B?CRQcM$P1ndJm>i0V&7g)){wKnLucZ=m~^+jI@ni_-9amSpB*UOZXYSovAx6cGOYu7dAqwFv9P;B&YhZOpRI16FE{0$ z8=yOwiV?;Um$wCaZwI0FN(VL~1*t^%acok_iB!{KDfTSLQp%X#K zEeqJ@i2$*+7hx5A27ZMI3k%ah=Z50SFi+lE>JqUHB)0(O8B`(?&G%)c6Xqn3K`4G2 zf!`L3tcVP+Z%Fe9iMKD7_L2udK_Zeu@*5Ogz>cJfQb`Lmm*qA$y1Nvb^n>c9Ddx*( zA&jK6f_3IkPq)!j=V)h&DqZc^lQ0NZIrCDTsjbkuy)gr`L87h zh%j+xgsaBAx6Ev68rF%Ds7ch&2+2cXV~=ksBNx$Yv0TC=L> zD@|Von-g`GOx?J)J5@wO=Lo2oJboyYsK60zef|86QXEqrO(F(|=_|hbJx(?k(YCmD zkc8}&QYjh}rz`{ghX4^-KDii7$FqZE>stuyFmaqlS^QGhz9-WGJV|WUNy;dxNq$-p1=GNkEvNX7zeD%JdS5RpVgE#6a(Q& z8KoxlfOZJJVj_NE9acmH9#QYds-@QHJH!ke{rm@sF}osKt(wLJq+0L=6$ax`f7YXi zstlzISjNgNkboVNpIc0vf@%Xt7N&D++H#&YAXKT{f?g;;$~Y@!^rG$;$JG)wvmcoy zQRN3}9qX_#Ba30V8n-iBxrtuJw?@CCfn8Ld^LDx#@Sjvi0gVjcQ1#TXpUii8A}YkuE~1!rqZA&VccAkRv?E%2Q(%E%Fx5Oo!xht z+yyrz_PT!C!pd{}qO`)BdE8*-7^79C+u%l2#C59rTV-C7<$}mO1^6K9&==+rmys9v zYHk-ig06yNe^aALrMv<_!Uvsgvisi4C&{}8p;R%0y(4yur^nJ>ED#UhKsGG9wLpoJ z0|7(Ac|y;`wbS*pV@i`%Z&9lHBh8>--`}2S>z(!^sbBy=DA|8|qW!iMVeF*;>uDC9 zApJRm4k6_11vTt+_8ue|G_poPS()2vRw6^9Q+VK3p;?b`iRz%6c9&3^MLwzE>m=vd zx1Wk!xQiris6!H!ET3ZJFdzn*Qp#6@)|zwp(0G#1pk6Ks(sH%xyHH2{Raz)8ibkg& zBrH22eL3dkA)HBne2i`=`@~Ncdd05Al!fRbrPw;s$Qp4tDj}&n3nhtDz%yteLd)z# z;S@6;a<#k94I{`jP+<#WIZUbQuSy0m2^3MsJ`Ci z;BtE_u13o@?dzyPKpNuAb?<#dJ`uzo02!fx`cq;dN|F4n-`Q3akv&i~8Pvo+0Wn1G zhg`2n#=}`60r%u0j6HK8RVx=%tu}igz}a|W#j^;H&DP}INXoH$Z7bhI(lNYc5H+NG zAngTaRWMS4M#iQ1Y4kmZGyDu$xR}sscHtp>JDz%*WWxG~pUxt0NZ)!tta$Gm_xD;{ z(ynE`wN%d8MmxuYDr%i|`gl$1v`VLP1=UckFC1?)degoE|C-CZ@J8+p0098h@AH{| zf1`r>HJ5R6w=({1CbOe8?y$s$)ODst{TSCI9Y;Q6q1Lv6@ASSjM zM9_k7tnMDpe)@EhV8zPkmn$?KOSOdgSsPV!A8bmuz}JtYlyz;jT468dl0bi97N6}3 z9B!NdjdWqt;{0Gs_H<``soB@FfA@+ge}Rgh_a!KGZP+=k3ne>!I{B@;aQtU|O z_(eFs4x0fL_9zC)GIps6bD$^#x2275JiBVpxL?cWdQIxtmtas|^P0~M6p-#{@-l3d zK0g_^TAUzCks!;;p;~@Z!4U+HcS+6}2IPZNCe0CR5Gvh^g{o!0{XKCrDi}QzGHFJr~?(JWm4#Ed@+bZ)~6VKPU(qNDlP+$!o`Gx>*d62jW(qo+4du*)P@j)y17R%h3} zfuRYvoo&RyAR!6-41$?DIwi zTxfr~9vwC$Q*$IFYVy-ExJsQ*w0X=P!XBxNc_io;1Q&*vcaCwPBxdzrfsnmO_fY6h1Y z?jj?+5v0sKVKiyC<6|Aw%rA>O`UfZ2FpYL%`1ZiPONcoXrjpZ%Lm&%ECzPMI6n{w1 zf~8@y5ReCE-i>QLD-nsgq$7mqh~O=|7zrx?ulC%%**IC0_}Ok83$9gTnM8d}HTUoL zMCh~4dt16YNLTURDjZg_kDV{4Z%$Uo$SMYlUr9U<9?+22vg`_kGAQ>Rx_cEz4{=m#arUg%M1IP z(|DZ_%S*%~=HG*Q%$54w7)YQ6oP3!vj#~38AIidO5j~`$qV;abqPQVcgM5pZ2kvn8 zS_CkzYd0^sL5RAxIP=b6ycgg~nvVdB3amXt#{0epnCC9-0gh}&piFTV*J#k+tz=R@ z@YBI$hU3i%L`{m4fxi0R5;J&})oV!Y8D*&uvnGX?qkvIn&?cDWx>Zctz>!Lijs>9^ z#OH4xGgcbz?R!ZjE{yjU-u0biO0&`>jyb5KkEKTDSmrtOg{OX@Pr~0|A16MFM`iDI z^gvQ1Q5R3Ab?zFUS68YUB+-n59br-5al?;yx+ZlUBKKFc_lKfJ1)Ve&Z?aRR66xNu zVYf8eCyFG`2C9`I7vbj@;&zmo4zMupeSl6Jy#!WtW2G98X}O)Mf?(%MYm>pxHRq>|r{#kebthj`xE8GQBl(6mh0sf1vt6^DV{G4Mm z{Ma>Z%N&^Rtxh4qoJ8<^uJ0>A6<&DVwf7_Slx-Xk#zrz*<1d`mq{`|bd7uq~Eh z91H|(;PjMWsL4shcMhVBTT5=L5V9q;-H7X#K`y%&-QvG$tN338Hn5 zib4$Wl`M)swYF~H(g~Zo=t=?blla`ImWj*(D47UNIgKdhC_2H+NuN=BYP!?bqCkAP z2?_O963@36N{F0QI>JP5rS?+-b&)Gm(zjD9SWmy{I3cT0E#_D};*mj)#o2Y=w--U| zO!_c=$NWi(k>$j1e);o{9|3OK`l6sP-uVHjL4}Nlk3!4X@#z;POla&sjs!jYaQj5} z`Hi4mMfRc;i-P7$@-{}8P{cTWow2a8uspKtInHH^^DlBJ<*>I82X>utIy|7E=vXsd&>+;Z3CanDW=mxd z##P)U%$p8CBxb2bfh8%P#w3s%pvH;w?eNv{ew4XE`7L8^QF@USiIB68K)HvL7-BX~ zH_)SMrjX_HEMO9S@@U<*>Qrk}-k8WV$SGUi<>bOag}jimo!81fH9v_l2Tf5v5o{ti zlI|=A*~Hysrov8{#Z@G=XpwH5v*;LoGlSeSMURB7VeT{h7H?_C$ zt0I++(6Ov7m?NAuM+%Nq`B97ugL*^9M9I4%t52k&`(Zj7-CG{+vLVMWi(|Cy1^b@?HXKyuOGX%RUU;AVkFsAV*8{%Nm7SwrH$>bW8f{6k zZL1pDKHUZRF^%ZM^5MLr$J3t9@2j^`E1L9;)r-p^R!#htM!58|X?8=?KXtGVt5)+C zONkm8IiRrPoZZVbV83IeGBF958*<{KYrj(eYRqWCzIR)hNnZl$5(@A2?~ z=qhKj8?f70m>zbi-V;dL3vQ9J2s?kS0ryGRNn@6ienpYES5{%mp^4@3O4<9?4a0GI zb-%fX*a!?-&AUobMMzjRx8XnkwmoZ}pOf#cMZ z*TgS7d;k|rp~cNW_TBNBTWbBB7_=%I8kgw&{U96m!c|xAAsSu*iH>tPj=1uaqY+|5 z>{ffk=!k1W_`tv*DQ6sr$VaxecpzC34vK89;+un(<}PMrGmaMj_~0f6F*OHUKS@bu zd&+{PqWeo*-M(UXQB<|n#ypj96#c46b9I!)i|naWr(qk#Xr9prWObsx2?t z2xVRA?M;9Ny~$QK;kekfd+)^)cR3X)WBVZ3>XBA`L=Es2isp{>%PzZLA4x~9sxOPK zePZbiXRPP7h7LBaJ7>|pNi=?%UGlBn7c%lWx344#%s#g0p-K{*R_%< zGHX)tP?TOyS?}Z8t(J>#_B2h5&QGL*ak&GjL>TcL@s~(13wK8eC;(buaru0qLkAR( zp}miY(ChsMwe4zDd8V`~U928K^2hn>J}_?jTmk;UR;;NjpM78NLuS6xX1^U^I((cD zaAm1Oa02(b{jy;G0jFW|~K=HA$3S6r}l; zb%xVU&_p!AFCZ!I&h@K z6&~*k`?d50?B)OriFb8Inoc68HXHj~Ohw~Yy4sBDosnBC#)$+p zH4sJ?11s63!X%{v!WWtt@{Es8bsijycRbA)hs{*>QNXEzIkgqc*X&UHhKYrXQ_4~~ zi-V`9@JuVgU)e0DJ`9^LfjQW5+iOkxoE%O2CknM&uP7he!&&&9wM>oeRHX)1eh~`0 zGW3rcWNj3`n$Ky4-7<2xKkYsdT-(-enH!cZYsgrJq--VMgnP+7 z-vFi%F+M&^Y}oAlkt6#+uo^=5-(5}G)I{*Q%!iZ-x#9Z|UQ9M+#B-KC*)-iu8bA#} z7RckgnfAimZkVZr`R7+jr9ButZD^2$nyk=7m&j4bUex<6LUZ@21k)aVXqio<=FL@1 z+mW(w^yZYNhsv%mw> zibd>vULb&Whrr&a{qShZMNo60L-+_oxJ>B6#F8W|Oq8S!W!2Z5!dVsS8bp7omEo>% zy*uTr4tarbjv#FlLvZIR1Vdu#aSF54tcz~H$#F!nL5P{(_(@VOH7=HX%LGxw!Et|L z_op0ZF}mT^NiOFaY(xp)wm&e$@)vw;Py{#enYz6^^8^Io&5mUmD+C6R2yAQ~AYwTfwU;^P$r zFaUQud7|r`*r9!ouZ}^;zVIuI?iX|-xya<1akd^NNmrSRP1ChY^J7+~F1rx+LgTTQ*ZzIbx50F9m+I|8V?>53V|6#=iL0L?;;H~c|1Om;lK--kl_I%Z|f9-9v?LBWU3{aaeCC75*Zb~H15 zRo&*CneP{W>g5gKBNcE=kYdB(CSa$@c-Gl<7Ttwm%lwHLF7LNm^S2e7j7HDUi)&0Z z&G4_`F`yui5zC5L9^)0=rN>;taC0o!p$>MkTowJducK_rONSo^%M3r1a}O<|l~h4h zh2%6+PxyS<>$#~tWzj=HH^l3F>~Y+p0%!z00*0vk!SV7^2E|LDCj&>wsEH=+pSG?I zH3s6)2hP5}J)QYR%F4S5kj~-WjWk!|egD1;N{l?EtU^!$sTAgF{=g0e(5@1NZsW2St`UVM zuCfI+|70Ec)W`}a&N_lu$@{uonmy;a73)>tvA7>6*j!d3F4RGaMlLHU&p2r9Wo>`n z2(Ih-YxDHT#&o|JSk3+WW_~=ccYgJLF=6OvrY?ZIZhkQIIJ;qQa^D`Rm71DcP2Ah@ z5;N*5@z^dLpkj;y7etfJN3SK)5cK0SKCrUhj!! z^G%H{z~ezw>or@8eN@=&&>&zK#$Ziaq?S4Kwl7egg}%^(M_hjh5#!s)$s!ADVA!%} zZE?$R(rPt=2xU}I9E8%Rr?jhuXaW@av~R+<^qQG_v?#1Qa&BtU zkfehyNrNP9wvn;gyqj<=(@`{3IWb~ruYIUc( z^o-?b#w3!G;%rvMp_a2{N8#z*^0d$R%f*cB(yU{O$l5qqR=4(^WCkLvhtJeiV0O0v z_nDAe>=H`HkJ9y|PAHviQ0WSadubVm;tkA`s`C&6`J1i-#|V{HgVl;aT^|q5%jFL? zssjlU1qtKBni`y~zLunz#Ezs_4j!h~Lx@T9e$#=yM{BFSdb?n-nYdD`dY51ctf?Ip zEbj9)jq;ev)mLpKt|~X%V?U2y8#|MPR&$TQwR!g+`R950wa@fboik!ov79>=zig@c zWz!E2BKBj&_9W_|VEGcyX}QxA>j$b1ib1rkJk@M!BzuR5_P1B0Z^XID8&3)v=`4Ba zBzULi?~~I#dOKe4kv(BQ=9!WV_R6}nBqYUeQGJcx4C<0i&t}|l?959wIKTEmYKR&W zD#EQ#`CK1yd_E#fm@wC{3DhtR)Nl#ZFbdRQk%V?-4*kiyox?kG9N2rKiKuCNc3Gl3 zweaTk!Fp<)L#l~sVQK{JOgvgENdlKrzdj;9pCT#sT=`L zU!zTzbFgclr~q)I#S8$z(_y}$SM~GCdgSrQ$A$HZ4w}egFx?Gfb^W_L_Cdra&qrc8 z(VY5nE(FVr*F*M-KFV}8iGUx*Mg8hWvd}H~v<@3(nBEVgf^&P{pHj}cIA*_BtokzP zAPyq=ZeRP%ZEh#+M!7@{C2)Rz{5(`(7d8lAd;bZIcVIJAjqCf5`0^C#8GEX4J`6`^SGXF!md+n*EGR*q8lfuE^kDs zEjMRztBgmP4K#lZS%sTTq-0Us>*au-bFY~QbUyV`pgAvB;GmeSJliepkRq64dgi~y zWa^gxIm(Ca3EL1;*`mTSmBwk`;O}EivO5Cfbe3iXY#FOcbi{``>S-!?Vw0yU7-dRV z6iALEb)Y8!t|+1Fn&Mw)S)!S@-!==Xz54n|H=6K-u?VB4^oV|uyaRNGOk~m0_-wlt@Pe!@ z?T?EiRrJIAcIca@m*B^j3z7u3S2IY*`J5kgQO%Ow%hFZTpY%_gzIwh?>?mA4C8`|Y zQGarnf@|)SX#Y9Ag-k!FNNm9JH9nnIb^%W^-fiV4wj=>7bX^`%V9G2FW(;o-orGX= zV(-_Xr1U<&^?SD50cZ`S!dj<@MbY*2xiVp~Xvwq{e9!A_#YEV3Y(ArDMxzgoa?Zr}L0Law$WuYM zj98tKNlKbyOB7t+UI)u1&$E`E)5ywJ(;l$Z1A^0gBoN-#tsL#WvT zp_!!#5xD~Z$1`Om!zEKphOE<2-UT5Lzs=`qpdDYt@qpW}&)Lf405skE*kU}k7 zvjuHyV2wpagb2eFg%rwlCGAy6lr!TGa)RYqSw+;qAz!U+WB`prvl>Q$%blc$#-%9` z)(wb$43NlMZ4+!%XuxG3eG?gmY}&YqUVo;tDx06il3B6h4o|OUAeBo^xp$5gT_>+4 zmNrL1CRjCd7cXIgoMP##wH*jprD{&U|`_nessaw#Nkq21Z6kCL|*8R1~f?RV}nv zZgqAaeCzM;9~&E+nVDI7m&?uV?d`+E!|j`ccUtZ1>+8Gex3{;?osN|N08FpA5WkY! z;z<@{H(?2E&kU1!dp03;C_*P2RPfmwBxUN-eAji#_?25Ptz{Z!x9O{x02cfx`~#;g zFB1Tk4`d4PKf5~(zRL0D7x{xEpwhJsYz_s8`S|=IX~aT2!$eFei<0DyLgbwdJ2)C@G@?5@;7sd- z(mf{FbQUM^qFLe(^k&yItya?08m1M|S0n7$$R37TF#A+s###Nx6Slx>9n65&Ai!{c z@P2&^4^dWK^XLM8fH(?A^t8lYJjexAim3+YDnCfy+@^M%k!v-lP;-JTH2%ii zH|-1gyNzI;N=O8Nx=7Dan4XaZ=EvZH_@opf=wgwG(4*9`G^j)N2uKglxQlOCQ#T+- zT|W@NlYQ0C1Ze3Me$Y=2?tm!GeoUt zQ39Ft?)yjqD^oWSX#VAN9Y?X@uPPui#=wPp9XZ8HFd&LnSWG1wcl}1d{rGMCN^##G zK%&MD@=beWs(`wKk-2K&bGK!;u0LxRseh@C_%d&^3`g~{ljI)Cx6e5ab~3lDT|2xh zfqK-()226LbKy56Y3-bAh2vn@N^!t=7E&ZmfuW(O^)VTjzm2@ zki|Hwch4d|x2^Qbce!#r``=~(*qrcmGF?$2dea!{)I@_aN?q$3XU@fMJq~H=KzOqp zY+(6nC(5u@zB_Krm4THe=(e5S_8pEJjq3`F7D@Aw+(hB!Z9Z)7-fapPvfmA$ z3yn;99~*K!S-hq-fQ$?+JGTEJo86&DZfnDR!pWDFJj%(2r;)aY6G2GE(oaa2SB$XaFYfKshuSFwsDBrpqN-;PaGbNBf}b=5$u^7R!q z_I_s`7ef|D} z=j?7#PswE3&*`Ud;4UcOxxLBv^<{1T1h4z~M#4DZ#Tkv^rU09*S`rwj$I$A!{q)mA zBi4D2?kgG(%k&Ly&BwadGp#24-<%Pos1^7AZq4oC$4{{lJ@p+qg=td zOjsWJFlLsH@8mWXm|*wCULG8qy)7}1YibU-UKa9NhTQIFNb{C2_E~8dKj`SKN@jcu zR6jzK^8+A2^INlY*3=mlC9`~$VSfUgK>6N@q;lrx=7y&C-KCK90Pbh_C{Ji^yZNUfkSr{V;*0gE(E$kaa05AD;Upv^$Ck=vhZ85^y3xD zT<7@?KFmd;Ie&Bra*Nzd^Rp|RGtlDl^VTi<;P;eEB6Rq;%Sy;t-<8T9jH|sgzoQdA zee9(u=YB`6ni;%quVdO8-kv0nK}%$}IHwjbk`fQ{FMghYK6=YE>o@yo$Q{?N9V|ch zUTA`Hj|vQGDG7+rQhM+`T;ElfAJDGUKy<+PrplVk##k}Kzi@KV`(yf5srKx(L86R` zLe5Es0S0Y{PE_BOm5I&{>b)U(zZl$pY2jVF=q>4eJmKn+HjWo7}6TC*DZ6 zyACmIt1|iTp$nBm9(SK9-bhX%jXL0tH3Le6hkXj8^n@^hx9dKqYrKeRd9&X~kRT+n zYYj@xSM*#Y$b6zNV?Qj-ca#T(@+arNCH|q@0;jMBjjsFsgB{dqUwm&lEMF!_p4ZWb z92Cd}5Vy>Wt{AX=DmiT8L{TRIGs1K!5OrOA-J>S}DOKHgSIpFVZj=isR~N}okt|-U zL#rkj3p|W06K!XD9t8HDkI|GKD2rd*0e)Jjb@K z*9+cneux({x4-Vh!F!Qs+PzrDHJoq^mW_tQcGAL8Ue#)ssWJi5{%Qesq z{?g3=I=bB4LZ zaYT~}Cm|m;i6~;oF`8TPIgV9s^2N5hfizeVHFCe=4u!WQKD-}nR zT>}b60O16j9}{1c1yIjXRZCkS)SS41{8u$*nf1r_e6}KC)3Mf+&Bp^w#69Own8#+RW@LGw2fUI*a0>C{V zqvjbzCA$ZhHrfX+|EXK#i?0Z69d@9vf3^Q-Ytwjx@p zBT|=?-8ocIYzh^*+Q8E3p~K?aXD{t1<#5gx10}Si`|uk~oPnSzVxsHf9>j5C!O=tMTG1!<*(sTnL6$P9qx_y5C=&Hf2*iEfUkX`A)^RSN(R7nTvK63`3ycm6Kw`&H7y zO8nH~G9t9r`Zne!#*V**fGZQnEc)pXgzp}p3Ob91DhM`>PyG7E=TzJW9NN3Cu?=q!=g+8Lw-2(;Gy8oPxFsh{yy60y!jX-n>$A`6P8+W zCSd2hZZ8HoXuXvGfrl_k;z|1AW1cXmG0k0bi9ZXuG6y0pvwmb(b_xJCf?@UJ4OkGvuz2`G9fJP5 zC2D}7XAUc|?hkVagNIWHwor3-kNVP)+Lv1{K@l&G!2%3{sOhnz>>JD%%UGc&nx3UI z#0T|=rNEIhaU3($VYz5HHJ@^nU5+P3>)w82&#TI)w(`Hf@dp0Jp#}m*0sMO^|F7@s zPx&LUUrzErT7LxX|C;Cei#_qD$iE-{d-neS#`$Ze+TU>)LH`Bkw;b93jq}(1fM0oS ze@ZLl-?QBQx8`5@<-hU={*=h~C5QKaYw%k(!T;9!E0g8ds=}WVjPzF~`X74##8>%y zf8}xiJL~1|-`bz{0DxaCls_f>A8`K8g86r(zqhFWACUgaiure(zqjb=A8`K8lKFR} zUyC|_3iCSw=AW2B{~?w9JIXKK%%5WU4=Dc!i{{_K{ys>h{{Z$^R?WZT{Jm@I|A6x+ z%jW+p3L5_ZIY581ZvKPv$1VKVRkdHdn?FSg|8Fdu|NR^O--~X4Du&Vh`x4wAD1Vs$ kt55pn{r?nUo_~23$Vq~}rvL!}Nbi5`@2+fw_gCrv0n?WeLjV8( literal 0 HcmV?d00001 diff --git a/docs/source/diagrams/svg/corundum_tx_scheduler_block.svg b/docs/source/diagrams/svg/corundum_tx_scheduler_block.svg new file mode 100644 index 000000000..d1a7d27bc --- /dev/null +++ b/docs/source/diagrams/svg/corundum_tx_scheduler_block.svg @@ -0,0 +1,530 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Round-robin transmit scheduler + + + + + + + + TXQMgr + + + + + + AXI lite + + + + + + + + TXengine + + + + + + + + Schedctrl + + + + + + + + + + + + + + Ind + + En + + GE + + SE + + Act + + Sch + + 0 + + 1 + + 0 + + 0 + + 1 + + 0 + + 1 + + 1 + + 0 + + 1 + + 1 + + 0 + + 2 + + 1 + + 0 + + 1 + + 0 + + 1 + + + + + + + + + + + + + + + + + + + + + + + + + Queue state (Block RAM) + + + + + + + + Startop + + + + + + + + + + + + + RD Addr + + + + + + + + Completeop + + + + + + + + + + + + + RD Data + + + + + + + + Delay + + + + + + + + + + + + + WR + + + + + + + Ind + + Act + + QI + + DB + + H + + N + + P + + 0 + + 0 + + 0 + + 1 + + 0 + + 1 + + - + + 1 + + 1 + + 0 + + 1 + + 1 + + - + + 0 + + 2 + + 1 + + 1 + + 0 + + 1 + + - + + - + + + + + + + + + + + + + + + + + + + Operation table + + + + + + + + + + + + req + + + + + + resp + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Sched FIFO + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + DB + + + + + + cmd + + + + \ No newline at end of file diff --git a/docs/source/diagrams/svg/queue_pointers.odg b/docs/source/diagrams/svg/queue_pointers.odg new file mode 100644 index 0000000000000000000000000000000000000000..72616156f37f113ca892dfb634f8f08bff98f1c0 GIT binary patch literal 11630 zcmdUVWmFwYx9!GVLI_UK5L|;xaCZ$7e4`t85AGV=U4sR8cXx;2?(UDAJ92Z*eL3H| zf8QFrd-Sf+b5>V%jjpxUT(S}nkmvvaECAq>N+X*FLEA|M004d-FHHb5V>6(wlO<5c z(h_8>r(A09SE=v`dQ`vThamEUsZ){hvhwk7KB#MDu%tniHI!9`-bCdj3v;m^ zGCUFu#md~y4~^%;A>6ke83o>$WzyJ9$i{;0fJ%D#T@(DLMOWeLRe@7l){zl$K%B5G zl2JI$32;=RYJDPx!2tj1ta%eF3F87iG^CiGgtTLVJ(;-}Y9^23ac5q2NNP8Fy&lwR zMRyaS%dj4Y!-K)mMMVX4yI4NH9^=t;)zi>Xy+dmvpKZ}X&ru#KE`e~-xwWt8D>BHN zDE2$gmRUru8uut+7OkM}SU>a=2M<_b==a9a`hAqrq8QtbZDcAD{nrRz^bLD7x~`}4knY=OXEu63 zHlE2;U0QP9RG-e^)7D>Wc=~CYfll-7F`7e{);|Q1V3aqANhpCY!+D1JX&PhTe`ui7CXDlapl<$^{O<8(O-Kygu=|pFTcP zULVmH!P1n7jH$AnSw@aVqYvzl^`LybSIIiodRmA2OfBkVNFg;%rD55!hQeOLwWFcP zB$OSk8O}I7po%(*hFN$S@*RlkzxqLTgk%ZkjlLbXT7|B#oKT^vr*duLsXkdf4nsvE zrj@KE*`-40RM3V13~Hl0zskE1#*uPP$v#V{5bK+ajyku+@>psq8&mHg1Dk4g)|(M; zU<7g41E&}3O2XS~Y^|!RfZPdV!U8|$oT9LdEC8FUD-5O5GzT2XI|2|v-yy%MEQ_Nv zWa$Ft7Uh1c?VCk_+*7TSX(U4>Hl@UgmudZ)ynQP2v3qL08+$VWh_r3NlJ^z3Hw|?> zu5ddZ(jm3uQuAXZ%W_a<@-*SxxZjCaNw8vFSybq;#9#nnhKB#eAa(}Wqd7a)&C++0 zvmM$nHA*pjbeVY+9+{@CGwKl5z-<_kce^7R=AQz zJY0mN6;xCOg*M9uUr>%A_|AnMR5iMsnR`H%vjS^NfxPi?QWjWdSe_I%vtKo@AUDN2 z(C;w>`}9`@j!D)OZIhUVjdF!6cc+NqA=;SED(^Yyz)yE_R4*~nl^>wQ%hJ|JOEsHi zt1E|6oiX-4hifu;X^(l;vLcq<@Jt?>tMHx1)dIM*)rD2^jVz;m-j_u$675id-dqH` zOSHbvE@)I&RHPAqK!~ZlyT(W}apjoO?(HI6f?QW1zYepWw_q4JMc-RY*PYwrj~%6H zbk1kfMpOR4?IA~jqT}Sl{3x?`E~}_-q{F>l zjV9+enaov~XiuhUBOqLO%v;+W(WR0rF*>aPYJ_`;XtdsiR9AV*C&@FsJd&wwEef6T zo`5-)PtmLRq_7q2YvgoN0BgtnigN$1(a3&usbzxAuvkyfR| zQ2haW#eYMB{Cq>CrMJZTPAMS^W(9bPcjulTcWN71$;X{0xfZ;u;&+#*qj8~QhYkIu zDfvC-SBx?_SEpA!TG$6J`x(WuD+C*@n7cyrTdfmpK_@EEcbvmE^}8MOKd``AyS^GD zEZ*LnQQ~>Lsjb1Lw3_m1wQSdEv@^FNy`ISmjEUi@m5~Ywom|%ikrs*Q9PrihAG|i8 zZB_{h#6}a-LRWWB|3Z4fPlw!eE?d>neMsh2y%;X2hvA>*0^R8LsSN{98ETc-`2qu_ z9HbcsEob}IX@b5aCSD`N7h?^x($H(wrl~N;(Q~@4q&gu+lun1f0w^36xjHrC4woc3 z$krBvE^j6Lah1a%jE=0gb_ra=`TY{@4hTZvwo>t3x!2uVopPZdqL>bnToh@w5s#XU zT8t=YOf0BCSI8-0?y_0~6HaR&PJ^9=x>{c;s3{^d*zs%|kz^KCr9Hi|9Pul0mu~p` zy@)6!(XW^NQ2kUO8t{9nGCUfpa%qQs^Eux$&WPzZSFD7Q=B^YPNx>D&=HyeCi6~pq z7eTJj&Rn#0paG$>jCUUbg}`ypX1A=M&+ks{4K)Z@YV(a6pQaFXwL8-NW`rlv5D#q( z$kn2W9&XH4(#f4RZkjKJ#NaCti-~ENNf!8NcH@C{Ze(~&sZ3f88- zj{IZIn04b1v{n|+NZ^+2B&0rhj1p#4?&U%x=wdaru6?n^LM=?g@{MyrUa- zWQ2eFeT8iH!^51akt;vKXKo1uDQSa@8_)wajja_ULdr*pdyUj#o{xPMW zdG9Mn&j%;u92s9rJ=LI92~2ZVr<<-%sWSuS$`(}NTTL)CYEbLP4hR#AP2$Eu6}ae2 z)C1ty%8yFyaB4Z50n~Wp(@iWfIqq$rWvkIdWDL-UW#tMa7@=rR2qic(cM=ia91af? ziA_0m?>Hyi)^ltJ%>vg@HZf#Gg!ofW%o+ybIMJ4@i+99lcodWPrYno)2#*LgJ1QBJ z4z0{|{T6oWSk`xMI`FPWctCjwsstv*^Rs*63s%#}?nAB@<+0f5`;L91rd0Nm&e80j zGZE#a%;wt|MmaaZnDj$+> zh9I}*z}F>1B=_L=YERg+uh%rTWr{u(tAi^4y|c@x(!%)eyL1w+TpMdyb2A=Io2I%#YX zhTT?Ei{Qq*^4!^gHm`P?)y!R>CPU2hz?@+y)l&N0ww8I&w^5)tgBgFz79zrw8+o#! z*>?t`J=E0EAAAxX32Zj3Yj2u0sbS24+|}*01jfUAJrF5fpFrEwf^r0p0snYkd#{=6 zMk#lVTe_u;DsjMz`y`241_Va)Ac3{2qO|&89ccUfIkn=1a<6veb^i#72^6%ZM>DgA zh=GP_2gLDB!Tp$kL6reFp1n+JRc-iSr3mR%!LP^;2fydVEGL8Sy}_nU=}#uh-R_E9$c{3C11~Xx z)J9roIg*^fdzTg~>H1m4LS(Au4RojJD7#@|8ULHO$WvL5Bt+goGOM>*9~-pPW%CcO zaqM@hngtvubrh$D5?lQ|#~S_heC4VzL`;0=$6*%OqUAn((pm4Oe$1u~oLN=mG(Kz} zu2FL}v`#7AGn->HdY&Modxrk$?P_dj5?8-Cvp7P(yxm`M(|^Y}FBdp3hk%8-fw7^T z^FDNbS!QD0~qfQU>!X@APD&K z3>Mb^0g3eoq^*SoNLS}Sq5jmLrL~2jHPFW9KO_G%JwN-i)G-Hw{=;+r31?%gV{7;S z=x{V#(EU#T3O%K*tkL-fjmH_uFR@;(PoOvA_YQD}CJ% z0xq->U;mX=c7M$Gr7Pc}KGli%2(vZGdzu%*i@V_o?mmb=o*r4ue{!>VY?xs~Bq5N3 zIM{|0PuN(1$%|1kK17-g_pQ!LBVC$&(jouF(Xn5;F>(54?KoEg5hGjpN&h&U@zKuaa;-*G zKto2`(J}B^=b$E?P4N_gjNO&?@k=YF?nJDgeTnZtlQLiem8(#GsJ~mjN4|95PD<4x zp9q(~xp|C#hlGaLU9waw?nl2tjw|MsX2Hr=*fYiPBw_YK#l~BGT3JCR{g)??GR?ZbKRb_m(CxiFOKCIgW`Fl{3K;!cD4Kyvqc- zQh31zqtaMrh5V2aRTuW0WsR>lZg9_J5#tGCC>>gf?-Eh+nR89z;~{Ey*)RMEba$4Cnzj)ivRl~vb9d-M_lPHN)~IT78n=TXDYEQDO^@;>>|1l}y8 zD`~Wg%qRcIspcJXZ40)B)$1=HN+pw{RF_(+c;DVw*!0~w(|g=xIk}A&55ox2(2Ia= z!3Wzw67C>`NFuw_ecvQSYXkD3w?x!9ZM6P@6TM%|R)Nq1j`|H67s!Om5 zKf`ppb_Xpk4Ug}GY0d8!eMJU~shGPca5xlFvuq9}!tc^=Tu<{^m&_*{>UTLtnD&yJfENdam!Yk z(7I&u8Ib(Fm*lgIQW(h(S(8iDxJeh1Q;bVowD9|S*udCY`FC`LGH9mH| zDyHPSKS0Ts^21uns-?uL;0k-CAf9QoeK&PhMQXej@g)BEkhZchiX)B#L9H)X9l9E# zpuc?GuTe`7gVV0A33Zcd;qVMEr3Hd7e&;0gI(na;&{mY709}>P4_jc;7Z9>;ml{>t`tgiz=-r6ibG>IA!<0 z4h>;TTj8qU=gKy-QXc)FoY#_rJ^ysT3=jP=ZxK0^R@<*WD5!zmsdt>?{oN-8-CB%~ z=Bpl-!~GD1PBj}b+bo%8Hty_iW%h*2Nya>x3s+`{!`1XwyG)#c!a#Cy%(b2fPZp6+ z8cE*}z)lecTeM5M6(zgCQ&rt&luMb+DfxK3cQxKZi7Ba`nzrK#!M2oks+&X)X(`;s zon@Rj!qoJ)vxz(K8(qHq1Zu&#F88Xp#f-l6qAbtG&Z;ip#r^=0)P$xIg>uHF#PZ;T zUeC{M-N@V5*CSmm%xz&AZD*i+Mb!MW8>f9l&YvNGE1~rI!QT6J z{I7W1C)9;6w81t%@;pD->vA8og9LnO|5KeJv%smydANF#-r3e5`Zt!h+5Dq<9_8R zY`@|5mV`cMvjvVQp3g7Zg4rs`EQ~DG%qc67=~kK|e$DICl##CtL!9Z;(FdUU+ig)Q z%j`uS)v_<_EDp*-V7|4s8;h+NTo&zOUDsrghieoikCoV-?w`3|;|c2c1fXu|O@ft~ z!Frew`U*FxkOU zd1TplxqZwCQ@c2Bd;qi5OGE1?P_$&-W@wbxN(j;^=;pyL6EJA;siGl*&cjEqa0r4E zy&ho4hiC(Xb5N1-;8^@xqj2=2Rd_+yw+I=6ahb+pcDN5Y?(tQ?f=37R>lvOOC&SbQ z_bH&eO|Vy;O0kq_PLt#L5AQuTKe#FF7wmA^oh%)zNo0Nyx3*$l5%4dHq6OFcNigTF>4?{@?B11hO8uta247vnd?_XGb?1ms^V#B|YOFak4_)2;JBOBy zqqs$GdWSUN`sNPu4#-i;4OGplxd|rMbjo-cxSyiaF#sE;hfZEmb+)>+U|7wU5+dSn zxpp62zR((Ip)E39VC5V~iQQ)7MeYXR-wbA3S<#>g$Xy$6HTq0}#r=HVarq4P(}#zs z7|M)*0{}SqfBEo#a^w(S961}Ht*x=S;a@rHCA9_XH8xbYl~Pf;3MHm0q1p{b&qR9Q z5h>}aF6;>z2np^Ps*DOua7lS5el&4K6Gpsxbs0iC;=1#~4loaLi^n$6awq)l;3(4pEostx+ zY2sFx6;o3Rw7CT!jueSvp7+x>s3gO7?XRgvB1suXBD{f`s!rUOzU0p~)biT8JY|+r z^>Ih8-sYFPvhA17D`j=y614=MtjPNp*e%L2qRVcJ4x5kD?hHMb5;SPz;E$RSV~{ia z$9K!cf;nkX!*=PW)sv-qFJ#lFcBUSx-Y@%zvdB3G`p40eyQ<5mK8m6F;^ULN-ycXk z-~hvo^&p=02uPK4Iu`L1F<(SQ(QP&K+nqAW0+u37JwUDD?0Hz_f_!i&BA%u2!z`a4;Cc8I{RAy~ zX$4C9mI{foxg|0 zOO=1f5051fvLOB5st)wb{p>itb94wt0QN>4uRWrjMmo-OA<>c#JJH-!tzI;*qhyO) z1y8+lIh;(5e`%pp1WhXyvkkUf5`FLK(vB?*L!ZZY&A`~&H0&^(LY_ci$wq*${%z0d ziHqi1bPFqkv!y*(kWk1a-}baRc~X+ESvQoAb1&PmE<85u0?tEzD-yO(JARFxrI0{u z9gH-sobHfRqwJfc1L{OMBXF;A#Bz$GlbB{69;rAq+yM*>{3DAM@AHUVw8rm12(L*j zYxU=6##?P{TzNA8i_mGacVRxAoW&#hEq;qrX+K;*;XayBP1PeOM!hb@$SB1^YT6e+ z-bxpPQ_$*ekn43rbd`H=!s2y&EDO`&+BfRu-84>wpU^efrXcfT8{J_D=zFz246z`u z8?IB)Moo@3I}JQpg$st;bbl=qS%t_~y}7pV^}1wXzvJNw46UeRfu#r`6;}N4_`(8l zCTAeEGcyF8!#$K14GMwT?#Cs^5Pp-dOj-{(vbx%lvzNPAA2u4>CNca#t4G?Kk8W43 zt#qak^73ukqz4LpJ*(fqfyAi2ta58%M__xy3251oY_*2 zUR;H(-%937LfdCXgP6LjiQsfUOQ5PK-KL%!`9M_Xjt3+FQU53y?<<9qy0zjtzwd`WC))bDjN)QC zALau_kfV6KIliB8KE2EFLq606$x|XP3s~8#^~#6E;mOGFYZAQ`Wr?R42pIZ9u|3qA zhup0qrj>hd)g#Z@smbiUh`(=H*>WyhAGo2sN`MqpI!p>bgkDqI@sg3fN>8 zY0VfrUUz;(rH}Vnb9~f~`gAaZjfG|f+9ms@iZ_QJ0z!@=^ix{VA?5aTIrv;;Q+zCt zy6nw+&N~1*iitcwFAO*L1>`B20KYL@pSs=|JorfwR=ULG7P{7hE_Cq?1IX# zld9ZUMn0AMqTqlgt)1-boDfM~!*xkV9lPD1Z7M^&S`&fbLQ^}@)xW@%hi4{IHVgke zus~)q`78!JW>S8XR;>!td+70)PjGLzK6omF>Dt~(XCshuKz92O!YNR#^^rAXFCWHZ zTd#K7&N^G6DA)h9o}6yiJLg8xaOlv*lW)iB10f#^kt}))S&v{#P+;&KlDS7NP3m$! zwFEUcyd9Xu&i%hWc3lH9wTTtp~LxLE*JsBJNWQd<*rIt%{EZ)3A zja$TS$a+4O9xuFkakgyfM%9pBJrPHLq)wW?;gvZq$)H&2@}HHgvYXE@>^z4FhGRMUdceeu}b4Vxofb z0KkiY3jlP(LcE;eO-jT70Ptv8Nkt)WaBx08J~1&dSt%)5MKyJ01zBZnwa>a*+WP9c zW?IHJKs8xi^-sq7nzn`-R>qq8KvPpwGeeNAiG`E4rK_o>ovE#xwYiy-g^9hTv4@R; zi?xZJt*wo{v%RCMv%QV0lbwr;i?M5fy{nhKN3g4#r=xp-`xhTKuVA-;7=LGTZx>r1 zSEoQ%s|Yt^Ur)DCPun0bmk3|iuYn%E-riomK|TS&K|Y=#fqo$&!Cw9m-XXCep%EdW ziQy4pp;0lxUy~yJzeELje+>&s3=fEm2v3X(iHiFAH6|%8K0YBfHYqVVG$u1VAv-*& zC@MMgYf^r~%Q?L;H9a#it0*?RGC8L_wV)!gs39XJI6E;aJteg;^=o!|Qc-&Jx2&YP zoS4SKr0ne6^o;zR!f)Slvq}nc3kwS~iz>3d)qnd|{;i@Yr>wrXyt1gGp`s|GtmLIN zv$8z5rmD30dtOsTL1kq{c~whIZGC-JMN?g6Q&Uq)No&UUuEMIelDdwPhQZ>NiK?c~ zhQ{{Bw$A#_p~n89@~+v&{;Ar5x#rQ;w&J|@>cZZt%;DzZuEvVi=7zrJs`2LH?$(;2 zw#vS?`pM3U$-aiAfrjqxuC~s;uD;=c?zXi3A;(1&=p|#1c)RoKDg>OEluKUaSaS zE;v%_vw!}0dXaj@#T9q5>O>RAiyi(_-|*W&=-16^eF7+SZQ2{+B-kmb>aa_n58{JO zza;rnI^)rnPBj!|GFQ~dvK=N5R?9V1c`V9MY3+b$7^B*2p^a{IZi8)*2Y9VQt6Zs> zR?6)fp29-6q|xD-EBOWq->Hrrp_T&9IZ_NAOir`ol@|-B*s0?YrJQqMNU!l~r9R%D zDvi8?QE8ne#d;zE*VD0ob8?TZ3bjGYA3Zs?r_kKn&ng}T`zq!@XHI;`1W4En@Irh2 zSx@iF^T{=0_Y5j*Mf2+o|2_zgQ5HA5n?1%```ViO>1@N(K?pyX(CouzYeGgaQ_+QiE>wn6e=J4OPf| zyBZ=vJM$)+@1FKWc|5ev0Rzbg^@EoxEiU2K`qr6AD26D~!yLv0<52t(+)p_5Q zd0^h6rMPz3J%$~mG4j~Mj^q_P(QdsdNqBZ8U6`NKl#P8;B01k>>Hr&UJ%o8-sMaZr zKc_;jc~kt7J32BK58W0@)4C4f?1}+3D8^Oi$}#7xmDYQ$_mS+5xc7-AD@@V?c+i5o zvl-J09<^Fgl~xRxz$N38s1SJqi2C!$`dLt$8i1wM zEm`dx_&lZbz&Uh$$Tj;CoTg?|g5{Ii{s2**T$AN%K^wPJbB5H^AOY#{}j6ts~EqC)rwK7WR&bJl5q@d^nN{ym3lH36vRvsp6nhluaV!MV$F^;P&(k=ZJ6Ur9F!`@ZF% z@HWkKfdOw_w4^0#%!#yzN=BzUoVr83lRZP|CW|y`N(C*EQLXSOsG1n^`Djteat{mZ z)ai{fiuX^gF|~uL%w%t88BS$qxFL7ss&3KJ^(Y~OozY08C(kJ$nk$F>^V?t9>8@ep zBY$6;>V;U=XbQA?KsV~gMA+RdbCH4eM|+Vs)E=0M6pfvPa1xx1(Ti-bSP=Km=H6swYl(fOXUO|Lk6Tf&!T`%sDtOOW1 zI^f^SCI7jueicvtLj6iK{Jm)IXHDd9lY6;*DY*QrLh^s2{w_ZL^xb}&IQZYo>we*W zxrM*)`!8wo_kT8||14DgbF9B{0KiYF^0!(3f#?4cF8`hErqC+dD7 zf35!i85I3SHoh$4e+!X*vHa@)@85u*pVQyg`~F`(uCfwPe{D3V08{|gi}y%F@e}la E04}y}Z~y=R literal 0 HcmV?d00001 diff --git a/docs/source/diagrams/svg/queue_pointers.svg b/docs/source/diagrams/svg/queue_pointers.svg new file mode 100644 index 000000000..79adfc4ee --- /dev/null +++ b/docs/source/diagrams/svg/queue_pointers.svg @@ -0,0 +1,148 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Producerpointer + + + + + + Consumerpointer + + + + + + “Shadow”consumerpointer + + + + + + + + + + + + + + + + + + Controlled by NIC + + + + + + In progress + + + + \ No newline at end of file diff --git a/docs/source/gettingstarted.rst b/docs/source/gettingstarted.rst new file mode 100644 index 000000000..08b15d996 --- /dev/null +++ b/docs/source/gettingstarted.rst @@ -0,0 +1,626 @@ +.. _gettingstarted: + +=============== +Getting Started +=============== + +Join the Corundum community +=========================== + +To stay up to date with the latest developments and get support, consider joining the `mailing list `_ and `Slack workspace `_. + +Obtaining the source code +========================= + +The main `upstream repository for Corundum `_ is located on `GitHub `_. There are two main ways to download the source code - downloading an archive, or cloning with git. + +To clone via HTTPS, run:: + + $ git clone https://github.com/corundum/corundum.git + +To clone via SSH, run:: + + $ git clone git@github.com:corundum/corundum.git + +Alternatively, download a zip file:: + + $ wget https://github.com/corundum/corundum/archive/refs/heads/master.zip + $ unzip master.zip + +Or a gzipped tar archive file:: + + $ wget https://github.com/corundum/corundum/archive/refs/heads/master.tar.gz + $ tar xvf master.tar.gz + +There is also a `mirror of the repository `_ on `gitee `_, here are the equivalent commands:: + + $ git clone https://gitee.com/alexforencich/corundum.git + $ git clone git@gitee.com:alexforencich/corundum.git + $ wget https://gitee.com/alexforencich/corundum/repository/archive/master.zip + $ wget https://gitee.com/alexforencich/corundum/repository/archive/master.tar.gz + +Setting up the FPGA development environment +=========================================== + +Corundum currently uses `Icarus Verilog `_ and `cocotb `_ for simulation. Linux is the recommended operating system for a development environment due to the use of symlinks (which can cause problems on Windows as they are not supported by windows filesystems), however WSL may also work well. + +The required system packages are: + +* Python 3 (``python`` or ``python3``, depending on distribution) +* Icarus Verilog (``iverilog``) +* GTKWave (``gtkwave``) + +The required python packages are: + +* ``cocotb`` +* ``cocotb-bus`` +* ``cocotb-test`` +* ``cocotbext-axi`` +* ``cocotbext-eth`` +* ``cocotbext-pcie`` +* ``pytest`` +* ``scapy`` + +Recommended additional python packages: + +* ``tox`` (to run pytest inside a python virtual environment) +* ``pytest-xdist`` (to run tests in parallel with `pytest -n auto`) +* ``pytest-sugar`` (makes pytest output a bit nicer) + +It is recommended to install the required system packages via the system package manager (``apt``, ``yum``, ``pacman``, etc.) and then install the required Python packages as user packages via ``pip`` (or ``pip3``, depending on distribution). + +Running tests +============= + +Once the packages are installed, you should be able to run the tests. There are several ways to do this. + +First, all tests can be run by runing ``tox`` in the repo root. In this case, tox will set up a python virtual environment and install all python dependencies inside the virtual environment. Additionally, tox will run pytest as ``pytest -n auto`` so it will run tests in parallel on multiple CPUs. :: + + $ cd /path/to/corundum/ + $ tox + py3 create: /home/alex/Projects/corundum/.tox/py3 + py3 installdeps: pytest == 6.2.5, pytest-xdist == 2.4.0, pytest-split == 0.4.0, cocotb == 1.6.1, cocotb-test == 0.2.1, cocotbext-axi == 0.1.18, cocotbext-eth == 0.1.18, cocotbext-pcie == 0.1.20, scapy == 2.4.5 + py3 installed: attrs==21.4.0,cocotb==1.6.1,cocotb-bus==0.2.1,cocotb-test==0.2.1,cocotbext-axi==0.1.18,cocotbext-eth==0.1.18,cocotbext-pcie==0.1.20,execnet==1.9.0,iniconfig==1.1.1,packaging==21.3,pluggy==1.0.0,py==1.11.0,pyparsing==3.0.7,pytest==6.2.5,pytest-forked==1.4.0,pytest-split==0.4.0,pytest-xdist==2.4.0,scapy==2.4.5,toml==0.10.2 + py3 run-test-pre: PYTHONHASHSEED='4023917175' + py3 run-test: commands[0] | pytest -n auto + ============================= test session starts ============================== + platform linux -- Python 3.9.7, pytest-6.2.5, py-1.11.0, pluggy-1.0.0 + cachedir: .tox/py3/.pytest_cache + rootdir: /home/alex/Projects/corundum, configfile: tox.ini, testpaths: fpga, fpga/app + plugins: forked-1.4.0, split-0.4.0, cocotb-test-0.2.1, xdist-2.4.0 + gw0 [69] / gw1 [69] / gw2 [69] / gw3 [69] / gw4 [69] / gw5 [69] / gw6 [69] / gw7 [69] / gw8 [69] / gw9 [69] / gw10 [69] / gw11 [69] / gw12 [69] / gw13 [69] / gw14 [69] / gw15 [69] / gw16 [69] / gw17 [69] / gw18 [69] / gw19 [69] / gw20 [69] / gw21 [69] / gw22 [69] / gw23 [69] / gw24 [69] / gw25 [69] / gw26 [69] / gw27 [69] / gw28 [69] / gw29 [69] / gw30 [69] / gw31 [69] / gw32 [69] / gw33 [69] / gw34 [69] / gw35 [69] / gw36 [69] / gw37 [69] / gw38 [69] / gw39 [69] / gw40 [69] / gw41 [69] / gw42 [69] / gw43 [69] / gw44 [69] / gw45 [69] / gw46 [69] / gw47 [69] / gw48 [69] / gw49 [69] / gw50 [69] / gw51 [69] / gw52 [69] / gw53 [69] / gw54 [69] / gw55 [69] / gw56 [69] / gw57 [69] / gw58 [69] / gw59 [69] / gw60 [69] / gw61 [69] / gw62 [69] / gw63 [69] + ..................................................................... [100%] + ======================= 69 passed in 1534.87s (0:25:34) ======================== + ___________________________________ summary ____________________________________ + py3: commands succeeded + congratulations :) + +Second, all tests can be run by running ``pytest`` in the repo root. Running as ``pytest -n auto`` is recommended to run multiple tests in parallel on multiple CPUs. :: + + $ cd /path/to/corundum/ + $ pytest -n auto + ============================= test session starts ============================== + platform linux -- Python 3.9.7, pytest-6.2.5, py-1.10.0, pluggy-0.13.1 + rootdir: /home/alex/Projects/corundum, configfile: tox.ini, testpaths: fpga, fpga/app + plugins: split-0.3.0, parallel-0.1.0, cocotb-test-0.2.0, forked-1.3.0, metadata-1.11.0, xdist-2.4.0, html-3.1.1, cov-2.12.1, flake8-1.0.7 + gw0 [69] / gw1 [69] / gw2 [69] / gw3 [69] / gw4 [69] / gw5 [69] / gw6 [69] / gw7 [69] / gw8 [69] / gw9 [69] / gw10 [69] / gw11 [69] / gw12 [69] / gw13 [69] / gw14 [69] / gw15 [69] / gw16 [69] / gw17 [69] / gw18 [69] / gw19 [69] / gw20 [69] / gw21 [69] / gw22 [69] / gw23 [69] / gw24 [69] / gw25 [69] / gw26 [69] / gw27 [69] / gw28 [69] / gw29 [69] / gw30 [69] / gw31 [69] / gw32 [69] / gw33 [69] / gw34 [69] / gw35 [69] / gw36 [69] / gw37 [69] / gw38 [69] / gw39 [69] / gw40 [69] / gw41 [69] / gw42 [69] / gw43 [69] / gw44 [69] / gw45 [69] / gw46 [69] / gw47 [69] / gw48 [69] / gw49 [69] / gw50 [69] / gw51 [69] / gw52 [69] / gw53 [69] / gw54 [69] / gw55 [69] / gw56 [69] / gw57 [69] / gw58 [69] / gw59 [69] / gw60 [69] / gw61 [69] / gw62 [69] / gw63 [69] + ..................................................................... [100%] + ======================= 69 passed in in 2032.42s (0:33:52) ===================== + +Third, groups of tests can be run by running ``pytest`` in a subdirectory. Running as ``pytest -n auto`` is recommended to run multiple tests in parallel on multiple CPUs. :: + + $ cd /path/to/corundum/fpga/common/tb/rx_hash + $ pytest -n 4 + ============================= test session starts ============================== + platform linux -- Python 3.9.7, pytest-6.2.5, py-1.10.0, pluggy-0.13.1 + rootdir: /home/alex/Projects/corundum, configfile: tox.ini + plugins: split-0.3.0, parallel-0.1.0, cocotb-test-0.2.0, forked-1.3.0, metadata-1.11.0, xdist-2.4.0, html-3.1.1, cov-2.12.1, flake8-1.0.7 + gw0 [2] / gw1 [2] / gw2 [2] / gw3 [2] + .. [100%] + ============================== 2 passed in 37.49s ============================== + +Finally, individual tests can be run by runing ``make``. This method provides the capability of overriding parameters and enabling waveform dumps in FST format that are viewable in gtkwave. :: + + $ cd /path/to/corundum/fpga/common/tb/rx_hash + $ make WAVES=1 + make -f Makefile results.xml + make[1]: Entering directory '/home/alex/Projects/corundum/fpga/common/tb/rx_hash' + echo 'module iverilog_dump();' > iverilog_dump.v + echo 'initial begin' >> iverilog_dump.v + echo ' $dumpfile("rx_hash.fst");' >> iverilog_dump.v + echo ' $dumpvars(0, rx_hash);' >> iverilog_dump.v + echo 'end' >> iverilog_dump.v + echo 'endmodule' >> iverilog_dump.v + /usr/bin/iverilog -o sim_build/sim.vvp -D COCOTB_SIM=1 -s rx_hash -P rx_hash.DATA_WIDTH=64 -P rx_hash.KEEP_WIDTH=8 -s iverilog_dump -f sim_build/cmds.f -g2012 ../../rtl/rx_hash.v iverilog_dump.v + MODULE=test_rx_hash TESTCASE= TOPLEVEL=rx_hash TOPLEVEL_LANG=verilog \ + /usr/bin/vvp -M /home/alex/.local/lib/python3.9/site-packages/cocotb/libs -m libcocotbvpi_icarus sim_build/sim.vvp -fst + -.--ns INFO cocotb.gpi ..mbed/gpi_embed.cpp:76 in set_program_name_in_venv Did not detect Python virtual environment. Using system-wide Python interpreter + -.--ns INFO cocotb.gpi ../gpi/GpiCommon.cpp:99 in gpi_print_registered_impl VPI registered + 0.00ns INFO Running on Icarus Verilog version 11.0 (stable) + 0.00ns INFO Running tests with cocotb v1.7.0.dev0 from /home/alex/.local/lib/python3.9/site-packages/cocotb + 0.00ns INFO Seeding Python random module with 1643529566 + 0.00ns INFO Found test test_rx_hash.run_test + 0.00ns INFO Found test test_rx_hash.run_test + 0.00ns INFO Found test test_rx_hash.run_test + 0.00ns INFO Found test test_rx_hash.run_test + 0.00ns INFO Found test test_rx_hash.run_test + 0.00ns INFO Found test test_rx_hash.run_test + 0.00ns INFO Found test test_rx_hash.run_test + 0.00ns INFO Found test test_rx_hash.run_test + 0.00ns INFO running run_test (1/8) + 0.00ns INFO AXI stream source + 0.00ns INFO cocotbext-axi version 0.1.19 + 0.00ns INFO Copyright (c) 2020 Alex Forencich + 0.00ns INFO https://github.com/alexforencich/cocotbext-axi + 0.00ns INFO AXI stream source configuration: + 0.00ns INFO Byte size: 8 bits + 0.00ns INFO Data width: 64 bits (8 bytes) + 0.00ns INFO AXI stream source signals: + 0.00ns INFO tdata width: 64 bits + 0.00ns INFO tdest: not present + 0.00ns INFO tid: not present + 0.00ns INFO tkeep width: 8 bits + 0.00ns INFO tlast width: 1 bits + 0.00ns INFO tready: not present + 0.00ns INFO tuser: not present + 0.00ns INFO tvalid width: 1 bits + 0.00ns INFO Reset de-asserted + 0.00ns INFO Reset de-asserted + FST info: dumpfile rx_hash.fst opened for output. + 4.00ns INFO Reset asserted + 4.00ns INFO Reset asserted + 12.00ns INFO Reset de-asserted + 12.00ns INFO Reset de-asserted + 20.00ns INFO TX frame: AxiStreamFrame(tdata=bytearray(b'\xda\xd1\xd2\xd3\xd4\xd5ZQRSTU\x90\x00\x00'), tkeep=None, tid=None, tdest=None, tuser=None, sim_time_start=20000, sim_time_end=None) + 28.00ns INFO TX frame: AxiStreamFrame(tdata=bytearray(b'\xda\xd1\xd2\xd3\xd4\xd5ZQRSTU\x90\x00\x00\x01'), tkeep=None, tid=None, tdest=None, tuser=None, sim_time_start=28000, sim_time_end=None) + 36.00ns INFO TX frame: AxiStreamFrame(tdata=bytearray(b'\xda\xd1\xd2\xd3\xd4\xd5ZQRSTU\x90\x00\x00\x01\x02'), tkeep=None, tid=None, tdest=None, tuser=None, sim_time_start=36000, sim_time_end=None) + 40.00ns INFO RX hash: 0x00000000 (expected: 0x00000000) type: HashType.0 (expected: HashType.0) + 48.00ns INFO TX frame: AxiStreamFrame(tdata=bytearray(b'\xda\xd1\xd2\xd3\xd4\xd5ZQRSTU\x90\x00\x00\x01\x02\x03'), tkeep=None, tid=None, tdest=None, tuser=None, sim_time_start=48000, sim_time_end=None) + 48.00ns INFO RX hash: 0x00000000 (expected: 0x00000000) type: HashType.0 (expected: HashType.0) + 56.00ns INFO RX hash: 0x00000000 (expected: 0x00000000) type: HashType.0 (expected: HashType.0) + + + ################ skip a very large number of lines ################ + + + 252652.01ns INFO TX frame: AxiStreamFrame(tdata=bytearray(b'\xda\xd1\xd2\xd3\xd4\xd5ZQRSTU\x08\x00E\x00\x00V\x00\x8b\x00\x00@\x06d\xff\n\x01\x00\x8b\n\x02\x00\x8b\x00\x8b\x10\x8b\x00\x00\x00\x00\x00\x00\x00\x00P\x02 \x00ms\x00\x00\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !"#$%&\'()*+,-'), tkeep=None, tid=None, tdest=None, tuser=None, sim_time_start=252652007, sim_time_end=None) + 252744.01ns INFO RX hash: 0xa2a55ee3 (expected: 0xa2a55ee3) type: HashType.TCP|IPV4 (expected: HashType.TCP|IPV4) + 252860.01ns INFO TX frame: AxiStreamFrame(tdata=bytearray(b'\xda\xd1\xd2\xd3\xd4\xd5ZQRSTU\x08\x00E\x00\x00V\x00\x8c\x00\x00@\x06d\xfc\n\x01\x00\x8c\n\x02\x00\x8c\x00\x8c\x10\x8c\x00\x00\x00\x00\x00\x00\x00\x00P\x02 \x00mo\x00\x00\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !"#$%&\'()*+,-'), tkeep=None, tid=None, tdest=None, tuser=None, sim_time_start=252860007, sim_time_end=None) + 252952.01ns INFO RX hash: 0x6308c813 (expected: 0x6308c813) type: HashType.TCP|IPV4 (expected: HashType.TCP|IPV4) + 252960.01ns INFO run_test passed + 252960.01ns INFO ************************************************************************************** + ** TEST STATUS SIM TIME (ns) REAL TIME (s) RATIO (ns/s) ** + ************************************************************************************** + ** test_rx_hash.run_test PASS 11144.00 1.14 9781.95 ** + ** test_rx_hash.run_test PASS 44448.00 3.80 11688.88 ** + ** test_rx_hash.run_test PASS 12532.00 1.40 8943.27 ** + ** test_rx_hash.run_test PASS 49984.00 4.42 11302.44 ** + ** test_rx_hash.run_test PASS 13088.00 1.54 8479.38 ** + ** test_rx_hash.run_test PASS 52208.00 4.62 11308.18 ** + ** test_rx_hash.run_test PASS 13940.00 1.65 8461.27 ** + ** test_rx_hash.run_test PASS 55616.00 5.03 11046.45 ** + ************************************************************************************** + ** TESTS=8 PASS=8 FAIL=0 SKIP=0 252960.01 25.11 10073.76 ** + ************************************************************************************** + + make[1]: Leaving directory '/home/alex/Projects/corundum/fpga/common/tb/rx_hash' + +Setting up the FPGA build environment (Vivado) +============================================== + +Building FPGA configurations for Xilinx devices requires `Vivado `_. Linux is the recommended operating system for a build environment due to the use of symlinks (which can cause problems on Windows) and makefiles for build automation. Additionally, Vivado uses more CPU cores for building on Linux than on Windows. It is not recommended to run Vivado inside of a virtual machine as Vivado uses a significant amount of RAM during the build process. Download and install the appropriate version of Vivado. Make sure to install device support for your target device; support for other devices can be disabled to save disk space. + +Licenses may be required, depending on the target device. A bare install of Vivado without any licenses runs in "WebPACK" mode and has limited device support. If your target device is on the `WebPACK device list `_, then no Vivado license is required. Otherwise, you will need access to a Vivado license to build the design. + +Additionally, the 100G MAC IP cores on UltraScale and UltraScale+ require separate licenses. These licenses are free of charge, and can be generated for `UltraScale `_ and `UltraScale+ `_. If your target design uses the 100G CMAC IP, then you will need one of these licenses to build the design. + +For example: if you want to build a 100G design for an Alveo U50, you will not need a Vivado license as the U50 is supported under WebPACK, but you will need to generate a (free-of-charge) license for the CMAC IP for UltraScale+. + +Before building a design with Vivado, you'll have to source the appropriate settings file. For example:: + + $ source /opt/Xilinx/Vivado/2020.2/settings64.sh + $ make + +Building the FPGA configuration +=============================== + +Each design contains a set of makefiles for automating the build process. To use the makefile, simply source the settings file for the required toolchain and then run ``make``. Note that the repository makes significant use of symbolic links, so it is highly recommended to build the design under Linux. + +For example:: + + $ cd /path/to/corundum/fpga/mqnic/[board]/fpga_[variant]/fpga + $ source /opt/Xilinx/Vivado/2020.2/settings64.sh + $ make + +Building the driver +=================== + +To build the driver, you will first need to install the required compiler and kernel source code packages. After these packages are installed, simply run ``make``. :: + + $ cd /path/to/corundum/modules/mqnic + $ make + +Note that the driver currently does not support RHEL, centos, and related distributions that use very old and significantly modified kernels where the reported kernel version number is not a reliable of the internal kernel API. + +Building the userspace tools +============================ + +To build the driver, you will first need to install the required compiler packages. After these packages are installed, simply run ``make``. :: + + $ cd /path/to/corundum/utils + $ make + +Loading the FPGA design +======================= + +There are three main ways for loading Corundum on to an FPGA board. The first is via JTAG, into volatile FPGA configuration memory. This is best for development and debugging, especially when complemented with a baseline design with the same PCIe interface configuration stored in flash. The second is via indirect JTAG, into nonvolatile on-card flash memory. This is quite slow. The third is via PCI express, into nonvolatile on-card memory. This is the fastest method of programming the flash, but it requires the board to already be running the Corundum design. + +For a card that's not already running Corundum, there are two options for programming the flash. The first is to use indirect JTAG, but this is very slow. The second is to first load the design via JTAG into volatile configuration memory, then perform a warm reboot, and finally write the design into flash via PCIe with the ``mqnic-fw`` utility. + +Loading the design via JTAG into volatile configuration memory with Vivado is straightforward: install the card into a host computer, attach the JTAG cable, power up the host computer, and use Vivado to connect and load the bit file into the FPGA. When using the makefile, run ``make program`` to program the device. If physical access is a problem, it is possible to run a hardware server instance on the host computer and connect to the hardware server over the network. Once the design is loaded into the FPGA, perform either a hot reset (via ``pcie_hot_reset.sh`` or ``mqnic-fw -t``, but only if the card was enumerated at boot and the PCIe configuration has not changed) or a warm reboot. + +Loading the design via indirect JTAG into nonvolatile memory with Vivado requires basically the same steps as loading it into volatile configuration memory, the main difference is that the configuration flash image must first be generated by running ``make fpga.mcs`` after using make to generate the bit file. Once this file is generated, connect with the hardware manager, add the configuration memory device (check the makefile for the part number), and program the flash. After the programming operation is complete, boot the FPGA from the configuration memory, either via Vivado (right click -> boot from configuration memory) or by performing a cold reboot (full shut down, then power on). When using the makefile, run ``make flash`` to generate the flash images, program the flash via indirect JTAG, and boot the FPGA from the configuration memory. Finally, reboot the host computer to re-enumerate the PCIe bus. + +Loading the design via PCI express is straightforward: use the ``mqnic-fw`` utility to load the bit file into flash, then trigger an FPGA reboot to load the new design. This does not require the kernel module to be loaded. With the kernel module loaded, point ``mqnic-fw`` either to ``/dev/mqnic`` or to one of the associated network interfaces. Without the kernel module loaded, point ``mqnic-fw`` either to the raw PCIe ID, or to ``/sys/bus/pci/devices//resource0``; check ``lspci`` for the PCIe ID. Use ``-w`` to specify the bit file to load, then ``-b`` to command the FPGA to reset and reload its configuration from flash. You can also use ``-t`` to trigger a hot reset to reset the design. + +Query device information with ``mqnic-fw``, with no kernel module loaded:: + + $ sudo ./mqnic-fw -d 81:00.0 + PCIe ID (device): 0000:81:00.0 + PCIe ID (upstream port): 0000:80:01.1 + FPGA ID: 0x04b77093 + FPGA part: XCU50 + FW ID: 0x00000000 + FW version: 0.0.1.0 + Board ID: 0x10ee9032 + Board version: 1.0.0.0 + Build date: 2022-01-05 08:33:23 UTC (raw 0x61d557d3) + Git hash: ddd7e639 + Release info: 00000000 + Flash type: SPI + Flash format: 0x00048100 + Data width: 4 + Manufacturer ID: 0x20 + Memory type: 0xbb + Memory capacity: 0x21 + Flash size: 128 MB + Write buffer size: 256 B + Erase block size: 4096 B + Flash segment 0: start 0x00000000 length 0x01002000 + Flash segment 1: start 0x01002000 length 0x06ffe000 + Selected: segment 1 start 0x01002000 length 0x06ffe000 + +Write design into nonvolatile flash memory with ``mqnic-fw``, with no kernel module loaded:: + + $ sudo ./mqnic-fw -d 81:00.0 -w ../fpga/mqnic/AU50/fpga_100g/fpga/fpga.bit + PCIe ID (device): 0000:81:00.0 + PCIe ID (upstream port): 0000:80:01.1 + FPGA ID: 0x04b77093 + FPGA part: XCU50 + FW ID: 0x00000000 + FW version: 0.0.1.0 + Board ID: 0x10ee9032 + Board version: 1.0.0.0 + Build date: 2022-01-05 08:33:23 UTC (raw 0x61d557d3) + Git hash: ddd7e639 + Release info: 00000000 + Flash type: SPI + Flash format: 0x00048100 + Data width: 4 + Manufacturer ID: 0x20 + Memory type: 0xbb + Memory capacity: 0x21 + Flash size: 128 MB + Write buffer size: 256 B + Erase block size: 4096 B + Flash segment 0: start 0x00000000 length 0x01002000 + Flash segment 1: start 0x01002000 length 0x06ffe000 + Selected: segment 1 start 0x01002000 length 0x06ffe000 + Erasing flash... + Start address: 0x01002000 + Length: 0x01913000 + Erase address 0x02910000, length 0x00005000 (99%) + Writing flash... + Start address: 0x01002000 + Length: 0x01913000 + Write address 0x02910000, length 0x00005000 (99%) + Verifying flash... + Start address: 0x01002000 + Length: 0x01913000 + Read address 0x02910000, length 0x00005000 (99%) + Programming succeeded! + +Reboot FPGA to load design from flash with ``mqnic-fw``, with no kernel module loaded:: + + $ sudo ./mqnic-fw -d 81:00.0 -b + PCIe ID (device): 0000:81:00.0 + PCIe ID (upstream port): 0000:80:01.1 + FPGA ID: 0x04b77093 + FPGA part: XCU50 + FW ID: 0x00000000 + FW version: 0.0.1.0 + Board ID: 0x10ee9032 + Board version: 1.0.0.0 + Build date: 2022-01-05 08:33:23 UTC (raw 0x61d557d3) + Git hash: ddd7e639 + Release info: 00000000 + Flash type: SPI + Flash format: 0x00048100 + Data width: 4 + Manufacturer ID: 0x20 + Memory type: 0xbb + Memory capacity: 0x21 + Flash size: 128 MB + Write buffer size: 256 B + Erase block size: 4096 B + Flash segment 0: start 0x00000000 length 0x01002000 + Flash segment 1: start 0x01002000 length 0x06ffe000 + Selected: segment 1 start 0x01002000 length 0x06ffe000 + Preparing to reset device... + Disabling PCIe fatal error reporting on port... + No driver bound + Triggering IPROG to reload FPGA... + Removing device... + Performing hot reset on upstream port... + Rescanning on upstream port... + Success, device is online! + +Loading the kernel module +========================= + +Once the kernel module is built, load it with ``insmod``:: + + $ sudo insmod mqnic.ko + +When the driver loads, it will print some debug information:: + + [ 1502.394486] mqnic 0000:81:00.0: mqnic PCI probe + [ 1502.394494] mqnic 0000:81:00.0: Vendor: 0x1234 + [ 1502.394496] mqnic 0000:81:00.0: Device: 0x1001 + [ 1502.394498] mqnic 0000:81:00.0: Subsystem vendor: 0x10ee + [ 1502.394500] mqnic 0000:81:00.0: Subsystem device: 0x9032 + [ 1502.394501] mqnic 0000:81:00.0: Class: 0x020000 + [ 1502.394504] mqnic 0000:81:00.0: PCI ID: 0000:81:00.0 + [ 1502.394511] mqnic 0000:81:00.0: Max payload size: 512 bytes + [ 1502.394513] mqnic 0000:81:00.0: Max read request size: 512 bytes + [ 1502.394515] mqnic 0000:81:00.0: Link capability: gen 3 x16 + [ 1502.394516] mqnic 0000:81:00.0: Link status: gen 3 x16 + [ 1502.394518] mqnic 0000:81:00.0: Relaxed ordering: enabled + [ 1502.394520] mqnic 0000:81:00.0: Phantom functions: disabled + [ 1502.394521] mqnic 0000:81:00.0: Extended tags: enabled + [ 1502.394522] mqnic 0000:81:00.0: No snoop: enabled + [ 1502.394523] mqnic 0000:81:00.0: NUMA node: 1 + [ 1502.394531] mqnic 0000:81:00.0: 126.016 Gb/s available PCIe bandwidth (8.0 GT/s PCIe x16 link) + [ 1502.394554] mqnic 0000:81:00.0: enabling device (0000 -> 0002) + [ 1502.394587] mqnic 0000:81:00.0: Control BAR size: 16777216 + [ 1502.396014] mqnic 0000:81:00.0: Device-level register blocks: + [ 1502.396016] mqnic 0000:81:00.0: type 0xffffffff (v 0.0.1.0) + [ 1502.396019] mqnic 0000:81:00.0: type 0x0000c000 (v 0.0.1.0) + [ 1502.396021] mqnic 0000:81:00.0: type 0x0000c004 (v 0.0.1.0) + [ 1502.396023] mqnic 0000:81:00.0: type 0x0000c080 (v 0.0.1.0) + [ 1502.396025] mqnic 0000:81:00.0: type 0x0000c120 (v 0.0.1.0) + [ 1502.396027] mqnic 0000:81:00.0: type 0x0000c140 (v 0.0.1.0) + [ 1502.396029] mqnic 0000:81:00.0: type 0x0000c150 (v 0.0.1.0) + [ 1502.396038] mqnic 0000:81:00.0: FPGA ID: 0x04b77093 + [ 1502.396040] mqnic 0000:81:00.0: FW ID: 0x00000000 + [ 1502.396041] mqnic 0000:81:00.0: FW version: 0.0.1.0 + [ 1502.396043] mqnic 0000:81:00.0: Board ID: 0x10ee9032 + [ 1502.396044] mqnic 0000:81:00.0: Board version: 1.0.0.0 + [ 1502.396046] mqnic 0000:81:00.0: Build date: 2022-03-03 07:39:57 UTC (raw: 0x622070cd) + [ 1502.396049] mqnic 0000:81:00.0: Git hash: 8851b3b1 + [ 1502.396051] mqnic 0000:81:00.0: Release info: 00000000 + [ 1502.396056] mqnic 0000:81:00.0: IF offset: 0x00000000 + [ 1502.396057] mqnic 0000:81:00.0: IF count: 1 + [ 1502.396059] mqnic 0000:81:00.0: IF stride: 0x01000000 + [ 1502.396060] mqnic 0000:81:00.0: IF CSR offset: 0x00080000 + [ 1502.396065] mqnic 0000:81:00.0: Resetting Alveo CMS + [ 1502.613317] mqnic 0000:81:00.0: Read 4 MACs from Alveo BMC + [ 1502.624743] mqnic 0000:81:00.0: registered PHC (index 5) + [ 1502.624748] mqnic 0000:81:00.0: Creating interface 0 + [ 1502.624798] mqnic 0000:81:00.0: Interface-level register blocks: + [ 1502.624799] mqnic 0000:81:00.0: type 0x0000c001 (v 0.0.2.0) + [ 1502.624801] mqnic 0000:81:00.0: type 0x0000c010 (v 0.0.1.0) + [ 1502.624803] mqnic 0000:81:00.0: type 0x0000c020 (v 0.0.1.0) + [ 1502.624804] mqnic 0000:81:00.0: type 0x0000c030 (v 0.0.1.0) + [ 1502.624805] mqnic 0000:81:00.0: type 0x0000c021 (v 0.0.1.0) + [ 1502.624806] mqnic 0000:81:00.0: type 0x0000c031 (v 0.0.1.0) + [ 1502.624807] mqnic 0000:81:00.0: type 0x0000c003 (v 0.0.1.0) + [ 1502.624811] mqnic 0000:81:00.0: IF features: 0x00000711 + [ 1502.624812] mqnic 0000:81:00.0: Max TX MTU: 9214 + [ 1502.624813] mqnic 0000:81:00.0: Max RX MTU: 9214 + [ 1502.624816] mqnic 0000:81:00.0: Event queue offset: 0x00100000 + [ 1502.624817] mqnic 0000:81:00.0: Event queue count: 32 + [ 1502.624818] mqnic 0000:81:00.0: Event queue stride: 0x00000020 + [ 1502.624822] mqnic 0000:81:00.0: TX queue offset: 0x00200000 + [ 1502.624823] mqnic 0000:81:00.0: TX queue count: 8192 + [ 1502.624824] mqnic 0000:81:00.0: TX queue stride: 0x00000020 + [ 1502.624827] mqnic 0000:81:00.0: TX completion queue offset: 0x00400000 + [ 1502.624828] mqnic 0000:81:00.0: TX completion queue count: 8192 + [ 1502.624829] mqnic 0000:81:00.0: TX completion queue stride: 0x00000020 + [ 1502.624832] mqnic 0000:81:00.0: RX queue offset: 0x00600000 + [ 1502.624833] mqnic 0000:81:00.0: RX queue count: 256 + [ 1502.624834] mqnic 0000:81:00.0: RX queue stride: 0x00000020 + [ 1502.624838] mqnic 0000:81:00.0: RX completion queue offset: 0x00700000 + [ 1502.624838] mqnic 0000:81:00.0: RX completion queue count: 256 + [ 1502.624839] mqnic 0000:81:00.0: RX completion queue stride: 0x00000020 + [ 1502.624841] mqnic 0000:81:00.0: Max desc block size: 8 + [ 1502.632850] mqnic 0000:81:00.0: Port-level register blocks: + [ 1502.632855] mqnic 0000:81:00.0: type 0x0000c040 (v 0.0.1.0) + [ 1502.632860] mqnic 0000:81:00.0: Scheduler type: 0x0000c040 + [ 1502.632861] mqnic 0000:81:00.0: Scheduler offset: 0x00800000 + [ 1502.632862] mqnic 0000:81:00.0: Scheduler channel count: 8192 + [ 1502.632863] mqnic 0000:81:00.0: Scheduler channel stride: 0x00000004 + [ 1502.632864] mqnic 0000:81:00.0: Scheduler count: 1 + [ 1502.632866] mqnic 0000:81:00.0: Port count: 1 + [ 1503.217179] mqnic 0000:81:00.0: Registered device mqnic0 + +The driver will attempt to read MAC addresses from the card. If it fails, it will fall back on random MAC addresses. On some cards, the MAC addresses are fixed and cannot be changed, on other cards they are written to use-accessible EEPROM and as such can be changed. Some cards with EEPROM come with blank EEPROMs, so if you want a persistent MAC address, you'll have to write a base MAC address into the EEPROM. And finally, some cards do not have an EEPROM for storing MAC addresses, and persistent MAC addresses are not currently supported on these cards. + +Testing the design +================== + +To test the design, connect it to another NIC, either directly with a DAC cable or similar, or via a switch. + +Before performing any testing, an IP address must be assigned through the Linux kernel. There are various ways to do this, depending on the distribution in question. For example, using ``iproute2``:: + + $ sudo ip link set dev enp129s0 up + $ sudo ip addr add 10.0.0.2/24 dev enp129s0 + +You can also change the MTU setting:: + + $ sudo ip link set mtu 9000 dev enp129s0 + +Note that NetworkManager can fight over the network interface configuration (depending on the linux distribution). If the IP address disappears from the interface, then this is likely the fault of NetworkManager as it attempts to dynamically configure the interface. One solution for this is simply to use NetworkManager to configure the interface instead of iproute2. Another is to statically configure the interface using configuration files in ``/etc/network/interfaces`` so that NetworkManager will leave it alone. + +One the card is configured, using ``ping`` is a good first test:: + + $ ping 10.0.0.1 + PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data. + 64 bytes from 10.0.0.1: icmp_seq=1 ttl=64 time=0.221 ms + 64 bytes from 10.0.0.1: icmp_seq=2 ttl=64 time=0.109 ms + ^C + --- 10.0.0.1 ping statistics --- + 2 packets transmitted, 2 received, 0% packet loss, time 1052ms + rtt min/avg/max/mdev = 0.109/0.165/0.221/0.056 ms + +If ``ping`` works, then try ``iperf``. + +On the server:: + + $ iperf3 -s + ----------------------------------------------------------- + Server listening on 5201 + ----------------------------------------------------------- + Accepted connection from 10.0.0.2, port 54316 + [ 5] local 10.0.0.1 port 5201 connected to 10.0.0.2 port 54318 + [ ID] Interval Transfer Bitrate + [ 5] 0.00-1.00 sec 2.74 GBytes 23.6 Gbits/sec + [ 5] 1.00-2.00 sec 2.85 GBytes 24.5 Gbits/sec + [ 5] 2.00-3.00 sec 2.82 GBytes 24.2 Gbits/sec + [ 5] 3.00-4.00 sec 2.83 GBytes 24.3 Gbits/sec + [ 5] 4.00-5.00 sec 2.82 GBytes 24.2 Gbits/sec + [ 5] 5.00-6.00 sec 2.76 GBytes 23.7 Gbits/sec + [ 5] 6.00-7.00 sec 2.63 GBytes 22.6 Gbits/sec + [ 5] 7.00-8.00 sec 2.81 GBytes 24.2 Gbits/sec + [ 5] 8.00-9.00 sec 2.73 GBytes 23.5 Gbits/sec + [ 5] 9.00-10.00 sec 2.73 GBytes 23.4 Gbits/sec + [ 5] 10.00-10.00 sec 384 KBytes 7.45 Gbits/sec + - - - - - - - - - - - - - - - - - - - - - - - - - + [ ID] Interval Transfer Bitrate + [ 5] 0.00-10.00 sec 27.7 GBytes 23.8 Gbits/sec receiver + ----------------------------------------------------------- + Server listening on 5201 + ----------------------------------------------------------- + +On the client:: + + $ iperf3 -c 10.0.0.1 + Connecting to host 10.0.0.1, port 5201 + [ 5] local 10.0.0.2 port 54318 connected to 10.0.0.1 port 5201 + [ ID] Interval Transfer Bitrate Retr Cwnd + [ 5] 0.00-1.00 sec 2.74 GBytes 23.6 Gbits/sec 0 2.18 MBytes + [ 5] 1.00-2.00 sec 2.85 GBytes 24.5 Gbits/sec 0 2.18 MBytes + [ 5] 2.00-3.00 sec 2.82 GBytes 24.2 Gbits/sec 0 2.29 MBytes + [ 5] 3.00-4.00 sec 2.83 GBytes 24.3 Gbits/sec 0 2.40 MBytes + [ 5] 4.00-5.00 sec 2.82 GBytes 24.2 Gbits/sec 0 2.40 MBytes + [ 5] 5.00-6.00 sec 2.76 GBytes 23.7 Gbits/sec 0 2.65 MBytes + [ 5] 6.00-7.00 sec 2.63 GBytes 22.6 Gbits/sec 0 2.65 MBytes + [ 5] 7.00-8.00 sec 2.81 GBytes 24.2 Gbits/sec 0 2.65 MBytes + [ 5] 8.00-9.00 sec 2.73 GBytes 23.5 Gbits/sec 0 2.65 MBytes + [ 5] 9.00-10.00 sec 2.73 GBytes 23.4 Gbits/sec 0 2.65 MBytes + - - - - - - - - - - - - - - - - - - - - - - - - - + [ ID] Interval Transfer Bitrate Retr + [ 5] 0.00-10.00 sec 27.7 GBytes 23.8 Gbits/sec 0 sender + [ 5] 0.00-10.00 sec 27.7 GBytes 23.8 Gbits/sec receiver + + iperf Done. + +Finally, test the PTP synchronization performance with ``ptp4l`` from ``linuxptp``. + +On the server:: + + $ sudo ptp4l -i enp193s0np0 --masterOnly=1 -m --logSyncInterval=-3 + ptp4l[4463.798]: selected /dev/ptp2 as PTP clock + ptp4l[4463.799]: port 1: INITIALIZING to LISTENING on INIT_COMPLETE + ptp4l[4463.799]: port 0: INITIALIZING to LISTENING on INIT_COMPLETE + ptp4l[4471.745]: port 1: LISTENING to MASTER on ANNOUNCE_RECEIPT_TIMEOUT_EXPIRES + ptp4l[4471.746]: selected local clock ec0d9a.fffe.6821d4 as best master + ptp4l[4471.746]: port 1: assuming the grand master role + +On the client:: + + $ sudo ptp4l -i enp129s0 --slaveOnly=1 -m + ptp4l[642.961]: selected /dev/ptp5 as PTP clock + ptp4l[642.962]: port 1: INITIALIZING to LISTENING on INIT_COMPLETE + ptp4l[642.962]: port 0: INITIALIZING to LISTENING on INIT_COMPLETE + ptp4l[643.477]: port 1: new foreign master ec0d9a.fffe.6821d4-1 + ptp4l[647.478]: selected best master clock ec0d9a.fffe.6821d4 + ptp4l[647.478]: port 1: LISTENING to UNCALIBRATED on RS_SLAVE + ptp4l[648.233]: port 1: UNCALIBRATED to SLAVE on MASTER_CLOCK_SELECTED + ptp4l[648.859]: rms 973559315 max 1947121298 freq -41295 +/- 15728 delay 643 +/- 0 + ptp4l[649.860]: rms 698 max 1236 freq -44457 +/- 949 delay 398 +/- 0 + ptp4l[650.861]: rms 1283 max 1504 freq -42099 +/- 257 delay 168 +/- 0 + ptp4l[651.862]: rms 612 max 874 freq -42059 +/- 85 delay 189 +/- 1 + ptp4l[652.863]: rms 127 max 245 freq -42403 +/- 85 + ptp4l[653.865]: rms 58 max 81 freq -42612 +/- 36 delay 188 +/- 0 + ptp4l[654.866]: rms 21 max 36 freq -42603 +/- 12 delay 181 +/- 0 + ptp4l[655.867]: rms 6 max 12 freq -42584 +/- 7 delay 174 +/- 1 + ptp4l[656.868]: rms 14 max 26 freq -42606 +/- 12 + ptp4l[657.869]: rms 19 max 23 freq -42631 +/- 11 delay 173 +/- 0 + ptp4l[658.870]: rms 24 max 35 freq -42660 +/- 12 delay 173 +/- 0 + ptp4l[659.870]: rms 23 max 35 freq -42679 +/- 16 delay 173 +/- 0 + ptp4l[660.872]: rms 18 max 20 freq -42696 +/- 5 delay 170 +/- 0 + ptp4l[661.873]: rms 18 max 30 freq -42714 +/- 8 delay 167 +/- 1 + ptp4l[662.874]: rms 26 max 36 freq -42747 +/- 10 delay 168 +/- 0 + ptp4l[663.875]: rms 18 max 21 freq -42757 +/- 10 delay 167 +/- 0 + ptp4l[664.876]: rms 14 max 17 freq -42767 +/- 8 delay 167 +/- 1 + ptp4l[665.877]: rms 9 max 12 freq -42741 +/- 7 delay 168 +/- 2 + +In this case, ``ptp4l`` has converged to an offset of well under 100 ns, reporting a frequency difference of about -43 ppm. + +While ``ptp4l`` is syncing the clock, the kernel module will print some debug information:: + + [ 642.943481] mqnic 0000:81:00.0: mqnic_phc_adjfine scaled_ppm: 0 + [ 642.943487] mqnic 0000:81:00.0: mqnic_phc_adjfine adj: 0x400000000 + [ 647.860040] mqnic 0000:81:00.0: mqnic_start_xmit TX TS requested + [ 647.860084] mqnic 0000:81:00.0: mqnic_process_tx_cq TX TS requested + [ 648.090566] mqnic 0000:81:00.0: mqnic_phc_adjfine scaled_ppm: 2795012 + [ 648.090572] mqnic 0000:81:00.0: mqnic_phc_adjfine adj: 0x4000b2e18 + [ 648.090575] mqnic 0000:81:00.0: mqnic_phc_adjtime delta: -1947115961 + [ 648.215705] mqnic 0000:81:00.0: mqnic_phc_adjfine scaled_ppm: 3241067 + [ 648.215711] mqnic 0000:81:00.0: mqnic_phc_adjfine adj: 0x4000cf6da + [ 648.340845] mqnic 0000:81:00.0: mqnic_phc_adjfine scaled_ppm: 3199401 + [ 648.340851] mqnic 0000:81:00.0: mqnic_phc_adjfine adj: 0x4000ccc30 + [ 648.465995] mqnic 0000:81:00.0: mqnic_phc_adjfine scaled_ppm: 3161092 + [ 648.466001] mqnic 0000:81:00.0: mqnic_phc_adjfine adj: 0x4000ca4f5 + [ 648.591129] mqnic 0000:81:00.0: mqnic_phc_adjfine scaled_ppm: 3121946 + [ 648.591135] mqnic 0000:81:00.0: mqnic_phc_adjfine adj: 0x4000c7cdf + [ 648.716275] mqnic 0000:81:00.0: mqnic_phc_adjfine scaled_ppm: 3082853 + [ 648.716281] mqnic 0000:81:00.0: mqnic_phc_adjfine adj: 0x4000c54d7 + [ 648.841425] mqnic 0000:81:00.0: mqnic_phc_adjfine scaled_ppm: 3048881 + [ 648.841431] mqnic 0000:81:00.0: mqnic_phc_adjfine adj: 0x4000c320e + [ 648.966550] mqnic 0000:81:00.0: mqnic_phc_adjfine scaled_ppm: 3012985 + [ 648.966556] mqnic 0000:81:00.0: mqnic_phc_adjfine adj: 0x4000c0d4c + [ 649.091601] mqnic 0000:81:00.0: mqnic_phc_adjfine scaled_ppm: 2980479 + [ 649.091607] mqnic 0000:81:00.0: mqnic_phc_adjfine adj: 0x4000bec03 + [ 649.216740] mqnic 0000:81:00.0: mqnic_phc_adjfine scaled_ppm: 2950457 + [ 649.216746] mqnic 0000:81:00.0: mqnic_phc_adjfine adj: 0x4000bcd45 + [ 649.341844] mqnic 0000:81:00.0: mqnic_phc_adjfine scaled_ppm: 2922995 + [ 649.341850] mqnic 0000:81:00.0: mqnic_phc_adjfine adj: 0x4000bb126 + [ 649.466966] mqnic 0000:81:00.0: mqnic_phc_adjfine scaled_ppm: 2897658 + [ 649.466972] mqnic 0000:81:00.0: mqnic_phc_adjfine adj: 0x4000b9734 + [ 649.592007] mqnic 0000:81:00.0: mqnic_phc_adjfine scaled_ppm: 2875145 + [ 649.592013] mqnic 0000:81:00.0: mqnic_phc_adjfine adj: 0x4000b8026 + [ 649.717159] mqnic 0000:81:00.0: mqnic_phc_adjfine scaled_ppm: 2854962 + [ 649.717165] mqnic 0000:81:00.0: mqnic_phc_adjfine adj: 0x4000b6b7b + [ 649.776717] mqnic 0000:81:00.0: mqnic_start_xmit TX TS requested + [ 649.776761] mqnic 0000:81:00.0: mqnic_process_tx_cq TX TS requested + [ 649.842186] mqnic 0000:81:00.0: mqnic_phc_adjfine scaled_ppm: 2813737 + [ 649.842191] mqnic 0000:81:00.0: mqnic_phc_adjfine adj: 0x4000b4144 + [ 649.967434] mqnic 0000:81:00.0: mqnic_phc_adjfine scaled_ppm: 2800052 + [ 649.967440] mqnic 0000:81:00.0: mqnic_phc_adjfine adj: 0x4000b3341 + +In this case, the core clock frequency is slightly less than 250 MHz. You can compute the clock frequency in GHz like so:: + + >>> 2**32/0x4000b3341 + 0.24998931910318553 diff --git a/docs/source/glossary.rst b/docs/source/glossary.rst new file mode 100644 index 000000000..d1282efa2 --- /dev/null +++ b/docs/source/glossary.rst @@ -0,0 +1,46 @@ +.. _glossary: + +======== +Glossary +======== + +.. glossary:: + + AXI + Advanced eXtensible Interface + + BAR + Base Address Register + + DMA + Direct Memory Access + + FPGA + Field-Programmable Gate Array + + JTAG + Joint Test Action Group + + MAC + Media Access Control(ler) + + MSI + Message-Signaled Interrupt + + NIC + Network Interface Controller + + PCI + Peripheral Component Interconnect + + PCIe + PCI Express + + PHY + PHYsical layer (interface) + + PTP + Precision Time Protocol (IEEE 1588) + + RSS + Receive Side Scaling diff --git a/docs/source/index.rst b/docs/source/index.rst new file mode 100644 index 000000000..6392e675c --- /dev/null +++ b/docs/source/index.rst @@ -0,0 +1,67 @@ +.. _intro: + +============ +Introduction +============ + +Corundum is an open-source, high-performance :term:`FPGA`-based :term:`NIC` and platform for in-network compute. Features include a high performance datapath, 10G/25G/100G Ethernet, PCI express gen 3, a custom, high performance, tightly-integrated :term:`PCIe` :term:`DMA` engine, many (1000+) transmit, receive, completion, and event queues, scatter/gather DMA, :term:`MSI`, multiple interfaces, multiple ports per interface, per-port transmit scheduling including high precision TDMA, flow hashing, :term:`RSS`, checksum offloading, and native IEEE 1588 :term:`PTP` timestamping. A Linux driver is included that integrates with the Linux networking stack. Development and debugging is facilitated by an extensive simulation framework that covers the entire system from a simulation model of the driver and PCI express interface on one side to the Ethernet interfaces on the other side. + +Corundum has several unique architectural features. First, transmit, receive, completion, and event queue states are stored efficiently in block RAM or ultra RAM, enabling support for thousands of individually-controllable queues. These queues are associated with interfaces, and each interface can have multiple ports, each with its own independent scheduler. This enables extremely fine-grained control over packet transmission. Coupled with PTP time synchronization, this enables high precision TDMA. + +Corundum also provides an application section for implementing custom logic. The application section has a dedicated PCIe BAR for control and a number of interfaces that provide access to the core datapath and DMA infrastructure. + +The latest source code is available from the `Corundum GitHub repository `_. To stay up to date with the latest developments and get support, consider joining the `mailing list `_ and `Slack workspace `_. + +Corundum currently supports devices from both Xilinx and Intel, on boards from several different manufacturers. Designs are included for the following FPGA boards; see :ref:`device_list` for more details: + +* Alpha Data ADM-PCIE-9V3 (Xilinx Virtex UltraScale+ XCVU3P) +* Exablaze ExaNIC X10/Cisco Nexus K35-S (Xilinx Kintex UltraScale XCKU035) +* Exablaze ExaNIC X25/Cisco Nexus K3P-S (Xilinx Kintex UltraScale+ XCKU3P) +* Silicom fb2CG\@KU15P (Xilinx Kintex UltraScale+ XCKU15P) +* NetFPGA SUME (Xilinx Virtex 7 XC7V690T) +* Intel Stratix 10 MX dev kit (Intel Stratix 10 MX 1SM21CHU1F53E1VG) +* Xilinx Alveo U50 (Xilinx Virtex UltraScale+ XCU50) +* Xilinx Alveo U200 (Xilinx Virtex UltraScale+ XCU200) +* Xilinx Alveo U250 (Xilinx Virtex UltraScale+ XCU250) +* Xilinx Alveo U280 (Xilinx Virtex UltraScale+ XCU280) +* Xilinx VCU108 (Xilinx Virtex UltraScale XCVU095) +* Xilinx VCU118 (Xilinx Virtex UltraScale+ XCVU9P) +* Xilinx VCU1525 (Xilinx Virtex UltraScale+ XCVU9P) +* Xilinx ZCU106 (Xilinx Zynq UltraScale+ XCZU7EV) + +Publications +============ + +- A. Forencich, A. C. Snoeren, G. Porter, G. Papen, *Corundum: An Open-Source 100-Gbps NIC,* in FCCM'20. (`FCCM Paper `_, `FCCM Presentation `_) + +- J. A. Forencich, *System-Level Considerations for Optical Switching in Data Center Networks*. (`Thesis `_) + +Citation +======== + +If you use Corundum in your project, please cite one of the following papers +and/or link to the project on GitHub:: + + @inproceedings{forencich2020fccm, + author = {Alex Forencich and Alex C. Snoeren and George Porter and George Papen}, + title = {Corundum: An Open-Source {100-Gbps} {NIC}}, + booktitle = {28th IEEE International Symposium on Field-Programmable Custom Computing Machines}, + year = {2020}, + } + + @phdthesis{forencich2020thesis, + author = {John Alexander Forencich}, + title = {System-Level Considerations for Optical Switching in Data Center Networks}, + school = {UC San Diego}, + year = {2020}, + url = {https://escholarship.org/uc/item/3mc9070t}, + } + +.. only:: html + + Indices and tables + ================== + + * :ref:`genindex` + * :ref:`modindex` + * :ref:`search` diff --git a/docs/source/macaddr.rst b/docs/source/macaddr.rst new file mode 100644 index 000000000..866ffb8d8 --- /dev/null +++ b/docs/source/macaddr.rst @@ -0,0 +1,42 @@ +.. _macaddr: + +======================== +Persistent MAC Addresses +======================== + +When registering network interfaces with the operating system, the driver must provide a MAC address for each interface. Ensuring that the configured MAC addresses are unique and consistent across driver reloads requires binding the addresses to the hardware in some way, usually through the use of some form of nonvolatile memory. It is relatively common for FPGA boards to provide small I2C EEPROMs for storing this sort of information. On other boards, the MAC address can be read out from the board management controller. If the driver fails to read a valid MAC address, it will fall back to using a randomly-generated MAC address. See the :ref:`device_list` for a summary of how persistent MAC addresses are implemented on each board. Boards that have pre-programmed MAC addresses should work "out of the box". However, boards that include blank EEPROMs need to have a MAC address written into the EEPROM for this functionality to work. + +Programming I2C EEPROM via kernel module +======================================== + +The driver registers all on-card I2C devices via the Linux I2C subsystem. Therefore, the MAC address EEPROM appears in sysfs, and a MAC address can easily be written using ``dd``. Note that accessing the EEPROM is a little bit different on each board. + +After loading the driver, the device can be accessed either directly (``/sys/bus/pci/devices/0000:xx:00.0/``) or from the corresponding network interface (``/sys/class/net/eth0/device/``) or miscdev (``/sys/class/misc/mqnic0/device/``). See the table below for the sysfs paths for each board. Note that the I2C bus numbers will vary. Also note that optical module I2C interfaces are registered as EEPROMs with I2C address 0x50, so ensure you have the correct EEPROM by dumping the contents with ``xxd`` or a hex editor before programming it. + +After determining the sysfs path and picking a MAC address, run a command similar to this one to program the MAC address into the EEPROM:: + + echo 02 aa bb 00 00 00 | xxd -r -p - | dd bs=1 count=6 of=/sys/class/net/eth0/device/i2c-4/4-0074/channel-2/7-0054/eeprom + +After reloading the driver, the interfaces should use the new MAC address:: + + 14: enp1s0: mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000 + link/ether 02:aa:bb:00:00:00 brd ff:ff:ff:ff:ff:ff + 15: enp1s0d1: mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000 + link/ether 02:aa:bb:00:00:01 brd ff:ff:ff:ff:ff:ff + +============ =================== ======================================== +Manufacturer Board sysfs path :sup:`1` +============ =================== ======================================== +Alpha Data ADM-PCIE-9V3 ``i2c-X/X-0050/eeprom`` :sup:`3` +Exablaze ExaNIC X10 :sup:`2` ``i2c-X/X-0050/eeprom`` :sup:`3` +Exablaze ExaNIC X25 :sup:`2` ``i2c-X/X-0050/eeprom`` :sup:`3` +Xilinx VCU108 ``i2c-X/X-0075/channel-3/Y-0054/eeprom`` +Xilinx VCU118 ``i2c-X/X-0075/channel-3/Y-0054/eeprom`` +Xilinx VCU1525 ``i2c-X/X-0074/channel-2/Y-0054/eeprom`` +Xilinx ZCU106 ``i2c-X/X-0074/channel-0/Y-0054/eeprom`` +============ =================== ======================================== + +Notes: +* :sup:`1` X and Y are i2c bus numbers that will vary +* :sup:`2` Card should come pre-programmed with a base MAC address +* :sup:`3` Optical module I2C interfaces may appear exactly the same way; confirm correct EEPROM by reading the contents with ``xxd`` or a hex editor. diff --git a/docs/source/modules/cpl_queue_manager.rst b/docs/source/modules/cpl_queue_manager.rst new file mode 100644 index 000000000..0d234412b --- /dev/null +++ b/docs/source/modules/cpl_queue_manager.rst @@ -0,0 +1,7 @@ +.. _mod_cpl_queue_manager: + +===================== +``cpl_queue_manager`` +===================== + +``cpl_queue_manager`` implements \ No newline at end of file diff --git a/docs/source/modules/cpl_write.rst b/docs/source/modules/cpl_write.rst new file mode 100644 index 000000000..4052d2811 --- /dev/null +++ b/docs/source/modules/cpl_write.rst @@ -0,0 +1,7 @@ +.. _mod_cpl_write: + +============= +``cpl_write`` +============= + +``cpl_write`` manages operations associated with completion writeback. It is responsible for enqueuing completion and event records into the completion queue managers and writing records into host memory via DMA. diff --git a/docs/source/modules/desc_fetch.rst b/docs/source/modules/desc_fetch.rst new file mode 100644 index 000000000..f9c3fc2f7 --- /dev/null +++ b/docs/source/modules/desc_fetch.rst @@ -0,0 +1,7 @@ +.. _mod_desc_fetch: + +============== +``desc_fetch`` +============== + +``desc_fetch`` manages operations associated with fetching descriptors. It is responsible for dequeuing descriptors from the queue managers and reading descriptors from host memory via DMA. diff --git a/docs/source/modules/index.rst b/docs/source/modules/index.rst new file mode 100644 index 000000000..2d1dbcb12 --- /dev/null +++ b/docs/source/modules/index.rst @@ -0,0 +1,15 @@ +.. _fpga_mod: + +======= +Modules +======= + + + +.. toctree:: + :maxdepth: 1 + :caption: Contents: + :glob: + + overview + * diff --git a/docs/source/modules/mqnic_app_block.rst b/docs/source/modules/mqnic_app_block.rst new file mode 100644 index 000000000..c516bf5bc --- /dev/null +++ b/docs/source/modules/mqnic_app_block.rst @@ -0,0 +1,1099 @@ +.. _mod_mqnic_app_block: + +=================== +``mqnic_app_block`` +=================== + +``mqnic_app_block`` is the top-level block for application logic. It is instantiated in :ref:`mod_mqnic_core`. This is a pluggable module, intended to be replaced by a customized implementation via the build system. See .... for more details. + +A number of interfaces are provided: + +* Clock and reset synchronous with core datapath +* Dedicated AXI-lite master interface for application control (``s_axil_app_ctrl``) +* AXI-lite slave interface for access to NIC control register space (``m_axil_ctrl``) +* Access to DMA subsystem (``*_axis_*_dma_*_desc``, ``*_dma_ram``) +* Access to PTP subsystem (``ptp_*``) +* Direct, MAC-synchronous, lowest-latency streaming interface (``*_axis_direct_*``) +* Direct, datapath-synchronous, low-latency streaming interface (``*_axis_sync_*``) +* Interface-level streaming interface (``*_axis_if_*``) +* Statistics interface (``m_axis_stat``) +* GPIO and JTAG passthrough (``gpio``, ``jtag``) + +Packet data from the host passes through all three streaming interfaces on its way to the network, and vise-versa. The three interfaces are: + +1. ``*_axis_direct_*``: Direct, MAC-synchronous, lowest-latency streaming interface. This interface is as close as possible to the main transmit and receive interfaces on :ref:`mod_mqnic_core`, and is synchronous to the TX and RX clocks instead of the core clock. Enabled/bypassed via ``APP_AXIS_DIRECT_ENABLE`` in ``config.tcl``. +2. ``*_axis_sync_*``: Direct, datapath-synchronous, low-latency streaming interface. This interface handles per-port data between the main transmit and receive FIFOs and the async FIFOs, and is synchronous to the core clock. Enabled/bypassed via ``APP_AXIS_SYNC_ENABLE`` in ``config.tcl``. +3. ``*_axis_if_*``: Interface-level streaming interface. This interface handles aggregated interface-level data between the host and the main receive and transmit FIFOs, and is synchronous to the core clock. Enabled/bypassed via ``APP_AXIS_IF_ENABLE`` in ``config.tcl``. + +On the transmit path, data flows as follows: + +1. :ref:`mod_mqnic_interface_tx`: data is read from host memory via DMA +2. :ref:`mod_mqnic_egress`: egress processing +3. ``s_axis_if_tx``: data is presented to the application section +4. ``m_axis_if_tx``: data is returned from the application section +5. Data passes enters per-interface transmit FIFO module and is divided into per-port, per-traffic-class FIFOs +6. ``s_axis_sync_tx``: data is presented to the application section +7. ``m_axis_sync_tx``: data is returned from the application section +8. Data passes through per-port transmit async FIFO module and is transferred to MAC TX clock domain +9. ``s_axis_sync_tx``: data is presented to the application section +10. ``m_axis_sync_tx``: data is returned from the application section +11. :ref:`mod_mqnic_l2_egress`: layer 2 egress processing +12. :ref:`mod_mqnic_core`: data leaves through transmit streaming interfaces + +On the receive path, data flows as follows: + +1. :ref:`mod_mqnic_core`: data enters through receive streaming interfaces +2. :ref:`mod_mqnic_l2_ingress`: layer 2 ingress processing +3. ``s_axis_sync_rx``: data is presented to the application section +4. ``m_axis_sync_rx``: data is returned from the application section +5. Data passes through per-port receive async FIFO module and is transferred to core clock domain +6. ``s_axis_sync_rx``: data is presented to the application section +7. ``m_axis_sync_rx``: data is returned from the application section +8. Data passes enters per-interface receive FIFO module and is placed into per-port FIFOs, then aggregated into a single stream +9. ``s_axis_if_rx``: data is presented to the application section +10. ``m_axis_if_rx``: data is returned from the application section +11. :ref:`mod_mqnic_ingress`: ingress processing +12. :ref:`mod_mqnic_interface_rx`: data is read from host memory via DMA + +Parameters +========== + +.. object:: IF_COUNT + + Interface count, default ``1``. + +.. object:: PORTS_PER_IF + + Ports per interface, default ``1``. + +.. object:: PORT_COUNT + + Total port count, must be set to ``IF_COUNT*PORTS_PER_IF``. + +.. object:: PTP_TS_WIDTH + + PTP timestamp width, must be ``96``. + +.. object:: PTP_TAG_WIDTH + + PTP tag signal width, default ``16``. + +.. object:: PTP_PERIOD_NS_WIDTH + + PTP period ns field width, default ``4``. + +.. object:: PTP_OFFSET_NS_WIDTH + + PTP offset ns field width, default ``32``. + +.. object:: PTP_FNS_WIDTH + + PTP fractional ns field width, default ``32``. + +.. object:: PTP_PERIOD_NS + + PTP nominal period, ns portion ``4'd4``. + +.. object:: PTP_PERIOD_FNS + + PTP nominal period, fractional ns portion ``32'd0``. + +.. object:: PTP_USE_SAMPLE_CLOCK + + Use external PTP sample clock, used to synchronize the PTP clock across clock domains. Default ``0``. + +.. object:: PTP_PEROUT_ENABLE + + Enable PTP period output module, default ``0``. + +.. object:: PTP_PEROUT_COUNT + + Number of PTP period output channels, default ``1``. + +.. object:: PTP_TS_ENABLE + + Enable PTP timestamping, default ``1``. + +.. object:: APP_CTRL_ENABLE + + Enable application section control connection to core NIC registers, default ``1``. + +.. object:: APP_DMA_ENABLE + + Enable application section connection to DMA subsystem, default ``1``. + +.. object:: APP_AXIS_DIRECT_ENABLE + + Enable lowest-latency asynchronous streaming connection to application section, default ``1`` + +.. object:: APP_AXIS_SYNC_ENABLE + + Enable low-latency synchronous streaming connection to application section, default ``1`` + +.. object:: APP_AXIS_IF_ENABLE + + Enable interface-level streaming connection to application section, default ``1`` + +.. object:: APP_STAT_ENABLE + + Enable application section connection to statistics collection subsystem, default ``1`` + +.. object:: APP_GPIO_IN_WIDTH + + Application section GPIO input signal width, default ``32`` + +.. object:: APP_GPIO_OUT_WIDTH + + Application section GPIO output signal width, default ``32`` + +.. object:: DMA_ADDR_WIDTH + + DMA interface address signal width, default ``64``. + +.. object:: DMA_LEN_WIDTH + + DMA interface length signal width, default ``16``. + +.. object:: DMA_TAG_WIDTH + + DMA interface tag signal width, default ``16``. + +.. object:: RAM_SEL_WIDTH + + Width of select signal per segment in DMA RAM interface, default ``4``. + +.. object:: RAM_ADDR_WIDTH + + Width of address signal for DMA RAM interface, default ``16``. + +.. object:: RAM_SEG_COUNT + + Number of segments in DMA RAM interface, default ``2``. Must be a power of 2, must be at least 2. + +.. object:: RAM_SEG_DATA_WIDTH + + Width of data signal per segment in DMA RAM interface, default ``256*2/RAM_SEG_COUNT``. + +.. object:: RAM_SEG_BE_WIDTH + + Width of byte enable signal per segment in DMA RAM interface, default ``RAM_SEG_DATA_WIDTH/8``. + +.. object:: RAM_SEG_ADDR_WIDTH + + Width of address signal per segment in DMA RAM interface, default ``RAM_ADDR_WIDTH-$clog2(RAM_SEG_COUNT*RAM_SEG_BE_WIDTH)``. + +.. object:: RAM_PIPELINE + + Number of output pipeline stages in segmented DMA RAMs, default ``2``. Tune for best usage of block RAM cascade registers. + +.. object:: AXIL_APP_CTRL_DATA_WIDTH + + AXI lite application control data signal width, default ``AXIL_CTRL_DATA_WIDTH``. Can be 32 or 64. + +.. object:: AXIL_APP_CTRL_ADDR_WIDTH + + AXI lite application control address signal width, default ``16``. + +.. object:: AXIL_APP_CTRL_STRB_WIDTH + + AXI lite application control byte enable signal width, must be set to ``AXIL_APP_CTRL_DATA_WIDTH/8``. + +.. object:: AXIL_CTRL_DATA_WIDTH + + AXI lite control data signal width, default ``32``. Must be 32. + +.. object:: AXIL_CTRL_ADDR_WIDTH + + AXI lite control address signal width, default ``16``. + +.. object:: AXIL_CTRL_STRB_WIDTH + + AXI lite control byte enable signal width, must be set to ``AXIL_CTRL_DATA_WIDTH/8``. + +.. object:: AXIS_DATA_WIDTH + + Asynchronous streaming interface ``tdata`` signal width, default ``512``. + +.. object:: AXIS_KEEP_WIDTH + + Asynchronous streaming interface ``tkeep`` signal width, must be set to ``AXIS_DATA_WIDTH/8``. + +.. object:: AXIS_TX_USER_WIDTH + + Asynchronous streaming transmit interface ``tuser`` signal width, default ``(PTP_TS_ENABLE ? PTP_TAG_WIDTH : 0) + 1``. + +.. object:: AXIS_RX_USER_WIDTH + + Asynchronous streaming receive interface ``tuser`` signal width, default ``(PTP_TS_ENABLE ? PTP_TS_WIDTH : 0) + 1``. + +.. object:: AXIS_RX_USE_READY + + Use ``tready`` signal on RX interfaces, default ``0``. If set, logic will exert backpressure with ``tready`` instead of dropping packets when RX FIFOs are full. + +.. object:: AXIS_SYNC_DATA_WIDTH + + Synchronous streaming interface ``tdata`` signal width, default ``AXIS_DATA_WIDTH``. + +.. object:: AXIS_SYNC_KEEP_WIDTH + + Synchronous streaming interface ``tkeep`` signal width, must be set to ``AXIS_SYNC_DATA_WIDTH/8``. + +.. object:: AXIS_SYNC_TX_USER_WIDTH + + Synchronous streaming transmit interface ``tuser`` signal width, default ``AXIS_TX_USER_WIDTH``. + +.. object:: AXIS_SYNC_RX_USER_WIDTH + + Synchronous streaming receive interface ``tuser`` signal width, default ``AXIS_RX_USER_WIDTH``. + +.. object:: AXIS_IF_DATA_WIDTH + + Interface streaming interface ``tdata`` signal width, default ``AXIS_SYNC_DATA_WIDTH*2**$clog2(PORTS_PER_IF)``. + +.. object:: AXIS_IF_KEEP_WIDTH + + Interface streaming interface ``tkeep`` signal width, must be set to ``AXIS_IF_DATA_WIDTH/8``. + +.. object:: AXIS_IF_TX_ID_WIDTH + + Interface transmit streaming interface ``tid`` signal width, default ``12``. + +.. object:: AXIS_IF_RX_ID_WIDTH + + Interface receive streaming interface ``tid`` signal width, default ``PORTS_PER_IF > 1 ? $clog2(PORTS_PER_IF) : 1``. + +.. object:: AXIS_IF_TX_DEST_WIDTH + + Interface transmit streaming interface ``tdest`` signal width, default ``$clog2(PORTS_PER_IF)+4``. + +.. object:: AXIS_IF_RX_DEST_WIDTH + + Interface receive streaming interface ``tdest`` signal width, default ``8``. + +.. object:: AXIS_IF_TX_USER_WIDTH + + Interface transmit streaming interface ``tuser`` signal width, default ``AXIS_SYNC_TX_USER_WIDTH``. + +.. object:: AXIS_IF_RX_USER_WIDTH + + Interface receive streaming interface ``tuser`` signal width, default ``AXIS_SYNC_RX_USER_WIDTH``. + +.. object:: STAT_ENABLE + + Enable statistics collection subsystem, default ``1``. + +.. object:: STAT_INC_WIDTH + + Statistics increment signal width, default ``24``. + +.. object:: STAT_ID_WIDTH + + Statistics ID signal width, default ``12``. Sets the number of statistics counters as ``2**STAT_ID_WIDTH``. + +Ports +===== + +.. object:: clk + + Logic clock. Most interfaces are synchronous to this clock. + + .. table:: + + ====== === ===== ================== + Signal Dir Width Description + ====== === ===== ================== + clk in 1 Logic clock + ====== === ===== ================== + +.. object:: rst + + Logic reset, active high + + .. table:: + + ====== === ===== ================== + Signal Dir Width Description + ====== === ===== ================== + rst in 1 Logic reset, active high + ====== === ===== ================== + +.. object:: s_axil_app_ctrl + + AXI-Lite slave interface (application control). + + .. table:: + + ======================= === ======================== =================== + Signal Dir Width Description + ======================= === ======================== =================== + s_axil_app_ctrl_awaddr in AXIL_APP_CTRL_ADDR_WIDTH Write address + s_axil_app_ctrl_awprot in 3 Write protect + s_axil_app_ctrl_awvalid in 1 Write address valid + s_axil_app_ctrl_awready out 1 Write address ready + s_axil_app_ctrl_wdata in AXIL_APP_CTRL_DATA_WIDTH Write data + s_axil_app_ctrl_wstrb in AXIL_APP_CTRL_STRB_WIDTH Write data strobe + s_axil_app_ctrl_wvalid in 1 Write data valid + s_axil_app_ctrl_wready out 1 Write data ready + s_axil_app_ctrl_bresp out 2 Write response status + s_axil_app_ctrl_bvalid out 1 Write response valid + s_axil_app_ctrl_bready in 1 Write response ready + s_axil_app_ctrl_araddr in AXIL_APP_CTRL_ADDR_WIDTH Read address + s_axil_app_ctrl_arprot in 3 Read protect + s_axil_app_ctrl_arvalid in 1 Read address valid + s_axil_app_ctrl_arready out 1 Read address ready + s_axil_app_ctrl_rdata out AXIL_APP_CTRL_DATA_WIDTH Read response data + s_axil_app_ctrl_rresp out 2 Read response status + s_axil_app_ctrl_rvalid out 1 Read response valid + s_axil_app_ctrl_rready in 1 Read response ready + ======================= === ======================== =================== + +.. object:: m_axil_ctrl + + AXI-Lite master interface (control). This interface provides access to the main NIC control register space. + + .. table:: + + =================== === ==================== =================== + Signal Dir Width Description + =================== === ==================== =================== + m_axil_ctrl_awaddr in AXIL_CTRL_ADDR_WIDTH Write address + m_axil_ctrl_awprot in 3 Write protect + m_axil_ctrl_awvalid in 1 Write address valid + m_axil_ctrl_awready out 1 Write address ready + m_axil_ctrl_wdata in AXIL_CTRL_DATA_WIDTH Write data + m_axil_ctrl_wstrb in AXIL_CTRL_STRB_WIDTH Write data strobe + m_axil_ctrl_wvalid in 1 Write data valid + m_axil_ctrl_wready out 1 Write data ready + m_axil_ctrl_bresp out 2 Write response status + m_axil_ctrl_bvalid out 1 Write response valid + m_axil_ctrl_bready in 1 Write response ready + m_axil_ctrl_araddr in AXIL_CTRL_ADDR_WIDTH Read address + m_axil_ctrl_arprot in 3 Read protect + m_axil_ctrl_arvalid in 1 Read address valid + m_axil_ctrl_arready out 1 Read address ready + m_axil_ctrl_rdata out AXIL_CTRL_DATA_WIDTH Read response data + m_axil_ctrl_rresp out 2 Read response status + m_axil_ctrl_rvalid out 1 Read response valid + m_axil_ctrl_rready in 1 Read response ready + =================== === ==================== =================== + +.. object:: m_axis_ctrl_dma_read_desc + + DMA read descriptor output (control) + + .. table:: + + ================================== === ============== =================== + Signal Dir Width Description + ================================== === ============== =================== + m_axis_ctrl_dma_read_desc_dma_addr out DMA_ADDR_WIDTH DMA address + m_axis_ctrl_dma_read_desc_ram_sel out RAM_SEL_WIDTH RAM select + m_axis_ctrl_dma_read_desc_ram_addr out RAM_ADDR_WIDTH RAM address + m_axis_ctrl_dma_read_desc_len out DMA_LEN_WIDTH Transfer length + m_axis_ctrl_dma_read_desc_tag out DMA_TAG_WIDTH Transfer tag + m_axis_ctrl_dma_read_desc_valid out 1 Request valid + m_axis_ctrl_dma_read_desc_ready in 1 Request ready + ================================== === ============== =================== + +.. object:: s_axis_ctrl_dma_read_desc_status + + DMA read descriptor status input (control) + + .. table:: + + ====================================== === ============= =================== + Signal Dir Width Description + ====================================== === ============= =================== + s_axis_ctrl_dma_read_desc_status_tag in DMA_TAG_WIDTH Status tag + s_axis_ctrl_dma_read_desc_status_error in 4 Status error code + s_axis_ctrl_dma_read_desc_status_valid in 1 Status valid + ====================================== === ============= =================== + +.. object:: m_axis_ctrl_dma_write_desc + + DMA write descriptor output (control) + + .. table:: + + =================================== === ============== =================== + Signal Dir Width Description + =================================== === ============== =================== + m_axis_ctrl_dma_write_desc_dma_addr out DMA_ADDR_WIDTH DMA address + m_axis_ctrl_dma_write_desc_ram_sel out RAM_SEL_WIDTH RAM select + m_axis_ctrl_dma_write_desc_ram_addr out RAM_ADDR_WIDTH RAM address + m_axis_ctrl_dma_write_desc_len out DMA_LEN_WIDTH Transfer length + m_axis_ctrl_dma_write_desc_tag out DMA_TAG_WIDTH Transfer tag + m_axis_ctrl_dma_write_desc_valid out 1 Request valid + m_axis_ctrl_dma_write_desc_ready in 1 Request ready + =================================== === ============== =================== + +.. object:: s_axis_ctrl_dma_write_desc_status + + DMA write descriptor status input (control) + + .. table:: + + ======================================= === ============= =================== + Signal Dir Width Description + ======================================= === ============= =================== + s_axis_ctrl_dma_write_desc_status_tag in DMA_TAG_WIDTH Status tag + s_axis_ctrl_dma_write_desc_status_error in 4 Status error code + s_axis_ctrl_dma_write_desc_status_valid in 1 Status valid + ======================================= === ============= =================== + +.. object:: m_axis_data_dma_read_desc + + DMA read descriptor output (data) + + .. table:: + + ================================== === ============== =================== + Signal Dir Width Description + ================================== === ============== =================== + m_axis_data_dma_read_desc_dma_addr out DMA_ADDR_WIDTH DMA address + m_axis_data_dma_read_desc_ram_sel out RAM_SEL_WIDTH RAM select + m_axis_data_dma_read_desc_ram_addr out RAM_ADDR_WIDTH RAM address + m_axis_data_dma_read_desc_len out DMA_LEN_WIDTH Transfer length + m_axis_data_dma_read_desc_tag out DMA_TAG_WIDTH Transfer tag + m_axis_data_dma_read_desc_valid out 1 Request valid + m_axis_data_dma_read_desc_ready in 1 Request ready + ================================== === ============== =================== + +.. object:: s_axis_data_dma_read_desc_status + + DMA read descriptor status input (data) + + .. table:: + + ====================================== === ============= =================== + Signal Dir Width Description + ====================================== === ============= =================== + s_axis_data_dma_read_desc_status_tag in DMA_TAG_WIDTH Status tag + s_axis_data_dma_read_desc_status_error in 4 Status error code + s_axis_data_dma_read_desc_status_valid in 1 Status valid + ====================================== === ============= =================== + +.. object:: m_axis_data_dma_write_desc + + DMA write descriptor output (data) + + .. table:: + + =================================== === ============== =================== + Signal Dir Width Description + =================================== === ============== =================== + m_axis_data_dma_write_desc_dma_addr out DMA_ADDR_WIDTH DMA address + m_axis_data_dma_write_desc_ram_sel out RAM_SEL_WIDTH RAM select + m_axis_data_dma_write_desc_ram_addr out RAM_ADDR_WIDTH RAM address + m_axis_data_dma_write_desc_len out DMA_LEN_WIDTH Transfer length + m_axis_data_dma_write_desc_tag out DMA_TAG_WIDTH Transfer tag + m_axis_data_dma_write_desc_valid out 1 Request valid + m_axis_data_dma_write_desc_ready in 1 Request ready + =================================== === ============== =================== + +.. object:: s_axis_data_dma_write_desc_status + + DMA write descriptor status input (data) + + .. table:: + + ======================================= === ============= =================== + Signal Dir Width Description + ======================================= === ============= =================== + s_axis_data_dma_write_desc_status_tag in DMA_TAG_WIDTH Status tag + s_axis_data_dma_write_desc_status_error in 4 Status error code + s_axis_data_dma_write_desc_status_valid in 1 Status valid + ======================================= === ============= =================== + +.. object:: ctrl_dma_ram + + DMA RAM interface (control) + + .. table:: + + ========================== === ================================ =================== + Signal Dir Width Description + ========================== === ================================ =================== + ctrl_dma_ram_wr_cmd_sel in RAM_SEG_COUNT*RAM_SEL_WIDTH Write command select + ctrl_dma_ram_wr_cmd_be in RAM_SEG_COUNT*RAM_SEG_BE_WIDTH Write command byte enable + ctrl_dma_ram_wr_cmd_addr in RAM_SEG_COUNT*RAM_SEG_ADDR_WIDTH Write command address + ctrl_dma_ram_wr_cmd_data in RAM_SEG_COUNT*RAM_SEG_DATA_WIDTH Write command data + ctrl_dma_ram_wr_cmd_valid in RAM_SEG_COUNT Write command valid + ctrl_dma_ram_wr_cmd_ready out RAM_SEG_COUNT Write command ready + ctrl_dma_ram_wr_done out RAM_SEG_COUNT Write done + ctrl_dma_ram_rd_cmd_sel in RAM_SEG_COUNT*RAM_SEL_WIDTH Read command select + ctrl_dma_ram_rd_cmd_addr in RAM_SEG_COUNT*RAM_SEG_ADDR_WIDTH Read command address + ctrl_dma_ram_rd_cmd_valid in RAM_SEG_COUNT Read command valid + ctrl_dma_ram_rd_cmd_ready out RAM_SEG_COUNT Read command ready + ctrl_dma_ram_rd_resp_data out RAM_SEG_COUNT*RAM_SEG_DATA_WIDTH Read response data + ctrl_dma_ram_rd_resp_valid out RAM_SEG_COUNT Read response valid + ctrl_dma_ram_rd_resp_ready in RAM_SEG_COUNT Read response ready + ========================== === ================================ =================== + +.. object:: data_dma_ram + + DMA RAM interface (data) + + .. table:: + + ========================== === ================================ =================== + Signal Dir Width Description + ========================== === ================================ =================== + data_dma_ram_wr_cmd_sel in RAM_SEG_COUNT*RAM_SEL_WIDTH Write command select + data_dma_ram_wr_cmd_be in RAM_SEG_COUNT*RAM_SEG_BE_WIDTH Write command byte enable + data_dma_ram_wr_cmd_addr in RAM_SEG_COUNT*RAM_SEG_ADDR_WIDTH Write command address + data_dma_ram_wr_cmd_data in RAM_SEG_COUNT*RAM_SEG_DATA_WIDTH Write command data + data_dma_ram_wr_cmd_valid in RAM_SEG_COUNT Write command valid + data_dma_ram_wr_cmd_ready out RAM_SEG_COUNT Write command ready + data_dma_ram_wr_done out RAM_SEG_COUNT Write done + data_dma_ram_rd_cmd_sel in RAM_SEG_COUNT*RAM_SEL_WIDTH Read command select + data_dma_ram_rd_cmd_addr in RAM_SEG_COUNT*RAM_SEG_ADDR_WIDTH Read command address + data_dma_ram_rd_cmd_valid in RAM_SEG_COUNT Read command valid + data_dma_ram_rd_cmd_ready out RAM_SEG_COUNT Read command ready + data_dma_ram_rd_resp_data out RAM_SEG_COUNT*RAM_SEG_DATA_WIDTH Read response data + data_dma_ram_rd_resp_valid out RAM_SEG_COUNT Read response valid + data_dma_ram_rd_resp_ready in RAM_SEG_COUNT Read response ready + ========================== === ================================ =================== + +.. object:: ptp + + PTP clock connections. + + .. table:: + + ================= === ================ =================== + Signal Dir Width Description + ================= === ================ =================== + ptp_sample_clk in 1 PTP sample clock + ptp_pps in 1 PTP pulse-per-second + ptp_ts_96 in PTP_TS_WIDTH current PTP time + ptp_ts_step in 1 PTP clock step + ptp_perout_locked in PTP_PEROUT_COUNT PTP period output locked + ptp_perout_error in PTP_PEROUT_COUNT PTP period output error + ptp_perout_pulse in PTP_PEROUT_COUNT PTP period output pulse + ================= === ================ =================== + +.. object:: direct_tx_clk + + Transmit clocks for direct asynchronous streaming interfaces, one per port + + .. table:: + + ============= === ========== ================== + Signal Dir Width Description + ============= === ========== ================== + direct_tx_clk in PORT_COUNT Transmit clock + ============= === ========== ================== + +.. object:: direct_tx_rst + + Transmit resets for direct asynchronous streaming interfaces, one per port + + .. table:: + + ============= === ========== ================== + Signal Dir Width Description + ============= === ========== ================== + direct_tx_rst in PORT_COUNT Transmit reset + ============= === ========== ================== + +.. object:: s_axis_direct_tx + + Streaming transmit data from host, one AXI stream interface per port. Lowest latency interface, synchronous with transmit clock. + + .. table:: + + ======================= === ============================= ================== + Signal Dir Width Description + ======================= === ============================= ================== + s_axis_direct_tx_tdata in PORT_COUNT*AXIS_DATA_WIDTH Streaming data + s_axis_direct_tx_tkeep in PORT_COUNT*AXIS_KEEP_WIDTH Byte enable + s_axis_direct_tx_tvalid in PORT_COUNT Data valid + s_axis_direct_tx_tready out PORT_COUNT Ready for data + s_axis_direct_tx_tlast in PORT_COUNT End of frame + s_axis_direct_tx_tuser in PORT_COUNT*AXIS_TX_USER_WIDTH Sideband data + ======================= === ============================= ================== + + ``s_axis_direct_tx_tuser`` bits, per port + + .. table:: + + =============== ========= ============= ============= + Bit Name Width Description + =============== ========= ============= ============= + 0 bad_frame 1 Invalid frame + PTP_TAG_WIDTH:1 ptp_tag PTP_TAG_WIDTH PTP tag + =============== ========= ============= ============= + +.. object:: m_axis_direct_tx + + Streaming transmit data towards network, one AXI stream interface per port. Lowest latency interface, synchronous with transmit clock. + + .. table:: + + ======================= === ============================= ================== + Signal Dir Width Description + ======================= === ============================= ================== + m_axis_direct_tx_tdata out PORT_COUNT*AXIS_DATA_WIDTH Streaming data + m_axis_direct_tx_tkeep out PORT_COUNT*AXIS_KEEP_WIDTH Byte enable + m_axis_direct_tx_tvalid out PORT_COUNT Data valid + m_axis_direct_tx_tready in PORT_COUNT Ready for data + m_axis_direct_tx_tlast out PORT_COUNT End of frame + m_axis_direct_tx_tuser out PORT_COUNT*AXIS_TX_USER_WIDTH Sideband data + ======================= === ============================= ================== + + ``m_axis_direct_tx_tuser`` bits, per port + + .. table:: + + =============== ========= ============= ============= + Bit Name Width Description + =============== ========= ============= ============= + 0 bad_frame 1 Invalid frame + PTP_TAG_WIDTH:1 ptp_tag PTP_TAG_WIDTH PTP tag + =============== ========= ============= ============= + +.. object:: s_axis_direct_tx_ptp_ts + + Transmit PTP timestamp from MAC, one AXI stream interface per port. + + .. table:: + + ============================= === ======================== =================== + Signal Dir Width Description + ============================= === ======================== =================== + s_axis_direct_tx_ptp_ts in PORT_COUNT*PTP_TS_WIDTH PTP timestamp + s_axis_direct_tx_ptp_ts_tag in PORT_COUNT*PTP_TAG_WIDTH PTP timestamp tag + s_axis_direct_tx_ptp_ts_valid in PORT_COUNT PTP timestamp valid + s_axis_direct_tx_ptp_ts_ready out PORT_COUNT PTP timestamp ready + ============================= === ======================== =================== + +.. object:: m_axis_direct_tx_ptp_ts + + Transmit PTP timestamp towards core logic, one AXI stream interface per port. + + .. table:: + + ============================= === ======================== =================== + Signal Dir Width Description + ============================= === ======================== =================== + s_axis_direct_tx_ptp_ts out PORT_COUNT*PTP_TS_WIDTH PTP timestamp + s_axis_direct_tx_ptp_ts_tag out PORT_COUNT*PTP_TAG_WIDTH PTP timestamp tag + s_axis_direct_tx_ptp_ts_valid out PORT_COUNT PTP timestamp valid + s_axis_direct_tx_ptp_ts_ready in PORT_COUNT PTP timestamp ready + ============================= === ======================== =================== + +.. object:: direct_rx_clk + + Receive clocks for direct asynchronous streaming interfaces, one per port + + .. table:: + + ============= === ========== ================== + Signal Dir Width Description + ============= === ========== ================== + direct_rx_clk in PORT_COUNT Receive clock + ============= === ========== ================== + +.. object:: direct_rx_rst + + Receive resets for direct asynchronous streaming interfaces, one per port + + .. table:: + + ============= === ========== ================== + Signal Dir Width Description + ============= === ========== ================== + direct_rx_rst in PORT_COUNT Receive reset + ============= === ========== ================== + +.. object:: s_axis_direct_rx + + Streaming receive data from network, one AXI stream interface per port. Lowest latency interface, synchronous with receive clock. + + .. table:: + + ======================= === ============================= ================== + Signal Dir Width Description + ======================= === ============================= ================== + s_axis_direct_rx_tdata in PORT_COUNT*AXIS_DATA_WIDTH Streaming data + s_axis_direct_rx_tkeep in PORT_COUNT*AXIS_KEEP_WIDTH Byte enable + s_axis_direct_rx_tvalid in PORT_COUNT Data valid + s_axis_direct_rx_tready out PORT_COUNT Ready for data + s_axis_direct_rx_tlast in PORT_COUNT End of frame + s_axis_direct_rx_tuser in PORT_COUNT*AXIS_RX_USER_WIDTH Sideband data + ======================= === ============================= ================== + + ``s_axis_direct_rx_tuser`` bits, per port + + .. table:: + + =============== ========= ============= ============= + Bit Name Width Description + =============== ========= ============= ============= + 0 bad_frame 1 Invalid frame + PTP_TS_WIDTH:1 ptp_ts PTP_TS_WIDTH PTP timestamp + =============== ========= ============= ============= + +.. object:: m_axis_direct_rx + + Streaming receive data towards host, one AXI stream interface per port. Lowest latency interface, synchronous with receive clock. + + .. table:: + + ======================= === ============================= ================== + Signal Dir Width Description + ======================= === ============================= ================== + m_axis_direct_rx_tdata out PORT_COUNT*AXIS_DATA_WIDTH Streaming data + m_axis_direct_rx_tkeep out PORT_COUNT*AXIS_KEEP_WIDTH Byte enable + m_axis_direct_rx_tvalid out PORT_COUNT Data valid + m_axis_direct_rx_tready in PORT_COUNT Ready for data + m_axis_direct_rx_tlast out PORT_COUNT End of frame + m_axis_direct_rx_tuser out PORT_COUNT*AXIS_RX_USER_WIDTH Sideband data + ======================= === ============================= ================== + + ``m_axis_direct_rx_tuser`` bits, per port + + .. table:: + + =============== ========= ============= ============= + Bit Name Width Description + =============== ========= ============= ============= + 0 bad_frame 1 Invalid frame + PTP_TS_WIDTH:1 ptp_ts PTP_TS_WIDTH PTP timestamp + =============== ========= ============= ============= + +.. object:: s_axis_sync_tx + + Streaming transmit data from host, one AXI stream interface per port. Low latency interface, synchronous with core clock. + + .. table:: + + ===================== === ================================== ================== + Signal Dir Width Description + ===================== === ================================== ================== + s_axis_sync_tx_tdata in PORT_COUNT*AXIS_SYNC_DATA_WIDTH Streaming data + s_axis_sync_tx_tkeep in PORT_COUNT*AXIS_SYNC_KEEP_WIDTH Byte enable + s_axis_sync_tx_tvalid in PORT_COUNT Data valid + s_axis_sync_tx_tready out PORT_COUNT Ready for data + s_axis_sync_tx_tlast in PORT_COUNT End of frame + s_axis_sync_tx_tuser in PORT_COUNT*AXIS_SYNC_TX_USER_WIDTH Sideband data + ===================== === ================================== ================== + + ``s_axis_sync_tx_tuser`` bits, per port + + .. table:: + + =============== ========= ============= ============= + Bit Name Width Description + =============== ========= ============= ============= + 0 bad_frame 1 Invalid frame + PTP_TAG_WIDTH:1 ptp_tag PTP_TAG_WIDTH PTP tag + =============== ========= ============= ============= + +.. object:: m_axis_sync_tx + + Streaming transmit data towards network, one AXI stream interface per port. Low latency interface, synchronous with core clock. + + .. table:: + + ===================== === ================================== ================== + Signal Dir Width Description + ===================== === ================================== ================== + m_axis_sync_tx_tdata out PORT_COUNT*AXIS_SYNC_DATA_WIDTH Streaming data + m_axis_sync_tx_tkeep out PORT_COUNT*AXIS_SYNC_KEEP_WIDTH Byte enable + m_axis_sync_tx_tvalid out PORT_COUNT Data valid + m_axis_sync_tx_tready in PORT_COUNT Ready for data + m_axis_sync_tx_tlast out PORT_COUNT End of frame + m_axis_sync_tx_tuser out PORT_COUNT*AXIS_SYNC_TX_USER_WIDTH Sideband data + ===================== === ================================== ================== + + ``m_axis_sync_tx_tuser`` bits, per port + + .. table:: + + =============== ========= ============= ============= + Bit Name Width Description + =============== ========= ============= ============= + 0 bad_frame 1 Invalid frame + PTP_TAG_WIDTH:1 ptp_tag PTP_TAG_WIDTH PTP tag + =============== ========= ============= ============= + +.. object:: s_axis_sync_tx_ptp_ts + + Transmit PTP timestamp from MAC, one AXI stream interface per port. + + .. table:: + + =========================== === ======================== =================== + Signal Dir Width Description + =========================== === ======================== =================== + s_axis_sync_tx_ptp_ts in PORT_COUNT*PTP_TS_WIDTH PTP timestamp + s_axis_sync_tx_ptp_ts_tag in PORT_COUNT*PTP_TAG_WIDTH PTP timestamp tag + s_axis_sync_tx_ptp_ts_valid in PORT_COUNT PTP timestamp valid + s_axis_sync_tx_ptp_ts_ready out PORT_COUNT PTP timestamp ready + =========================== === ======================== =================== + +.. object:: m_axis_sync_tx_ptp_ts + + Transmit PTP timestamp towards core logic, one AXI stream interface per port. + + .. table:: + + =========================== === ======================== =================== + Signal Dir Width Description + =========================== === ======================== =================== + s_axis_sync_tx_ptp_ts out PORT_COUNT*PTP_TS_WIDTH PTP timestamp + s_axis_sync_tx_ptp_ts_tag out PORT_COUNT*PTP_TAG_WIDTH PTP timestamp tag + s_axis_sync_tx_ptp_ts_valid out PORT_COUNT PTP timestamp valid + s_axis_sync_tx_ptp_ts_ready in PORT_COUNT PTP timestamp ready + =========================== === ======================== =================== + +.. object:: s_axis_sync_rx + + Streaming receive data from network, one AXI stream interface per port. Low latency interface, synchronous with core clock. + + .. table:: + + ===================== === ================================== ================== + Signal Dir Width Description + ===================== === ================================== ================== + s_axis_sync_rx_tdata in PORT_COUNT*AXIS_SYNC_DATA_WIDTH Streaming data + s_axis_sync_rx_tkeep in PORT_COUNT*AXIS_SYNC_KEEP_WIDTH Byte enable + s_axis_sync_rx_tvalid in PORT_COUNT Data valid + s_axis_sync_rx_tready out PORT_COUNT Ready for data + s_axis_sync_rx_tlast in PORT_COUNT End of frame + s_axis_sync_rx_tuser in PORT_COUNT*AXIS_SYNC_RX_USER_WIDTH Sideband data + ===================== === ================================== ================== + + ``s_axis_sync_rx_tuser`` bits, per port + + .. table:: + + =============== ========= ============= ============= + Bit Name Width Description + =============== ========= ============= ============= + 0 bad_frame 1 Invalid frame + PTP_TS_WIDTH:1 ptp_ts PTP_TS_WIDTH PTP timestamp + =============== ========= ============= ============= + +.. object:: m_axis_sync_rx + + Streaming receive data towards host, one AXI stream interface per port. Low latency interface, synchronous with core clock. + + .. table:: + + ===================== === ================================== ================== + Signal Dir Width Description + ===================== === ================================== ================== + m_axis_sync_rx_tdata out PORT_COUNT*AXIS_SYNC_DATA_WIDTH Streaming data + m_axis_sync_rx_tkeep out PORT_COUNT*AXIS_SYNC_KEEP_WIDTH Byte enable + m_axis_sync_rx_tvalid out PORT_COUNT Data valid + m_axis_sync_rx_tready in PORT_COUNT Ready for data + m_axis_sync_rx_tlast out PORT_COUNT End of frame + m_axis_sync_rx_tuser out PORT_COUNT*AXIS_SYNC_RX_USER_WIDTH Sideband data + ===================== === ================================== ================== + + ``m_axis_sync_rx_tuser`` bits, per port + + .. table:: + + =============== ========= ============= ============= + Bit Name Width Description + =============== ========= ============= ============= + 0 bad_frame 1 Invalid frame + PTP_TS_WIDTH:1 ptp_ts PTP_TS_WIDTH PTP timestamp + =============== ========= ============= ============= + +.. object:: s_axis_if_tx + + Streaming transmit data from host, one AXI stream interface per interface. Closest interface to host, synchronous with core clock. + + .. table:: + + =================== === ================================ ================== + Signal Dir Width Description + =================== === ================================ ================== + s_axis_if_tx_tdata in PORT_COUNT*AXIS_IF_DATA_WIDTH Streaming data + s_axis_if_tx_tkeep in PORT_COUNT*AXIS_IF_KEEP_WIDTH Byte enable + s_axis_if_tx_tvalid in PORT_COUNT Data valid + s_axis_if_tx_tready out PORT_COUNT Ready for data + s_axis_if_tx_tlast in PORT_COUNT End of frame + s_axis_if_tx_tid in PORT_COUNT*AXIS_IF_TX_ID_WIDTH Source queue + s_axis_if_tx_tdest in PORT_COUNT*AXIS_IF_TX_DEST_WIDTH Destination port + s_axis_if_tx_tuser in PORT_COUNT*AXIS_IF_TX_USER_WIDTH Sideband data + =================== === ================================ ================== + + ``s_axis_if_tx_tuser`` bits, per interface + + .. table:: + + =============== ========= ============= ============= + Bit Name Width Description + =============== ========= ============= ============= + 0 bad_frame 1 Invalid frame + PTP_TAG_WIDTH:1 ptp_tag PTP_TAG_WIDTH PTP tag + =============== ========= ============= ============= + +.. object:: m_axis_if_tx + + Streaming transmit data towards network, one AXI stream interface per interface. Closest interface to host, synchronous with core clock. + + .. table:: + + =================== === ================================ ================== + Signal Dir Width Description + =================== === ================================ ================== + m_axis_if_tx_tdata out PORT_COUNT*AXIS_IF_DATA_WIDTH Streaming data + m_axis_if_tx_tkeep out PORT_COUNT*AXIS_IF_KEEP_WIDTH Byte enable + m_axis_if_tx_tvalid out PORT_COUNT Data valid + m_axis_if_tx_tready in PORT_COUNT Ready for data + m_axis_if_tx_tlast out PORT_COUNT End of frame + m_axis_if_tx_tid out PORT_COUNT*AXIS_IF_TX_ID_WIDTH Source queue + m_axis_if_tx_tdest out PORT_COUNT*AXIS_IF_TX_DEST_WIDTH Destination port + m_axis_if_tx_tuser out PORT_COUNT*AXIS_IF_TX_USER_WIDTH Sideband data + =================== === ================================ ================== + + ``m_axis_if_tx_tuser`` bits, per interface + + .. table:: + + =============== ========= ============= ============= + Bit Name Width Description + =============== ========= ============= ============= + 0 bad_frame 1 Invalid frame + PTP_TAG_WIDTH:1 ptp_tag PTP_TAG_WIDTH PTP tag + =============== ========= ============= ============= + +.. object:: s_axis_if_tx_ptp_ts + + Transmit PTP timestamp from MAC, one AXI stream interface per interface. + + .. table:: + + ========================= === ======================== =================== + Signal Dir Width Description + ========================= === ======================== =================== + s_axis_if_tx_ptp_ts in PORT_COUNT*PTP_TS_WIDTH PTP timestamp + s_axis_if_tx_ptp_ts_tag in PORT_COUNT*PTP_TAG_WIDTH PTP timestamp tag + s_axis_if_tx_ptp_ts_valid in PORT_COUNT PTP timestamp valid + s_axis_if_tx_ptp_ts_ready out PORT_COUNT PTP timestamp ready + ========================= === ======================== =================== + +.. object:: m_axis_if_tx_ptp_ts + + Transmit PTP timestamp towards core logic, one AXI stream interface per interface. + + .. table:: + + ========================= === ======================== =================== + Signal Dir Width Description + ========================= === ======================== =================== + s_axis_if_tx_ptp_ts out PORT_COUNT*PTP_TS_WIDTH PTP timestamp + s_axis_if_tx_ptp_ts_tag out PORT_COUNT*PTP_TAG_WIDTH PTP timestamp tag + s_axis_if_tx_ptp_ts_valid out PORT_COUNT PTP timestamp valid + s_axis_if_tx_ptp_ts_ready in PORT_COUNT PTP timestamp ready + ========================= === ======================== =================== + +.. object:: s_axis_if_rx + + Streaming receive data from network, one AXI stream interface per interface. Closest interface to host, synchronous with core clock. + + .. table:: + + =================== === ================================ ================== + Signal Dir Width Description + =================== === ================================ ================== + s_axis_if_rx_tdata in PORT_COUNT*AXIS_IF_DATA_WIDTH Streaming data + s_axis_if_rx_tkeep in PORT_COUNT*AXIS_IF_KEEP_WIDTH Byte enable + s_axis_if_rx_tvalid in PORT_COUNT Data valid + s_axis_if_rx_tready out PORT_COUNT Ready for data + s_axis_if_rx_tlast in PORT_COUNT End of frame + s_axis_if_rx_tid in PORT_COUNT*AXIS_IF_RX_ID_WIDTH Source port + s_axis_if_rx_tdest in PORT_COUNT*AXIS_IF_RX_DEST_WIDTH Destination queue + s_axis_if_rx_tuser in PORT_COUNT*AXIS_IF_RX_USER_WIDTH Sideband data + =================== === ================================ ================== + + ``s_axis_if_rx_tuser`` bits, per interface + + .. table:: + + =============== ========= ============= ============= + Bit Name Width Description + =============== ========= ============= ============= + 0 bad_frame 1 Invalid frame + PTP_TS_WIDTH:1 ptp_ts PTP_TS_WIDTH PTP timestamp + =============== ========= ============= ============= + +.. object:: m_axis_if_rx + + Streaming receive data towards host, one AXI stream interface per interface. Closest interface to host, synchronous with core clock. + + .. table:: + + =================== === ================================ ================== + Signal Dir Width Description + =================== === ================================ ================== + m_axis_if_rx_tdata out PORT_COUNT*AXIS_IF_DATA_WIDTH Streaming data + m_axis_if_rx_tkeep out PORT_COUNT*AXIS_IF_KEEP_WIDTH Byte enable + m_axis_if_rx_tvalid out PORT_COUNT Data valid + m_axis_if_rx_tready in PORT_COUNT Ready for data + m_axis_if_rx_tlast out PORT_COUNT End of frame + m_axis_if_rx_tid out PORT_COUNT*AXIS_IF_RX_ID_WIDTH Source port + m_axis_if_rx_tdest out PORT_COUNT*AXIS_IF_RX_DEST_WIDTH Destination queue + m_axis_if_rx_tuser out PORT_COUNT*AXIS_IF_RX_USER_WIDTH Sideband data + =================== === ================================ ================== + + ``m_axis_if_rx_tuser`` bits, per interface + + .. table:: + + =============== ========= ============= ============= + Bit Name Width Description + =============== ========= ============= ============= + 0 bad_frame 1 Invalid frame + PTP_TS_WIDTH:1 ptp_ts PTP_TS_WIDTH PTP timestamp + =============== ========= ============= ============= + +.. object:: m_axis_stat + + Statistics increment output + + .. table:: + + ================== === ============== =================== + Signal Dir Width Description + ================== === ============== =================== + m_axis_stat_tdata in STAT_INC_WIDTH Statistic increment + m_axis_stat_tid in STAT_ID_WIDTH Statistic ID + m_axis_stat_tvalid in 1 Statistic valid + m_axis_stat_tready out 1 Statistic ready + ================== === ============== =================== + +.. object:: gpio + + Application section GPIO + + .. table:: + + ======== === ================== =================== + Signal Dir Width Description + ======== === ================== =================== + gpio_in in APP_GPIO_IN_WIDTH GPIO inputs + gpio_out out APP_GPIO_OUT_WIDTH GPIO outputs + ======== === ================== =================== + +.. object:: jtag + + Application section JTAG scan chain + + .. table:: + + ======== === ===== =================== + Signal Dir Width Description + ======== === ===== =================== + jtag_tdi in 1 JTAG TDI + jtag_tdo out 1 JTAG TDO + jtag_tms in 1 JTAG TMS + jtag_tck in 1 JTAG TCK + ======== === ===== =================== diff --git a/docs/source/modules/mqnic_core.rst b/docs/source/modules/mqnic_core.rst new file mode 100644 index 000000000..9142fdc26 --- /dev/null +++ b/docs/source/modules/mqnic_core.rst @@ -0,0 +1,870 @@ +.. _mod_mqnic_core: + +============== +``mqnic_core`` +============== + +``mqnic_core`` is the core integration-level module for mqnic for all host interfaces. Contains the interfaces, asynchronous FIFOs, PTP subsystem, statistics collection subsystem, and application block. + +For maximum flexibility, this module does not contain the actual host-facing DMA engine, so a wrapper is required to provide the DMA engine with the proper host-facing interface. The available wrappers are: + +* :ref:`mod_mqnic_core_pcie` for PCI express +* :ref:`mod_mqnic_core_axi` for AXI + +``mqnic_core`` integrates the following modules: + +* ``stats_counter``: statistics aggregation +* :ref:`mod_mqnic_ptp`: PTP subsystem +* :ref:`mod_mqnic_interface`: NIC interface +* :ref:`mod_mqnic_app_block`: Application block + +Parameters +========== + +.. object:: FPGA_ID + + FPGA JTAG ID, default is ``32'hDEADBEEF``. Reported in :ref:`rb_fw_id`. + +.. object:: FW_ID + + Firmware ID, default is ``32'h00000000``. Reported in :ref:`rb_fw_id`. + +.. object:: FW_VER + + Firmware version, default is ``32'h00_00_01_00``. Reported in :ref:`rb_fw_id`. + +.. object:: BOARD_ID + + Board ID, default is ``16'h1234_0000``. Reported in :ref:`rb_fw_id`. + +.. object:: BOARD_VER + + Board version, default is ``32'h01_00_00_00``. Reported in :ref:`rb_fw_id`. + +.. object:: BUILD_DATE + + Build date as a 32-bit unsigned Unix timestamp, default is ``32'd602976000``. Reported in :ref:`rb_fw_id`. + +.. object:: GIT_HASH + + 32 bits of the git commit hash, default is ``32'hdce357bf``. Reported in :ref:`rb_fw_id`. + +.. object:: RELEASE_INFO + + Additional release info, default is ``32'h00000000``. Reported in :ref:`rb_fw_id`. + +.. object:: IF_COUNT + + Interface count, default ``1``. + +.. object:: PORTS_PER_IF + + Ports per interface, default ``1``. + +.. object:: PORT_COUNT + + Total port count, must be set to ``IF_COUNT*PORTS_PER_IF``. + +.. object:: PTP_TS_WIDTH + + PTP timestamp width, must be ``96``. + +.. object:: PTP_TAG_WIDTH + + PTP tag signal width, default ``16``. + +.. object:: PTP_PERIOD_NS_WIDTH + + PTP period ns field width, default ``4``. + +.. object:: PTP_OFFSET_NS_WIDTH + + PTP offset ns field width, default ``32``. + +.. object:: PTP_FNS_WIDTH + + PTP fractional ns field width, default ``32``. + +.. object:: PTP_PERIOD_NS + + PTP nominal period, ns portion, default ``4'd4``. + +.. object:: PTP_PERIOD_FNS + + PTP nominal period, fractional ns portion, default ``32'd0``. + +.. object:: PTP_USE_SAMPLE_CLOCK + + Use external PTP sample clock, used to synchronize the PTP clock across clock domains, default ``0``. + +.. object:: PTP_SEPARATE_RX_CLOCK + + Use ``rx_ptp_clk`` instead of ``rx_clk`` for providing current PTP time if set, default ``0``. + +.. object:: PTP_PEROUT_ENABLE + + Enable PTP period output module, default ``0``. + +.. object:: PTP_PEROUT_COUNT + + Number of PTP period output channels, default ``1``. + +.. object:: EVENT_QUEUE_OP_TABLE_SIZE + + Event queue manager operation table size, default ``32``. + +.. object:: TX_QUEUE_OP_TABLE_SIZE + + Transmit queue manager operation table size, default ``32``. + +.. object:: RX_QUEUE_OP_TABLE_SIZE + + Receive queue manager operation table size, default ``32``. + +.. object:: TX_CPL_QUEUE_OP_TABLE_SIZE + + Transmit completion queue operation table size, default ``TX_QUEUE_OP_TABLE_SIZE``. + +.. object:: RX_CPL_QUEUE_OP_TABLE_SIZE + + Receive completion queue operation table size, default ``RX_QUEUE_OP_TABLE_SIZE``. + +.. object:: TX_QUEUE_INDEX_WIDTH + + Transmit queue index width, default ``13``. Sets the number of transmit queues on each interfaces as ``2**TX_QUEUE_INDEX_WIDTH``. + +.. object:: RX_QUEUE_INDEX_WIDTH + + Receive queue index width, default ``8``. Sets the number of receive queues on each interfaces as ``2**RX_QUEUE_INDEX_WIDTH``. + +.. object:: TX_CPL_QUEUE_INDEX_WIDTH + + Transmit completion queue index width, default ``TX_QUEUE_INDEX_WIDTH``. Sets the number of transmit completion queues on each interfaces as ``2**TX_CPL_QUEUE_INDEX_WIDTH``. + +.. object:: RX_CPL_QUEUE_INDEX_WIDTH + + Receive completion queue index width, default ``RX_QUEUE_INDEX_WIDTH``. Sets the number of receive completion queues on each interfaces as ``2**RX_CPL_QUEUE_INDEX_WIDTH``. + +.. object:: EVENT_QUEUE_PIPELINE + + Event queue manager pipeline length, default ``3``. Tune for best usage of block RAM cascade registers for specified queue count. + +.. object:: TX_QUEUE_PIPELINE + + Transmit queue manager pipeline stages, default ``3+(TX_QUEUE_INDEX_WIDTH > 12 ? TX_QUEUE_INDEX_WIDTH-12 : 0)``. Tune for best usage of block RAM cascade registers for specified queue count. + +.. object:: RX_QUEUE_PIPELINE + + Receive queue manager pipeline stages, default ``3+(RX_QUEUE_INDEX_WIDTH > 12 ? RX_QUEUE_INDEX_WIDTH-12 : 0)``. Tune for best usage of block RAM cascade registers for specified queue count. + +.. object:: TX_CPL_QUEUE_PIPELINE + + Transmit completion queue manager pipeline stages, default ``TX_QUEUE_PIPELINE``. Tune for best usage of block RAM cascade registers for specified queue count. + +.. object:: RX_CPL_QUEUE_PIPELINE + + Receive completion queue maanger pipeline stages, default ``RX_QUEUE_PIPELINE``. Tune for best usage of block RAM cascade registers for specified queue count. + +.. object:: TX_DESC_TABLE_SIZE + + Transmit engine descriptor table size, default ``32``. + +.. object:: RX_DESC_TABLE_SIZE + + Receive engine descriptor table size, default ``32``. + +.. object:: TX_SCHEDULER_OP_TABLE_SIZE + + Transmit scheduler operation table size, default ``TX_DESC_TABLE_SIZE``. + +.. object:: TX_SCHEDULER_PIPELINE + + Transmit scheduler pipeline stages, default ``TX_QUEUE_PIPELINE``. Tune for best usage of block RAM cascade registers for specified queue count. + +.. object:: TDMA_INDEX_WIDTH + + TDMA index width, default ``6``. Sets the number of TDMA timeslots as ``2**TDMA_INDEX_WIDTH``. + +.. object:: PTP_TS_ENABLE + + Enable PTP timestamping, default ``1``. + +.. object:: TX_PTP_TS_FIFO_DEPTH + + Depth of TX PTP timestamp FIFO, default ``32``. + +.. object:: RX_PTP_TS_FIFO_DEPTH + + Depth of RX PTP timestamp FIFO, default ``32``. + +.. object:: TX_CHECKSUM_ENABLE + + Enable TCP/UDP checksum offloading on transmit path, default ``1``. + +.. object:: RX_RSS_ENABLE + + Enable receive side scaling, default ``1``. Requires ``RX_HASH_ENABLE`` to be set. + +.. object:: RX_HASH_ENABLE + + Enable Toeplitz flow hashing for RX traffic, default ``1``. + +.. object:: RX_CHECKSUM_ENABLE + + Enable TCP/UDP checksum offloading on receive path, default ``1`` + +.. object:: TX_FIFO_DEPTH + + Transmit FIFO depth in bytes, per output port, per traffic class, default ``32768``. + +.. object:: RX_FIFO_DEPTH + + Receive FIFO depth in bytes, per output port, default ``32768``. + +.. object:: MAX_TX_SIZE + + Maximum packet size on transmit path, default ``9214``. + +.. object:: MAX_RX_SIZE + + Maximum packet size on receive path, default ``9214``. + +.. object:: TX_RAM_SIZE + + Transmit scratchpad RAM size per interface, default ``32768``. + +.. object:: RX_RAM_SIZE + + Receive scratchpad RAM size per interface, default ``32768``. + +.. object:: APP_ENABLE + + Enable application section, default ``0``. + +.. object:: APP_CTRL_ENABLE + + Enable application section control connection to core NIC registers, default ``1``. + +.. object:: APP_DMA_ENABLE + + Enable application section connection to DMA subsystem, default ``1``. + +.. object:: APP_AXIS_DIRECT_ENABLE + + Enable lowest-latency asynchronous streaming connection to application section, default ``1`` + +.. object:: APP_AXIS_SYNC_ENABLE + + Enable low-latency synchronous streaming connection to application section, default ``1`` + +.. object:: APP_AXIS_IF_ENABLE + + Enable interface-level streaming connection to application section, default ``1`` + +.. object:: APP_STAT_ENABLE + + Enable application section connection to statistics collection subsystem, default ``1`` + +.. object:: APP_GPIO_IN_WIDTH + + Application section GPIO input signal width, default ``32`` + +.. object:: APP_GPIO_OUT_WIDTH + + Application section GPIO output signal width, default ``32`` + +.. object:: DMA_ADDR_WIDTH + + DMA interface address signal width, default ``64``. + +.. object:: DMA_LEN_WIDTH + + DMA interface length signal width, default ``16``. + +.. object:: DMA_TAG_WIDTH + + DMA interface tag signal width, default ``16``. + +.. object:: IF_RAM_SEL_WIDTH + + Width of interface-level select signal, default ``1``. + +.. object:: RAM_SEL_WIDTH + + Width of select signal per segment in DMA RAM interface, default ``$clog2(IF_COUNT+(APP_ENABLE && APP_DMA_ENABLE ? 1 : 0))+IF_RAM_SEL_WIDTH+1``. + +.. object:: RAM_ADDR_WIDTH + + Width of address signal for DMA RAM interface, default ``$clog2(TX_RAM_SIZE > RX_RAM_SIZE ? TX_RAM_SIZE : RX_RAM_SIZE)``. + +.. object:: RAM_SEG_COUNT + + Number of segments in DMA RAM interface, default ``2``. Must be a power of 2, must be at least 2. + +.. object:: RAM_SEG_DATA_WIDTH + + Width of data signal per segment in DMA RAM interface, default ``256*2/RAM_SEG_COUNT``. + +.. object:: RAM_SEG_BE_WIDTH + + Width of byte enable signal per segment in DMA RAM interface, must be set to ``RAM_SEG_DATA_WIDTH/8``. + +.. object:: RAM_SEG_ADDR_WIDTH + + Width of address signal per segment in DMA RAM interface, default ``RAM_ADDR_WIDTH-$clog2(RAM_SEG_COUNT*RAM_SEG_BE_WIDTH)``. + +.. object:: RAM_PIPELINE + + Number of output pipeline stages in segmented DMA RAMs, default ``2``. Tune for best usage of block RAM cascade registers. + +.. object:: MSI_COUNT + + Number of interrupt channels, default ``32``. + +.. object:: AXIL_CTRL_DATA_WIDTH + + AXI lite control data signal width, must be set to ``32``. + +.. object:: AXIL_CTRL_ADDR_WIDTH + + AXI lite control address signal width, default ``16``. + +.. object:: AXIL_CTRL_STRB_WIDTH + + AXI lite control byte enable signal width, must be set to ``AXIL_CTRL_DATA_WIDTH/8``. + +.. object:: AXIL_IF_CTRL_ADDR_WIDTH + + AXI lite interface control address signal width, default ``AXIL_CTRL_ADDR_WIDTH-$clog2(IF_COUNT)`` + +.. object:: AXIL_CSR_ADDR_WIDTH + + AXI lite interface CSR address signal width, default ``AXIL_IF_CTRL_ADDR_WIDTH-5-$clog2((PORTS_PER_IF+3)/8)`` + +.. object:: AXIL_CSR_PASSTHROUGH_ENABLE + + Enable NIC control register space passthrough, default ``0``. + +.. object:: RB_NEXT_PTR + + Next pointer of last register block in the NIC-level CSR space, default ``0``. + +.. object:: AXIL_APP_CTRL_DATA_WIDTH + + AXI lite application control data signal width, default ``AXIL_CTRL_DATA_WIDTH``. Can be 32 or 64. + +.. object:: AXIL_APP_CTRL_ADDR_WIDTH + + AXI lite application control address signal width, default ``16``. + +.. object:: AXIL_APP_CTRL_STRB_WIDTH + + AXI lite application control byte enable signal width, must be set to ``AXIL_APP_CTRL_DATA_WIDTH/8``. + +.. object:: AXIS_DATA_WIDTH + + Streaming interface ``tdata`` signal width, default ``512``. + +.. object:: AXIS_KEEP_WIDTH + + Streaming interface ``tkeep`` signal width, must be set to ``AXIS_DATA_WIDTH/8``. + +.. object:: AXIS_SYNC_DATA_WIDTH + + Synchronous streaming interface ``tdata`` signal width, default ``AXIS_DATA_WIDTH``. + +.. object:: AXIS_IF_DATA_WIDTH + + Interface streaming interface ``tdata`` signal width, default ``AXIS_SYNC_DATA_WIDTH*2**$clog2(PORTS_PER_IF)``. + +.. object:: AXIS_TX_USER_WIDTH + + Transmit streaming interface ``tuser`` signal width, default ``(PTP_TS_ENABLE ? PTP_TAG_WIDTH : 0) + 1``. + +.. object:: AXIS_RX_USER_WIDTH + + Receive streaming interface ``tuser`` signal width, default ``(PTP_TS_ENABLE ? PTP_TS_WIDTH : 0) + 1``. + +.. object:: AXIS_RX_USE_READY + + Use ``tready`` signal on RX interfaces, default ``0``. If set, logic will exert backpressure with ``tready`` instead of dropping packets when RX FIFOs are full. + +.. object:: AXIS_TX_PIPELINE + + Number of stages in transmit path pipeline FIFO, default ``0``. Useful for SLR crossings. + +.. object:: AXIS_TX_FIFO_PIPELINE + + Number of output pipeline stages in transmit FIFO, default ``2``. Tune for best usage of block RAM cascade registers. + +.. object:: AXIS_TX_TS_PIPELINE + + Number of stages in transmit path PTP timestamp pipeline FIFO, default ``0``. Useful for SLR crossings. + +.. object:: AXIS_RX_PIPELINE + + Number of stages in receive path pipeline FIFO, default ``0``. Useful for SLR crossings. + +.. object:: AXIS_RX_FIFO_PIPELINE + + Number of output pipeline stages in receive FIFO, default ``2``. Tune for best usage of block RAM cascade registers. + +.. object:: STAT_ENABLE + + Enable statistics collection subsystem, default ``1``. + +.. object:: STAT_INC_WIDTH + + Statistics increment signal width, default ``24``. + +.. object:: STAT_ID_WIDTH + + Statistics ID signal width, default ``12``. Sets the number of statistics counters as ``2**STAT_ID_WIDTH``. + +Ports +===== + +.. object:: clk + + Logic clock. Most interfaces are synchronous to this clock. + + .. table:: + + ====== === ===== ================== + Signal Dir Width Description + ====== === ===== ================== + clk in 1 Logic clock + ====== === ===== ================== + +.. object:: rst + + Logic reset, active high + + .. table:: + + ====== === ===== ================== + Signal Dir Width Description + ====== === ===== ================== + rst in 1 Logic reset, active high + ====== === ===== ================== + +.. object:: s_axil_ctrl + + AXI-Lite slave interface (control). This interface provides access to the main NIC control register space. + + .. table:: + + =================== === ==================== =================== + Signal Dir Width Description + =================== === ==================== =================== + s_axil_ctrl_awaddr in AXIL_CTRL_ADDR_WIDTH Write address + s_axil_ctrl_awprot in 3 Write protect + s_axil_ctrl_awvalid in 1 Write address valid + s_axil_ctrl_awready out 1 Write address ready + s_axil_ctrl_wdata in AXIL_CTRL_DATA_WIDTH Write data + s_axil_ctrl_wstrb in AXIL_CTRL_STRB_WIDTH Write data strobe + s_axil_ctrl_wvalid in 1 Write data valid + s_axil_ctrl_wready out 1 Write data ready + s_axil_ctrl_bresp out 2 Write response status + s_axil_ctrl_bvalid out 1 Write response valid + s_axil_ctrl_bready in 1 Write response ready + s_axil_ctrl_araddr in AXIL_CTRL_ADDR_WIDTH Read address + s_axil_ctrl_arprot in 3 Read protect + s_axil_ctrl_arvalid in 1 Read address valid + s_axil_ctrl_arready out 1 Read address ready + s_axil_ctrl_rdata out AXIL_CTRL_DATA_WIDTH Read response data + s_axil_ctrl_rresp out 2 Read response status + s_axil_ctrl_rvalid out 1 Read response valid + s_axil_ctrl_rready in 1 Read response ready + =================== === ==================== =================== + +.. object:: s_axil_app_ctrl + + AXI-Lite slave interface (application control). This interface is directly passed through to the application section. + + .. table:: + + ======================= === ======================== =================== + Signal Dir Width Description + ======================= === ======================== =================== + s_axil_app_ctrl_awaddr in AXIL_APP_CTRL_ADDR_WIDTH Write address + s_axil_app_ctrl_awprot in 3 Write protect + s_axil_app_ctrl_awvalid in 1 Write address valid + s_axil_app_ctrl_awready out 1 Write address ready + s_axil_app_ctrl_wdata in AXIL_APP_CTRL_DATA_WIDTH Write data + s_axil_app_ctrl_wstrb in AXIL_APP_CTRL_STRB_WIDTH Write data strobe + s_axil_app_ctrl_wvalid in 1 Write data valid + s_axil_app_ctrl_wready out 1 Write data ready + s_axil_app_ctrl_bresp out 2 Write response status + s_axil_app_ctrl_bvalid out 1 Write response valid + s_axil_app_ctrl_bready in 1 Write response ready + s_axil_app_ctrl_araddr in AXIL_APP_CTRL_ADDR_WIDTH Read address + s_axil_app_ctrl_arprot in 3 Read protect + s_axil_app_ctrl_arvalid in 1 Read address valid + s_axil_app_ctrl_arready out 1 Read address ready + s_axil_app_ctrl_rdata out AXIL_APP_CTRL_DATA_WIDTH Read response data + s_axil_app_ctrl_rresp out 2 Read response status + s_axil_app_ctrl_rvalid out 1 Read response valid + s_axil_app_ctrl_rready in 1 Read response ready + ======================= === ======================== =================== + +.. object:: m_axil_csr + + AXI-Lite master interface (passthrough for NIC control and status). This interface can be used to implement additional components in the main NIC control register space. + + .. table:: + + ================== === ==================== =================== + Signal Dir Width Description + ================== === ==================== =================== + m_axil_csr_awaddr in AXIL_CSR_ADDR_WIDTH Write address + m_axil_csr_awprot in 3 Write protect + m_axil_csr_awvalid in 1 Write address valid + m_axil_csr_awready out 1 Write address ready + m_axil_csr_wdata in AXIL_CTRL_DATA_WIDTH Write data + m_axil_csr_wstrb in AXIL_CTRL_STRB_WIDTH Write data strobe + m_axil_csr_wvalid in 1 Write data valid + m_axil_csr_wready out 1 Write data ready + m_axil_csr_bresp out 2 Write response status + m_axil_csr_bvalid out 1 Write response valid + m_axil_csr_bready in 1 Write response ready + m_axil_csr_araddr in AXIL_CTRL_ADDR_WIDTH Read address + m_axil_csr_arprot in 3 Read protect + m_axil_csr_arvalid in 1 Read address valid + m_axil_csr_arready out 1 Read address ready + m_axil_csr_rdata out AXIL_CTRL_DATA_WIDTH Read response data + m_axil_csr_rresp out 2 Read response status + m_axil_csr_rvalid out 1 Read response valid + m_axil_csr_rready in 1 Read response ready + ================== === ==================== =================== + +.. object:: ctrl_reg + + Control register interface. This interface can be used to implement additional control registers and register blocks in the main NIC control register space. + + .. table:: + + ================= === ==================== =================== + Signal Dir Width Description + ================= === ==================== =================== + ctrl_reg_wr_addr out AXIL_CSR_ADDR_WIDTH Write address + ctrl_reg_wr_data out AXIL_CTRL_DATA_WIDTH Write data + ctrl_reg_wr_strb out AXIL_CTRL_STRB_WIDTH Write strobe + ctrl_reg_wr_en out 1 Write enable + ctrl_reg_wr_wait in 1 Write wait + ctrl_reg_wr_ack in 1 Write acknowledge + ctrl_reg_rd_addr out AXIL_CSR_ADDR_WIDTH Read address + ctrl_reg_rd_en out 1 Read enable + ctrl_reg_rd_data in AXIL_CTRL_DATA_WIDTH Read data + ctrl_reg_rd_wait in 1 Read wait + ctrl_reg_rd_ack in 1 Read acknowledge + ================= === ==================== =================== + +.. object:: m_axis_dma_read_desc + + DMA read descriptor output + + .. table:: + + ============================= === ============== =================== + Signal Dir Width Description + ============================= === ============== =================== + m_axis_dma_read_desc_dma_addr out DMA_ADDR_WIDTH DMA address + m_axis_dma_read_desc_ram_sel out RAM_SEL_WIDTH RAM select + m_axis_dma_read_desc_ram_addr out RAM_ADDR_WIDTH RAM address + m_axis_dma_read_desc_len out DMA_LEN_WIDTH Transfer length + m_axis_dma_read_desc_tag out DMA_TAG_WIDTH Transfer tag + m_axis_dma_read_desc_valid out 1 Request valid + m_axis_dma_read_desc_ready in 1 Request ready + ============================= === ============== =================== + +.. object:: s_axis_dma_read_desc_status + + DMA read descriptor status input + + .. table:: + + ================================= === ============= =================== + Signal Dir Width Description + ================================= === ============= =================== + s_axis_dma_read_desc_status_tag in DMA_TAG_WIDTH Status tag + s_axis_dma_read_desc_status_error in 4 Status error code + s_axis_dma_read_desc_status_valid in 1 Status valid + ================================= === ============= =================== + +.. object:: m_axis_dma_write_desc + + DMA write descriptor output + + .. table:: + + ============================== === ============== =================== + Signal Dir Width Description + ============================== === ============== =================== + m_axis_dma_write_desc_dma_addr out DMA_ADDR_WIDTH DMA address + m_axis_dma_write_desc_ram_sel out RAM_SEL_WIDTH RAM select + m_axis_dma_write_desc_ram_addr out RAM_ADDR_WIDTH RAM address + m_axis_dma_write_desc_len out DMA_LEN_WIDTH Transfer length + m_axis_dma_write_desc_tag out DMA_TAG_WIDTH Transfer tag + m_axis_dma_write_desc_valid out 1 Request valid + m_axis_dma_write_desc_ready in 1 Request ready + ============================== === ============== =================== + +.. object:: s_axis_dma_write_desc_status + + DMA write descriptor status input + + .. table:: + + ================================== === ============= =================== + Signal Dir Width Description + ================================== === ============= =================== + s_axis_dma_write_desc_status_tag in DMA_TAG_WIDTH Status tag + s_axis_dma_write_desc_status_error in 4 Status error code + s_axis_dma_write_desc_status_valid in 1 Status valid + ================================== === ============= =================== + +.. object:: dma_ram + + DMA RAM interface + + .. table:: + + ===================== === ================================ =================== + Signal Dir Width Description + ===================== === ================================ =================== + dma_ram_wr_cmd_sel in RAM_SEG_COUNT*RAM_SEL_WIDTH Write command select + dma_ram_wr_cmd_be in RAM_SEG_COUNT*RAM_SEG_BE_WIDTH Write command byte enable + dma_ram_wr_cmd_addr in RAM_SEG_COUNT*RAM_SEG_ADDR_WIDTH Write command address + dma_ram_wr_cmd_data in RAM_SEG_COUNT*RAM_SEG_DATA_WIDTH Write command data + dma_ram_wr_cmd_valid in RAM_SEG_COUNT Write command valid + dma_ram_wr_cmd_ready out RAM_SEG_COUNT Write command ready + dma_ram_wr_done out RAM_SEG_COUNT Write done + dma_ram_rd_cmd_sel in RAM_SEG_COUNT*RAM_SEL_WIDTH Read command select + dma_ram_rd_cmd_addr in RAM_SEG_COUNT*RAM_SEG_ADDR_WIDTH Read command address + dma_ram_rd_cmd_valid in RAM_SEG_COUNT Read command valid + dma_ram_rd_cmd_ready out RAM_SEG_COUNT Read command ready + dma_ram_rd_resp_data out RAM_SEG_COUNT*RAM_SEG_DATA_WIDTH Read response data + dma_ram_rd_resp_valid out RAM_SEG_COUNT Read response valid + dma_ram_rd_resp_ready in RAM_SEG_COUNT Read response ready + ===================== === ================================ =================== + +.. object:: msi_irq + + MSI request outputs + + .. table:: + + ======= === ========= =================== + Signal Dir Width Description + ======= === ========= =================== + msi_irq out MSI_COUNT Interrupt request + ======= === ========= =================== + +.. object:: ptp + + PTP clock connections. + + .. table:: + + ================= === ================ =================== + Signal Dir Width Description + ================= === ================ =================== + ptp_sample_clk in 1 PTP sample clock + ptp_pps out 1 PTP pulse-per-second + ptp_ts_96 out PTP_TS_WIDTH current PTP time + ptp_ts_step out 1 PTP clock step + ptp_perout_locked out PTP_PEROUT_COUNT PTP period output locked + ptp_perout_error out PTP_PEROUT_COUNT PTP period output error + ptp_perout_pulse out PTP_PEROUT_COUNT PTP period output pulse + ================= === ================ =================== + +.. object:: tx_clk + + Transmit clocks, one per port + + .. table:: + + ====== === ========== ================== + Signal Dir Width Description + ====== === ========== ================== + tx_clk in PORT_COUNT Transmit clock + ====== === ========== ================== + +.. object:: tx_rst + + Transmit resets, one per port + + .. table:: + + ====== === ========== ================== + Signal Dir Width Description + ====== === ========== ================== + tx_rst in PORT_COUNT Transmit reset + ====== === ========== ================== + +.. object:: tx_ptp_ts + + Reference PTP time for transmit timestamping synchronous to each transmit clock, one per port. + + .. table:: + + ============== === ======================= ================== + Signal Dir Width Description + ============== === ======================= ================== + tx_ptp_ts_96 out PORT_COUNT*PTP_TS_WIDTH current PTP time + tx_ptp_ts_step out PORT_COUNT PTP clock step + ============== === ======================= ================== + +.. object:: m_axis_tx + + Streaming transmit data towards network, one AXI stream interface per port. + + .. table:: + + ================ === ============================= ================== + Signal Dir Width Description + ================ === ============================= ================== + m_axis_tx_tdata out PORT_COUNT*AXIS_DATA_WIDTH Streaming data + m_axis_tx_tkeep out PORT_COUNT*AXIS_KEEP_WIDTH Byte enable + m_axis_tx_tvalid out PORT_COUNT Data valid + m_axis_tx_tready in PORT_COUNT Ready for data + m_axis_tx_tlast out PORT_COUNT End of frame + m_axis_tx_tuser out PORT_COUNT*AXIS_TX_USER_WIDTH Sideband data + ================ === ============================= ================== + + ``s_axis_tx_tuser`` bits, per port + + .. table:: + + =============== ========= ============= ============= + Bit Name Width Description + =============== ========= ============= ============= + 0 bad_frame 1 Invalid frame + PTP_TAG_WIDTH:1 ptp_tag PTP_TAG_WIDTH PTP tag + =============== ========= ============= ============= + +.. object:: s_axis_tx_ptp_ts + + Transmit PTP timestamp, one AXI stream interface per port. + + .. table:: + + ====================== === ======================== =================== + Signal Dir Width Description + ====================== === ======================== =================== + s_axis_tx_ptp_ts in PORT_COUNT*PTP_TS_WIDTH PTP timestamp + s_axis_tx_ptp_ts_tag in PORT_COUNT*PTP_TAG_WIDTH PTP timestamp tag + s_axis_tx_ptp_ts_valid in PORT_COUNT PTP timestamp valid + s_axis_tx_ptp_ts_ready out PORT_COUNT PTP timestamp ready + ====================== === ======================== =================== + +.. object:: rx_clk + + Receive clocks, one per port + + .. table:: + + ====== === ========== ================== + Signal Dir Width Description + ====== === ========== ================== + rx_clk in PORT_COUNT Receive clock + ====== === ========== ================== + +.. object:: rx_rst + + Receive resets, one per port + + .. table:: + + ====== === ========== ================== + Signal Dir Width Description + ====== === ========== ================== + rx_rst in PORT_COUNT Receive reset + ====== === ========== ================== + +.. object:: rx_ptp_ts + + Reference PTP time for receive timestamping synchronous to each receive clock, one per port. Synchronous to ``rx_ptp_clk`` if ``PTP_SEPARATE_RX_CLOCK`` is set. + + .. table:: + + ============== === ======================= ================== + Signal Dir Width Description + ============== === ======================= ================== + rx_ptp_clk in PORT_COUNT clock for PTP time + rx_ptp_rst in PORT_COUNT reset for PTP time + rx_ptp_ts_96 out PORT_COUNT*PTP_TS_WIDTH current PTP time + rx_ptp_ts_step out PORT_COUNT PTP clock step + ============== === ======================= ================== + +.. object:: s_axis_rx + + Streaming receive data from network, one AXI stream interface per port. + + .. table:: + + ================ === ============================= ================== + Signal Dir Width Description + ================ === ============================= ================== + s_axis_rx_tdata in PORT_COUNT*AXIS_DATA_WIDTH Streaming data + s_axis_rx_tkeep in PORT_COUNT*AXIS_KEEP_WIDTH Byte enable + s_axis_rx_tvalid in PORT_COUNT Data valid + s_axis_rx_tready out PORT_COUNT Ready for data + s_axis_rx_tlast in PORT_COUNT End of frame + s_axis_rx_tuser in PORT_COUNT*AXIS_TX_USER_WIDTH Sideband data + ================ === ============================= ================== + + ``s_axis_rx_tuser`` bits, per port + + .. table:: + + ============== ========= ============ ============= + Bit Name Width Description + ============== ========= ============ ============= + 0 bad_frame 1 Invalid frame + PTP_TS_WIDTH:1 ptp_ts PTP_TS_WIDTH PTP timestamp + ============== ========= ============ ============= + +.. object:: s_axis_stat + + Statistics increment input + + .. table:: + + ================== === ============== =================== + Signal Dir Width Description + ================== === ============== =================== + s_axis_stat_tdata in STAT_INC_WIDTH Statistic increment + s_axis_stat_tid in STAT_ID_WIDTH Statistic ID + s_axis_stat_tvalid in 1 Statistic valid + s_axis_stat_tready out 1 Statistic ready + ================== === ============== =================== + +.. object:: app_gpio + + Application section GPIO + + .. table:: + + ============ === ================== =================== + Signal Dir Width Description + ============ === ================== =================== + app_gpio_in in APP_GPIO_IN_WIDTH GPIO inputs + app_gpio_out out APP_GPIO_OUT_WIDTH GPIO outputs + ============ === ================== =================== + +.. object:: app_jtag + + Application section JTAG scan chain + + .. table:: + + ============ === ===== =================== + Signal Dir Width Description + ============ === ===== =================== + app_jtag_tdi in 1 JTAG TDI + app_jtag_tdo out 1 JTAG TDO + app_jtag_tms in 1 JTAG TMS + app_jtag_tck in 1 JTAG TCK + ============ === ===== =================== diff --git a/docs/source/modules/mqnic_core_axi.rst b/docs/source/modules/mqnic_core_axi.rst new file mode 100644 index 000000000..32ef8573e --- /dev/null +++ b/docs/source/modules/mqnic_core_axi.rst @@ -0,0 +1,128 @@ +.. _mod_mqnic_core_axi: + +================== +``mqnic_core_axi`` +================== + +``mqnic_core_axi`` is the core integration-level module for mqnic for the AXI host interface. Wrapper around :ref:`mod_mqnic_core`, adding the AXI DMA interface module. + +``mqnic_core_axi`` integrates the following modules: + +* ``dma_if_axi``: AXI DMA engine +* :ref:`mod_mqnic_core`: core logic + +Parameters +========== + +Only parameters implemented in the wrapper are described here, for the other parameters see :ref:`mod_mqnic_core`. + +.. object:: AXI_DATA_WIDTH + + AXI master interface data signal width, default ``128``. + +.. object:: AXI_ADDR_WIDTH + + AXI master interface address signal width, default ``32``. + +.. object:: AXI_STRB_WIDTH + + AXI master interface byte enable signal width, default ``(AXI_DATA_WIDTH/8)``. + +.. object:: AXI_ID_WIDTH + + AXI master interface ID signal width, default ``8``. + +.. object:: AXI_DMA_MAX_BURST_LEN + + AXI DMA maximum burst length, default ``256``. + +.. object:: AXI_DMA_READ_USE_ID + + Use ID field for AXI DMA reads, default ``0``. + +.. object:: AXI_DMA_WRITE_USE_ID + + Use ID field for AXI DMA writes, default ``1``. + +.. object:: AXI_DMA_READ_OP_TABLE_SIZE + + AXI read DMA operation table size, default ``2**(AXI_ID_WIDTH)``. + +.. object:: AXI_DMA_WRITE_OP_TABLE_SIZE + + AXI write DMA operation table size, default ``2**(AXI_ID_WIDTH)``. + +.. object:: IRQ_COUNT + + IRQ channel count, default ``32``. + +.. object:: STAT_DMA_ENABLE + + Enable DMA-related statistics, default ``1``. + +.. object:: STAT_AXI_ENABLE + + Enable AXI-related statistics, default ``1``. + +Ports +===== + +Only ports implemented in the wrapper are described here, for the other ports see :ref:`mod_mqnic_core`. + +.. object:: m_axi + + AXI master interface (DMA). + + .. table:: + + ============= === ============== =================== + Signal Dir Width Description + ============= === ============== =================== + m_axi_awid out AXI_ID_WIDTH Write ID + m_axi_awaddr out AXI_ADDR_WIDTH Write address + m_axi_awlen out 8 Write burst length + m_axi_awsize out 3 Write burst size + m_axi_awburst out 2 Write burst type + m_axi_awlock out 1 Write lock + m_axi_awcache out 4 Write cache + m_axi_awprot out 3 Write protect + m_axi_awvalid out 1 Write valid + m_axi_awready in 1 Write ready + m_axi_wdata out AXI_DATA_WIDTH Write data data + m_axi_wstrb out AXI_STRB_WIDTH Write data strobe + m_axi_wlast out 1 Write data last + m_axi_wvalid out 1 Write data valid + m_axi_wready in 1 Write data ready + m_axi_bid in AXI_ID_WIDTH Write response ID + m_axi_bresp in 2 Write response status + m_axi_bvalid in 1 Write response valid + m_axi_bready out 1 Write response ready + m_axi_arid out AXI_ID_WIDTH Read ID + m_axi_araddr out AXI_ADDR_WIDTH Read address + m_axi_arlen out 8 Read burst length + m_axi_arsize out 3 Read burst size + m_axi_arburst out 2 Read burst type + m_axi_arlock out 1 Read lock + m_axi_arcache out 4 Read cache + m_axi_arprot out 3 Read protect + m_axi_arvalid out 1 Read address valid + m_axi_arready in 1 Read address ready + m_axi_rid in AXI_ID_WIDTH Read response ID + m_axi_rdata in AXI_DATA_WIDTH Read response data + m_axi_rresp in 2 Read response status + m_axi_rlast in 1 Read response last + m_axi_rvalid in 1 Read response valid + m_axi_rready out 1 Read response ready + ============= === ============== =================== + +.. object:: msi_irq + + Interrupt outputs + + .. table:: + + ====== === ========= =================== + Signal Dir Width Description + ====== === ========= =================== + irq out IRQ_COUNT Interrupt request + ====== === ========= =================== diff --git a/docs/source/modules/mqnic_core_pcie.rst b/docs/source/modules/mqnic_core_pcie.rst new file mode 100644 index 000000000..10ad4711b --- /dev/null +++ b/docs/source/modules/mqnic_core_pcie.rst @@ -0,0 +1,289 @@ +.. _mod_mqnic_core_pcie: + +=================== +``mqnic_core_pcie`` +=================== + +``mqnic_core_pcie`` is the core integration-level module for mqnic for the PCIe host interface. Wrapper around :ref:`mod_mqnic_core`, adding PCIe DMA interface module and PCIe-AXI Lite masters for the NIC and application control BARs. + +This module implements a generic PCIe host interface, which must be adapted to the target device with a wrapper. The available wrappers are: + +* :ref:`mod_mqnic_core_pcie_us` for Xilinx Virtex 7, UltraScale, and UltraScale+ +* :ref:`mod_mqnic_core_pcie_s10` for Intel Stratix 10 H-tile/L-tile + +``mqnic_core_pcie`` integrates the following modules: + +* ``dma_if_pcie``: PCIe DMA engine +* ``pcie_axil_master``: AXI lite master module for control registers +* ``stats_pcie_if``: statistics collection for PCIe TLP traffic +* ``stats_dma_if_pcie``: statistics collection for PCIe DMA engine +* :ref:`mod_mqnic_core`: core logic + +Parameters +========== + +Only parameters implemented in the wrapper are described here, for the other parameters see :ref:`mod_mqnic_core`. + +.. object:: TLP_SEG_COUNT + + Number of segments in the TLP interfaces, default ``1``. + +.. object:: TLP_SEG_DATA_WIDTH + + TLP segment data width, default ``256``. + +.. object:: TLP_SEG_STRB_WIDTH + + TLP segment byte enable width, must be set to ``TLP_SEG_DATA_WIDTH/32``. + +.. object:: TLP_SEG_HDR_WIDTH + + TLP segment header width, must be ``128``. + +.. object:: TX_SEQ_NUM_COUNT + + Number of transmit sequence number inputs, default ``1``. + +.. object:: TX_SEQ_NUM_WIDTH + + Transmit sequence number width, default ``5``. + +.. object:: TX_SEQ_NUM_ENABLE + + Use transmit sequence numbers, default ``0``. + +.. object:: PF_COUNT + + PCIe PF count, default ``1``. + +.. object:: VF_COUNT + + PCIe VF count, default ``0``. + +.. object:: F_COUNT + + PCIe function count, must be ``PF_COUNT+VF_COUNT``. + +.. object:: PCIE_TAG_COUNT + + PCIe tag count, default ``256``. + +.. object:: PCIE_DMA_READ_OP_TABLE_SIZE + + PCIe read DMA operation table size, default ``PCIE_TAG_COUNT``. + +.. object:: PCIE_DMA_READ_TX_LIMIT + + PCIe read DMA transmit operation limit, default ``2**TX_SEQ_NUM_WIDTH``. + +.. object:: PCIE_DMA_READ_TX_FC_ENABLE + + Use transmit flow control credits in PCIe read DMA, default ``0``. + +.. object:: PCIE_DMA_WRITE_OP_TABLE_SIZE + + PCIe write DMA operation table size, default ``2**TX_SEQ_NUM_WIDTH``. + +.. object:: PCIE_DMA_WRITE_TX_LIMIT + + PCIe write DMA transmit operation limit, default ``2**TX_SEQ_NUM_WIDTH``. + +.. object:: PCIE_DMA_WRITE_TX_FC_ENABLE + + Use transmit flow control credits in PCIe write DMA, default ``0``. + +.. object:: TLP_FORCE_64_BIT_ADDR + + Force 64 bit address field for all TLPs, default ``0``. + +.. object:: CHECK_BUS_NUMBER + + Check bus number in received TLPs, default ``1``. + +.. object:: MSI_COUNT + + Number of MSI channels, default ``32``. + +.. object:: STAT_DMA_ENABLE + + Enable DMA-related statistics, default ``1``. + +.. object:: STAT_PCIE_ENABLE + + Enable PCIe-related statistics, default ``1``. + +Ports +===== + +Only ports implemented in the wrapper are described here, for the other ports see :ref:`mod_mqnic_core`. + +.. object:: pcie_rx_req_tlp + + TLP input (request to BAR) + + .. table:: + + ======================== === ================================ =================== + Signal Dir Width Description + ======================== === ================================ =================== + pcie_rx_req_tlp_data in TLP_SEG_COUNT*TLP_SEG_DATA_WIDTH TLP payload + pcie_rx_req_tlp_hdr in TLP_SEG_COUNT*TLP_SEG_HDR_WIDTH TLP header + pcie_rx_req_tlp_bar_id in TLP_SEG_COUNT*3 BAR ID + pcie_rx_req_tlp_func_num in TLP_SEG_COUNT*8 Function + pcie_rx_req_tlp_valid in TLP_SEG_COUNT Valid + pcie_rx_req_tlp_sop in TLP_SEG_COUNT Start of packet + pcie_rx_req_tlp_eop in TLP_SEG_COUNT End of packet + pcie_rx_req_tlp_ready out 1 Ready + ======================== === ================================ =================== + +.. object:: pcie_rx_cpl_tlp + + TLP input (completion to DMA) + + .. table:: + + ===================== === ================================ =================== + Signal Dir Width Description + ===================== === ================================ =================== + pcie_rx_cpl_tlp_data in TLP_SEG_COUNT*TLP_SEG_DATA_WIDTH TLP payload + pcie_rx_cpl_tlp_hdr in TLP_SEG_COUNT*TLP_SEG_HDR_WIDTH TLP header + pcie_rx_cpl_tlp_error in TLP_SEG_COUNT*4 Error + pcie_rx_cpl_tlp_valid in TLP_SEG_COUNT Valid + pcie_rx_cpl_tlp_sop in TLP_SEG_COUNT Start of packet + pcie_rx_cpl_tlp_eop in TLP_SEG_COUNT End of packet + pcie_rx_cpl_tlp_ready out 1 Ready + ===================== === ================================ =================== + +.. object:: pcie_tx_rd_req_tlp + + TLP output (read request from DMA) + + .. table:: + + ======================== === =============================== =================== + Signal Dir Width Description + ======================== === =============================== =================== + pcie_tx_rd_req_tlp_hdr out TLP_SEG_COUNT*TLP_SEG_HDR_WIDTH TLP header + pcie_tx_rd_req_tlp_seq out TLP_SEG_COUNT*TX_SEQ_NUM_WIDTH TX seq num + pcie_tx_rd_req_tlp_valid out TLP_SEG_COUNT Valid + pcie_tx_rd_req_tlp_sop out TLP_SEG_COUNT Start of packet + pcie_tx_rd_req_tlp_eop out TLP_SEG_COUNT End of packet + pcie_tx_rd_req_tlp_ready in 1 Ready + ======================== === =============================== =================== + +.. object:: s_axis_pcie_rd_req_tx_seq_num + + Transmit sequence number input (DMA read request) + + .. table:: + + =================================== === ================================= =================== + Signal Dir Width Description + =================================== === ================================= =================== + s_axis_pcie_rd_req_tx_seq_num in TX_SEQ_NUM_COUNT*TX_SEQ_NUM_WIDTH TX seq num + s_axis_pcie_rd_req_tx_seq_num_valid in TX_SEQ_NUM_COUNT Valid + =================================== === ================================= =================== + +.. object:: pcie_tx_wr_req_tlp + + TLP output (read request from DMA) + + .. table:: + + ======================== === ================================ =================== + Signal Dir Width Description + ======================== === ================================ =================== + pcie_tx_wr_req_tlp_data out TLP_SEG_COUNT*TLP_SEG_DATA_WIDTH TLP payload + pcie_tx_wr_req_tlp_strb out TLP_SEG_COUNT*TLP_SEG_STRB_WIDTH TLP byte enable + pcie_tx_wr_req_tlp_hdr out TLP_SEG_COUNT*TLP_SEG_HDR_WIDTH TLP header + pcie_tx_wr_req_tlp_seq out TLP_SEG_COUNT*TX_SEQ_NUM_WIDTH TX seq num + pcie_tx_wr_req_tlp_valid out TLP_SEG_COUNT Valid + pcie_tx_wr_req_tlp_sop out TLP_SEG_COUNT Start of packet + pcie_tx_wr_req_tlp_eop out TLP_SEG_COUNT End of packet + pcie_tx_wr_req_tlp_ready in 1 Ready + ======================== === ================================ =================== + +.. object:: s_axis_pcie_wr_req_tx_seq_num + + Transmit sequence number input (DMA write request) + + .. table:: + + =================================== === ================================= =================== + Signal Dir Width Description + =================================== === ================================= =================== + s_axis_pcie_wr_req_tx_seq_num in TX_SEQ_NUM_COUNT*TX_SEQ_NUM_WIDTH TX seq num + s_axis_pcie_wr_req_tx_seq_num_valid in TX_SEQ_NUM_COUNT Valid + =================================== === ================================= =================== + +.. object:: pcie_tx_cpl_tlp + + TLP output (completion from BAR) + + .. table:: + + ===================== === ================================ =================== + Signal Dir Width Description + ===================== === ================================ =================== + pcie_tx_cpl_tlp_data out TLP_SEG_COUNT*TLP_SEG_DATA_WIDTH TLP payload + pcie_tx_cpl_tlp_strb out TLP_SEG_COUNT*TLP_SEG_STRB_WIDTH TLP byte enable + pcie_tx_cpl_tlp_hdr out TLP_SEG_COUNT*TLP_SEG_HDR_WIDTH TLP header + pcie_tx_cpl_tlp_valid out TLP_SEG_COUNT Valid + pcie_tx_cpl_tlp_sop out TLP_SEG_COUNT Start of packet + pcie_tx_cpl_tlp_eop out TLP_SEG_COUNT End of packet + pcie_tx_cpl_tlp_ready in 1 Ready + ===================== === ================================ =================== + +.. object:: pcie_tx_fc + + Flow control credits + + .. table:: + + ================= === ===== =================== + Signal Dir Width Description + ================= === ===== =================== + pcie_tx_fc_ph_av in 8 Available posted header credits + pcie_tx_fc_pd_av in 12 Available posted data credits + pcie_tx_fc_nph_av in 8 Available non-posted header credits + ================= === ===== =================== + +.. object:: config + + Configuration inputs + + .. table:: + + ===================== === ========= =================== + Signal Dir Width Description + ===================== === ========= =================== + bus_num in 8 Bus number + ext_tag_enable in F_COUNT Extended tag enable + max_read_request_size in F_COUNT*3 Max read request size + max_payload_size in F_COUNT*3 Max payload size + ===================== === ========= =================== + +.. object:: pcie_error + + PCIe error outputs + + .. table:: + + ================ === ===== =================== + Signal Dir Width Description + ================ === ===== =================== + pcie_error_cor out 1 Correctable error + pcie_error_uncor out 1 Uncorrectable error + ================ === ===== =================== + +.. object:: msi_irq + + MSI request outputs + + .. table:: + + ======= === ========= =================== + Signal Dir Width Description + ======= === ========= =================== + msi_irq out MSI_COUNT Interrupt request + ======= === ========= =================== diff --git a/docs/source/modules/mqnic_core_pcie_s10.rst b/docs/source/modules/mqnic_core_pcie_s10.rst new file mode 100644 index 000000000..c5a9d2f07 --- /dev/null +++ b/docs/source/modules/mqnic_core_pcie_s10.rst @@ -0,0 +1,135 @@ +.. _mod_mqnic_core_pcie_s10: + +======================= +``mqnic_core_pcie_s10`` +======================= + +``mqnic_core_pcie_s10`` is the core integration-level module for mqnic for the PCIe host interface on Intel Stratix 10 GX, SX, MX, and TX series devices with H-tiles or L-tiles. Wrapper around :ref:`mod_mqnic_core_pcie`, adding device-specific shims for the PCIe interface. + +``mqnic_core_pcie_s10`` integrates the following modules: + +* ``pcie_s10_if``: PCIe interface shim +* :ref:`mod_mqnic_core_pcie`: core logic for PCI express + +Parameters +========== + +Only parameters implemented in the wrapper are described here, for the other parameters see :ref:`mod_mqnic_core_pcie`. + +.. object:: SEG_COUNT + + TLP segment count, default ``1``. + +.. object:: SEG_DATA_WIDTH + + TLP segment data signal width, default ``256``. + +.. object:: SEG_EMPTY_WIDTH + + TLP segment empty signal width, must be set to ``$clog2(SEG_DATA_WIDTH/32)``. + +.. object:: TX_SEQ_NUM_WIDTH + + Transmit sequence number width, default ``6``. + +.. object:: TX_SEQ_NUM_ENABLE + + Transmit sequence number enable, default ``1``. + +.. object:: L_TILE + + Tile select, ``0`` for H-tile, ``1`` for L-tile, default ``0``. + +Ports +===== + +Only ports implemented in the wrapper are described here, for the other ports see :ref:`mod_mqnic_core_pcie`. + +.. object:: rx_st + + H-Tile/L-Tile RX AVST interface + + .. table:: + + =============== === ========================= =================== + Signal Dir Width Description + =============== === ========================= =================== + rx_st_data in SEG_COUNT*SEG_DATA_WIDTH TLP data + rx_st_empty in SEG_COUNT*SEG_EMPTY_WIDTH Empty + rx_st_sop in SEG_COUNT Start of packet + rx_st_eop in SEG_COUNT End of packet + rx_st_valid in SEG_COUNT Valid + rx_st_ready out 1 Ready + rx_st_vf_active in SEG_COUNT VF active + rx_st_func_num in SEG_COUNT*2 Function number + rx_st_vf_num in SEG_COUNT*11 VF number + rx_st_bar_range in SEG_COUNT*3 BAR range + =============== === ========================= =================== + +.. object:: tx_st + + H-Tile/L-Tile TX AVST interface + + .. table:: + + =========== === ======================== =================== + Signal Dir Width Description + =========== === ======================== =================== + tx_st_data out SEG_COUNT*SEG_DATA_WIDTH TLP data + tx_st_sop out SEG_COUNT Start of packet + tx_st_eop out SEG_COUNT End of packet + tx_st_valid out SEG_COUNT Valid + tx_st_ready in 1 Ready + tx_st_err out SEG_COUNT Error + =========== === ======================== =================== + +.. object:: tx_fc + + H-Tile/L-Tile TX flow control + + .. table:: + + ===================== === =========== =================== + Signal Dir Width Description + ===================== === =========== =================== + tx_ph_cdts in 8 Posted header credits + tx_pd_cdts in 12 Posted data credits + tx_nph_cdts in 8 Non-posted header credits + tx_npd_cdts in 12 Non-posted data credits + tx_cplh_cdts in 8 Completion header credits + tx_cpld_cdts in 12 Completion data credits + tx_hdr_cdts_consumed in SEG_COUNT Header credits consumed + tx_data_cdts_consumed in SEG_COUNT Data credits consumed + tx_cdts_type in SEG_COUNT*2 Credit type + tx_cdts_data_value in SEG_COUNT*1 Credit data value + ===================== === =========== =================== + +.. object:: app_msi + + H-Tile/L-Tile MSI interrupt interface + + .. table:: + + ================ === ===== =================== + Signal Dir Width Description + ================ === ===== =================== + app_msi_req out 1 MSI request + app_msi_ack in 1 MSI acknowledge + app_msi_tc out 3 MSI traffic class + app_msi_num out 5 MSI number + app_msi_func_num out 2 Function number + ================ === ===== =================== + +.. object:: tl_cfg + + H-Tile/L-Tile configuration interface + + .. table:: + + =========== === ===== =================== + Signal Dir Width Description + =========== === ===== =================== + tl_cfg_ctl in 32 Config data + tl_cfg_add in 5 Config address + tl_cfg_func in 2 Config function + =========== === ===== =================== diff --git a/docs/source/modules/mqnic_core_pcie_us.rst b/docs/source/modules/mqnic_core_pcie_us.rst new file mode 100644 index 000000000..b550121a4 --- /dev/null +++ b/docs/source/modules/mqnic_core_pcie_us.rst @@ -0,0 +1,224 @@ +.. _mod_mqnic_core_pcie_us: + +====================== +``mqnic_core_pcie_us`` +====================== + +``mqnic_core_pcie_us`` is the core integration-level module for mqnic for the PCIe host interface on Xilinx Virtex 7, UltraScale, and UltraScale+ series devices. Wrapper around :ref:`mod_mqnic_core_pcie`, adding device-specific shims for the PCIe interface. + +``mqnic_core_pcie_us`` integrates the following modules: + +* ``pcie_us_if``: PCIe interface shim +* :ref:`mod_mqnic_core_pcie`: core logic for PCI express + +Parameters +========== + +Only parameters implemented in the wrapper are described here, for the other parameters see :ref:`mod_mqnic_core_pcie`. + +.. object:: AXIS_PCIE_DATA_WIDTH + + PCIe AXI stream ``tdata`` signal width, default ``256``. + +.. object:: AXIS_PCIE_KEEP_WIDTH + + PCIe AXI stream ``tkeep`` signal width, must be set to ``(AXIS_PCIE_DATA_WIDTH/32)``. + +.. object:: AXIS_PCIE_RC_USER_WIDTH + + PCIe AXI stream RC ``tuser`` signal width, default ``AXIS_PCIE_DATA_WIDTH < 512 ? 75 : 161``. + +.. object:: AXIS_PCIE_RQ_USER_WIDTH + + PCIe AXI stream RQ ``tuser`` signal width, default ``AXIS_PCIE_DATA_WIDTH < 512 ? 62 : 137``. + +.. object:: AXIS_PCIE_CQ_USER_WIDTH + + PCIe AXI stream CQ ``tuser`` signal width, default ``AXIS_PCIE_DATA_WIDTH < 512 ? 85 : 183``. + +.. object:: AXIS_PCIE_CC_USER_WIDTH + + PCIe AXI stream CC ``tuser`` signal width, default ``AXIS_PCIE_DATA_WIDTH < 512 ? 33 : 81``. + +.. object:: RQ_SEQ_NUM_WIDTH + + PCIe RQ sequence number width, default ``AXIS_PCIE_RQ_USER_WIDTH == 60 ? 4 : 6``. + +Ports +===== + +Only ports implemented in the wrapper are described here, for the other ports see :ref:`mod_mqnic_core_pcie`. + +.. object:: s_axis_rc + + AXI input (RC) + + .. table:: + + ================ === ======================= =================== + Signal Dir Width Description + ================ === ======================= =================== + s_axis_rc_tdata in AXIS_PCIE_DATA_WIDTH TLP data + s_axis_rc_tkeep in AXIS_PCIE_KEEP_WIDTH Byte enable + s_axis_rc_tvalid in 1 Valid + s_axis_rc_tready out 1 Ready + s_axis_rc_tlast in 1 End of frame + s_axis_rc_tuser in AXIS_PCIE_RC_USER_WIDTH Sideband data + ================ === ======================= =================== + +.. object:: m_axis_rq + + AXI output (RQ) + + .. table:: + + ================ === ======================= =================== + Signal Dir Width Description + ================ === ======================= =================== + m_axis_rq_tdata out AXIS_PCIE_DATA_WIDTH TLP data + m_axis_rq_tkeep out AXIS_PCIE_KEEP_WIDTH Byte enable + m_axis_rq_tvalid out 1 Valid + m_axis_rq_tready in 1 Ready + m_axis_rq_tlast out 1 End of frame + m_axis_rq_tuser out AXIS_PCIE_RQ_USER_WIDTH Sideband data + ================ === ======================= =================== + +.. object:: s_axis_cq + + AXI input (CQ) + + .. table:: + + ================ === ======================= =================== + Signal Dir Width Description + ================ === ======================= =================== + s_axis_cq_tdata in AXIS_PCIE_DATA_WIDTH TLP data + s_axis_cq_tkeep in AXIS_PCIE_KEEP_WIDTH Byte enable + s_axis_cq_tvalid in 1 Valid + s_axis_cq_tready out 1 Ready + s_axis_cq_tlast in 1 End of frame + s_axis_cq_tuser in AXIS_PCIE_CQ_USER_WIDTH Sideband data + ================ === ======================= =================== + +.. object:: m_axis_cc + + AXI output (CC) + + .. table:: + + ================ === ======================= =================== + Signal Dir Width Description + ================ === ======================= =================== + m_axis_cc_tdata out AXIS_PCIE_DATA_WIDTH TLP data + m_axis_cc_tkeep out AXIS_PCIE_KEEP_WIDTH Byte enable + m_axis_cc_tvalid out 1 Valid + m_axis_cc_tready in 1 Ready + m_axis_cc_tlast out 1 End of frame + m_axis_cc_tuser out AXIS_PCIE_CC_USER_WIDTH Sideband data + ================ === ======================= =================== + +.. object:: s_axis_rq_seq_num + + Transmit sequence number input + + .. table:: + + ========================= === ================ =================== + Signal Dir Width Description + ========================= === ================ =================== + s_axis_rq_seq_num_0 in RQ_SEQ_NUM_WIDTH Sequence number + s_axis_rq_seq_num_valid_0 in 1 Valid + s_axis_rq_seq_num_1 in RQ_SEQ_NUM_WIDTH Sequence number + s_axis_rq_seq_num_valid_1 in 1 Valid + ========================= === ================ =================== + +.. object:: cfg_fc_ph + + Flow control + + .. table:: + + =========== === ===== =================== + Signal Dir Width Description + =========== === ===== =================== + cfg_fc_ph in 8 Posted header credits + cfg_fc_pd in 12 Posted data credits + cfg_fc_nph in 8 Non-posted header credits + cfg_fc_npd in 12 Non-posted data credits + cfg_fc_cplh in 8 Completion header credits + cfg_fc_cpld in 12 Completion data credits + cfg_fc_sel out 3 Credit select + =========== === ===== =================== + +.. object:: cfg_max_read_req + + Configuration inputs + + .. table:: + + ================ === ========= =================== + Signal Dir Width Description + ================ === ========= =================== + cfg_max_read_req in F_COUNT*3 Max read request + cfg_max_payload in F_COUNT*3 Max payload + ================ === ========= =================== + +.. object:: cfg_mgmt_addr + + Configuration interface + + .. table:: + + ======================== === ===== =================== + Signal Dir Width Description + ======================== === ===== =================== + cfg_mgmt_addr out 10 Address + cfg_mgmt_function_number out 8 Function number + cfg_mgmt_write out 1 Write enable + cfg_mgmt_write_data out 32 Write data + cfg_mgmt_byte_enable out 4 Byte enable + cfg_mgmt_read out 1 Read enable + cfg_mgmt_read_data in 32 Read data + cfg_mgmt_read_write_done in 1 Write done + ======================== === ===== =================== + +.. object:: cfg_interrupt_msi_enable + + Interrupt interface + + .. table:: + + ============================================= === ===== =================== + Signal Dir Width Description + ============================================= === ===== =================== + cfg_interrupt_msi_enable in 4 MSI enable + cfg_interrupt_msi_vf_enable in 8 VF enable + cfg_interrupt_msi_mmenable in 12 MM enable + cfg_interrupt_msi_mask_update in 1 Mask update + cfg_interrupt_msi_data in 32 Data + cfg_interrupt_msi_select out 4 Select + cfg_interrupt_msi_int out 32 Interrupt request + cfg_interrupt_msi_pending_status out 32 Pending status + cfg_interrupt_msi_pending_status_data_enable out 1 Pending status enable + cfg_interrupt_msi_pending_status_function_num out 4 Pending status function + cfg_interrupt_msi_sent in 1 MSI sent + cfg_interrupt_msi_fail in 1 MSI fail + cfg_interrupt_msi_attr out 3 MSI attr + cfg_interrupt_msi_tph_present out 1 TPH present + cfg_interrupt_msi_tph_type out 2 TPH type + cfg_interrupt_msi_tph_st_tag out 9 TPH ST tag + cfg_interrupt_msi_function_number out 4 MSI function number + ============================================= === ===== =================== + +.. object:: status_error_cor + + PCIe error outputs + + .. table:: + + ================== === ===== =================== + Signal Dir Width Description + ================== === ===== =================== + status_error_cor out 1 Correctable error + status_error_uncor out 1 Uncorrectable error + ================== === ===== =================== diff --git a/docs/source/modules/mqnic_egress.rst b/docs/source/modules/mqnic_egress.rst new file mode 100644 index 000000000..36424948b --- /dev/null +++ b/docs/source/modules/mqnic_egress.rst @@ -0,0 +1,13 @@ +.. _mod_mqnic_egress: + +================ +``mqnic_egress`` +================ + +``mqnic_egress`` implements egress processing on the transmit side. This consists of: + +1. Transmit checksum offloading + +``mqnic_egress`` integrates the following modules: + +* :ref:`mod_tx_checksum`: transmit checksum offloading diff --git a/docs/source/modules/mqnic_ingress.rst b/docs/source/modules/mqnic_ingress.rst new file mode 100644 index 000000000..ae293135e --- /dev/null +++ b/docs/source/modules/mqnic_ingress.rst @@ -0,0 +1,15 @@ +.. _mod_mqnic_ingress: + +================= +``mqnic_ingress`` +================= + +``mqnic_ingress`` implements ingress processing on the receive path. This consists of: + +1. Receive checksum offloading +2. RSS flow hashing + +``mqnic_ingress`` integrates the following modules: + +* :ref:`mod_rx_checksum`: receive checksum offloading +* :ref:`mod_rx_hash`: RSS flow hash computation diff --git a/docs/source/modules/mqnic_interface.rst b/docs/source/modules/mqnic_interface.rst new file mode 100644 index 000000000..00bf9b688 --- /dev/null +++ b/docs/source/modules/mqnic_interface.rst @@ -0,0 +1,17 @@ +.. _mod_mqnic_interface: + +=================== +``mqnic_interface`` +=================== + +``mqnic_interface`` implements one NIC interface, including the queue management logic, descriptor, completion, and event handling, transmit scheduler, and the transmit and receive datapaths. + +``mqnic_interface`` integrates the following modules: + +* :ref:`mod_queue_manager`: transmit and receive queues +* :ref:`mod_cpl_queue_manager`: transmit and receive completion queues, event queues +* :ref:`mod_desc_fetch`: descriptor fetch +* :ref:`mod_cpl_write`: completion write +* :ref:`mod_mqnic_tx_scheduler_block`: transmit scheduler +* :ref:`mod_mqnic_interface_rx`: receive datapath +* :ref:`mod_mqnic_interface_tx`: transmit datapath diff --git a/docs/source/modules/mqnic_interface_rx.rst b/docs/source/modules/mqnic_interface_rx.rst new file mode 100644 index 000000000..b34caf522 --- /dev/null +++ b/docs/source/modules/mqnic_interface_rx.rst @@ -0,0 +1,13 @@ +.. _mod_mqnic_interface_rx: + +====================== +``mqnic_interface_rx`` +====================== + +``mqnic_interface_rx`` implements the host-side receive datapath. + +``mqnic_interface_rx`` integrates the following modules: + +* :ref:`mod_rx_engine`: receive engine +* :ref:`mod_mqnic_ingress`: ingress datapath +* ``dma_client_axis_sink``: internal DMA engine diff --git a/docs/source/modules/mqnic_interface_tx.rst b/docs/source/modules/mqnic_interface_tx.rst new file mode 100644 index 000000000..75459e8a9 --- /dev/null +++ b/docs/source/modules/mqnic_interface_tx.rst @@ -0,0 +1,13 @@ +.. _mod_mqnic_interface_tx: + +====================== +``mqnic_interface_tx`` +====================== + +``mqnic_interface_tx`` implements the host-side transmit datapath. + +``mqnic_interface_tx`` integrates the following modules: + +* :ref:`mod_tx_engine`: transmit engine +* ``dma_client_axis_source``: internal DMA engine +* :ref:`mod_mqnic_egress`: egress datapath diff --git a/docs/source/modules/mqnic_l2_egress.rst b/docs/source/modules/mqnic_l2_egress.rst new file mode 100644 index 000000000..5c9e03ef7 --- /dev/null +++ b/docs/source/modules/mqnic_l2_egress.rst @@ -0,0 +1,105 @@ +.. _mod_mqnic_l2_egress: + +=================== +``mqnic_l2_egress`` +=================== + +``mqnic_l2_egress`` contains layer 2 egress processing components, and operates synchronous to the MAC TX clock. Currently, this module is a placeholder, passing through streaming data without modification. + +Parameters +========== + +.. object:: AXIS_DATA_WIDTH + + Streaming interface ``tdata`` signal width, default ``512``. + +.. object:: AXIS_KEEP_WIDTH + + Streaming interface ``tkeep`` signal width, must be set to ``AXIS_DATA_WIDTH/8``. + +.. object:: AXIS_USER_WIDTH + + Streaming interface ``tuser`` signal width, default ``1``. + +Ports +===== + +.. object:: clk + + Logic clock. + + .. table:: + + ====== === ===== ================== + Signal Dir Width Description + ====== === ===== ================== + clk in 1 Logic clock + ====== === ===== ================== + +.. object:: rst + + Logic reset, active high + + .. table:: + + ====== === ===== ================== + Signal Dir Width Description + ====== === ===== ================== + rst in 1 Logic reset, active high + ====== === ===== ================== + +.. object:: s_axis + + Streaming transmit data from host + + .. table:: + + ============= === =============== ================== + Signal Dir Width Description + ============= === =============== ================== + s_axis_tdata in AXIS_DATA_WIDTH Streaming data + s_axis_tkeep in AXIS_KEEP_WIDTH Byte enable + s_axis_tvalid in Data valid + s_axis_tready out Ready for data + s_axis_tlast in End of frame + s_axis_tuser in AXIS_USER_WIDTH Sideband data + ============= === =============== ================== + + ``s_axis_tuser`` bits + + .. table:: + + ============== ========= ============ ============= + Bit Name Width Description + ============== ========= ============ ============= + 0 bad_frame 1 Invalid frame + PTP_TS_WIDTH:1 ptp_ts PTP_TS_WIDTH PTP timestamp + ============== ========= ============ ============= + +.. object:: m_axis + + Streaming transmit data towards network + + .. table:: + + ============= === =============== ================== + Signal Dir Width Description + ============= === =============== ================== + m_axis_tdata out AXIS_DATA_WIDTH Streaming data + m_axis_tkeep out AXIS_KEEP_WIDTH Byte enable + m_axis_tvalid out Data valid + m_axis_tready in Ready for data + m_axis_tlast out End of frame + m_axis_tuser out AXIS_USER_WIDTH Sideband data + ============= === =============== ================== + + ``m_axis_tuser`` bits + + .. table:: + + ============== ========= ============ ============= + Bit Name Width Description + ============== ========= ============ ============= + 0 bad_frame 1 Invalid frame + PTP_TS_WIDTH:1 ptp_ts PTP_TS_WIDTH PTP timestamp + ============== ========= ============ ============= diff --git a/docs/source/modules/mqnic_l2_ingress.rst b/docs/source/modules/mqnic_l2_ingress.rst new file mode 100644 index 000000000..b97745371 --- /dev/null +++ b/docs/source/modules/mqnic_l2_ingress.rst @@ -0,0 +1,109 @@ +.. _mod_mqnic_l2_ingress: + +==================== +``mqnic_l2_ingress`` +==================== + +``mqnic_l2_ingress`` contains layer 2 ingress processing components, and operates synchronous to the MAC RX clock. Currently, this module is a placeholder, passing through streaming data without modification. + +Parameters +========== + +.. object:: AXIS_DATA_WIDTH + + Streaming interface ``tdata`` signal width, default ``512``. + +.. object:: AXIS_KEEP_WIDTH + + Streaming interface ``tkeep`` signal width, must be set to ``AXIS_DATA_WIDTH/8``. + +.. object:: AXIS_USER_WIDTH + + Streaming interface ``tuser`` signal width, default ``1``. + +.. object:: AXIS_USE_READY + + Use ``tready`` signal, default ``0``. If set, logic will exert backpressure with ``tready`` instead of dropping packets when RX FIFOs are full. + +Ports +===== + +.. object:: clk + + Logic clock. + + .. table:: + + ====== === ===== ================== + Signal Dir Width Description + ====== === ===== ================== + clk in 1 Logic clock + ====== === ===== ================== + +.. object:: rst + + Logic reset, active high + + .. table:: + + ====== === ===== ================== + Signal Dir Width Description + ====== === ===== ================== + rst in 1 Logic reset, active high + ====== === ===== ================== + +.. object:: s_axis + + Streaming receive data from network + + .. table:: + + ============= === =============== ================== + Signal Dir Width Description + ============= === =============== ================== + s_axis_tdata in AXIS_DATA_WIDTH Streaming data + s_axis_tkeep in AXIS_KEEP_WIDTH Byte enable + s_axis_tvalid in Data valid + s_axis_tready out Ready for data + s_axis_tlast in End of frame + s_axis_tuser in AXIS_USER_WIDTH Sideband data + ============= === =============== ================== + + ``s_axis_tuser`` bits + + .. table:: + + ============== ========= ============ ============= + Bit Name Width Description + ============== ========= ============ ============= + 0 bad_frame 1 Invalid frame + PTP_TS_WIDTH:1 ptp_ts PTP_TS_WIDTH PTP timestamp + ============== ========= ============ ============= + +.. object:: m_axis + + Streaming receive data towards host + + .. table:: + + ============= === =============== ================== + Signal Dir Width Description + ============= === =============== ================== + m_axis_tdata out AXIS_DATA_WIDTH Streaming data + m_axis_tkeep out AXIS_KEEP_WIDTH Byte enable + m_axis_tvalid out Data valid + m_axis_tready in Ready for data + m_axis_tlast out End of frame + m_axis_tuser out AXIS_USER_WIDTH Sideband data + ============= === =============== ================== + + ``m_axis_tuser`` bits + + .. table:: + + ============== ========= ============ ============= + Bit Name Width Description + ============== ========= ============ ============= + 0 bad_frame 1 Invalid frame + PTP_TS_WIDTH:1 ptp_ts PTP_TS_WIDTH PTP timestamp + ============== ========= ============ ============= diff --git a/docs/source/modules/mqnic_ptp.rst b/docs/source/modules/mqnic_ptp.rst new file mode 100644 index 000000000..2435045db --- /dev/null +++ b/docs/source/modules/mqnic_ptp.rst @@ -0,0 +1,129 @@ +.. _mod_mqnic_ptp: + +============= +``mqnic_ptp`` +============= + +``mqnic_ptp`` implements the PTP subsystem, including PTP clock and period output modules. + +``mqnic_ptp`` integrates the following modules: + +* :ref:`mod_mqnic_ptp_clock`: PTP clock (:ref:`rb_phc`) +* :ref:`mod_mqnic_ptp_perout`: PTP period output (:ref:`rb_phc_perout`) + +Parameters +========== + +.. object:: PTP_PERIOD_NS_WIDTH + + PTP period ns field width, default ``4``. + +.. object:: PTP_OFFSET_NS_WIDTH + + PTP offset ns field width, default ``32``. + +.. object:: PTP_FNS_WIDTH + + PTP fractional ns field width, default ``32``. + +.. object:: PTP_PERIOD_NS + + PTP nominal period, ns portion ``4'd4``. + +.. object:: PTP_PERIOD_FNS + + PTP nominal period, fractional ns portion ``32'd0``. + +.. object:: PTP_PEROUT_ENABLE + + Enable PTP period output module, default ``0``. + +.. object:: PTP_PEROUT_COUNT + + Number of PTP period output channels, default ``1``. + +.. object:: REG_ADDR_WIDTH + + Register interface address width, default ``7+(PTP_PEROUT_ENABLE ? $clog2((PTP_PEROUT_COUNT+1)/2) + 1 : 0)``. + +.. object:: REG_DATA_WIDTH + + Register interface data width, default ``32``. + +.. object:: REG_STRB_WIDTH + + Register interface byte enable width, must be set to ``(REG_DATA_WIDTH/8)``. + +.. object:: RB_BASE_ADDR + + Base address of control register block, default ``0``. + +.. object:: RB_NEXT_PTR + + Address of next control register block, default ``0``. + +Ports +===== + +.. object:: clk + + Logic clock. + + .. table:: + + ====== === ===== ================== + Signal Dir Width Description + ====== === ===== ================== + clk in 1 Logic clock + ====== === ===== ================== + +.. object:: rst + + Logic reset, active high + + .. table:: + + ====== === ===== ================== + Signal Dir Width Description + ====== === ===== ================== + rst in 1 Logic reset, active high + ====== === ===== ================== + +.. object:: reg + + Control register interface + + .. table:: + + =========== === =============== =================== + Signal Dir Width Description + =========== === =============== =================== + reg_wr_addr in REG_ADDR_WIDTH Write address + reg_wr_data in REG_DATA_WIDTH Write data + reg_wr_strb in REG_STRB_WIDTH Write byte enable + reg_wr_en in 1 Write enable + reg_wr_wait out 1 Write wait + reg_wr_ack out 1 Write acknowledge + reg_rd_addr in REG_ADDR_WIDTH Read address + reg_rd_en in 1 Read enable + reg_rd_data out REG_DATA_WIDTH Read data + reg_rd_wait out 1 Read wait + reg_rd_ack out 1 Read acknowledge + =========== === =============== =================== + +.. object:: ptp + + PTP signals + + .. table:: + + ================= === ================ =================== + Signal Dir Width Description + ================= === ================ =================== + ptp_pps out 1 Pulse-per-second + ptp_ts_96 out 96 PTP timestamp + ptp_ts_step out 1 PTP timestamp step + ptp_perout_locked out PTP_PEROUT_COUNT Period output channel locked + ptp_perout_error out PTP_PEROUT_COUNT Period output channel error + ptp_perout_pulse out PTP_PEROUT_COUNT Period output channel pulse + ================= === ================ =================== diff --git a/docs/source/modules/mqnic_ptp_clock.rst b/docs/source/modules/mqnic_ptp_clock.rst new file mode 100644 index 000000000..2c6211590 --- /dev/null +++ b/docs/source/modules/mqnic_ptp_clock.rst @@ -0,0 +1,122 @@ +.. _mod_mqnic_ptp_clock: + +=================== +``mqnic_ptp_clock`` +=================== + +``mqnic_ptp_clock`` implements the PTP hardware clock. It wraps ``ptp_clock`` and provides a register interface for control, see :ref:`rb_phc`. + +Parameters +========== + +.. object:: PTP_PERIOD_NS_WIDTH + + PTP period ns field width, default ``4``. + +.. object:: PTP_OFFSET_NS_WIDTH + + PTP offset ns field width, default ``32``. + +.. object:: PTP_FNS_WIDTH + + PTP fractional ns field width, default ``32``. + +.. object:: PTP_PERIOD_NS + + PTP nominal period, ns portion ``4'd4``. + +.. object:: PTP_PERIOD_FNS + + PTP nominal period, fractional ns portion ``32'd0``. + +.. object:: PTP_PEROUT_ENABLE + + Enable PTP period output module, default ``0``. + +.. object:: PTP_PEROUT_COUNT + + Number of PTP period output channels, default ``1``. + +.. object:: REG_ADDR_WIDTH + + Register interface address width, default ``7``. + +.. object:: REG_DATA_WIDTH + + Register interface data width, default ``32``. + +.. object:: REG_STRB_WIDTH + + Register interface byte enable width, must be set to ``(REG_DATA_WIDTH/8)``. + +.. object:: RB_BASE_ADDR + + Base address of control register block, default ``0``. + +.. object:: RB_NEXT_PTR + + Address of next control register block, default ``0``. + +Ports +===== + +.. object:: clk + + Logic clock. + + .. table:: + + ====== === ===== ================== + Signal Dir Width Description + ====== === ===== ================== + clk in 1 Logic clock + ====== === ===== ================== + +.. object:: rst + + Logic reset, active high + + .. table:: + + ====== === ===== ================== + Signal Dir Width Description + ====== === ===== ================== + rst in 1 Logic reset, active high + ====== === ===== ================== + +.. object:: reg + + Control register interface + + .. table:: + + =========== === =============== =================== + Signal Dir Width Description + =========== === =============== =================== + reg_wr_addr in REG_ADDR_WIDTH Write address + reg_wr_data in REG_DATA_WIDTH Write data + reg_wr_strb in REG_STRB_WIDTH Write byte enable + reg_wr_en in 1 Write enable + reg_wr_wait out 1 Write wait + reg_wr_ack out 1 Write acknowledge + reg_rd_addr in REG_ADDR_WIDTH Read address + reg_rd_en in 1 Read enable + reg_rd_data out REG_DATA_WIDTH Read data + reg_rd_wait out 1 Read wait + reg_rd_ack out 1 Read acknowledge + =========== === =============== =================== + +.. object:: ptp + + PTP signals + + .. table:: + + ================= === ===== =================== + Signal Dir Width Description + ================= === ===== =================== + ptp_pps out 1 Pulse-per-second + ptp_ts_96 out 96 PTP timestamp + ptp_ts_step out 1 PTP timestamp step + ================= === ===== =================== + diff --git a/docs/source/modules/mqnic_ptp_perout.rst b/docs/source/modules/mqnic_ptp_perout.rst new file mode 100644 index 000000000..24c5bab6d --- /dev/null +++ b/docs/source/modules/mqnic_ptp_perout.rst @@ -0,0 +1,95 @@ +.. _mod_mqnic_ptp_perout: + +==================== +``mqnic_ptp_perout`` +==================== + +``mqnic_ptp_perout`` implements the PTP period output functionality. It wraps ``ptp_perout`` and provides a register interface for control, see :ref:`rb_phc_perout`. + +Parameters +========== + +.. object:: REG_ADDR_WIDTH + + Register interface address width, default ``6``. + +.. object:: REG_DATA_WIDTH + + Register interface data width, default ``32``. + +.. object:: REG_STRB_WIDTH + + Register interface byte enable width, must be set to ``(REG_DATA_WIDTH/8)``. + +.. object:: RB_BASE_ADDR + + Base address of control register block, default ``0``. + +.. object:: RB_NEXT_PTR + + Address of next control register block, default ``0``. + +Ports +===== + +.. object:: clk + + Logic clock. + + .. table:: + + ====== === ===== ================== + Signal Dir Width Description + ====== === ===== ================== + clk in 1 Logic clock + ====== === ===== ================== + +.. object:: rst + + Logic reset, active high + + .. table:: + + ====== === ===== ================== + Signal Dir Width Description + ====== === ===== ================== + rst in 1 Logic reset, active high + ====== === ===== ================== + +.. object:: reg + + Control register interface + + .. table:: + + =========== === =============== =================== + Signal Dir Width Description + =========== === =============== =================== + reg_wr_addr in REG_ADDR_WIDTH Write address + reg_wr_data in REG_DATA_WIDTH Write data + reg_wr_strb in REG_STRB_WIDTH Write byte enable + reg_wr_en in 1 Write enable + reg_wr_wait out 1 Write wait + reg_wr_ack out 1 Write acknowledge + reg_rd_addr in REG_ADDR_WIDTH Read address + reg_rd_en in 1 Read enable + reg_rd_data out REG_DATA_WIDTH Read data + reg_rd_wait out 1 Read wait + reg_rd_ack out 1 Read acknowledge + =========== === =============== =================== + +.. object:: ptp + + PTP signals + + .. table:: + + ================= === ===== =================== + Signal Dir Width Description + ================= === ===== =================== + ptp_ts_96 in 96 PTP timestamp + ptp_ts_step in 1 PTP timestamp step + ptp_perout_locked out 1 Period output locked + ptp_perout_error out 1 Period output error + ptp_perout_pulse out 1 Period output pulse + ================= === ===== =================== diff --git a/docs/source/modules/mqnic_tx_scheduler_block.rst b/docs/source/modules/mqnic_tx_scheduler_block.rst new file mode 100644 index 000000000..826116527 --- /dev/null +++ b/docs/source/modules/mqnic_tx_scheduler_block.rst @@ -0,0 +1,242 @@ +.. _mod_mqnic_tx_scheduler_block: + +============================ +``mqnic_tx_scheduler_block`` +============================ + +``mqnic_tx_scheduler_block`` is the top-level block for the transmit scheduler. It is instantiated in :ref:`mod_mqnic_interface`. This is a pluggable module, intended to be replaced by a customized implementation via the build system. See .... for more details. + +Two variations are provided: + +* ``mqnic_tx_scheduler_block_rr``: round-robin transmit scheduler (:ref:`mod_tx_scheduler_rr`) +* ``mqnic_tx_scheduler_block_rr_tdma``: round-robin transmit scheduler (:ref:`mod_tx_scheduler_rr`) with TDMA scheduler controller + +Parameters +========== + +.. object:: PORTS + + Number of ports, default ``1``. + +.. object:: INDEX + + Scheduler index, default ``0``. + +.. object:: REG_ADDR_WIDTH + + Width of control register interface address in bits, default ``16``. + +.. object:: REG_DATA_WIDTH + + Width of control register interface data in bits, default ``32``. + +.. object:: REG_STRB_WIDTH + + Width of control register interface strb, must be set to ``(REG_DATA_WIDTH/8)``. + +.. object:: RB_BASE_ADDR + + Register block base address, default ``0``. + +.. object:: RB_NEXT_PTR + + Register block next pointer, default ``0``. + +.. object:: AXIL_DATA_WIDTH + + Width of AXI lite data bus in bits, default ``32``. + +.. object:: AXIL_ADDR_WIDTH + + Width of AXI lite address bus in bits, default ``16``. + +.. object:: AXIL_STRB_WIDTH + + Width of AXI lite wstrb (width of data bus in words), must be set to ``AXIL_DATA_WIDTH/8``. + +.. object:: AXIL_OFFSET + + Offset to AXI lite interface, default ``0``. + +.. object:: LEN_WIDTH + + Length field width, default ``16``. + +.. object:: REQ_TAG_WIDTH + + Transmit request tag field width, default ``8``. + +.. object:: OP_TABLE_SIZE + + Number of outstanding operations, default ``16``. + +.. object:: QUEUE_INDEX_WIDTH + + Queue index width, default ``6``. + +.. object:: PIPELINE + + Pipeline setting, default ``3``. + +.. object:: TDMA_INDEX_WIDTH + + Scheduler TDMA index width, default ``8``. + +.. object:: PTP_TS_WIDTH + + PTP timestamp width, default ``96``. + +.. object:: AXIS_TX_DEST_WIDTH + + AXI stream tdest signal width, default ``$clog2(PORTS)+4``. + +.. object:: MAX_TX_SIZE + + Max transmit packet size, default ``2048``. + +Ports +===== + +.. object:: clk + + Logic clock. Most interfaces are synchronous to this clock. + + .. table:: + + ====== === ===== ================== + Signal Dir Width Description + ====== === ===== ================== + clk in 1 Logic clock + ====== === ===== ================== + +.. object:: rst + + Logic reset, active high + + .. table:: + + ====== === ===== ================== + Signal Dir Width Description + ====== === ===== ================== + rst in 1 Logic reset, active high + ====== === ===== ================== + +.. object:: ctrl_reg + + Control register interface + + .. table:: + + ================ === =============== =================== + Signal Dir Width Description + ================ === =============== =================== + ctrl_reg_wr_addr in REG_ADDR_WIDTH Write address + ctrl_reg_wr_data in REG_DATA_WIDTH Write data + ctrl_reg_wr_strb in REG_STRB_WIDTH Write byte enable + ctrl_reg_wr_en in 1 Write enable + ctrl_reg_wr_wait out 1 Write wait + ctrl_reg_wr_ack out 1 Write acknowledge + ctrl_reg_rd_addr in REG_ADDR_WIDTH Read address + ctrl_reg_rd_en in 1 Read enable + ctrl_reg_rd_data out REG_DATA_WIDTH Read data + ctrl_reg_rd_wait out 1 Read wait + ctrl_reg_rd_ack out 1 Read acknowledge + ================ === =============== =================== + +.. object:: s_axil + + AXI-Lite slave interface. This interface provides access to memory-mapped per-queue control registers. + + .. table:: + + ============== === =============== =================== + Signal Dir Width Description + ============== === =============== =================== + s_axil_awaddr in AXIL_ADDR_WIDTH Write address + s_axil_awprot in 3 Write protect + s_axil_awvalid in 1 Write address valid + s_axil_awready out 1 Write address ready + s_axil_wdata in AXIL_DATA_WIDTH Write data + s_axil_wstrb in AXIL_STRB_WIDTH Write data strobe + s_axil_wvalid in 1 Write data valid + s_axil_wready out 1 Write data ready + s_axil_bresp out 2 Write response status + s_axil_bvalid out 1 Write response valid + s_axil_bready in 1 Write response ready + s_axil_araddr in AXIL_ADDR_WIDTH Read address + s_axil_arprot in 3 Read protect + s_axil_arvalid in 1 Read address valid + s_axil_arready out 1 Read address ready + s_axil_rdata out AXIL_DATA_WIDTH Read response data + s_axil_rresp out 2 Read response status + s_axil_rvalid out 1 Read response valid + s_axil_rready in 1 Read response ready + ============== === =============== =================== + +.. object:: m_axis_tx_req + + Transmit request output, for transmit requests to the transmit engine. + + .. table:: + + =================== === ================== =================== + Signal Dir Width Description + =================== === ================== =================== + m_axis_tx_req_queue out QUEUE_INDEX_WIDTH Queue index + m_axis_tx_req_tag out REQ_TAG_WIDTH Tag + m_axis_tx_req_dest out AXIS_TX_DEST_WIDTH Destination port and TC + m_axis_tx_req_valid out 1 Valid + m_axis_tx_req_ready in 1 Ready + =================== === ================== =================== + +.. object:: s_axis_tx_req_status + + Transmit request status input, for responses from the transmit engine. + + .. table:: + + ========================== === ============= =================== + Signal Dir Width Description + ========================== === ============= =================== + s_axis_tx_req_status_len in LEN_WIDTH Packet length + s_axis_tx_req_status_tag in REQ_TAG_WIDTH Tag + s_axis_tx_req_status_valid in 1 Valid + ========================== === ============= =================== + +.. object:: s_axis_doorbell + + Doorbell input, for enqueue notifications from the transmit queue manager. + + .. table:: + + ===================== === ================= =================== + Signal Dir Width Description + ===================== === ================= =================== + s_axis_doorbell_queue in QUEUE_INDEX_WIDTH Queue index + s_axis_doorbell_valid in 1 Valid + ===================== === ================= =================== + +.. object:: ptp_ts + + PTP time input from PTP clock + + .. table:: + + =========== === ============ =================== + Signal Dir Width Description + =========== === ============ =================== + ptp_ts_96 in PTP_TS_WIDTH PTP time + ptp_ts_step in 1 PTP clock step + =========== === ============ =================== + +.. object:: config + + Configuration signals + + .. table:: + + =========== === ============ =================== + Signal Dir Width Description + =========== === ============ =================== + mtu in LEN_WIDTH MTU + =========== === ============ =================== diff --git a/docs/source/modules/overview.rst b/docs/source/modules/overview.rst new file mode 100644 index 000000000..9a347900b --- /dev/null +++ b/docs/source/modules/overview.rst @@ -0,0 +1,29 @@ +.. _modules_overview: + +======== +Overview +======== + +Corundum has several unique architectural features. First, hardware queue states are stored efficiently in FPGA block RAM, enabling support for thousands of individually-controllable queues. These queues are associated with interfaces, and each interface can have multiple ports, each with its own independent transmit scheduler. This enables extremely fine-grained control over packet transmission. The scheduler module is designed to be modified or swapped out completely to implement different transmit scheduling schemes, including experimental schedulers. Coupled with PTP time synchronization, this enables time-based scheduling, including high precision TDMA. + +The design of Corundum is modular and highly parametrized. Many configuration and structural options can be set at synthesis time by Verilog parameters, including interface and port counts, queue counts, memory sizes, etc. These design parameters are exposed in configuration registers that the driver reads to determine the NIC configuration, enabling the same driver to support many different boards and configurations without modification. + +High-level overview +=================== + +.. _fig_overview_block: +.. figure:: /diagrams/svg/corundum_block.svg + + Block diagram of the Corundum NIC. PCIe HIP: PCIe hard IP core; AXIL M: AXI lite master; DMA IF: DMA interface; AXI M: AXI master; PHC: PTP hardware clock; TXQ: transmit queue manager; TXCQ: transmit completion queue manager; RXQ: receive queue manager; RXCQ: receive completion queue manager; EQ: event queue manager; MAC + PHY: Ethernet media access controller (MAC) and physical interface layer (PHY). + +A block diagram of the Corundum NIC is shown in :numref:`fig_overview_block`. At a high level, the NIC consists of several hierarchy levels. The top-level module primarily contains support and interfacing components. These components include the PCI express hard IP core and Ethernet interface components including MACs, PHYs, and associated serializers, along with an instance of an appropriate :ref:`mod_mqnic_core` wrapper, which provides the DMA interface. This core module contains the PTP clock (:ref:`mod_mqnic_ptp`), application section (:ref:`mod_mqnic_app_block`), and one or more :ref:`mod_mqnic_interface` module instances. Each interface module corresponds to an operating-system-level network interface (e.g. ``eth0``), and contains the queue management logic, descriptor and completion handling logic, transmit schedulers, transmit and receive engines, transmit and receive datapaths, and a scratchpad RAM for temporarily storing incoming and outgoing packets during DMA operations. The queue management logic maintains the queue state for all of the NIC queues---transmit, transmit completion, receive, receive completion, and event queues. + +For each interface, the transmit scheduler (:ref:`mod_mqnic_tx_scheduler_block`) in the interface module decides which queues are designated for transmission. The transmit scheduler generates commands for the transmit engine, which coordinates operations on the transmit datapath. The scheduler module is a flexible functional block that can be modified or replaced to support arbitrary schedules, which may be event driven. The default implementation of the scheduler in :ref:`mod_tx_scheduler_rr` is simple round robin. All ports associated with the same interface module share the same set of transmit queues and appear as a single, unified interface to the operating system. This enables flows to be migrated between ports or load-balanced across multiple ports by changing only the transmit scheduler settings without affecting the rest of the network stack. This dynamic, scheduler-defined mapping of queues to ports is a unique feature of Corundum that can enable research into new protocols and network architectures, including parallel networks and optically-switched networks. + +In the receive direction, incoming packets pass through a flow hash module to determine the target receive queue and generate commands for the receive engine, which coordinates operations on the receive datapath. Because all ports in the same interface module share the same set of receive queues, incoming flows on different ports are merged together into the same set of queues. + +An application block (:ref:`mod_mqnic_app_block`) is provided for customization, including packet processing, routing, and in-network compute applications. The application block has connections to several different subsystems. + +The components on the NIC are interconnected with several different interfaces including AXI lite, AXI stream, and a custom segmented memory interface for DMA operations. AXI lite is used for the control path from the driver to the NIC. It is used to initialize and configure the NIC components and to control the queue pointers during transmit and receive operations. AXI stream interfaces are used for transferring packetized data within the NIC. The segmented memory interface serves to connect the PCIe DMA interface to the NIC datapath and to the descriptor and completion handling logic. + +The majority of the NIC logic runs in the PCIe user clock domain, which is nominally 250 MHz for all of the current design variants. Asynchronous FIFOs are used to interface with the MACs, which run in the serializer transmit and receive clock domains as appropriate---156.25 MHz for 10G, 390.625 MHz for 25G, and 322.265625 MHz for 100G. diff --git a/docs/source/modules/queue_manager.rst b/docs/source/modules/queue_manager.rst new file mode 100644 index 000000000..7595c60b1 --- /dev/null +++ b/docs/source/modules/queue_manager.rst @@ -0,0 +1,40 @@ +.. _mod_queue_manager: + +================= +``queue_manager`` +================= + +``queue_manager`` implements the queue management logic for the transmit and receive queues. It stores host to device queue state in block RAM or ultra RAM. + +Operation +========= + +Communication of packet data between the Corundum NIC and the driver is mediated via descriptor and completion queues. Descriptor queues form the host-to-NIC communications channel, carrying information about where individual packets are stored in system memory. Completion queues form the NIC-to-host communications channel, carrying information about completed operations and associated metadata. The descriptor and completion queues are implemented as ring buffers that reside in DMA-accessible system memory, while the NIC hardware maintains the necessary queue state information. This state information consists of a pointer to the DMA address of the ring buffer, the size of the ring buffer, the producer and consumer pointers, and a reference to the associated completion queue. The required state for each queue fits into 128 bits. + +The queue management logic for the Corundum NIC must be able to efficiently store and manage the state for thousands of queues. This means that the queue state must be completely stored in block RAM (BRAM) or ultra RAM (URAM) on the FPGA. Since a 128 bit RAM is required and URAM blocks are 72x4096, storing the state for 4096 queues requires only 2 URAM instances. Utilizing URAM instances enables scaling the queue management logic to handle at least 32,768 queues per interface. + +In order to support high throughput, the NIC must be able to process multiple descriptors in parallel. Therefore, the queue management logic must track multiple in-progress operations, reporting updated queue pointers to the driver as the operations are completed. The state required to track in-process operations is much smaller than the state required to describe the queue state itself. Therefore the in-process operation state is stored in flip-flops and distributed RAM. + +The NIC design uses two queue manager modules: ``queue_manager`` is used to manage host-to-NIC descriptor queues, while ``cpl_queue_manager`` is used to manage NIC-to-host completion queues. The modules are similar except for a few minor differences in terms of pointer handling, fill handling, and doorbell/event generation. Because of the similarities, this section will discuss only the operation of the ``queue_manager`` module. + +The BRAM or URAM array used to store the queue state information requires several cycles of latency for each read operation, so the ``queue_manager`` is built with a pipelined architecture to facilitate multiple concurrent operations. The pipeline supports four different operations: register read, register write, dequeue/enqueue request, and dequeue/enqueue commit. Register-access operations over an AXI lite interface enable the driver to initialize the queue state and provide pointers to the allocated host memory as well as access the producer and consumer pointers during normal operation. + +.. _fig_queue_manager_block: +.. figure:: /diagrams/svg/corundum_queue_manager_block.svg + + Block diagram of the queue manager module, showing the queue state RAM and operation table. Ind = index, Addr = DMA address, Op = index in operation table, Act = active, LS = log base 2 of queue size, Cpl = completion queue index, Tail = tail or consumer pointer, Head = head or producer pointer, Com = committed; QI = queue index; Ptr = new queue pointer + +A block diagram of the queue manager module is shown in :numref:`fig_queue_manager_block`. The BRAM or URAM array used to store the queue state information requires several cycles of latency for each read operation, so the ``queue_manager`` is built with a pipelined architecture to facilitate multiple concurrent operations. The pipeline supports four different operations: register read, register write, dequeue/enqueue request, and dequeue/enqueue commit. Register-access operations over an AXI lite interface enable the driver to initialize the queue state and provide pointers to the allocated host memory as well as access the producer and consumer pointers during normal operation. + +.. _fig_queue_pointers: +.. figure:: /diagrams/svg/queue_pointers.svg + + Queue pointers on software ring buffers. + +Each queue has three pointers associated with it, as shown in :numref:`fig_queue_pointers`---the producer pointer, the host-facing consumer pointer, and the shadow consumer pointer. The driver has control over the producer pointer and can read the host-facing consumer pointer. Entries between the consumer pointer and the producer pointer are under the control of the NIC and must not be modified by the driver. The driver enqueues a descriptor by writing it into the ring buffer at the index indicated by the producer pointer, issuing a memory barrier, then incrementing the producer pointer in the queue manager. The NIC dequeues descriptors by reading them out of the descriptor ring via DMA and incrementing the consumer pointer. The host-facing consumer pointer must not be incremented until the descriptor read operation completes, so the queue manager maintains an internal shadow consumer pointer to keep track of read operations that have started in addition to the host-facing pointer that is updated as the read operations are completed. + +The dequeue request operation on the queue manager pipeline initiates a dequeue operation on a queue. If the target queue is disabled or empty, the operation is rejected with an *empty* or *error* status. Otherwise, the shadow consumer pointer is incremented and the physical address of the queue element is returned, along with the queue element index and an operation tag. Operations on any combination of queues can be initiated until the operation table is full. The dequeue request input is stalled when the table is full. As the read operations complete, the dequeue operations are committed to free the operation table entry and update the host-facing consumer pointer. Operations can be committed in any order, simply setting the commit flag in the operation table, but the operation table entries will be freed and host-facing consumer pointer will be updated in-order to ensure descriptors being processed are not modified by the driver. + +The operation table tracks in-process queue operations that have yet to be committed. Entries in the table consist of an active flag, a commit flag, the queue index, and the index of the next element in the queue. The queue state also contains a pointer to the most recent entry for that queue in the operation table. During an enqueue operation, the operation table is checked to see if there are any outstanding operations on that queue. If so, the consumer pointer for the most recent operation is incremented and stored in the new operation table entry. Otherwise, the current consumer pointer is incremented. When a dequeue commit request is received, the commit bit is set for the corresponding entry. The entries are then committed in-order, updating the host-facing consumer pointer with the pointer from the operation table and clearing the active bit in the operation table entry. + +Both the queue manager and completion queue manager modules generate notifications during enqueue operations. In a queue manager, when the driver updates a producer pointer on an enabled queue, the module issues a doorbell event that is passed to the transmit schedulers for the associated ports. Similarly, completion queue managers generate events on hardware enqueue operations, which are passed to the event subsystem and ultimately generate interrupts. To reduce the number of events and interrupts, completion queues also have an *armed* status. An armed completion queue will generate a single event, disarming itself in the process. The driver must re-arm the queue after handling the event. diff --git a/docs/source/modules/rx_checksum.rst b/docs/source/modules/rx_checksum.rst new file mode 100644 index 000000000..167bffc69 --- /dev/null +++ b/docs/source/modules/rx_checksum.rst @@ -0,0 +1,7 @@ +.. _mod_rx_checksum: + +=============== +``rx_checksum`` +=============== + +``rx_checksum`` implements the receive checksum offloading support. It computes 16 bit checksum of Ethernet frame payload to aid in IP checksum offloading by the host network stack. diff --git a/docs/source/modules/rx_engine.rst b/docs/source/modules/rx_engine.rst new file mode 100644 index 000000000..158be2f08 --- /dev/null +++ b/docs/source/modules/rx_engine.rst @@ -0,0 +1,7 @@ +.. _mod_rx_engine: + +============= +``rx_engine`` +============= + +``rx_engine`` manages receive datapath operations including descriptor dequeue and fetch via DMA, packet reception, data writeback via DMA, and completion enqueue and writeback via DMA. It also handles PTP timestamps for inclusion in completion records. diff --git a/docs/source/modules/rx_hash.rst b/docs/source/modules/rx_hash.rst new file mode 100644 index 000000000..b98605315 --- /dev/null +++ b/docs/source/modules/rx_hash.rst @@ -0,0 +1,7 @@ +.. _mod_rx_hash: + +=========== +``rx_hash`` +=========== + +``rx_hash`` implements flow hashing on the receive path. It extracts IP addresses and ports from packet headers and computes a 32-bit Toeplitz flow hash. diff --git a/docs/source/modules/tx_checksum.rst b/docs/source/modules/tx_checksum.rst new file mode 100644 index 000000000..d861a439f --- /dev/null +++ b/docs/source/modules/tx_checksum.rst @@ -0,0 +1,7 @@ +.. _mod_tx_checksum: + +=============== +``tx_checksum`` +=============== + +``tx_checksum`` implements the transmit checksum offloading support. It computes 16 bit checksum of frame data with specified start offset, then inserts computed checksum at the position specified by the host network stack. diff --git a/docs/source/modules/tx_engine.rst b/docs/source/modules/tx_engine.rst new file mode 100644 index 000000000..6e0666d07 --- /dev/null +++ b/docs/source/modules/tx_engine.rst @@ -0,0 +1,7 @@ +.. _mod_tx_engine: + +============= +``tx_engine`` +============= + +``tx_engine`` manages transmit datapath operations including descriptor dequeue and fetch via DMA, packet data fetch via DMA, packet transmission, and completion enqueue and writeback via DMA. It also handles PTP timestamps for inclusion in completion records. diff --git a/docs/source/modules/tx_scheduler_rr.rst b/docs/source/modules/tx_scheduler_rr.rst new file mode 100644 index 000000000..69c833cc3 --- /dev/null +++ b/docs/source/modules/tx_scheduler_rr.rst @@ -0,0 +1,196 @@ +.. _mod_tx_scheduler_rr: + +=================== +``tx_scheduler_rr`` +=================== + +Operation +========= + +The default transmit scheduler used in the Corundum NIC is a simple round-robin scheduler implemented in the ``tx_scheduler_rr`` module. The scheduler sends commands to the transmit engine to initiate transmit operations out of the NIC transmit queues. The round-robin scheduler contains basic queue state for all queues, a FIFO to store currently-active queues and enforce the round-robin schedule, and an operation table to track in-process transmit operations. + +Similar to the queue management logic, the round-robin transmit scheduler also stores queue state information in BRAM or URAM on the FPGA so that it can scale to support a large number of queues. The transmit scheduler also uses a processing pipeline to hide the memory access latency. + +The transmit scheduler module has four main interfaces: an AXI lite register interface and three streaming interfaces. The AXI lite interface permits the driver to change scheduler parameters and enable/disable queues. The first streaming interface provides doorbell events from the queue management logic when the driver enqueues packets for transmission. The second streaming interface carries transmit commands generated by the scheduler to the transmit engine. Each command consists of a queue index to transmit from, along with a tag for tracking in-process operations. The final streaming interface returns transmit operation status information back to the scheduler. The status information informs the scheduler of the length of the transmitted packet, or if the transmit operation failed due to an empty or disabled queue. + +The transmit scheduler module can be extended or replaced to implement arbitrary scheduling algorithms. This enables Corundum to be used as a platform to evaluate experimental scheduling algorithms. It is also possible to provide additional inputs to the transmit scheduler module, including feedback from the receive path, which can be used to implement new protocols and congestion control techniques. Connecting the scheduler to the PTP hardware clock can be used to support TDMA, which can be used to implement circuit-switched architectures. + +The structure of the transmit scheduler logic is similar to the queue management logic in that it stores queue state in BRAM or URAM and uses a processing pipeline. However there are a number of significant differences. First, the scheduler logic is designed so that the scheduler does not stall when a queue is empty and a subsequent dequeue operation fails. Second, the scheduler contains a FIFO to enforce the round-robin schedule. The use of this FIFO requires an explicit reset routine to make the internal state (namely the scheduled flag bits) consistent after a reset. Third, the scheduler also contains logic to track the active state of each queue based on incoming doorbell requests and dequeue failures. + +.. _fig_tx_scheduler_block: +.. figure:: /diagrams/svg/corundum_tx_scheduler_block.svg + + Block diagram of the transmit scheduler module, showing queue state RAM and operation table. Ind = index, En = queue enable, GE = global enable, SE = schedule enable, Act = active, Sch = scheduled, QI = queue index, DB = doorbell, H = head, N = next, P = previous + +A block diagram of the transmit scheduler module is shown in :numref:`fig_tx_scheduler_block`. The transmit scheduler is built around a *scheduled queue* FIFO. This FIFO stores the indices of the currently-scheduled queues. An active queue is one that is presumed to have at least one packet available for transmission, an enabled queue is one that has been enabled for transmission, and a scheduled queue is one that has an entry in the scheduler FIFO. A queue will be scheduled (marked as scheduled and inserted into the FIFO) if it is both active and enabled. A queue will be descheduled when it reaches the front of the schedule FIFO, but is not enabled or not active. Queue enable states are controlled via three different enable bits per queue: queue enable, global enable, and schedule enable. The queue enable and global enable bits are writable via AXI lite, while the schedule enable bit is controlled from the scheduler control module via an internal interface. A queue is enabled when the queue enable bit and either the global enable or schedule enable bits are set. Queues become active when doorbell events are received, and queues become inactive when a transmit request fails due to an empty queue. + +Tracking the queue active states must be done carefully for several reasons. First, the driver can update the producer pointer after enqueuing more than one packet, so the number of generated doorbell events does not necessarily correspond to the number of packets that were enqueued. Second, because the queues are shared among all ports on the same interface, multiple ports can attempt to send packets from the same queue, and the port transmit schedulers have no visibility into what the other schedulers are doing. Therefore, the most reliable method for determining that a queue is empty is to try sending from it, and flagging the failure. Note that the cost of an error is much higher when the queue is active than when the queue is empty. Attempting to send from an empty queue costs a few clock cycles and temporarily occupies a few slots in corresponding operation tables. However, assuming a queue is empty when it is not will result in packets getting stuck in the queue. Fixing this stuck queue will not occur until the OS sends another packet on that queue and triggers another doorbell. Therefore, it is imperative to properly track doorbell events during transmit operations, as it is possible for a doorbell event to arrive after a dequeue attempt has failed, but before the failed transmit status arrives at the transmit scheduler module. + +The pipeline in the transmit scheduler supports seven different operations: initialize, register read, register write, handle doorbell, transmit complete, scheduler control, and transmit request. The initialize operation is used to ensure the scheduler state is consistent after a reset. Register access operations over an AXI lite interface enable the driver to read all of the per-queue state and set the queue enable and global enable bits. The pipeline also handles incoming doorbell requests from the transmit queue manager module as well as queue enable/disable requests from the scheduler control module. Finally, the transmit request and transmit complete operations are used to generate transmit requests and handle the necessary queue state updates when the transmit operations complete. + +Queues can become scheduled based on a register write that enables an active queue, a doorbell that activates an enabled queue, a scheduler operation that enables an active queue, and a transmit completion on an enabled queue that is either successful or has the doorbell bit set in the operation table. Queues can only be descheduled when the queue index advances to the front of the scheduler FIFO. If this occurs when the queue is both active and enabled, then the queue can be rescheduled and a transmit request generated. When the transmit operation completes, the transmit status response will be temporarily stored in a small FIFO and then processed by the pipeline to update the corresponding operation table entry and, if necessary, reschedule the queue. + +The operation table tracks in-process transmit operations. Entries in the table consist of an active flag, the queue index, a doorbell flag, a head flag, a next pointer, and a previous pointer. The next and previous pointers form a linked list, enabling entries to be removed in any order while preserving the doorbell flag in the table. This prevents doorbells from getting 'lost' and the queue being mistakenly marked as inactive. A separate linked list is formed for each queue with active transmit operations. The operation table is implemented in such a way that it fits in distributed RAM. + +Parameters +========== + +.. object:: AXIL_DATA_WIDTH + + Width of AXI lite data bus in bits, default ``32``. + +.. object:: AXIL_ADDR_WIDTH + + Width of AXI lite address bus in bits, default ``16``. + +.. object:: AXIL_STRB_WIDTH + + Width of AXI lite wstrb (width of data bus in words), must be set to ``AXIL_DATA_WIDTH/8``. + +.. object:: LEN_WIDTH + + Length field width, default ``16``. + +.. object:: REQ_TAG_WIDTH + + Transmit request tag field width, default ``8``. + +.. object:: OP_TABLE_SIZE + + Number of outstanding operations, default ``16``. + +.. object:: QUEUE_INDEX_WIDTH + + Queue index width, default ``6``. + +.. object:: PIPELINE + + Pipeline setting, default ``3``. + +Ports +===== + +.. object:: clk + + Logic clock. Most interfaces are synchronous to this clock. + + .. table:: + + ====== === ===== ================== + Signal Dir Width Description + ====== === ===== ================== + clk in 1 Logic clock + ====== === ===== ================== + +.. object:: rst + + Logic reset, active high + + .. table:: + + ====== === ===== ================== + Signal Dir Width Description + ====== === ===== ================== + rst in 1 Logic reset, active high + ====== === ===== ================== + +.. object:: m_axis_tx_req + + Transmit request output, for transmit requests to the transmit engine. + + .. table:: + + =================== === ================== =================== + Signal Dir Width Description + =================== === ================== =================== + m_axis_tx_req_queue out QUEUE_INDEX_WIDTH Queue index + m_axis_tx_req_tag out REQ_TAG_WIDTH Tag + m_axis_tx_req_dest out AXIS_TX_DEST_WIDTH Destination port and TC + m_axis_tx_req_valid out 1 Valid + m_axis_tx_req_ready in 1 Ready + =================== === ================== =================== + +.. object:: s_axis_tx_req_status + + Transmit request status input, for responses from the transmit engine. + + .. table:: + + ========================== === ============= =================== + Signal Dir Width Description + ========================== === ============= =================== + s_axis_tx_req_status_len in LEN_WIDTH Packet length + s_axis_tx_req_status_tag in REQ_TAG_WIDTH Tag + s_axis_tx_req_status_valid in 1 Valid + ========================== === ============= =================== + +.. object:: s_axis_doorbell + + Doorbell input, for enqueue notifications from the transmit queue manager. + + .. table:: + + ===================== === ================= =================== + Signal Dir Width Description + ===================== === ================= =================== + s_axis_doorbell_queue in QUEUE_INDEX_WIDTH Queue index + s_axis_doorbell_valid in 1 Valid + ===================== === ================= =================== + +.. object:: s_axis_sched_ctrl + + Scheduler control input, to permit user logic to dynamically enable/disable queues. + + .. table:: + + ======================== === ================= =================== + Signal Dir Width Description + ======================== === ================= =================== + s_axis_sched_ctrl_queue in QUEUE_INDEX_WIDTH Queue index + s_axis_sched_ctrl_enable in 1 Queue enable + s_axis_sched_ctrl_valid in 1 Valid + s_axis_sched_ctrl_ready out 1 Ready + ======================== === ================= =================== + +.. object:: s_axil + + AXI-Lite slave interface. This interface provides access to memory-mapped per-queue control registers. + + .. table:: + + ============== === =============== =================== + Signal Dir Width Description + ============== === =============== =================== + s_axil_awaddr in AXIL_ADDR_WIDTH Write address + s_axil_awprot in 3 Write protect + s_axil_awvalid in 1 Write address valid + s_axil_awready out 1 Write address ready + s_axil_wdata in AXIL_DATA_WIDTH Write data + s_axil_wstrb in AXIL_STRB_WIDTH Write data strobe + s_axil_wvalid in 1 Write data valid + s_axil_wready out 1 Write data ready + s_axil_bresp out 2 Write response status + s_axil_bvalid out 1 Write response valid + s_axil_bready in 1 Write response ready + s_axil_araddr in AXIL_ADDR_WIDTH Read address + s_axil_arprot in 3 Read protect + s_axil_arvalid in 1 Read address valid + s_axil_arready out 1 Read address ready + s_axil_rdata out AXIL_DATA_WIDTH Read response data + s_axil_rresp out 2 Read response status + s_axil_rvalid out 1 Read response valid + s_axil_rready in 1 Read response ready + ============== === =============== =================== + +.. object:: control + + Control and status signals + + .. table:: + + =========== === ============ =================== + Signal Dir Width Description + =========== === ============ =================== + enable in enable Enable + active out enable Active + =========== === ============ =================== diff --git a/docs/source/operations.rst b/docs/source/operations.rst new file mode 100644 index 000000000..bdadddac5 --- /dev/null +++ b/docs/source/operations.rst @@ -0,0 +1,246 @@ +.. _operations: + +========== +Operations +========== + +This is a list of all of the operations involved in sending and receiving packets, across both hardware and software. + +Packet transmission +=================== + +#. linux: The linux kernel calls ``mqnic_start_xmit()`` (via ``ndo_start_xmit()``) with an ``sk_buff`` for transmission +#. ``mqnic_start_xmit()`` (``mqnic_tx.c``): The driver determines the destination transmit queue with ``skb_get_queue_mapping`` +#. ``mqnic_start_xmit()`` (``mqnic_tx.c``): The driver marks the ``sk_buff`` for timestamping, if requested +#. ``mqnic_start_xmit()`` (``mqnic_tx.c``): The driver generates the hardware IP checksum command and writes it into the descriptor +#. ``mqnic_map_skb()`` (``mqnic_tx.c``): The driver writes a reference to the ``sk_buff`` into ``ring->tx_info`` +#. ``mqnic_map_skb()`` (``mqnic_tx.c``): The driver generates DMA mappings for the ``sk_buff`` (``skb_frag_dma_map()``/``dma_map_single()``) and builds the descriptor +#. ``mqnic_start_xmit()`` (``mqnic_tx.c``): The driver enqueues the packet by incrementing its local copy of the producer pointer +#. ``mqnic_start_xmit()`` (``mqnic_tx.c``): At the end of a batch of packets, the driver writes the updated producer pointer to the NIC via MMIO +#. :ref:`mod_queue_manager` ``s_axil_*``: The MMIO write arrives at the queue manager via AXI lite +#. :ref:`mod_queue_manager` ``m_axis_doorbell_*``: The queue manager updates the producer pointer and generates a doorbell event +#. :ref:`mod_tx_scheduler_rr` ``s_axis_doorbell_*``: The doorbell event arrives at the port schedulers +#. :ref:`mod_tx_scheduler_rr`: The scheduler marks the queue as active and schedules it if necessary +#. :ref:`mod_tx_scheduler_rr`: The scheduler decides to send a packet +#. :ref:`mod_tx_scheduler_rr` ``m_axis_tx_req_*``: The scheduler generates a transmit request +#. :ref:`mod_tx_engine` ``s_axis_tx_req_*``: The transmit request arrives at the transmit engine +#. :ref:`mod_tx_engine` ``m_axis_desc_req_*``: The transmit engine issues a descriptor request +#. :ref:`mod_desc_fetch` ``s_axis_desc_req_*``: The descriptor request arrives at the descriptor fetch module +#. :ref:`mod_desc_fetch` ``m_axis_desc_dequeue_req_*``: The descriptor fetch module issues a dequeue request to the queue manager +#. :ref:`mod_queue_manager` ``s_axis_dequeue_req_*``: The dequeue request arrives at the queue manager module +#. :ref:`mod_queue_manager`: If the queue is not empty, the queue manager starts a dequeue operation on the queue +#. :ref:`mod_queue_manager` ``m_axis_dequeue_resp_*``: The queue manager sends a response containing the operation status and DMA address +#. :ref:`mod_desc_fetch` ``s_axis_desc_dequeue_resp_*``: The response arrives at the descriptor fetch module +#. :ref:`mod_desc_fetch` ``m_axis_req_status_*``: The descriptor module reports the descriptor fetch status +#. :ref:`mod_desc_fetch` ``m_axis_dma_read_desc_*``: The descriptor module issues a DMA read request +#. ``dma_if_pcie_rd`` ``s_axis_read_desc_*``: The requst arrives at the DMA read interface +#. ``dma_if_pcie_rd``: The DMA read interface issues a PCIe read request +#. ``dma_if_pcie_rd``: The read data comes back in a completion packet and is written to the descriptor fetch local DMA RAM +#. ``dma_if_pcie_rd`` ``m_axis_read_desc_status_*``: The DMA read interface issues a status message +#. :ref:`mod_desc_fetch` ``m_axis_desc_dequeue_commit_*``: The descriptor fetch module issues a dequeue commit message +#. :ref:`mod_queue_manager`: The queue manager commits the dequeue operation and updates the consumer pointer +#. :ref:`mod_desc_fetch` ``dma_read_desc_*``: The descriptor fetch module issues a read request to its internal DMA module +#. :ref:`mod_desc_fetch` ``m_axis_desc_*``: The internal DMA module reads the descriptor and transfers it via AXI stream +#. :ref:`mod_tx_engine`: The descriptor arrives at the transmit engine +#. :ref:`mod_tx_engine`: The transmit engine stores the descriptor data +#. :ref:`mod_tx_engine` ``m_axis_dma_read_desc_*``: The transmit engine issues a DMA read request +#. ``dma_if_pcie_rd`` ``s_axis_read_desc_*``: The requst arrives at the DMA read interface +#. ``dma_if_pcie_rd``: The DMA read interface issues a PCIe read request +#. ``dma_if_pcie_rd``: The read data comes back in a completion packet and is written to the interface local DMA RAM +#. ``dma_if_pcie_rd`` ``m_axis_read_desc_status_*``: The DMA read interface issues a status message +#. :ref:`mod_tx_engine` ``m_axis_tx_desc_*``: The transmit engine issues a read request to the interface DMA engine +#. :ref:`mod_tx_engine` ``m_axis_tx_csum_cmd_*``: The transmit engine issues a transmit checksum command +#. :ref:`mod_mqnic_interface_tx` ``tx_axis_*``: The interface DMA module reads the packet data from interface local DMA RAM and transfers it via AXI stream +#. :ref:`mod_mqnic_egress`: egress processing +#. :ref:`mod_tx_checksum`: The transmit checksum module computes and inserts the checksum +#. :ref:`mod_mqnic_app_block` ``s_axis_if_tx``: data is presented to the application section +#. :ref:`mod_mqnic_app_block` ``m_axis_if_tx``: data is returned from the application section +#. :ref:`mod_mqnic_core`: Data passes enters per-interface transmit FIFO module and is divided into per-port, per-traffic-class FIFOs +#. :ref:`mod_mqnic_app_block` ``s_axis_sync_tx``: data is presented to the application section +#. :ref:`mod_mqnic_app_block` ``m_axis_sync_tx``: data is returned from the application section +#. :ref:`mod_mqnic_core`: Data passes through per-port transmit async FIFO module and is transferred to MAC TX clock domain +#. :ref:`mod_mqnic_app_block` ``s_axis_sync_tx``: data is presented to the application section +#. :ref:`mod_mqnic_app_block` ``m_axis_sync_tx``: data is returned from the application section +#. :ref:`mod_mqnic_l2_egress`: layer 2 egress processing +#. :ref:`mod_mqnic_core`: data leaves through transmit streaming interfaces +#. The packet arrives at the MAC +#. The MAC produces a PTP timestamp +#. :ref:`mod_tx_engine`: The PTP timestamp arrives at the transmit engine +#. :ref:`mod_tx_engine` ``m_axis_cpl_req_*``: The transmit engine issues a completion write request +#. :ref:`mod_cpl_write`: The completion write module writes the completion data into its local DMA RAM +#. :ref:`mod_cpl_write` ``m_axis_cpl_enqueue_req_*``: The completion write module issues an enqueue request to the completion queue manager +#. :ref:`mod_cpl_queue_manager` ``m_axis_enqueue_req_*``: The enqueue request arrives at the completion queue manager module +#. :ref:`mod_cpl_queue_manager`: If the queue is not full, the queue manager starts an enqueue operation on the queue +#. :ref:`mod_cpl_queue_manager` ``m_axis_enqueue_resp_*``: The completion queue manager sends a response containing the operation status and DMA address +#. :ref:`mod_cpl_write`: The response arrives at the completion write module +#. :ref:`mod_cpl_write` ``m_axis_req_status_*``: The completion write module reports the completion write status +#. :ref:`mod_desc_fetch` ``m_axis_dma_write_desc_*``: The completion write module issues a DMA write request +#. ``dma_if_pcie_wr`` ``s_axis_write_desc_*``: The requst arrives at the DMA write interface +#. ``dma_if_pcie_wr``: The DMA write interface reads the completion data from the completion write module local DMA RAM +#. ``dma_if_pcie_wr``: The DMA write interface issues a PCIe write request +#. ``dma_if_pcie_wr`` ``m_axis_write_desc_status_*``: The DMA write interface issues a status message +#. :ref:`mod_cpl_write` ``m_axis_desc_enqueue_commit_*``: The completion write module issues an enqueue commit message +#. :ref:`mod_cpl_queue_manager`: The completion queue manager commits the enqueue operation and updates the producer pointer +#. :ref:`mod_cpl_queue_manager` ``m_axis_event_*``: The completion queue manager issues an event, if armed +#. :ref:`mod_cpl_write`: The event arrives at the completion write module +#. :ref:`mod_cpl_write`: The completion write module writes the event data into its local DMA RAM +#. :ref:`mod_cpl_write` ``m_axis_cpl_enqueue_req_*``: The completion write module issues an enqueue request to the completion queue manager +#. :ref:`mod_cpl_queue_manager` ``s_axis_enqueue_req_*``: The enqueue request arrives at the completion queue manager module +#. :ref:`mod_cpl_queue_manager`: If the queue is not full, the queue manager starts an enqueue operation on the queue +#. :ref:`mod_cpl_queue_manager` ``m_axis_enqueue_resp_*``: The completion queue manager sends a response containing the operation status and DMA address +#. :ref:`mod_cpl_write` ``s_axis_cpl_enqueue_resp_*``: The response arrives at the completion write module +#. :ref:`mod_cpl_write` ``m_axis_req_status_*``: The completion write module reports the completion write status +#. :ref:`mod_desc_fetch` ``m_axis_dma_write_desc_*``: The completion write module issues a DMA write request +#. ``dma_if_pcie_wr`` ``s_axis_write_desc_*``: The requst arrives at the DMA write interface +#. ``dma_if_pcie_wr``: The DMA write interface reads the event data from the completion write module local DMA RAM +#. ``dma_if_pcie_wr``: The DMA write interface issues a PCIe write request +#. ``dma_if_pcie_wr`` ``m_axis_write_desc_status_*``: The DMA write interface issues a status message +#. :ref:`mod_cpl_write` ``m_axis_desc_enqueue_commit_*``: The completion write module issues an enqueue commit message +#. :ref:`mod_cpl_queue_manager`: The completion queue manager commits the enqueue operation and updates the producer pointer +#. :ref:`mod_cpl_queue_manager` ``m_axis_event_*``: The completion queue manager issues an interrupt, if armed +#. linux: The linux kernel calls ``mqnic_irq_handler()`` +#. ``mqnic_irq_handler()`` (``mqnic_irq.c``): The driver calls the EQ handler via the notifier chain (``atomic_notifier_call_chain()``) +#. ``mqnic_eq_int()`` (``mqnic_eq.c``): The driver calls ``mqnic_process_eq()`` +#. ``mqnic_process_eq()`` (``mqnic_eq.c``): The driver processes the event queue, which calls the appropriate handler (``mqnic_tx_irq()``) +#. ``mqnic_tx_irq()`` (``mqnic_tx.c``): The driver enables NAPI polling on the queue (``napi_schedule_irqoff()``) +#. ``mqnic_eq_int()`` (``mqnic_eq.c``): The driver rearms the EQ (``mqnic_arm_eq()``) +#. NAPI: The linux kernel calls ``mqnic_poll_tx_cq()`` +#. ``mqnic_poll_tx_cq()`` (``mqnic_tx.c``): The driver calls ``mqnic_process_tx_cq()`` +#. ``mqnic_process_tx_cq()`` (``mqnic_tx.c``): The driver reads the completion queue producer pointer from the NIC +#. ``mqnic_process_tx_cq()`` (``mqnic_tx.c``): The driver reads the completion record +#. ``mqnic_process_tx_cq()`` (``mqnic_tx.c``): The driver reads the ``sk_buff`` from ``ring->tx_info`` +#. ``mqnic_process_tx_cq()`` (``mqnic_tx.c``): The driver completes the transmit timestamp operation +#. ``mqnic_process_tx_cq()`` (``mqnic_tx.c``): The driver calls ``mqnic_free_tx_desc()`` +#. ``mqnic_free_tx_desc()`` (``mqnic_tx.c``): The driver unmaps the ``sk_buff`` (``dma_unmap_single()``/``dma_unmap_page()``) +#. ``mqnic_free_tx_desc()`` (``mqnic_tx.c``): The driver frees the ``sk_buff`` (``napi_consume_skb()``) +#. ``mqnic_process_tx_cq()`` (``mqnic_tx.c``): The driver dequeues the completion record by incrementing the completion queue consumer pointer +#. ``mqnic_process_tx_cq()`` (``mqnic_tx.c``): The driver writes the updated consumer pointer via MMIO +#. ``mqnic_process_tx_cq()`` (``mqnic_tx.c``): The driver reads the queue consumer pointer from the NIC +#. ``mqnic_process_tx_cq()`` (``mqnic_tx.c``): The driver increments the ring consumer pointer for in-order freed descriptors +#. ``mqnic_process_tx_cq()`` (``mqnic_tx.c``): The driver wakes the queue if it was stopped (``netif_tx_wake_queue()``) +#. ``mqnic_poll_tx_cq()`` (``mqnic_tx.c``): The driver disables NAPI polling, when idle (``napi_complete()``) +#. ``mqnic_poll_tx_cq()`` (``mqnic_tx.c``): The driver rearms the CQ (``mqnic_arm_cq()``) + +Packet reception +================ + +init: + +#. ``mqnic_activate_rx_ring()`` (``mqnic_rx.c``): The driver calls ``mqnic_refill_rx_buffers()`` +#. ``mqnic_refill_rx_buffers()`` (``mqnic_rx.c``): The driver calls ``mqnic_prepare_rx_desc()`` for each empty location in the ring +#. ``mqnic_prepare_rx_desc()`` (``mqnic_rx.c``): The driver allocates memory pages (``dev_alloc_pages()``) +#. ``mqnic_prepare_rx_desc()`` (``mqnic_rx.c``): The driver maps the pages (``dev_alloc_pages()``) +#. ``mqnic_prepare_rx_desc()`` (``mqnic_rx.c``): The driver writes a pointer to the page struct in ``ring->rx_info`` +#. ``mqnic_prepare_rx_desc()`` (``mqnic_rx.c``): The driver writes a descriptor with the DMA pointer and length +#. ``mqnic_refill_rx_buffers()`` (``mqnic_rx.c``): The driver enqueues the descriptor by incrementing its local copy of the producer pointer +#. ``mqnic_refill_rx_buffers()`` (``mqnic_rx.c``): At the end of the loop, the driver writes the updated producer pointer to the NIC via MMIO + +receive: + +#. A packet arrives at the MAC +#. The MAC produces a PTP timestamp +#. :ref:`mod_mqnic_core`: data enters through receive streaming interfaces +#. :ref:`mod_mqnic_l2_ingress`: layer 2 ingress processing +#. :ref:`mod_mqnic_app_block` ``s_axis_sync_rx``: data is presented to the application section +#. :ref:`mod_mqnic_app_block` ``m_axis_sync_rx``: data is returned from the application section +#. :ref:`mod_mqnic_core`: Data passes through per-port receive async FIFO module and is transferred to core clock domain +#. :ref:`mod_mqnic_app_block` ``s_axis_sync_rx``: data is presented to the application section +#. :ref:`mod_mqnic_app_block` ``m_axis_sync_rx``: data is returned from the application section +#. :ref:`mod_mqnic_core`: Data passes enters per-interface receive FIFO module and is placed into per-port FIFOs, then aggregated into a single stream +#. :ref:`mod_mqnic_app_block` ``s_axis_if_rx``: data is presented to the application section +#. :ref:`mod_mqnic_app_block` ``m_axis_if_rx``: data is returned from the application section +#. :ref:`mod_mqnic_ingress`: ingress processing +#. :ref:`mod_rx_hash`: The receive hash module computes the packet flow hash +#. :ref:`mod_rx_checksum`: The receive checksum module computes the packet payload checksum +#. :ref:`mod_mqnic_interface_rx`: A receive request is generated +#. :ref:`mod_rx_engine`: The receive hash arrives at the receive engine +#. :ref:`mod_rx_engine`: The receive checksum arrives at the receive engine +#. :ref:`mod_rx_engine`: The receive request arrives at the receive engine +#. :ref:`mod_rx_engine` ``m_axis_rx_desc_*``: The receive engine issues a write request to the interface DMA engine +#. :ref:`mod_mqnic_interface_rx` ``rx_axis_*``: The interface DMA module writes the packet data from AXI stream to the interface local DMA RAM +#. :ref:`mod_rx_engine` ``m_axis_desc_req_*``: The receive engine issues a descriptor request +#. :ref:`mod_desc_fetch`: The descriptor request arrives at the descriptor fetch module +#. :ref:`mod_desc_fetch` ``m_axis_desc_dequeue_req_*``: The descriptor fetch module issues a dequeue request to the queue manager +#. :ref:`mod_queue_manager` ``s_axis_dequeue_req_*``: The dequeue request arrives at the queue manager module +#. :ref:`mod_queue_manager`: If the queue is not empty, the queue manager starts a dequeue operation on the queue +#. :ref:`mod_queue_manager` ``m_axis_dequeue_resp_*``: The queue manager sends a response containing the operation status and DMA address +#. :ref:`mod_desc_fetch` ``m_axis_desc_dequeue_resp_*``: The response arrives at the descriptor fetch module +#. :ref:`mod_desc_fetch` ``m_axis_req_status_*``: The descriptor module reports the descriptor fetch status +#. :ref:`mod_desc_fetch` ``m_axis_dma_read_desc_*``: The descriptor module issues a DMA read request +#. ``dma_if_pcie_us_rd`` ``s_axis_read_desc_*``: The requst arrives at the DMA read interface +#. ``dma_if_pcie_us_rd``: The DMA read interface issues a PCIe read request +#. ``dma_if_pcie_us_rd``: The read data comes back in a completion packet and is written to the descriptor fetch local DMA RAM +#. ``dma_if_pcie_us_rd`` ``m_axis_read_desc_status_*``: The DMA read interface issues a status message +#. :ref:`mod_desc_fetch` ``m_axis_desc_dequeue_commit_*``: The descriptor fetch module issues a dequeue commit message +#. :ref:`mod_queue_manager`: The queue manager commits the dequeue operation and updates the consumer pointer +#. :ref:`mod_desc_fetch` ``dma_read_desc_*``: The descriptor fetch module issues a read request to its internal DMA module +#. :ref:`mod_desc_fetch` ``m_axis_desc_*``: The internal DMA module reads the descriptor and transfers it via AXI stream +#. :ref:`mod_rx_engine`: The descriptor arrives at the receive engine +#. :ref:`mod_rx_engine`: The receive engine stores the descriptor data +#. :ref:`mod_rx_engine` ``m_axis_dma_write_desc_*``: The receive engine issues a DMA write request +#. ``dma_if_pcie_us_wr`` ``s_axis_write_desc_*``: The requst arrives at the DMA write interface +#. ``dma_if_pcie_us_wr``: The DMA write interface reads the packet data from the interface local DMA RAM +#. ``dma_if_pcie_us_wr``: The DMA write interface issues a PCIe write request +#. ``dma_if_pcie_us_wr`` ``m_axis_write_desc_status_*``: The DMA write interface issues a status message +#. :ref:`mod_rx_engine` ``m_axis_cpl_req_*``: The receive engine issues a completion write request +#. :ref:`mod_cpl_write`: The completion write module writes the completion data into its local DMA RAM +#. :ref:`mod_cpl_write` ``m_axis_cpl_enqueue_req_*``: The completion write module issues an enqueue request to the completion queue manager +#. :ref:`mod_cpl_queue_manager` ``s_axis_enqueue_req_*``: The enqueue request arrives at the completion queue manager module +#. :ref:`mod_cpl_queue_manager`: If the queue is not full, the queue manager starts an enqueue operation on the queue +#. :ref:`mod_cpl_queue_manager` ``m_axis_enqueue_resp_*``: The completion queue manager sends a response containing the operation status and DMA address +#. :ref:`mod_cpl_write` ``s_axis_cpl_enqueue_resp_*``: The response arrives at the completion write module +#. :ref:`mod_cpl_write` ``m_axis_req_status_*``: The completion write module reports the completion write status +#. :ref:`mod_desc_fetch` ``m_axis_dma_write_desc_*``: The completion write module issues a DMA write request +#. ``dma_if_pcie_us_wr`` ``s_axis_write_desc_*``: The requst arrives at the DMA write interface +#. ``dma_if_pcie_us_wr``: The DMA write interface reads the completion data from the completion write module local DMA RAM +#. ``dma_if_pcie_us_wr``: The DMA write interface issues a PCIe write request +#. ``dma_if_pcie_us_wr`` ``m_axis_write_desc_status_*``: The DMA write interface issues a status message +#. :ref:`mod_cpl_write` ``m_axis_desc_enqueue_commit_*``: The completion write module issues an enqueue commit message +#. :ref:`mod_cpl_queue_manager`: The completion queue manager commits the enqueue operation and updates the producer pointer +#. :ref:`mod_cpl_queue_manager` ``m_axis_event_*``: The completion queue manager issues an event, if armed +#. :ref:`mod_cpl_write`: The event arrives at the completion write module +#. :ref:`mod_cpl_write`: The completion write module writes the event data into its local DMA RAM +#. :ref:`mod_cpl_write` ``m_axis_cpl_enqueue_req_*``: The completion write module issues an enqueue request to the completion queue manager +#. :ref:`mod_cpl_queue_manager` ``s_axis_enqueue_req_*``: The enqueue request arrives at the completion queue manager module +#. :ref:`mod_cpl_queue_manager`: If the queue is not full, the queue manager starts an enqueue operation on the queue +#. :ref:`mod_cpl_queue_manager` ``m_axis_enqueue_resp_*``: The completion queue manager sends a response containing the operation status and DMA address +#. :ref:`mod_cpl_write` ``s_axis_cpl_enqueue_resp_*``: The response arrives at the completion write module +#. :ref:`mod_cpl_write` ``m_axis_req_status_*``: The completion write module reports the completion write status +#. :ref:`mod_desc_fetch` ``m_axis_dma_write_desc_*``: The completion write module issues a DMA write request +#. ``dma_if_pcie_us_wr`` ``s_axis_write_desc_*``: The requst arrives at the DMA write interface +#. ``dma_if_pcie_us_wr``: The DMA write interface reads the event data from the completion write module local DMA RAM +#. ``dma_if_pcie_us_wr``: The DMA write interface issues a PCIe write request +#. ``dma_if_pcie_us_wr`` ``m_axis_write_desc_status_*``: The DMA write interface issues a status message +#. :ref:`mod_cpl_write` ``m_axis_desc_enqueue_commit_*``: The completion write module issues an enqueue commit message +#. :ref:`mod_cpl_queue_manager`: The completion queue manager commits the enqueue operation and updates the producer pointer +#. :ref:`mod_cpl_queue_manager` ``m_axis_event_*``: The completion queue manager issues an interrupt, if armed +#. linux: The linux kernel calls ``mqnic_irq_handler()`` +#. ``mqnic_irq_handler()`` (``mqnic_irq.c``): The driver calls the EQ handler via the notifier chain (``atomic_notifier_call_chain()``) +#. ``mqnic_eq_int()`` (``mqnic_eq.c``): The driver calls ``mqnic_process_eq()`` +#. ``mqnic_process_eq()`` (``mqnic_eq.c``): The driver processes the event queue, which calls the appropriate handler (``mqnic_rx_irq()``) +#. ``mqnic_rx_irq()`` (``mqnic_rx.c``): The driver enables NAPI polling on the queue (``napi_schedule_irqoff()``) +#. ``mqnic_eq_int()`` (``mqnic_eq.c``): The driver rearms the EQ (``mqnic_arm_eq()``) +#. NAPI: The linux kernel calls ``mqnic_poll_rx_cq()`` +#. ``mqnic_poll_rx_cq()`` (``mqnic_rx.c``): The driver calls ``mqnic_process_rx_cq()`` +#. ``mqnic_process_rx_cq()`` (``mqnic_rx.c``): The driver reads the CQ producer pointer from the NIC +#. ``mqnic_process_rx_cq()`` (``mqnic_rx.c``): The driver reads the completion record +#. ``mqnic_process_rx_cq()`` (``mqnic_rx.c``): The driver fetches a fresh ``sk_buff`` (``napi_get_frags()``) +#. ``mqnic_process_rx_cq()`` (``mqnic_rx.c``): The driver sets the ``sk_buff`` hardware timestamp +#. ``mqnic_process_rx_cq()`` (``mqnic_rx.c``): The driver unmaps the pages (``dma_unmap_page()``) +#. ``mqnic_process_rx_cq()`` (``mqnic_rx.c``): The driver associates the pages with the ``sk_buff`` (``__skb_fill_page_desc()``) +#. ``mqnic_process_rx_cq()`` (``mqnic_rx.c``): The driver sets the ``sk_buff`` length +#. ``mqnic_process_rx_cq()`` (``mqnic_rx.c``): The driver hands off the ``sk_buff`` to ``napi_gro_frags()`` +#. ``mqnic_process_rx_cq()`` (``mqnic_rx.c``): The driver dequeues the completion record by incrementing the CQ consumer pointer +#. ``mqnic_process_rx_cq()`` (``mqnic_rx.c``): The driver writes the updated CQ consumer pointer via MMIO +#. ``mqnic_process_rx_cq()`` (``mqnic_rx.c``): The driver reads the queue consumer pointer from the NIC +#. ``mqnic_process_rx_cq()`` (``mqnic_rx.c``): The driver increments the ring consumer pointer for in-order freed descriptors +#. ``mqnic_process_rx_cq()`` (``mqnic_rx.c``): The driver calls ``mqnic_refill_rx_buffers()`` +#. ``mqnic_refill_rx_buffers()`` (``mqnic_rx.c``): The driver calls ``mqnic_prepare_rx_desc()`` for each empty location in the ring +#. ``mqnic_prepare_rx_desc()`` (``mqnic_rx.c``): The driver allocates memory pages (``dev_alloc_pages()``) +#. ``mqnic_prepare_rx_desc()`` (``mqnic_rx.c``): The driver maps the pages (``dev_alloc_pages()``) +#. ``mqnic_prepare_rx_desc()`` (``mqnic_rx.c``): The driver writes a pointer to the page struct in ``ring->rx_info`` +#. ``mqnic_prepare_rx_desc()`` (``mqnic_rx.c``): The driver writes a descriptor with the DMA pointer and length +#. ``mqnic_refill_rx_buffers()`` (``mqnic_rx.c``): The driver enqueues the descriptor by incrementing its local copy of the producer pointer +#. ``mqnic_refill_rx_buffers()`` (``mqnic_rx.c``): At the end of the loop, the driver writes the updated producer pointer to the NIC via MMIO +#. ``mqnic_poll_rx_cq()`` (``mqnic_rx.c``): The driver disables NAPI polling, when idle (``napi_complete()``) +#. ``mqnic_poll_rx_cq()`` (``mqnic_rx.c``): The driver rearms the CQ (``mqnic_arm_cq()``) diff --git a/docs/source/porting.rst b/docs/source/porting.rst new file mode 100644 index 000000000..73ecf9f2a --- /dev/null +++ b/docs/source/porting.rst @@ -0,0 +1,62 @@ +.. _porting: + +======= +Porting +======= + +This guide is a high-level overview for how to port Corundum to new hardware. In general, this guide only applies to FPGA families that are already supported by Corundum, new FPGA families can require significant interfacing changes, especially for the PCI express interface as this can vary significantly between different FPGA families. + +The only interfaces that the Corundum datapath requires are the PCI express interface and the Ethernet interfaces. Ancillary features such as firmware updates, persistent MAC addresses, and optical module communication are optional---the core datapath will still function if these features are not implemented. In general the PCI express and Ethernet interfaces are dependent almost completely on the FPGA family, while ancillary features tend to be much more board-dependent. + +Preparation +=========== + +Before porting Corundum to a new board, it is recommended to create example designs for both verilog-ethernet and verilog-pcie for the target board. The verilog-ethernet design will bring up the Ethernet interfaces at 10 Gbps and ensures the transceivers, reference clocks, and module control pins are properly configured for the Ethernet interfaces to operate. Some boards may require additional code to configure clocking logic to supply the proper reference clocks to the transceivers on the FPGA, which will generally be one of 156.25 MHz, 161.1328125 MHz, 322.265625 MHz, or 644.53125 MHz. The verilog-pcie design brings up the PCI express interface, validating that all of the pin assignments and transceiver site locations are correct. Once both of these designs are working, then porting corundum is straightforward. + +Porting Corundum +================ + +Start by making a copy of a Corundum design that targets a similar board. Priority goes to a chip in the same family, then similar ancillary interfaces. + +Board ID +-------- + +Each board should have a unique board ID specified in ``mqnic_hw.h``. These IDs are used by the driver for any board-specific initialization and interfacing. These IDs are arbitrary, but making something relatively predictable is a good idea to reduce the possibility of collisions. Most of the current IDs are a combination of the PCIe vendor ID of the board manufacturer, combined with a board-specific portion. For example, the board IDs for ExaNICs are simply the original ExaNIC PCIe VID and PID, and the Xilinx board IDs are a combination of the Xilinx PCIe VID, the part series (7 for 7 series, 8 for UltraScale, 9 for UltraScale+, etc.) and the hex version of the board part number (VCU108 = 6c, VCU1525 = 5f5, etc.). Pick a board ID, add it to ``mqnic_hw.h``, and set the ``BOARD_ID`` parameter in fpga_core.v. + +FPGA ID +------- + +The FPGA ID is used by the firmware update tool as a simple sanity check to prevent firmware for a different board from being loaded accidentally. Set the FPGA_ID parameter in fpga_core.v to the JTAG ID of the FPGA on the board. The IDs are located in ``fpga_id.h``/``fpga_id.c``. If you do not want to implement the firmware update feature, ``FPGA_ID`` can be set to 0. + +PCIe interface +-------------- + +Ensure that the PCIe hard IP core settings are correct for the target board. In many cases, the default settings are correct, but in some cases the transceiver sites need to be changed. Edit the TCL file appropriately, or generate the IP in vivado and extract the TCL commands from the Vivado journal file. If you previously ported the verilog-pcie design, then the settings can be copied over, with the PCIe IDs, BARs, and MSI settings configured appropriately. + +Check that the ``BAR0_APERTURE`` setting and PCIE_AXIS settings in ``fpga.v`` and ``fpga_core.v`` match the PCIe core configuration. + +Ethernet interfaces +------------------- + +For 100G interfaces, use Xilinx CMAC instances. A free license can be generated on the Xilinx website. The cores must be configured for CAUI-4. Select the appropriate reference clock and transceiver sites for the interfaces on the board. It may be necessary to adjust the CMAC site selections depending on which transceiver sites are used. Implement the design, open the implemented design, check the relative positions of the transceiver sites and CMAC sites, and adjust as appropriate. You can actually look at any synthesized or implemented design for the same chip to look at the relative positions of the sites. + +For 10G or 25G interfaces, you can either use the MAC modules from verilog-ethernet or Xilinx-provided MAC modules. For the included MACs, the main thing to adjust is the gtwizard instance. This needs to be set up to use the correct transceiver sites and reference clock inputs. The internal interface must be the 64 bit asynchronous gearbox. Check the connection ordering; the gtwizard instance is always in the order of the site names, but this may not match the board, and connections may need to be re-ordered to match. In particular, double check that the RX clocks are connected correctly. + +Update the interfaces between ``fpga.v`` and ``fpga_core.v`` to match the module configuration. Update the code in ``fpga_core.v`` to connect the PHYs in ``fpga.v`` to the appropriate MACs in ``fpga_core.v``. Also set ``IF_COUNT`` and ``PORTS_PER_IF`` appropriately in ``fpga_core.v``. + +I2C interfaces +-------------- + +MAC address EEPROMs and optical modules are accessed via I2C. This is highly board-dependent. On some boards, there is a single I2C interface and a number of I2C multiplexers to connect everything. On other boards, each optical module has a dedicated I2C interface. On other boards, the I2C bus sits behind a board management controller. The core datapath will work fine without setting up I2C, but having the I2C buses operational can be a useful debugging feature. If I2C access is not required, simply do not implement the registers and ensure that the selected board ID does not correspond to any I2C init code in ``mqnic_i2c.c``. + +All corundum designs that directly connect I2C interfaces to the FPGA pins make use of bit-bang I2C support in the Linux kernel. There are a set of registers set aside for controlling up to four I2C buses in ``mqnic_hw.h``. These should be appropriately implemented in the NIC CSR register space in ``fpga_core.v``. Driver code also needs to be added to ``mqnic_i2c.c`` to initialize everything appropriately based on the board ID. + +Flash access +------------ + +Firmware updates require access to the FPGA configuration flash. Depending on the flash type, this either requires connections to dedicated pins via specific device primitives, normal FPGA IO pins, or both. The flash interface is a very simple bit-bang interface that simply exposes these pins over PCIe via NIC CSR register space. The register definitions are in ``mqnic_hw.h``. Take a look at existing designs that implement QSPI or BPI flash and implement the same register configuration in ``fpga_core.v``. If firmware update support is not required, simply do not implement the flash register block. + +Module control pins +------------------- + +Optical modules have several low-speed control pins in addition to the I2C interface. For DAC cables, these pins have no effect, but for AOC cables or optical modules, these pins are very important. Specifically, SFP+ and SFP28 modules need to have the correct level on ``tx_disable`` in order to turn the laser on. Similarly, QSFP+ and QSFP28 modules need to have the ``reset`` and ``lpmode`` pins set correctly. These pins can be statically tied off with the modules enabled, or they can be exposed to the driver via standard registers specified in ``mqnic_hw.h`` and implemented in the NIC CSRs in ``fpga_core.v``. \ No newline at end of file diff --git a/docs/source/rb/bmc_alveo.rst b/docs/source/rb/bmc_alveo.rst new file mode 100644 index 000000000..fafb8e1bc --- /dev/null +++ b/docs/source/rb/bmc_alveo.rst @@ -0,0 +1,49 @@ +.. _rb_bmc_alveo: + +======================== +Alveo BMC register block +======================== + +The Alveo BMC register block has a header with type 0x0000C140, version 0x00000100, and contains control registers for the `Xilinx Alveo CMS IP `_. + +.. table:: + + ======== ============= ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ======== ============= ====== ====== ====== ====== ============= + RBB+0x00 Type Vendor ID Type RO 0x0000C140 + -------- ------------- -------------- -------------- ------------- + RBB+0x04 Version Major Minor Patch Meta RO 0x00000100 + -------- ------------- ------ ------ ------ ------ ------------- + RBB+0x08 Next pointer Pointer to next register block RO - + -------- ------------- ------------------------------ ------------- + RBB+0x0C Address Address RW 0x00000000 + -------- ------------- ------------------------------ ------------- + RBB+0x10 Data Data RW 0x00000000 + ======== ============= ============================== ============= + +See :ref:`rb_overview` for definitions of the standard register block header fields. + +.. object:: Address + + The address field controls the address bus to the CMS IP core. Writing to this register triggers a read of the corresponding address via the AXI-lite interface to the CMS IP. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x0C Address RW 0x00000000 + ======== ============================== ============= + +.. object:: Data + + The data field controls the data bus to the CMS IP core. Writing to this register triggers a write to the address specified by the address register via the AXI-lite interface to the CMS IP. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x10 Data RW 0x00000000 + ======== ============================== ============= diff --git a/docs/source/rb/bmc_gecko.rst b/docs/source/rb/bmc_gecko.rst new file mode 100644 index 000000000..1e6cb33d3 --- /dev/null +++ b/docs/source/rb/bmc_gecko.rst @@ -0,0 +1,73 @@ +.. _rb_bmc_gecko: + +======================== +Gecko BMC register block +======================== + +The Gecko BMC register block has a header with type 0x0000C141, version 0x00000100, and contains control registers for the Silicom Gecko BMC. + +.. table:: + + ======== ============= ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ======== ============= ====== ====== ====== ====== ============= + RBB+0x00 Type Vendor ID Type RO 0x0000C141 + -------- ------------- -------------- -------------- ------------- + RBB+0x04 Version Major Minor Patch Meta RO 0x00000100 + -------- ------------- ------ ------ ------ ------ ------------- + RBB+0x08 Next pointer Pointer to next register block RO - + -------- ------------- ------------------------------ ------------- + RBB+0x0C Status Status Read data RO 0x00000000 + -------- ------------- -------------- -------------- ------------- + RBB+0x10 Data Write data RW 0x00000000 + -------- ------------- ------------------------------ ------------- + RBB+0x14 Command Command RW 0x00000000 + ======== ============= ============== ============== ============= + +See :ref:`rb_overview` for definitions of the standard register block header fields. + +.. object:: Status + + The status field provides status information and the read data from the BMC. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x0C Status Read data RO 0x00000000 + ======== ============== ============== ============= + + .. table:: + + === ======== + Bit Function + === ======== + 16 Done + 18 Timeout + 19 Idle + === ======== + +.. object:: Data + + The data field provides the write data to the BMC. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x10 Write data RW 0x00000000 + ======== ============================== ============= + +.. object:: Command + + The command field provides the command to the BMC. Writing to the command field triggers an SPI transfer to the BMC. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x14 Command RW 0x00000000 + ======== ============== ============== ============= diff --git a/docs/source/rb/cqm_event.rst b/docs/source/rb/cqm_event.rst new file mode 100644 index 000000000..d298d24c1 --- /dev/null +++ b/docs/source/rb/cqm_event.rst @@ -0,0 +1,86 @@ +.. _rb_cqm_event: + +================================== +Event queue manager register block +================================== + +The event queue manager register block has a header with type 0x0000C010, version 0x00000100, and indicates the location of the event queue manager registers and number of event queues. + +.. table:: + + ======== ============= ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ======== ============= ====== ====== ====== ====== ============= + RBB+0x00 Type Vendor ID Type RO 0x0000C010 + -------- ------------- -------------- -------------- ------------- + RBB+0x04 Version Major Minor Patch Meta RO 0x00000100 + -------- ------------- ------ ------ ------ ------ ------------- + RBB+0x08 Next pointer Pointer to next register block RO - + -------- ------------- ------------------------------ ------------- + RBB+0x0C Offset Offset to queue manager RO - + -------- ------------- ------------------------------ ------------- + RBB+0x10 Count Queue count RO - + -------- ------------- ------------------------------ ------------- + RBB+0x14 Stride Queue control register stride RO 0x00000020 + ======== ============= ============================== ============= + +See :ref:`rb_overview` for definitions of the standard register block header fields. + +.. object:: Offset + + The offset field contains the offset to the start of the event queue manager region, relative to the start of the current region. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x0C Offset to queue manager RO - + ======== ============================== ============= + +.. object:: Count + + The count field contains the number of queues. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x10 Queue count RO - + ======== ============================== ============= + +.. object:: Stride + + The stride field contains the size of the control registers associated with each queue. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x14 Queue control register stride RO 0x00000020 + ======== ============================== ============= + +Event queue manager CSRs +======================== + +Each queue has several associated control registers, detailed in this table: + +.. table:: + + ========= ============== ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ========= ============== ====== ====== ====== ====== ============= + Base+0x00 Base address L Ring base address (lower 32) RW - + --------- -------------- ------------------------------ ------------- + Base+0x04 Base address H Ring base address (upper 32) RW - + --------- -------------- ------------------------------ ------------- + Base+0x08 Control 1 Active Size RW - + --------- -------------- ------ ------ ------ ------ ------------- + Base+0x0C Control 2 Arm Int index RW - + --------- -------------- ------ ------ -------------- ------------- + Base+0x10 Head pointer Head pointer RW - + --------- -------------- -------------- -------------- ------------- + Base+0x14 Tail pointer Tail pointer RW - + ========= ============== ============== ============== ============= diff --git a/docs/source/rb/cqm_rx.rst b/docs/source/rb/cqm_rx.rst new file mode 100644 index 000000000..3a2bbdc82 --- /dev/null +++ b/docs/source/rb/cqm_rx.rst @@ -0,0 +1,86 @@ +.. _rb_cqm_rx: + +================================================ +Receive completion queue manager register block +================================================ + +The receive completion queue manager register block has a header with type 0x0000C031, version 0x00000100, and indicates the location of the receive completion queue manager registers and number of completion queues. + +.. table:: + + ======== ============= ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ======== ============= ====== ====== ====== ====== ============= + RBB+0x00 Type Vendor ID Type RO 0x0000C031 + -------- ------------- -------------- -------------- ------------- + RBB+0x04 Version Major Minor Patch Meta RO 0x00000100 + -------- ------------- ------ ------ ------ ------ ------------- + RBB+0x08 Next pointer Pointer to next register block RO - + -------- ------------- ------------------------------ ------------- + RBB+0x0C Offset Offset to queue manager RO - + -------- ------------- ------------------------------ ------------- + RBB+0x10 Count Queue count RO - + -------- ------------- ------------------------------ ------------- + RBB+0x14 Stride Queue control register stride RO 0x00000020 + ======== ============= ============================== ============= + +See :ref:`rb_overview` for definitions of the standard register block header fields. + +.. object:: Offset + + The offset field contains the offset to the start of the receive completion queue manager region, relative to the start of the current region. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x0C Offset to queue manager RO - + ======== ============================== ============= + +.. object:: Count + + The count field contains the number of queues. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x10 Queue count RO - + ======== ============================== ============= + +.. object:: Stride + + The stride field contains the size of the control registers associated with each queue. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x14 Queue control register stride RO 0x00000020 + ======== ============================== ============= + +Completion queue manager CSRs +============================= + +Each queue has several associated control registers, detailed in this table: + +.. table:: + + ========= ============== ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ========= ============== ====== ====== ====== ====== ============= + Base+0x00 Base address L Ring base address (lower 32) RW - + --------- -------------- ------------------------------ ------------- + Base+0x04 Base address H Ring base address (upper 32) RW - + --------- -------------- ------------------------------ ------------- + Base+0x08 Control 1 Active Size RW - + --------- -------------- ------ ------ ------ ------ ------------- + Base+0x0C Control 2 Arm Event index RW - + --------- -------------- ------ ------ -------------- ------------- + Base+0x10 Head pointer Head pointer RW - + --------- -------------- -------------- -------------- ------------- + Base+0x14 Tail pointer Tail pointer RW - + ========= ============== ============== ============== ============= diff --git a/docs/source/rb/cqm_tx.rst b/docs/source/rb/cqm_tx.rst new file mode 100644 index 000000000..611cec69a --- /dev/null +++ b/docs/source/rb/cqm_tx.rst @@ -0,0 +1,86 @@ +.. _rb_cqm_tx: + +================================================ +Transmit completion queue manager register block +================================================ + +The transmit completion queue manager register block has a header with type 0x0000C030, version 0x00000100, and indicates the location of the transmit completion queue manager registers and number of completion queues. + +.. table:: + + ======== ============= ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ======== ============= ====== ====== ====== ====== ============= + RBB+0x00 Type Vendor ID Type RO 0x0000C030 + -------- ------------- -------------- -------------- ------------- + RBB+0x04 Version Major Minor Patch Meta RO 0x00000100 + -------- ------------- ------ ------ ------ ------ ------------- + RBB+0x08 Next pointer Pointer to next register block RO - + -------- ------------- ------------------------------ ------------- + RBB+0x0C Offset Offset to queue manager RO - + -------- ------------- ------------------------------ ------------- + RBB+0x10 Count Queue count RO - + -------- ------------- ------------------------------ ------------- + RBB+0x14 Stride Queue control register stride RO 0x00000020 + ======== ============= ============================== ============= + +See :ref:`rb_overview` for definitions of the standard register block header fields. + +.. object:: Offset + + The offset field contains the offset to the start of the transmit completion queue manager region, relative to the start of the current region. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x0C Offset to queue manager RO - + ======== ============================== ============= + +.. object:: Count + + The count field contains the number of queues. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x10 Queue count RO - + ======== ============================== ============= + +.. object:: Stride + + The stride field contains the size of the control registers associated with each queue. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x14 Queue control register stride RO 0x00000020 + ======== ============================== ============= + +Completion queue manager CSRs +============================= + +Each queue has several associated control registers, detailed in this table: + +.. table:: + + ========= ============== ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ========= ============== ====== ====== ====== ====== ============= + Base+0x00 Base address L Ring base address (lower 32) RW - + --------- -------------- ------------------------------ ------------- + Base+0x04 Base address H Ring base address (upper 32) RW - + --------- -------------- ------------------------------ ------------- + Base+0x08 Control 1 Active Size RW - + --------- -------------- ------ ------ ------ ------ ------------- + Base+0x0C Control 2 Arm Event index RW - + --------- -------------- ------ ------ -------------- ------------- + Base+0x10 Head pointer Head pointer RW - + --------- -------------- -------------- -------------- ------------- + Base+0x14 Tail pointer Tail pointer RW - + ========= ============== ============== ============== ============= diff --git a/docs/source/rb/drp.rst b/docs/source/rb/drp.rst new file mode 100644 index 000000000..c1cdc213b --- /dev/null +++ b/docs/source/rb/drp.rst @@ -0,0 +1,105 @@ +.. _rb_drp: + +================== +DRP register block +================== + +The DRP register block has a header with type 0x0000C150, version 0x00000100, and contains control registers for a Xilinx dynamic reconfiguration port (DRP). + +.. table:: + + ======== ============= ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ======== ============= ====== ====== ====== ====== ============= + RBB+0x00 Type Vendor ID Type RO 0x0000C150 + -------- ------------- -------------- -------------- ------------- + RBB+0x04 Version Major Minor Patch Meta RO 0x00000100 + -------- ------------- ------ ------ ------ ------ ------------- + RBB+0x08 Next pointer Pointer to next register block RO - + -------- ------------- ------------------------------ ------------- + RBB+0x0C DRP info DRP info RO - + -------- ------------- ------------------------------ ------------- + RBB+0x10 Control Control RW 0x00000000 + -------- ------------- ------------------------------ ------------- + RBB+0x14 Address Address RW 0x00000000 + -------- ------------- ------------------------------ ------------- + RBB+0x18 Write data Write data RW 0x00000000 + -------- ------------- ------------------------------ ------------- + RBB+0x1C Read data Read data RO 0x00000000 + ======== ============= ============================== ============= + +See :ref:`rb_overview` for definitions of the standard register block header fields. + +.. object:: DRP info + + The DRP info field contains identifying information about the component(s) accessible via the DRP interface. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x0C DRP info RO - + ======== ============================== ============= + +.. object:: Control + + The control field is used to trigger read and write operations on the DRP interface. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x10 Control RW 0x00000000 + ======== ============================== ============= + + .. table:: + + === ======== + Bit Function + === ======== + 0 Enable + 1 Write + 8 Busy + === ======== + + To issue a read operation, set the address register and then write 0x00000001 to the control register. Wait for the enable and busy bits to self-clear, then read the data from the read data register. + + To issue a write operation, set the address register and write data register appropriately, then write 0x00000003 to the control register. Wait for the enable and busy bits to self-clear. + +.. object:: Address + + The address field controls the address for DRP operations. This address is directly presented on the DRP interface. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x14 Address RW 0x00000000 + ======== ============================== ============= + +.. object:: Write data + + The write data field contains the data used for DRP write operations. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x18 Write data RW 0x00000000 + ======== ============================== ============= + +.. object:: Read data + + The read data field contains the data returned by DRP read operations. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x1C Read data RO 0x00000000 + ======== ============================== ============= diff --git a/docs/source/rb/flash_bpi.rst b/docs/source/rb/flash_bpi.rst new file mode 100644 index 000000000..6419c84a8 --- /dev/null +++ b/docs/source/rb/flash_bpi.rst @@ -0,0 +1,102 @@ +.. _rb_flash_bpi: + +======================== +BPI flash register block +======================== + +The BPI flash register block has a header with type 0x0000C121, version 0x00000100, and contains control registers for a BPI flash chip. + +.. table:: + + ======== ============= ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ======== ============= ====== ====== ====== ====== ============= + RBB+0x00 Type Vendor ID Type RO 0x0000C121 + -------- ------------- -------------- -------------- ------------- + RBB+0x04 Version Major Minor Patch Meta RO 0x00000100 + -------- ------------- ------ ------ ------ ------ ------------- + RBB+0x08 Next pointer Pointer to next register block RO - + -------- ------------- ------------------------------ ------------- + RBB+0x0C Format AW DW config RO - + -------- ------------- ------ ------ ------ ------ ------------- + RBB+0x10 Address Address RW 0x00000000 + -------- ------------- ------------------------------ ------------- + RBB+0x14 Data Data RW 0x00000000 + -------- ------------- ------------------------------ ------------- + RBB+0x18 Control REGION DQ_OE CTRL RW 0x0000000F + ======== ============= ====== ====== ====== ====== ============= + +See :ref:`rb_overview` for definitions of the standard register block header fields. + +.. object:: Format + + The format field contains information about the type and layout of the flash memory. AW and DW indicate the address and data interface widths in bits, and config indicates the layout of the flash. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x0C AW DW config RO - + ======== ====== ====== ====== ====== ============= + + .. table:: + + ====== ======================= + config Configuration + ====== ======================= + 0x00 1 segment + 0x01 1 segment + 0x02 2 segments (even split) + 0x04 4 segments (even split) + 0x08 8 segments (even split) + ====== ======================= + +.. object:: Address + + The address field controls the address bus to the flash chip. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x10 Address RW 0x00000000 + ======== ============================== ============= + +.. object:: Data + + The data field controls the data bus to the flash chip. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x14 Data RW 0x00000000 + ======== ============================== ============= + +.. object:: Control + + The control field contains registers to drive all of the other flash control lines, as well as registers for output enables. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x18 REGION DQ_OE CTRL RW 0x0000000F + ======== ====== ====== ====== ====== ============= + + .. table:: + + === ========= + Bit Function + === ========= + 0 CE_N + 1 OE_N + 2 WE_N + 3 ADV_N + 8 DQ_OE + 16 REGION_OE + === ========= diff --git a/docs/source/rb/flash_spi.rst b/docs/source/rb/flash_spi.rst new file mode 100644 index 000000000..ce0f0953a --- /dev/null +++ b/docs/source/rb/flash_spi.rst @@ -0,0 +1,93 @@ +.. _rb_flash_spi: + +======================== +SPI flash register block +======================== + +The SPI flash register block has a header with type 0x0000C120, version 0x00000100, and contains control registers for up to two SPI or QSPI flash chips. + +.. table:: + + ======== ============= ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ======== ============= ====== ====== ====== ====== ============= + RBB+0x00 Type Vendor ID Type RO 0x0000C120 + -------- ------------- -------------- -------------- ------------- + RBB+0x04 Version Major Minor Patch Meta RO 0x00000100 + -------- ------------- ------ ------ ------ ------ ------------- + RBB+0x08 Next pointer Pointer to next register block RO - + -------- ------------- ------------------------------ ------------- + RBB+0x0C Format AW DW config RO - + -------- ------------- ------ ------ ------ ------ ------------- + RBB+0x10 Control 0 CS/CLK OE D RW 0x00000000 + -------- ------------- ------ ------ ------ ------ ------------- + RBB+0x14 Control 1 CS/CLK OE D RW 0x00000000 + ======== ============= ====== ====== ====== ====== ============= + +See :ref:`rb_overview` for definitions of the standard register block header fields. + +.. object:: Format + + The format field contains information about the type and layout of the flash memory. AW and DW indicate the address and data interface widths in bits, and config indicates the layout of the flash. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x0C AW DW config RO - + ======== ====== ====== ====== ====== ============= + + .. table:: + + ====== ================================ + config Configuration + ====== ================================ + 0x00 1 segment + 0x01 1 segment + 0x02 2 segments (even split) + 0x04 4 segments (even split) + 0x08 8 segments (even split) + 0x81 2 segments (split at 0x01002000) + ====== ================================ + + .. table:: + + == ========== + DW Flash type + == ========== + 1 SPI + 4 QSPI + 8 Dual QSPI + == ========== + +.. object:: Control 0 and 1 + + The control 0 and 1 fields each control one SPI/QSPI flash interface. The second interface is only used in dual QSPI mode. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x10 CS/CLK OE D RW 0x00000000 + -------- ------ ------ ------ ------ ------------- + RBB+0x14 CS/CLK OE D RW 0x00000000 + ======== ====== ====== ====== ====== ============= + + .. table:: + + === ========= + Bit Function + === ========= + 0 D0 + 1 D1 + 2 D2 + 3 D3 + 8 OE for D0 + 9 OE for D1 + 10 OE for D2 + 11 OE for D3 + 16 CLK + 17 CS_N + === ========= diff --git a/docs/source/rb/fw_id.rst b/docs/source/rb/fw_id.rst new file mode 100644 index 000000000..55678b7ca --- /dev/null +++ b/docs/source/rb/fw_id.rst @@ -0,0 +1,133 @@ +.. _rb_fw_id: + +========================== +Firmware ID register block +========================== + +The firmware ID register block has a header with type 0xFFFFFFFF, version 0x00000100, and carries several pieces of information related to the firmware version and build. + +.. table:: + + ======== ============= ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ======== ============= ====== ====== ====== ====== ============= + RBB+0x00 Type Vendor ID Type RO 0xFFFFFFFF + -------- ------------- -------------- -------------- ------------- + RBB+0x04 Version Major Minor Patch Meta RO 0x00000100 + -------- ------------- ------ ------ ------ ------ ------------- + RBB+0x08 Next pointer Pointer to next register block RO - + -------- ------------- ------------------------------ ------------- + RBB+0x0C FPGA ID JTAG ID RO - + -------- ------------- ------------------------------ ------------- + RBB+0x10 FW ID Vendor ID Firmware ID RO - + -------- ------------- -------------- -------------- ------------- + RBB+0x14 FW Version Major Minor Patch Meta RO - + -------- ------------- ------ ------ ------ ------ ------------- + RBB+0x18 Board ID Vendor ID Board ID RO - + -------- ------------- -------------- -------------- ------------- + RBB+0x1C Board Version Major Minor Patch Meta RO - + -------- ------------- ------ ------ ------ ------ ------------- + RBB+0x20 Build date Build date RO - + -------- ------------- ------------------------------ ------------- + RBB+0x24 Git hash Commit hash RO - + -------- ------------- ------------------------------ ------------- + RBB+0x28 Release info Release info RO - + ======== ============= ============================== ============= + +See :ref:`rb_overview` for definitions of the standard register block header fields. + +.. object:: FPGA ID + + The FPGA ID field contains the JTAG ID of the target device. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x0C JTAG ID RO - + ======== ============================== ============= + +.. object:: Firmware ID + + The firmware ID field consists of a vendor ID in the upper 16 bits, and the firmware ID in the lower 16 bits. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x10 Vendor ID Firmware ID RO - + ======== ============== ============== ============= + +.. object:: Firmware version + + The firmware version field consists of four fields, major, minor, patch, and meta. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x14 Major Minor Patch Meta RO - + ======== ====== ====== ====== ====== ============= + +.. object:: Board ID + + The board ID field consists of a vendor ID in the upper 16 bits, and the board ID in the lower 16 bits. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x18 Vendor ID Board ID RO - + ======== ============== ============== ============= + +.. object:: Board version + + The board version field consists of four fields, major, minor, patch, and meta. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x1C Major Minor Patch Meta RO - + ======== ====== ====== ====== ====== ============= + +.. object:: Build date + + The build date field contains the Unix timestamp of the start of the build as an unsigned 32-bit integer. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x20 Build date RO - + ======== ============================== ============= + +.. object:: Git hash + + The git hash field contains the upper 32 bits of the git commit hash. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x24 Commit hash RO - + ======== ============================== ============= + +.. object:: Release info + + The release info field is reserved for additional release information. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x28 Release info RO - + ======== ============================== ============= diff --git a/docs/source/rb/gpio.rst b/docs/source/rb/gpio.rst new file mode 100644 index 000000000..66d16f280 --- /dev/null +++ b/docs/source/rb/gpio.rst @@ -0,0 +1,43 @@ +.. _rb_gpio: + +=================== +GPIO register block +=================== + +The GPIO register block has a header with type 0x0000C100, version 0x00000100, and contains GPIO control registers. + +======== ============= ====== ====== ====== ====== ============= +Address Field 31..24 23..16 15..8 7..0 Reset value +======== ============= ====== ====== ====== ====== ============= +RBB+0x00 Type Vendor ID Type RO 0x0000C100 +-------- ------------- -------------- -------------- ------------- +RBB+0x04 Version Major Minor Patch Meta RO 0x00000100 +-------- ------------- ------ ------ ------ ------ ------------- +RBB+0x08 Next pointer Pointer to next register block RO - +-------- ------------- ------------------------------ ------------- +RBB+0x0C GPIO in GPIO in RO - +-------- ------------- ------------------------------ ------------- +RBB+0x10 GPIO out GPIO out RW - +======== ============= ============================== ============= + +See :ref:`rb_overview` for definitions of the standard register block header fields. + +.. object:: GPIO in + + The GPIO in field reads the input signal states. + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x0C GPIO in RO - + ======== ============================== ============= + +.. object:: GPIO out + + The GPIO out field controls the output signal states. + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x10 GPIO out RW - + ======== ============================== ============= diff --git a/docs/source/rb/i2c.rst b/docs/source/rb/i2c.rst new file mode 100644 index 000000000..4dc8bd80c --- /dev/null +++ b/docs/source/rb/i2c.rst @@ -0,0 +1,46 @@ +.. _rb_i2c: + +=================== +I2C register block +=================== + +The I2C register block has a header with type 0x0000C110, version 0x00000100, and contains registers to control an I2C interface. + +.. table:: + + ======== ============= ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ======== ============= ====== ====== ====== ====== ============= + RBB+0x00 Type Vendor ID Type RO 0x0000C110 + -------- ------------- -------------- -------------- ------------- + RBB+0x04 Version Major Minor Patch Meta RO 0x00000100 + -------- ------------- ------ ------ ------ ------ ------------- + RBB+0x08 Next pointer Pointer to next register block RO - + -------- ------------- ------------------------------ ------------- + RBB+0x0C Control Mux control SDA SCL RW 0x00000303 + ======== ============= ============== ====== ====== ============= + +See :ref:`rb_overview` for definitions of the standard register block header fields. + +.. object:: Control + + The control field has bits to control SCL, SDA, and any associated multiplexers/switches. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x0C Mux control SDA SCL RW 0x00000303 + ======== ============== ====== ====== ============= + + .. table:: + + === ======== + Bit Function + === ======== + 0 SCL in + 1 SCL out + 8 SDA in + 9 SDA out + === ======== diff --git a/docs/source/rb/if.rst b/docs/source/rb/if.rst new file mode 100644 index 000000000..91595d176 --- /dev/null +++ b/docs/source/rb/if.rst @@ -0,0 +1,77 @@ +.. _rb_if: + +======================== +Interface register block +======================== + +The interface register block has a header with type 0x0000C000, version 0x00000100, and indicates the number of interfaces present and where they are located in the control register space. + +.. table:: + + ======== ============= ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ======== ============= ====== ====== ====== ====== ============= + RBB+0x00 Type Vendor ID Type RO 0x0000C000 + -------- ------------- -------------- -------------- ------------- + RBB+0x04 Version Major Minor Patch Meta RO 0x00000100 + -------- ------------- ------ ------ ------ ------ ------------- + RBB+0x08 Next pointer Pointer to next register block RO - + -------- ------------- ------------------------------ ------------- + RBB+0x0C Offset Offset to first interface RO - + -------- ------------- ------------------------------ ------------- + RBB+0x10 Count Interface count RO - + -------- ------------- ------------------------------ ------------- + RBB+0x14 Stride Interface stride RO - + -------- ------------- ------------------------------ ------------- + RBB+0x18 CSR offset Interface CSR offset RO - + ======== ============= ============================== ============= + +See :ref:`rb_overview` for definitions of the standard register block header fields. + +.. object:: Offset + + The offset field contains the offset to the start of the first interface region, relative to the start of the current region. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x0C Offset to first interface RO - + ======== ============================== ============= + +.. object:: Count + + The count field contains the number of interfaces. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x10 Interface count RO - + ======== ============================== ============= + +.. object:: Stride + + The stride field contains the size of the region for each interface. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x14 Interface stride RO - + ======== ============================== ============= + +.. object:: CSR offset + + The CSR offset field contains the offset to the head of the register block chain inside of each interface's region. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x18 Interface CSR offset RO - + ======== ============================== ============= diff --git a/docs/source/rb/if_ctrl.rst b/docs/source/rb/if_ctrl.rst new file mode 100644 index 000000000..41d6e284c --- /dev/null +++ b/docs/source/rb/if_ctrl.rst @@ -0,0 +1,119 @@ +.. _rb_if_ctrl: + +================================ +Interface control register block +================================ + +The interface control register block has a header with type 0x0000C001, version 0x00000200, and contains several interface-level control registers. + +.. table:: + + ======== ============= ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ======== ============= ====== ====== ====== ====== ============= + RBB+0x00 Type Vendor ID Type RO 0x0000C001 + -------- ------------- -------------- -------------- ------------- + RBB+0x04 Version Major Minor Patch Meta RO 0x00000200 + -------- ------------- ------ ------ ------ ------ ------------- + RBB+0x08 Next pointer Pointer to next register block RO - + -------- ------------- ------------------------------ ------------- + RBB+0x0C Features Interface feature bits RO - + -------- ------------- ------------------------------ ------------- + RBB+0x10 Max TX MTU Max TX MTU RO - + -------- ------------- ------------------------------ ------------- + RBB+0x14 Max RX MTU Max RX MTU RO - + -------- ------------- ------------------------------ ------------- + RBB+0x18 TX MTU TX MTU RW - + -------- ------------- ------------------------------ ------------- + RBB+0x1C RX MTU RX MTU RW - + -------- ------------- ------------------------------ ------------- + RBB+0x20 RSS mask RSS mask RW 0x00000000 + ======== ============= ============================== ============= + +See :ref:`rb_overview` for definitions of the standard register block header fields. + +.. object:: Features + + The features field contains all of the interface-level feature bits, indicating the state of various optional features that can be enabled via Verilog parameters during synthesis. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x0C Interface feature bits RO - + ======== ============================== ============= + + Currently implemented feature bits: + + .. table:: + + === ======================= + Bit Feature + === ======================= + 0 RSS + 4 PTP timestamping + 8 TX checksum offloading + 9 RX checksum offloading + 10 RX flow hash offloading + === ======================= + +.. object:: Max TX MTU + + The max TX MTU field contains the maximum frame size on the transmit path, as configured via Verilog parameters during synthesis. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x10 Max TX MTU RO - + ======== ============================== ============= + +.. object:: Max RX MTU + + The max RX MTU field contains the maximum frame size on the receive path, as configured via Verilog parameters during synthesis. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x14 Max RX MTU RO - + ======== ============================== ============= + +.. object:: TX MTU + + The TX MTU field controls the maximum frame size on the transmit path. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x18 TX MTU RW - + ======== ============================== ============= + +.. object:: RX MTU + + The RX MTU field controls the maximum frame size on the receive path. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x1C RX MTU RW - + ======== ============================== ============= + +.. object:: RSS mask + + The RSS mask field controls which receive queues will be selected by the computed RSS flow hash. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x20 RSS mask RW 0x00000000 + ======== ============================== ============= diff --git a/docs/source/rb/index.rst b/docs/source/rb/index.rst new file mode 100644 index 000000000..f7e9664ef --- /dev/null +++ b/docs/source/rb/index.rst @@ -0,0 +1,94 @@ +.. _rb_overview: + +=============== +Register blocks +=============== + +The NIC register space is constructed from a linked list of register blocks. Each block starts with a header that contains type, version, and next header fields. Blocks must be DWORD aligned in the register space. All fields must be naturally aligned. All pointers in the register blocks are relative to the start of the region. The list is terminated with a next pointer of 0x00000000. See :numref:`tbl_rb_list` for a list of all currently-defined register blocks. + +.. table:: + + ======== ============ ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ======== ============ ====== ====== ====== ====== ============= + RBB+0x00 Type Vendor ID Type RO - + -------- ------------ -------------- -------------- ------------- + RBB+0x04 Version Major Minor Patch Meta RO - + -------- ------------ ------ ------ ------ ------ ------------- + RBB+0x08 Next pointer Pointer to next register block RO - + ======== ============ ============================== ============= + +.. object:: Type + + The type field consists of a vendor ID in the upper 16 bits, and the sub type in the lower 16 bits. Vendor ID 0x0000 is used for all standard register blocks used by Corundum. See :numref:`tbl_rb_list` for a list of all currently-defined register blocks. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x00 Vendor ID Type RO - + ======== ============== ============== ============= + +.. object:: Version + + The version field consists of four fields, major, minor, patch, and meta. Version numbers must be changed when backwards-incompatible changes are made to register blocks. See :numref:`tbl_rb_list` for a list of all currently-defined register blocks. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x04 Major Minor Patch Meta RO - + ======== ====== ====== ====== ====== ============= + +.. object:: Next pointer + + The next pointer field contains a block-relative offset to the start of the header of the next register block in the chain. A next pointer of 0x00000000 indicates the end of the chain. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x08 Pointer to next register block RO - + ======== ============================== ============= + +.. _tbl_rb_list: +.. table:: List of all currently-defined register blocks + + ========== ========== ====================== + Type Version Block + ========== ========== ====================== + 0x00000000 \- :ref:`rb_null` + 0xFFFFFFFF 0x00000100 :ref:`rb_fw_id` + 0x0000C000 0x00000100 :ref:`rb_if` + 0x0000C001 0x00000200 :ref:`rb_if_ctrl` + 0x0000C002 0x00000200 port + 0x0000C003 0x00000100 :ref:`rb_sched_block` + 0x0000C004 0x00000100 stats + 0x0000C010 0x00000100 :ref:`rb_cqm_event` + 0x0000C020 0x00000100 :ref:`rb_qm_tx` + 0x0000C021 0x00000100 :ref:`rb_qm_rx` + 0x0000C030 0x00000100 :ref:`rb_cqm_tx` + 0x0000C031 0x00000100 :ref:`rb_cqm_rx` + 0x0000C040 0x00000100 :ref:`rb_sched_rr` + 0x0000C050 0x00000100 :ref:`rb_sched_ctrl_tdma` + 0x0000C060 0x00000100 :ref:`rb_tdma_sch` + 0x0000C080 0x00000100 :ref:`rb_phc` + 0x0000C081 0x00000100 :ref:`rb_phc_perout` + 0x0000C100 0x00000100 :ref:`rb_gpio` + 0x0000C110 0x00000100 :ref:`rb_i2c` + 0x0000C120 0x00000100 :ref:`rb_flash_spi` + 0x0000C121 0x00000100 :ref:`rb_flash_bpi` + 0x0000C140 0x00000100 :ref:`rb_bmc_alveo` + 0x0000C141 0x00000100 :ref:`rb_bmc_gecko` + 0x0000C150 0x00000100 :ref:`rb_drp` + ========== ========== ====================== + +.. toctree:: + :maxdepth: 1 + :hidden: + :glob: + + * diff --git a/docs/source/rb/null.rst b/docs/source/rb/null.rst new file mode 100644 index 000000000..f46c4669d --- /dev/null +++ b/docs/source/rb/null.rst @@ -0,0 +1,21 @@ +.. _rb_null: + +=================== +Null register block +=================== + +The null register block has a header with type 0x00000000 and no additional fields after the header. + +.. table:: + + ======== ============ ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ======== ============ ====== ====== ====== ====== ============= + RBB+0x00 Type Vendor ID Type RO 0x00000000 + -------- ------------ -------------- -------------- ------------- + RBB+0x04 Version Major Minor Patch Meta RO - + -------- ------------ ------ ------ ------ ------ ------------- + RBB+0x08 Next pointer Pointer to next register block RO - + ======== ============ ============================== ============= + +See :ref:`rb_overview` for definitions of the standard register block header fields. diff --git a/docs/source/rb/phc.rst b/docs/source/rb/phc.rst new file mode 100644 index 000000000..557cfc213 --- /dev/null +++ b/docs/source/rb/phc.rst @@ -0,0 +1,163 @@ +.. _rb_phc: + +================================= +PTP hardware clock register block +================================= + +The PTP hardware clock register block has a header with type 0x0000C080, version 0x00000100, and carries several control registers for the PTP clock. + +.. table:: + + ======== ============== ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ======== ============== ====== ====== ====== ====== ============= + RBB+0x00 Type Vendor ID Type RO 0x0000C080 + -------- -------------- -------------- -------------- ------------- + RBB+0x04 Version Major Minor Patch Meta RO 0x00000100 + -------- -------------- ------ ------ ------ ------ ------------- + RBB+0x08 Next pointer Pointer to next register block RO - + -------- -------------- ------------------------------ ------------- + RBB+0x0C Control Control RO - + -------- -------------- ------------------------------ ------------- + RBB+0x10 Current time Current time (fractional ns) RO - + -------- -------------- ------------------------------ ------------- + RBB+0x14 Current time Current time (ns) RO - + -------- -------------- ------------------------------ ------------- + RBB+0x18 Current time Current time (sec, lower 32) RO - + -------- -------------- ------------------------------ ------------- + RBB+0x1C Current time Current time (sec, upper 32) RO - + -------- -------------- ------------------------------ ------------- + RBB+0x20 Get time Get time (fractional ns) RO - + -------- -------------- ------------------------------ ------------- + RBB+0x24 Get time Get time (ns) RO - + -------- -------------- ------------------------------ ------------- + RBB+0x28 Get time Get time (sec, lower 32) RO - + -------- -------------- ------------------------------ ------------- + RBB+0x2C Get time Get time (sec, upper 32) RO - + -------- -------------- ------------------------------ ------------- + RBB+0x30 Set time Set time (fractional ns) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x34 Set time Set time (ns) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x38 Set time Set time (sec, lower 32) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x3C Set time Set time (sec, upper 32) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x40 Period Period (fractional ns) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x44 Period Period (ns) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x48 Nominal period Nominal period (fractional ns) RO - + -------- -------------- ------------------------------ ------------- + RBB+0x4C Nominal period Nominal period (ns) RO - + -------- -------------- ------------------------------ ------------- + RBB+0x50 Adj time Adj time (fractional ns) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x54 Adj time Adj time (ns) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x58 Adj time count Adj time cycle count RW - + -------- -------------- ------------------------------ ------------- + RBB+0x5C Adj time act Adj time active RO - + ======== ============== ============================== ============= + +See :ref:`rb_overview` for definitions of the standard register block header fields. + +.. object:: Current time + + The current time registers read the current time from the PTP clock, with no double-buffering. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x10 Current time (fractional ns) RO - + -------- ------------------------------ ------------- + RBB+0x14 Current time (ns) RO - + -------- ------------------------------ ------------- + RBB+0x18 Current time (sec, lower 32) RO - + -------- ------------------------------ ------------- + RBB+0x1C Current time (sec, upper 32) RO - + ======== ============================== ============= + +.. object:: Get time + + The get time registers read the current time from the PTP clock, with all values latched coincident with reading the fractional ns register. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x20 Get time (fractional ns) RO - + -------- ------------------------------ ------------- + RBB+0x24 Get time (ns) RO - + -------- ------------------------------ ------------- + RBB+0x28 Get time (sec, lower 32) RO - + -------- ------------------------------ ------------- + RBB+0x2C Get time (sec, upper 32) RO - + ======== ============================== ============= + +.. object:: Set time + + The set time registers set the current time on the PTP clock, with all values latched coincident with writing the upper 32 bits of the seconds field. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x30 Set time (fractional ns) RW - + -------- ------------------------------ ------------- + RBB+0x34 Set time (ns) RW - + -------- ------------------------------ ------------- + RBB+0x38 Set time (sec, lower 32) RW - + -------- ------------------------------ ------------- + RBB+0x3C Set time (sec, upper 32) RW - + ======== ============================== ============= + +.. object:: Period + + The period registers control the period of the PTP clock, with all values latched coincident with writing the ns field. The period value is accumulated into the PTP clock on every clock cycle. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x40 Period (fractional ns) RW - + -------- ------------------------------ ------------- + RBB+0x44 Period (ns) RW - + ======== ============================== ============= + +.. object:: Nominal period + + The nominal period registers contain the nominal period of the PTP clock, which corresponds to zero frequency offset in the ideal case. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x48 Nominal period (fractional ns) RO - + -------- ------------------------------ ------------- + RBB+0x4C Nominal period (ns) RO - + ======== ============================== ============= + +.. object:: Adjust time + + The adjust time registers can be used to slew the clock over some time period. An adjustment can be specified with some amount of time added every clock cycle for N cycles. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x50 Adj time (fractional ns) RW - + -------- ------------------------------ ------------- + RBB+0x54 Adj time (ns) RW - + -------- ------------------------------ ------------- + RBB+0x58 Adj time cycle count RW - + -------- ------------------------------ ------------- + RBB+0x5C Adj time active RO - + ======== ============================== ============= diff --git a/docs/source/rb/phc_perout.rst b/docs/source/rb/phc_perout.rst new file mode 100644 index 000000000..7ccb99434 --- /dev/null +++ b/docs/source/rb/phc_perout.rst @@ -0,0 +1,134 @@ +.. _rb_phc_perout: + +================================ +PTP period output register block +================================ + +The PTP period output register block has a header with type 0x0000C081, version 0x00000100, and carries several control registers for the PTP period output module. + +.. table:: + + ======== ============== ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ======== ============== ====== ====== ====== ====== ============= + RBB+0x00 Type Vendor ID Type RO 0x0000C081 + -------- -------------- -------------- -------------- ------------- + RBB+0x04 Version Major Minor Patch Meta RO 0x00000100 + -------- -------------- ------ ------ ------ ------ ------------- + RBB+0x08 Next pointer Pointer to next register block RO - + -------- -------------- ------------------------------ ------------- + RBB+0x0C Control Control RW - + -------- -------------- ------------------------------ ------------- + RBB+0x10 Start time Start time (fractional ns) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x14 Start time Start time (ns) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x18 Start time Start time (sec, lower 32) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x1C Start time Start time (sec, upper 32) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x20 Period Period (fractional ns) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x24 Period Period (ns) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x28 Period Period (sec, lower 32) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x2C Period Period (sec, upper 32) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x30 Width Width (fractional ns) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x34 Width Width (ns) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x38 Width Width (sec, lower 32) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x3C Width Width (sec, upper 32) RW - + ======== ============== ============================== ============= + +See :ref:`rb_overview` for definitions of the standard register block header fields. + +.. object:: Control + + The control register contains several control and status bits relating to the operation of the period output module. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x0C Control RW - + ======== ============================== ============= + + .. table:: + + === ======== + Bit Function + === ======== + 0 Enable + 8 Pulse + 16 Locked + 24 Error + === ======== + + The enable bit enables/disables output of the period output module. Note that this bit does not cause the module to lose lock when clear, only to stop generating pulses. + + The pulse bit reflects the current output of the PTP period output module. + + The locked bit indicates that the period output module has locked on to the current PTP time and is ready to generate pulses. The output is disabled while the period output module is unlocked, so it is not necessary to wait for the module to lock before enabling the output. The module will unlock whenever the start time, period, or width setting is changed. + + The error bit indicates that the period output module came out of lock due to the PTP clock being stepped. The error bit is self-clearing on either reacquisition of lock or a setting change. + + The period output module keeps track of the times for the next rising edge and next falling edge. Initially, it starts with the specified start time for the rising edge, and start time plus width for the falling edge. If the computed next rising edge time is in the past, the period will be added and it will be checked again, repeating this process until the next rising edge is in the future. Note that the period is added once per clock cycle, so it is recommended to compute a start time that is close to the current time, particularly when using a small period setting, so that the period output module can lock quickly. + +.. object:: Start time + + The start time registers determine the absolute start time for the output waveform (rising edge), with all values latched coincident with writing the upper 32 bits of the seconds field. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x10 Start time (fractional ns) RW - + -------- ------------------------------ ------------- + RBB+0x14 Start time (ns) RW - + -------- ------------------------------ ------------- + RBB+0x18 Start time (sec, lower 32) RW - + -------- ------------------------------ ------------- + RBB+0x1C Start time (sec, upper 32) RW - + ======== ============================== ============= + +.. object:: Period + + The period registers control the period of the output waveform (rising edge to rising edge), with all values latched coincident with writing the upper 32 bits of the seconds field. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x20 Period (fractional ns) RW - + -------- ------------------------------ ------------- + RBB+0x24 Period (ns) RW - + -------- ------------------------------ ------------- + RBB+0x28 Period (sec, lower 32) RW - + -------- ------------------------------ ------------- + RBB+0x2C Period (sec, upper 32) RW - + ======== ============================== ============= + +.. object:: Width + + The width registers control the width of the output waveform (rising edge to falling edge), with all values latched coincident with writing the upper 32 bits of the seconds field. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x30 Width (fractional ns) RW - + -------- ------------------------------ ------------- + RBB+0x34 Width (ns) RW - + -------- ------------------------------ ------------- + RBB+0x38 Width (sec, lower 32) RW - + -------- ------------------------------ ------------- + RBB+0x3C Width (sec, upper 32) RW - + ======== ============================== ============= diff --git a/docs/source/rb/qm_rx.rst b/docs/source/rb/qm_rx.rst new file mode 100644 index 000000000..d43cbe7ca --- /dev/null +++ b/docs/source/rb/qm_rx.rst @@ -0,0 +1,86 @@ +.. _rb_qm_rx: + +===================================== +Receive queue manager register block +===================================== + +The receive queue manager register block has a header with type 0x0000C021, version 0x00000100, and indicates the location of the receive queue manager registers and number of queues. + +.. table:: + + ======== ============= ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ======== ============= ====== ====== ====== ====== ============= + RBB+0x00 Type Vendor ID Type RO 0x0000C021 + -------- ------------- -------------- -------------- ------------- + RBB+0x04 Version Major Minor Patch Meta RO 0x00000100 + -------- ------------- ------ ------ ------ ------ ------------- + RBB+0x08 Next pointer Pointer to next register block RO - + -------- ------------- ------------------------------ ------------- + RBB+0x0C Offset Offset to queue manager RO - + -------- ------------- ------------------------------ ------------- + RBB+0x10 Count Queue count RO - + -------- ------------- ------------------------------ ------------- + RBB+0x14 Stride Queue control register stride RO 0x00000020 + ======== ============= ============================== ============= + +See :ref:`rb_overview` for definitions of the standard register block header fields. + +.. object:: Offset + + The offset field contains the offset to the start of the receive queue manager region, relative to the start of the current region. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x0C Offset to queue manager RO - + ======== ============================== ============= + +.. object:: Count + + The count field contains the number of queues. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x10 Queue count RO - + ======== ============================== ============= + +.. object:: Stride + + The stride field contains the size of the control registers associated with each queue. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x14 Queue control register stride RO 0x00000020 + ======== ============================== ============= + +Queue manager CSRs +================== + +Each queue has several associated control registers, detailed in this table: + +.. table:: + + ========= ============== ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ========= ============== ====== ====== ====== ====== ============= + Base+0x00 Base address L Ring base address (lower 32) RW - + --------- -------------- ------------------------------ ------------- + Base+0x04 Base address H Ring base address (upper 32) RW - + --------- -------------- ------------------------------ ------------- + Base+0x08 Control 1 Active Size RW - + --------- -------------- ------ ------ ------ ------ ------------- + Base+0x0C Control 2 CQ index RW - + --------- -------------- -------------- -------------- ------------- + Base+0x10 Head pointer Head pointer RW - + --------- -------------- -------------- -------------- ------------- + Base+0x14 Tail pointer Tail pointer RW - + ========= ============== ============== ============== ============= diff --git a/docs/source/rb/qm_tx.rst b/docs/source/rb/qm_tx.rst new file mode 100644 index 000000000..9c7e87316 --- /dev/null +++ b/docs/source/rb/qm_tx.rst @@ -0,0 +1,86 @@ +.. _rb_qm_tx: + +===================================== +Transmit queue manager register block +===================================== + +The transmit queue manager register block has a header with type 0x0000C020, version 0x00000100, and indicates the location of the transmit queue manager registers and number of queues. + +.. table:: + + ======== ============= ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ======== ============= ====== ====== ====== ====== ============= + RBB+0x00 Type Vendor ID Type RO 0x0000C020 + -------- ------------- -------------- -------------- ------------- + RBB+0x04 Version Major Minor Patch Meta RO 0x00000100 + -------- ------------- ------ ------ ------ ------ ------------- + RBB+0x08 Next pointer Pointer to next register block RO - + -------- ------------- ------------------------------ ------------- + RBB+0x0C Offset Offset to queue manager RO - + -------- ------------- ------------------------------ ------------- + RBB+0x10 Count Queue count RO - + -------- ------------- ------------------------------ ------------- + RBB+0x14 Stride Queue control register stride RO 0x00000020 + ======== ============= ============================== ============= + +See :ref:`rb_overview` for definitions of the standard register block header fields. + +.. object:: Offset + + The offset field contains the offset to the start of the transmit queue manager region, relative to the start of the current region. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x0C Offset to queue manager RO - + ======== ============================== ============= + +.. object:: Count + + The count field contains the number of queues. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x10 Queue count RO - + ======== ============================== ============= + +.. object:: Stride + + The stride field contains the size of the control registers associated with each queue. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x14 Queue control register stride RO 0x00000020 + ======== ============================== ============= + +Queue manager CSRs +================== + +Each queue has several associated control registers, detailed in this table: + +.. table:: + + ========= ============== ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ========= ============== ====== ====== ====== ====== ============= + Base+0x00 Base address L Ring base address (lower 32) RW - + --------- -------------- ------------------------------ ------------- + Base+0x04 Base address H Ring base address (upper 32) RW - + --------- -------------- ------------------------------ ------------- + Base+0x08 Control 1 Active Size RW - + --------- -------------- ------ ------ ------ ------ ------------- + Base+0x0C Control 2 CQ index RW - + --------- -------------- -------------- -------------- ------------- + Base+0x10 Head pointer Head pointer RW - + --------- -------------- -------------- -------------- ------------- + Base+0x14 Tail pointer Tail pointer RW - + ========= ============== ============== ============== ============= diff --git a/docs/source/rb/sched_block.rst b/docs/source/rb/sched_block.rst new file mode 100644 index 000000000..d26df7577 --- /dev/null +++ b/docs/source/rb/sched_block.rst @@ -0,0 +1,35 @@ +.. _rb_sched_block: + +============================== +Scheduler block register block +============================== + +The scheduler block register block has a header with type 0x0000C003, version 0x00000100, and indicates the offset to the scheduler block control registers. + +.. table:: + + ======== ============= ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ======== ============= ====== ====== ====== ====== ============= + RBB+0x00 Type Vendor ID Type RO 0x0000C003 + -------- ------------- -------------- -------------- ------------- + RBB+0x04 Version Major Minor Patch Meta RO 0x00000100 + -------- ------------- ------ ------ ------ ------ ------------- + RBB+0x08 Next pointer Pointer to next register block RO - + -------- ------------- ------------------------------ ------------- + RBB+0x0C Offset Offset to scheduler block CSRs RO - + ======== ============= ============================== ============= + +See :ref:`rb_overview` for definitions of the standard register block header fields. + +.. object:: Offset + + The offset field contains the offset to the start of scheduler block control registers, relative to the start of the current region. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x0C Offset to scheduler block CSRs RO - + ======== ============================== ============= diff --git a/docs/source/rb/sched_ctrl_tdma.rst b/docs/source/rb/sched_ctrl_tdma.rst new file mode 100644 index 000000000..4c8015c47 --- /dev/null +++ b/docs/source/rb/sched_ctrl_tdma.rst @@ -0,0 +1,127 @@ +.. _rb_sched_ctrl_tdma: + +======================================== +TDMA scheduler controller register block +======================================== + +The TDMA scheduler controller register block has a header with type 0x0000C050, version 0x00000100, and indicates the location of the scheduler controller in the register space, as well as containing some control, status, and informational registers. + +.. table:: + + ======== ============= ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ======== ============= ====== ====== ====== ====== ============= + RBB+0x00 Type Vendor ID Type RO 0x0000C050 + -------- ------------- -------------- -------------- ------------- + RBB+0x04 Version Major Minor Patch Meta RO 0x00000100 + -------- ------------- ------ ------ ------ ------ ------------- + RBB+0x08 Next pointer Pointer to next register block RO - + -------- ------------- ------------------------------ ------------- + RBB+0x0C Offset Offset to scheduler RO - + -------- ------------- ------------------------------ ------------- + RBB+0x10 CH count Channel count RO - + -------- ------------- ------------------------------ ------------- + RBB+0x14 CH stride Channel stride RO - + -------- ------------- ------------------------------ ------------- + RBB+0x18 Control Control RW 0x00000000 + -------- ------------- ------------------------------ ------------- + RBB+0x1C TS count TS count RW - + ======== ============= ============================== ============= + +See :ref:`rb_overview` for definitions of the standard register block header fields. + +.. object:: Offset + + The offset field contains the offset to the start of the scheduler, relative to the start of the current region. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x0C Offset to scheduler RO - + ======== ============================== ============= + +.. object:: Channel count + + The channel count field contains the number of channels. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x10 Channel count RO - + ======== ============================== ============= + +.. object:: Channel stride + + The channel stride field contains the size of the region for each channel. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x14 Channel stride RO - + ======== ============================== ============= + +.. object:: Control + + The control field contains scheduler-related control bits. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x18 Control RW 0x00000000 + ======== ============================== ============= + +.. object:: Timeslot count + + The timeslot count register contains the number of time slots supported. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x1C Timeslot count RO - + ======== ============================== ============= + +TDMA scheduler controller CSRs +============================== + +Each scheduler control channel has several associated control registers, detailed in this table: + +.. table:: + + ========= ============== ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ========= ============== ====== ====== ====== ====== ============= + Base+0x00 Enable bits Enable bits RW - + --------- -------------- ------------------------------ ------------- + Base+N Enable bits Enable bits RW - + ========= ============== ============================== ============= + +.. object:: Enable bits + + The enable bits field contains per-timeslot channel enable bits. + + .. table:: + + ========= ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ========= ====== ====== ====== ====== ============= + Base+0x00 Enable bits RW 0x00000000 + ========= ============================== ============= + + .. table:: + + === ================= + Bit Function + === ================= + 0 Timeslot 0 enable + N Timeslot N enable + === ================= diff --git a/docs/source/rb/sched_rr.rst b/docs/source/rb/sched_rr.rst new file mode 100644 index 000000000..e496e047a --- /dev/null +++ b/docs/source/rb/sched_rr.rst @@ -0,0 +1,136 @@ +.. _rb_sched_rr: + +==================================== +Round-robin scheduler register block +==================================== + +The round-robin scheduler register block has a header with type 0x0000C040, version 0x00000100, and indicates the location of the scheduler in the register space, as well as containing some control, status, and informational registers. + +.. table:: + + ======== ============= ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ======== ============= ====== ====== ====== ====== ============= + RBB+0x00 Type Vendor ID Type RO 0x0000C040 + -------- ------------- -------------- -------------- ------------- + RBB+0x04 Version Major Minor Patch Meta RO 0x00000100 + -------- ------------- ------ ------ ------ ------ ------------- + RBB+0x08 Next pointer Pointer to next register block RO - + -------- ------------- ------------------------------ ------------- + RBB+0x0C Offset Offset to scheduler RO - + -------- ------------- ------------------------------ ------------- + RBB+0x10 CH count Channel count RO - + -------- ------------- ------------------------------ ------------- + RBB+0x14 CH stride Channel stride RO 0x00000004 + -------- ------------- ------------------------------ ------------- + RBB+0x18 Control Control RW 0x00000000 + -------- ------------- ------------------------------ ------------- + RBB+0x1C Dest Dest RW - + ======== ============= ============================== ============= + +See :ref:`rb_overview` for definitions of the standard register block header fields. + +.. object:: Offset + + The offset field contains the offset to the start of the scheduler, relative to the start of the current region. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x0C Offset to scheduler RO - + ======== ============================== ============= + +.. object:: Channel count + + The channel count field contains the number of channels. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x10 Channel count RO - + ======== ============================== ============= + +.. object:: Channel stride + + The channel stride field contains the size of the region for each channel. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x14 Channel stride RO 0x00000004 + ======== ============================== ============= + +.. object:: Control + + The control field contains scheduler-related control bits. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x18 Control RW 0x00000000 + ======== ============================== ============= + + .. table:: + + === ======== + Bit Function + === ======== + 0 Enable + === ======== + +.. object:: Dest + + The dest field controls the destination port and traffic class of the scheduler. It is initialized with the scheduler's index with traffic class 0. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x1C Dest RW - + ======== ============================== ============= + +Round-robin scheduler CSRs +========================== + +Each scheduler channel has several associated control registers, detailed in this table: + +.. table:: + + ========= ============== ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ========= ============== ====== ====== ====== ====== ============= + Base+0x00 Control Control RW 0x00000000 + ========= ============== ============================== ============= + +.. object:: Control + + The control field contains scheduler-related control bits. + + .. table:: + + ========= ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ========= ====== ====== ====== ====== ============= + Base+0x00 Control RW 0x00000000 + ========= ============================== ============= + + .. table:: + + === ============= + Bit Function + === ============= + 0 Enable + 1 Global enable + 2 Control enable + 16 Active + 24 Scheduled + === ============= diff --git a/docs/source/rb/tdma_sch.rst b/docs/source/rb/tdma_sch.rst new file mode 100644 index 000000000..bdc3837bb --- /dev/null +++ b/docs/source/rb/tdma_sch.rst @@ -0,0 +1,210 @@ +.. _rb_tdma_sch: + +============================= +TDMA scheduler register block +============================= + +The TDMA scheduler register block has a header with type 0x0000C060, version 0x00000100, and carries several control registers for the TDMA scheduler module. + +.. table:: + + ======== ============== ====== ====== ====== ====== ============= + Address Field 31..24 23..16 15..8 7..0 Reset value + ======== ============== ====== ====== ====== ====== ============= + RBB+0x00 Type Vendor ID Type RO 0x0000C060 + -------- -------------- -------------- -------------- ------------- + RBB+0x04 Version Major Minor Patch Meta RO 0x00000100 + -------- -------------- ------ ------ ------ ------ ------------- + RBB+0x08 Next pointer Pointer to next register block RO - + -------- -------------- ------------------------------ ------------- + RBB+0x0C TS count Timeslot count RO - + -------- -------------- ------------------------------ ------------- + RBB+0x10 Control Control RW - + -------- -------------- ------------------------------ ------------- + RBB+0x14 Status Status RO - + -------- -------------- ------------------------------ ------------- + RBB+0x20 Sch start Sch start time (fractional ns) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x24 Sch start Sch start time (ns) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x28 Sch start Sch start time (sec, lower 32) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x2C Sch start Sch start time (sec, upper 32) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x30 Sch period Sch period (fractional ns) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x34 Sch period Sch period (ns) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x38 Sch period Sch period (sec, lower 32) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x3C Sch period Sch period (sec, upper 32) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x40 TS period TS period (fractional ns) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x44 TS period TS period (ns) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x48 TS period TS period (sec, lower 32) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x4C TS period TS period (sec, upper 32) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x50 Active period Active period (fractional ns) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x54 Active period Active period (ns) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x58 Active period Active period (sec, lower 32) RW - + -------- -------------- ------------------------------ ------------- + RBB+0x5C Active period Active period (sec, upper 32) RW - + ======== ============== ============================== ============= + +See :ref:`rb_overview` for definitions of the standard register block header fields. + +.. object:: Timeslot count + + The timeslot count register contains the number of time slots supported. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x0C Timeslot count RO - + ======== ============================== ============= + +.. object:: Control + + The control register contains several control bits relating to the operation of the TDMA scheduler module. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x10 Control RW - + ======== ============================== ============= + + .. table:: + + === ======== + Bit Function + === ======== + 0 Enable + === ======== + +.. object:: Status + + The control register contains several status bits relating to the operation of the TDMA scheduler module. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x14 Status RO - + ======== ============================== ============= + + .. table:: + + === ======== + Bit Function + === ======== + 0 Locked + 1 Error + === ======== + +.. object:: Schedule start time + + The schedule start time registers determine the absolute start time for the schedule, with all values latched coincident with writing the upper 32 bits of the seconds field. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x20 Sch start time (fractional ns) RW - + -------- ------------------------------ ------------- + RBB+0x24 Sch start time (ns) RW - + -------- ------------------------------ ------------- + RBB+0x28 Sch start time (sec, lower 32) RW - + -------- ------------------------------ ------------- + RBB+0x2C Sch start time (sec, upper 32) RW - + ======== ============================== ============= + +.. object:: Schedule period + + The schedule period registers control the period of the schedule, with all values latched coincident with writing the upper 32 bits of the seconds field. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x30 Sch period (fractional ns) RW - + -------- ------------------------------ ------------- + RBB+0x34 Sch period (ns) RW - + -------- ------------------------------ ------------- + RBB+0x38 Sch period (sec, lower 32) RW - + -------- ------------------------------ ------------- + RBB+0x3C Sch period (sec, upper 32) RW - + ======== ============================== ============= + +.. object:: Timeslot period + + The timeslot period registers control the period of each time slot, with all values latched coincident with writing the upper 32 bits of the seconds field. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x40 TS period (fractional ns) RW - + -------- ------------------------------ ------------- + RBB+0x44 TS period (ns) RW - + -------- ------------------------------ ------------- + RBB+0x48 TS period (sec, lower 32) RW - + -------- ------------------------------ ------------- + RBB+0x4C TS period (sec, upper 32) RW - + ======== ============================== ============= + +.. object:: Active period + + The active period registers control the active period of each time slot, with all values latched coincident with writing the upper 32 bits of the seconds field. + + .. table:: + + ======== ====== ====== ====== ====== ============= + Address 31..24 23..16 15..8 7..0 Reset value + ======== ====== ====== ====== ====== ============= + RBB+0x50 Active period (fractional ns) RW - + -------- ------------------------------ ------------- + RBB+0x54 Active period (ns) RW - + -------- ------------------------------ ------------- + RBB+0x58 Active period (sec, lower 32) RW - + -------- ------------------------------ ------------- + RBB+0x5C Active period (sec, upper 32) RW - + ======== ============================== ============= + +TDMA timing parameters +====================== + +The TDMA schedule is defined by several parameters - the schedule start time, schedule period, timeslot period, and timeslot active period. This figure depicts the relationship between these parameters:: + + schedule + start + | + V + |<-------- schedule period -------->| + -----+--------+--------+--------+--------+--------+--- + | SLOT 0 | SLOT 1 | SLOT 2 | SLOT 3 | SLOT 0 | + -----+--------+--------+--------+--------+--------+--- + |<------>| + timeslot + period + + + |<-------- timeslot period -------->| + -----+-----------------------------------+------------ + | SLOT 0 | SLOT 1 + -----+-----------------------------------+------------ + |<---- active period ----->| + +The schedule start time is the absolute start time. Each subsequent schedule will start on a multiple of the schedule period after the start time. Each schedule starts on timeslot 0, and advances to the next timeslot each timeslot period. The timeslot active period is the active period for each timeslot, forming a guard period at the end of the timeslot. It is recommended that the timeslot period divide evenly into the schedule period, but rounding errors will not accumulate as the schedule period takes precedence over the timeslot period. Similarly, the timeslot period takes precedence over the timeslot active period. diff --git a/docs/source/tuning.rst b/docs/source/tuning.rst new file mode 100644 index 000000000..d8a6fc6b2 --- /dev/null +++ b/docs/source/tuning.rst @@ -0,0 +1,101 @@ +.. _tuning: + +================== +Performance Tuning +================== + +Here are some tips and tricks to get the best possible performance with Corundum. + +First, it's always a good idea to test with a commercial 100G NIC as a sanity check - if a commercial 100G NIC doesn't run near 100G line rate, then Corundum will definitely have issues. + +Second, check the PCIe configuration with ``lspci``. You'll want to make sure that the card is actually running with the full PCIe bandwidth. Some x16 PCIe slots don't have all of the lanes physically wired, and in many cases lanes can be switched between slots depending on which slots are used---for example, two slots may share 16 lanes, if the second slot is empty, the first slot will use all 16 lanes, but if the second slot has a card installed, both slots will run with 8 lanes. + +This is what ``lspci`` reports for Corundum in a gen 3 x16 configuration in a machine with an AMD EPYC 7302P CPU:: + + $ sudo lspci -d 1234:1001 -vvv + 81:00.0 Ethernet controller: Device 1234:1001 + Subsystem: Silicom Denmark Device a00e + Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ + Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- 0002) + [ 349.460802] mqnic 0000:81:00.0: Control BAR size: 16777216 + [ 349.462723] mqnic 0000:81:00.0: Configured 32 IRQs + +Note that ``lspci`` reports ``LnkSta: Speed 8GT/s (ok), Width x16 (ok)``, indicating that the link is running at the max supported speed and max supported link width. If one of those is reported as ``(degraded)``, then further investigation is required. If ``(ok)`` or ``(degraded)`` is not shown, then compare ``LnkSta`` with ``LnkCap`` to see if ``LnkSta`` reports lower values. In this case, ``lspci`` reports ``LnkCap: Port #0, Speed 8GT/s, Width x16``, which matches ``LnkSta``. It also reports ``MSI: Enable+ Count=32/32``, indicating that all 32 MSI channels are active. Some motherboards do not fully implement MSI and limit devices to a single channel. Eventually, Corundum will migrate to MSI-X to mitigate this issue, as well as support more interrupt channels. Also note that ``lspci`` reports ``MaxPayload 512 bytes``---this is the largest that I have seen so far (on AMD EPYC), most modern systems report 256 bytes. Obviously, the larger, the better in terms of PCIe overhead. + +Any processes that access the NIC should be pinned to the NIC's NUMA node. The NUMA node is shown both in the ``lspci`` and driver output output (``NUMA node: 3``) and it can be read from sysfs (``/sys/class/net//device/numa_node``). Use ``numactl -N `` to run programs on that NUMA node. For example, ``numactl -N 3 iperf3 -s``. If you're testing with ``iperf``, you'll want to run both the client and the server on the correct NUMA node. + +It's also advisable to go into BIOS setup and disable any power-management features to get the system into its highest-performance state. + +Notes on the performance evaluation for the FCCM paper: the servers used are Dell R540 machines with dual Intel Xeon 6138 CPUs and all memory channels populated, and ``lspci`` reports ``MaxPayload 256 bytes``. The machines have two NUMA nodes, so only one CPU is used for performance evaluation to prevent traffic from traversing the UPI link (can use Intel PCM to confirm use of the correct NUMA node). On these machines, a single ``iperf`` process would run at 20-30 Gbps with 1500 byte MTU, or 40-50 Gbps with 9000 byte MTU. The Corundum design for those tests was configured with 8192 TX queues and 256 RX queues.