Verification with Chisel and UVM

This repo is for the project to explore the combination and interaction of Chisel and UVM. The ultimate goal is a verification framework within Scala for digital hardware described in Chisel also supporting legacy components in VHDL, Verilog, or SystemVerilog.

Run tests with

make

UVM Examples

In the sv directory, a number of UVM examples are located.

Simple examples

In sv/uvm-simple-examples a number of simple examples are located. These start with a very basic testbench with no DUT attached, and gradually transition into a complete testbench.

Vivado UVM Examples

These examples assume that a copy of Xilinx Vivado is installed and present in the PATH. The examples are currently tested only on Linux.

• The first example is taken from Vivado Design Suite Tutorial - Logic Simulation

Leros ALU

In the directory sv/leros, the Leros ALU is tested using UVM, to showcase that Chisel and UVM can work together. This testbench is reused to also test a VHDL implementation of the ALU, to show that UVM is usable on mixed-language designs (when using a mixed-language simulator).

The VHDL implementaion is run by setting the makefile argument TOP=top_vhd.

Using the SV DPI and Javas JNI

Using the SystemVerilog DPI (Direct Programming Interface) to cosimulate with a golden model described in C is explored in the scoreboard_dpi.svh file. The C-model is implemented in scoreboard.c, and the checking functionality is called from the SystemVerilog code.

Implementing a similar functionality in Scala/Chisel has been explored via the JNI (Java Native Interface). In the directory native, the necessary code for a simple Leros tester using the JNI is implemented.

To use the JNI functionality, first run make jni to generate the correct header files and shared libraries. Then, open sbt and type project native to access the native project. Then run sbt test to test the Leros ALU using a C model called from within Scala. To switch back, type project chisel-uvm.

Functional Coverage in Chisel

The idea is to implement functional coverage features from SystemVerilog such as covergroup, coverpoint and bins directly in Chisel.

Work

The interesting part of the code can be found in Functional_coverage_test/src/test/scala/coverage and a test use case can be found in Functional_coverage_test/src/test/scala/gcd/GcdTesters2.scala

What is done

So far a basic version of a verification plan has been implemented. This allows one to define constructs similar to SystemVerilog's covergroup, coverpoint and bins. The structure of the system can be seen in the diagram below.

Coverage Reporter

This is the heart of the system. It handles everything from registering the Cover Points to managing the Coverage DataBase. It will also generate the final coverage report. Registering Cover Points together will group them into a same Cover Group.

Coverage DB

This DataBase handles the maintenance of the values that were sampled for each of the Cover Point bins. This allows us to know how much of the verification plan was tested.
The DB also keeps mappings linking Cover Groups to their contained Cover Points.

How to use it

The Functional coverage system is compatible with the chisel testers2 framework.

1. The CoverageReporter must first be instanciated within a chisel test.
2. CoverGroups can then be created by using the register method of the coverage reporter. This takes as parameter a list of CoverPoints that contain a port that will be sampled, a portName that will be shown in the report and a List[Bins] created from a name and a scala range.
3. The port must then be manually sampled by calling the sample method of the coverage reporter.
4. Once the test is done, a coverage report can be generated by calling the printReport or report methods of the coverage reporter.

An example of this can be found here.

Timed Cross Coverage

Idea: We want to check the relationship between two ports with a delay of a certain amount of cycles.
Example: We have the following situation, imagine we have a device that breaks if:

• dut.io.a takes the value of 1.U at cycle 1
• dut.io.b takes the value of 1.U the following cycle

We want to verify that the above case was tested. This can be done by defining a TimedCross between the two points:

        val cr = new CoverageReporter(dut)
        cr.register(
            //Declare CoverPoints
            CoverPoint(dut.io.a, "a")())::
            CoverPoint(dut.io.b, "b")())::
            Nil,
            //Declare timed cross point with a delay of 1 cycle
            TimedCross("timedAB", "a", "b", Exactly(1))(
                CrossBin("both1", 1 to 1, 1 to 1)::Nil)::
            Nil)

Using that, we can check that we tested the above case in our test suite.
This construct can be used to check delay between two cover points.

Use

To be able to use the timed coverage, stepping the clock must be done through the coverage reporter:

dut.clock.step(nCycles) //Will trigger an exception if used with Timed Cross Coverage
cr.step(nCycles) //Works

This is done in order ensure that the coverage database will always remain synchronized with the DUT's internal clock.

Delay Types

The current implementation allows for the following special types of timing:

• Eventually: This sees if a cross hit was detected at any point in the next given amount of cycles.
• Always: This only considers a hit if the it was detected every cycle in the next given amount of cycles.
• Exactly: This only considers a hit if it was detected exactly after a given amount of cycles.

Constraint Random Verification

The CRV package inside this project aims to mimic the functionality of SystemVerilog constraint programming and integrates them into chisel-tester2. The CRV package combines a Constraint Satisfactory Problem Solver, with some helper classes to create and use random objects inside your tests. Currently, only the jacop backend is supported, but in the future other backends can be added.

Comparison

System Verilog

class frame_t;
rand pkt_type ptype;
rand integer len;
randc bit [1:0] no_repeat;
// Constraint the members
constraint legal {
  len >= 2;
  len <= 5;
}

CRV / jacop backend

class Frame extends RandObj(new Model) {
  val pkType: Rand = new Rand(0, 3)
  val len: Rand = new Rand(0, 10)
  val noRepeat: Randc = new Randc(0, 1)

  val legal: ConstraintGroup = new ConstraintGroup {
    len #>= 2
    len #<= 5
  }
}

Random Objects

Random objects can be created by extending the RandObj trait. This class accepts one parameter which is a Model. A model correspond to a database in which all the random variable and constraint declared inside the RandObj are stored.

class Frame extends RandObj(new Model)

A model can be initialized with a seed new Model(42), which allows the user to create reproducible tests.

Random Fields

A random field can be added to a RandObj by declaring a Rand variable.

  val len: Rand = new Rand(0, 10)

Random-cyclic variable can be added by declaring a Randc field inside a RandObj

  val noRepeat: Randc = Randc(0, 1)

Constraints

Each variable can have one or multiple constraint. Constraint relations are usually preceded by the # symbol.

len #>= 2

In the previous block of code we are specifying that the variable len can only take values that are grater then 2. Each constraint can be assigned to a variable and enabled or disabled at any time during the test

val lenConstraint = len #> 2
[....]
lenConstraint.disable()
[....]
lenConstraint.enable()

Constraint can also be grouped together in a ConstraintGrup and the group itself can be enabled or disabled.

val legal: ConstraintGroup = new ConstraintGroup {
  len #>= 2
  len #<= 5
  payload.size #= len
}
[...]
legal.disable()
[...]
legal.enable()

By default, constraints and constraints groups are enabled when they are declared.

The list of operator used to construct constraint is the following: #<, #<=, #>, #>=,#=, div, #*, mod, #+, -, #\=, #^, in, inside

It is also possible to declare conditional constraints with constructors like IfCon and IfElseCon.

val constraint1: crv.Constraint = IfCon(len #= 1) {
        payload.size #= 3
    } ElseC {
        payload.size #= 10
    }

Usage

As in SystemVerilog, each random class exposes a method called randomize() this method automatically solves the constraint specified in the class and assign to each random filed a random value. The method returns true only if the CSP found a set of values that satisfy the current constraints.

val myPacket = new Frame(new Model)
assert(myPacket.randomize)

Other usage examples can be found in src/test/scala/backends/jacopsrc/test/scala/verifyTests/crv/backends/jacop/

Example Use Cases

We will explore a handful of use cases to explore verification.

• Leros ALU (basically done)
• Heap priority queue (from MicroSemi), see also https://www.hackerearth.com/practice/notes/heaps-and-priority-queues/
• Network-on-chip (in Chisel), see https://github.com/schoeberl/soc-comm
• Decimation filter from WSA (VHDL code plus testbench given)

Resources

If you're interested in learning more about the UVM, we recommend that you explore the repository, as well as some of the following links:

• First steps with UVM
• UVM Cookbook (requires an account)
• ChipVerify.com UVM Tutorials
• Ray Salemi's UVM Primer videos

Documents

Collect pointers to relevant documents.

General Verification Documents

OVM Documents

Related Work

• https://github.com/ekiwi/paso
• https://github.com/TsaiAnson/verif
• Layering RTL, SAFL, Handel-C and BluespecConstructs on Chisel HCL, David J Greaves, see http://koo.corpus.cam.ac.uk/drafts/tndjg-008-transactional-modelling-in-chisel.html

CRV

• Choco-Solver Java library for solving CSP problems

Testing Framewrok / Simulation tools

Cocotb -- coroutine based cosimulation python library library for hardware development in Python

Cocotb repository: cocotb is a coroutine based cosimulation library for writing VHDL and Verilog testbenches in Python.

Resources Related to Cocotb

• Philipp Wagner (FOSSi Foundation, lowRISC) "Cocotb: Python-powered hardware verification" (WOSH 2019) (video)
• Ben Rosser (University of Pennsylvania) "Cocotb: a Python-based digital logic verification framework" (CERN 2018) (pdf)
• Torbjørn Viem Ness (NTNU) "Low Power Floating-Point Unit for RISC-V" (2018) [PDF]
• Andrey Filippov (Elphel) "I will not have to learn SystemVerilog" (2016) [Blog]
• Chris Higgs (Potential Ventures) "Applying agile techniques to FPGA development" Video, Paper
• Chris Higgs (Potential Ventures) "Rapid FPGA Verification" (NMI, February 2014) [Slides]
• Smith, Andrew Michael; Mayo, Jackson; Armstrong, Robert C.; Schiek, Richard; Sholander, Peter E.; Mei, Ting (Sandia National Lab): "Digital/Analog Cosimulation using CocoTB and Xyce" (paper)

Extension of Cocotb

• cocotb-coverage: Extension that enables coverage and constrained random verification
  • Publication in iEEE Paper
• python-uvm: port of SystemVerilog (SV) Universal Verification Methodology (UVM) 1.2 to Python and cocotb

Hwt -- Python library for hardware development

hwt: one of the golas of this library is to implement some simulation feature similar to UVM

Not strictly relevant resources

• CRAVE: An advanced constrained random verification environment for SystemC

• EnrichingUVM in SystemC with AMS extensions for randomization and functional coverage

• Coverage directed test generation for functional verification using Bayesian network

• Introducing XCS to Coverage Directed test Generation

• LiveHD LiveHD is an infrastructure designed for Live Hardware Development. By live, we mean that small changes in the design should have the synthesis and simulation results in a few seconds, as the fast interactive systems usually response in sub-second.