DV Methodology

Chapter 11 of 15

The Test Plan as a Living, Traceable Document

Closing the module: a chain linking spec requirement to plan item to coverage item to actual test, traced against axil_regfile's real open gaps -- and a verification plan that absorbs new information from measurement and from its own planning process, rather than being written once and frozen.

Ch1 through ch4 built a verification plan one pass at a time: what it is, a feature list, a taxonomy of scenarios, a risk ranking. Every one of those was a snapshot, frozen at the moment it was written. This closing chapter is the piece that makes a plan a plan rather than a document that happened once: a mechanism for tracing each item all the way from the spec to an actual test, and a plan that keeps absorbing new information instead of staying exactly as written.

A single chain, per scenario: spec requirement → plan item → coverage item → test. Each link matters on its own, independent of whether the others exist:

  • Spec → plan item: ch2 and ch3's job — a promise the spec makes, decomposed into a concrete scenario.
  • Plan item → coverage item: the SystemVerilog construct that would actually measure whether the scenario ran — a covergroup's coverpoint, a cross, a branch a line/branch report would track. dv-methodology ch2 made the point this connects to directly: coverage is authored, not automatic. Something only gets measured because a plan item told someone to declare a coverpoint for it in the first place — a coverage item floating with no plan item behind it is measuring whatever a covergroup's author happened to think of, not what the spec actually requires.
  • Coverage item → test: the actual test that exercises the scenario, which is what would make the coverage item's number move at all.

A plan item missing its coverage item is a scenario nobody would notice going untested even if a test somehow ran anyway. A coverage item missing its plan item is measuring something nobody decided mattered. Both gaps are invisible from only one end of the chain — which is exactly why tracing the whole thing, not just asking "is there a test," is the point.

Two real items, traced end to end

STATUS/COUNT/an unmapped write (B2/B3, illegal category, ch2-3): the spec requirement is the response-code section uvm-advanced ch1 grounds in the real AXI4-Lite text. The plan item is ch3's illegal-category scenario. The coverage item already exists, in two places — the addr_x_resp cross's legal, currently-unsampled bins (dv-methodology ch2) and the write case's SLVERR/DECERR branches (ch3). The test is the only missing link, and Coverage ch6's waiver log already marked it ready: four lines of directed code, no new infrastructure.

AW and W arriving on different cycles (A5/A6, concurrent category, ch2-3): same first three links — spec requirement (the protocol's independence rule), plan item (ch3), coverage item, this time in two different places (aw_have/w_have's toggle check, ch4 of Coverage; the AW_ONLY/W_ONLY FSM states, ch5 of Coverage). The test is missing here too, but for a different reason: Coverage ch6's waiver log already flagged that closing it needs new driver or sequence capability, not a quick directed addition. Same chain, same missing link, a genuinely different amount of work behind it — exactly the distinction ch4 of this module argued for keeping separate from risk itself.

Both chains have three of four links already built. Neither is finished. Tracing them this way is what makes "not finished" a specific, checkable claim instead of a vague sense that more testing would probably be good.

The plan absorbs three kinds of new information

A plan written once and left alone stops being trustworthy the moment anything downstream of it changes. Three sources of change, all real ones this project has actually produced:

  • Coverage measurement finds a gap the plan didn't have. dv-methodology ch3-5 found the STATUS/COUNT/unmapped-write gap and the aw_have/w_have gap by running actual reports — before this module existed, nothing had formally written either one down as a plan item at all. A living plan absorbs that finding as a new row, not as a fact that stays parked inside a coverage report nobody revisits.
  • The plan's own systematic process finds a gap measurement never isolated. This module's ch2 and ch3 found four of these on their own: the ENABLE-gated DATA write, reading CTRL back, the single-outstanding-transaction restriction (A7), and reset mid-transaction (D2) — none of them caught by any covergroup, branch check, toggle check, or FSM check built anywhere in Coverage ch1-6, because nothing in those chapters happened to isolate any of the four as its own item. Planning and measurement each catch gaps the other doesn't; a living plan is where both kinds land.
  • Priorities shift. This project hasn't produced a real case of this yet — no related bug report or design change has come in to reorder ch4's ranking — but it's the third legitimate reason a plan changes, worth naming even without a concrete example: new information about likelihood or impact (a related bug found elsewhere, a spec revision) can move an item up or down without the scenario itself changing at all.

The living verification plan

Extending Coverage ch6's waiver log with every scenario this module traced, not just the two it already tracked:

Verification Plan Status -- axil_regfile.sv
=====================================================================================================
Scenario                            Priority (ch4)   Coverage item                        Test
-----------------------------------  ---------------  ------------------------------------  --------
AW/W arrive on different cycles      Highest          aw_have/w_have toggle (Cov ch4);      Open --
                                                       AW_ONLY/W_ONLY FSM states (Cov ch5)   backlog
A7: new AW/W while B outstanding     High             none built in Coverage module         Open --
                                                       -- new row, added by this module      new
D2: reset mid-transaction            Notable (impact) none built in Coverage module         Open --
                                                       -- new row, added by this module      new
DATA write while ENABLE=0            Medium           none built in Coverage module         Open --
                                                       -- new row, added by this module      new
STATUS/COUNT/unmapped write          Medium           addr_x_resp cross legal bins,          Open --
                                                       write-case branches (Cov ch2-3)       ready
Reading CTRL back                    Lowest           none built in Coverage module         Open --
                                                       -- new row, added by this module      new

Four of six rows didn't exist as tracked plan items anywhere in this project until this module's own systematic pass found them. That's the concrete shape of "the plan absorbs new information" — not a hypothetical, a table with rows in it that weren't there before ch2 and ch3 of this module ran.

A waiver isn't a dead end

Coverage ch6 marked the AW/W-independence gap "Backlog — needs new driver/sequence capability, not a quick fix" and moved on, because that was the right call for a chapter about reading a coverage report. Under this module's framework, that waiver entry and the top row of the table above are the same fact, viewed from two different documents. "Backlog" was never an ending — it's a plan item with a known priority (ch4's highest ranking), a known reason it's not done yet (missing test infrastructure, not missing understanding), and a place it lives that gets checked again, not a line that quietly stops mattering the moment nobody's looking at that particular report anymore.

When is a plan done?

Never, in the same sense Coverage ch6 argued 100% coverage was never really the goal. A plan's native state is current — traceable, prioritized, and honest about which links in each chain exist and which don't — not final. This module opened with an interview question about how to verify a block given only its spec. The honest answer, after five chapters, isn't a syntax list or a percentage: it's this — a plan that started from the spec, generated its own scenarios systematically, ranked them by what actually matters, and kept changing every time new information showed up, from measurement or from its own next pass. That, not a frozen document, is what "coverage-driven judgment" and "test-planning mindset" turn out to be the same skill underneath.

Summary

  • Traceability is a four-link chain — spec requirement, plan item, coverage item, test — and each link is independently checkable; a broken link anywhere is invisible if you only look from one end of the chain.
  • Tracing STATUS/COUNT/unmapped writes and AW/W independence end to end shows both have three of four links built already, with the same missing link (a test) blocked for genuinely different reasons.
  • A living plan absorbs three kinds of new information: gaps coverage measurement finds that the plan lacked, gaps the plan's own systematic process finds that coverage measurement never isolated, and priority shifts driven by new information.
  • Four of the six scenarios in this module's closing plan table (A7, D2, the ENABLE-gated DATA write, CTRL readback) are new rows this module's own planning process found — proof the plan absorbs information measurement alone never produced.
  • A coverage waiver marked "backlog" isn't an ending; it's a plan item with a known priority and a known reason it isn't done, living in a document that gets revisited, not a fact that quietly stops mattering.

A covergroup exists in the codebase measuring some DUT behavior, but no plan item anywhere documents why that specific behavior was chosen to measure. What's the problem with this, according to the traceability chain this chapter describes?

Four of the six scenarios in this chapter's closing plan table (A7, D2, the ENABLE-gated DATA write, CTRL readback) have no coverage item built anywhere in the Coverage module. What does this demonstrate?

Coverage ch6 marked the AW/W-independence gap 'Backlog.' Under this chapter's framework, what does that waiver entry actually represent?

This chapter closes by arguing a verification plan is never really 'done.' What's the actual target, if not completion?