Agile Methods for ASIC Verification

Why use agile methods?

Agile methods embrace change.

If you invest time in a plan, you surely hope to realize benefits from that investment. A change in the basis of your plan will tend to invalidate that plan. In an attempt to protect your investment, you argue against the change. You explain that if marketing cant decide what product they want, then engineering will never be able to create any product. If the change comes from the design team, then you explain that they must not strive for perfection: they must stabilize the design so that the business can reap the rewards of managing the time to market. You explain that the verification process is already the bottleneck on the critical path of the project. You beg them to make your life easy.

Are these people really trying to ruin your life? The marketing group contains intelligent people: they understand the impact of change upon engineering team. They will only ask for a change if they think that that change will improve the performance of the product in the market. If only because they covet their bonuses, they are not trying to torpedo the business.

The design group also wants to bring the product to market. Every engineer experiences a feeling of pride on seeing his or her design used in a customers product. Even seeing their design pass your tests will give them pleasure. However, if that design is perceived as being poor quality, then they will not be happy. An engineer understands that the design process must end: but within the time available, they want to make the best product that they can. They are human, and make mistakes. An initial design will rarely be the best. A designer must have the flexibility to undo their mistakes, right up to the end. You surely do not want to provide an excuse for releasing a design known to be shoddy: down that road leads low morale, and ultimately ruin.

Verification has been estimated to comprise 70% of the engineering effort. Terms such as verification bottleneck are intended to strike fear into the project manager. To convince them that they must make excessive compromises to ensure that verification problems dont lead to schedule slips. But in making this argument, you reinforce the image of verification as a necessary evil.

This image, of verification as a necessary evil, is one that I do not wish to promote. Verification should be seen as an integral part of the team: improving the product to ensure that engineers see their work used by happy customers and that the sales and marketing team are laughing all the way to the bank. When the dagger of layoffs strikes, then your team will not be seen as the boat anchor they has dragged down the business.

So how can we change the image of verification? How do we meld the perception from being a wart to being a valuable asset? I believe that, not only is it possible, but that doing so will enhance the very act of verifying. Not only will we improve our image: we will improve ourselves.

Wheres the beef?

So how do we bring about this change: in both perception and actuality?

I will propose two tracks to follow. One concentrates on methodology. Its goal is to ensure that verification will affect the schedule only as needed to demonstrate that the design works or to identify things that cause designers to carry out further work. The act of verification, itself, should not be a major consideration. There is, of course, work to be done. Someone must create test environments and models, and that creation is the responsibility of the verification team. But we will not create inertia. Our methodology will embrace change indeed, it will expect and be strengthened by it.

The other track is concerned with the scope of the verification task. The skills of the verification team should be harnessed to create maximum value for the business. We must assume responsibilities that make us positively indispensable in contrast to our current negative indispensability. What responsibilities are these? I will make my suggestions later, but to prime your imagination, let me point out that the verification team should be the group that best understands how to use the product: that this understanding is a necessary component of the verification task, and a valuable asset when a customer struggles.

Methodology Principles

The act of planning is useful: it forces us to understand the scope of our undertakings. But a plan itself is often more of a hindrance than a help. It can constrain our activities, encouraging us not to reopen discussions whose conclusions are embedded in the plan. A plan commits us to a course of action. But plans are not our only source of inertia. Any work product that makes assumptions about the future will tend to lead us in the direction of that future. Most engineers have found themselves in the position of advocating a course of action that feels wrong, but that is forced by the state of the infrastructure on which they build.

In any endeavor in which we build on our previous work, we will see this force in action. It is a fundamental property of evolution. They role of history in the design of every organism alive today cannot be ignored.

We know that our work will always involve adaptations of our previous work. We must embrace this fact. We must not ask to build on a blank sheet: whenever our goals are changed, we must ask ourselves how to build on what we have. We must re-evaluate our work, in the context of our new goals. We must identify which aspects will be strengths in our new context. We must be prepared to continue to build on our newly created perception of our foundations.

If we accept that we will always build upon what we have, then we must build what we have in such a way that it does not overly constrain our efforts. We must resist the temptation to commit to a specific course of evolution. We do not know how our goals will change. Effort expended to bring a specific goal closer will likely not be perceived as a strength if that goal goes away. If we have the conceit to predict the future, then we should consider a career in Las Vegas! At least there, we understand that the house always has an edge.

The preceding arguments suggest that we should write code that is easy to adapt, but which has no specific adaptations. This would be an incorrect interpretation. Our code must always meet its current goals. It must have the functionality needed to support the testing of the RTL that currently exists. But we must avoid hooks. We must avoid writing code that we believe will make our lives easy. This may sound wrong-headed, but the thing we are avoiding is not to make our lives easy: we are avoiding doing things that we believe will. Specifications and RTL implementations will change. The future is unpredictable: and just as in Las Vegas, the house wins in the end.

Writing the code that we need today is enough to keep us busy. We have no need to add to our workload by writing code that we think we will need tomorrow. The software world has developed methodologies that enable us to avoid committing to a specific future. Code that is well factored is evolvable to any future: not just the currently favored: but also the one that will be desired tomorrow.

Scope of Responsibilities

What do verification engineers do well? The answer should be that they understand and can use the product that has been designed. Designers may also have this understanding, but they may be too close to the implementation. The role of the verification engineer is not only to demonstrate that the design works: it is to understand how a customer will use the product: and to see the frustrations that they will encounter.

No business can afford to create a fully staffed applications engineering team before there are customers who need support from that team. But by the time the customers need that support, it is too late to hire and train the team. The verification team can bridge this gap.

I argue that a verification team is the engineering interface between the design team and the customer. The team validates not only the design, but the specification. And not just the specification, but also the requirements. If the verification team cannot find a way to use the design in the context of the marketing teams requirements, then something is surely wrong. After the chip is available for a customer to design into their product, then the teams expertise is called upon to support the initial customers.

The act of verification requires the team to build models of the chip. Any automated test environment requires these models to define the expected behavior of the design. These models may take many forms, and be written in many languages. A model may be realized as a set of assertions in directed tests, or it may be an explicit, cycle accurate, representation of the designs micro architecture. Whatever the form, it must exist independently of the design.

Customers also ask for models of the design. Their requirements are different to those of the verification group, so there is no guarantee that a good verification model will serve as a good model for customers. Indeed, the customer may be happier with a model derived from the design (which is undeniably an accurate representation of the design) than with a model that is merely acted as a reference for comparison.

But in my experience, models created by a verification group that are usable by customers will generally have better characteristics than models that are derived from the RTL design. There are exceptions, but these exceptions provide insight into the requirements of a customer-oriented model.

Creating an Agile Methodology