|
Ch
1 - Plan from Hell |
REFERENCE - THE JANE SERIES CHAPTER 6 - SETTING GOALS |
|||
|
"I remember you mentioned something about defining our verification quality goals. I've put a little thought into it and I'd like to see what you think." "So what have you come up with?" Josh answered. "Well, I'd really like to set a high standard for the team, something that will keep them motivated and aiming for excellence during the project. I'm thinking about aiming the team at a 0-bug goal! My management would like to see me set aggressive goals, so it really gives us credit all over. So what do you think about this?" Jane asked, pleased with her ambitious goal. "OK, let me answer by asking some leading questions." Jane smiled; Josh had a knack for answering a question with another question. "Has your group ever succeeded in delivering a perfect design to fabrication?", he asked. "Well, we were pretty close a few projects ago on the THX50, there were only some minor bugs and we found a way to work-around the bugs in software, so it didn't delay shipping of the product," she answered. "If you discovered these bugs after freezing the data-base, would it have been worth delaying the tape-out to fix them?" he continued. "No, not really, since the overall project would have suffered a delay," she answered. "So, are you saying that for the THX50 project, time-to-market (TTM) was more important than achieving a 0-bug tape-out?" "I guess, but it still doesn't mean that we shouldn't set 0-bugs as a goal," she answered. "But, what you are saying is that TTM was the ultimate goal of the verification of the project." "Correct," she said. "And when there is a trade-off between bugs and TTM, will you choose TTM?" "Yes." "What about your
current project? How important is 0-bugs to that project? And how much
of a schedule hit would your division be willing to take to make that
happen?" "So who's going to decide which bugs might get left inside?" Josh asked. "Well, we won't really know, since we will not have uncovered those bugs yet when we need to tape-out." "So your division, absolutely needs to have a demo of this product to survive in the THX market segment, and you won't really know if you have a high enough quality level to demo the product until it comes back from fabrication." "Well, that's not entirely true; we will be focusing on some of the main-path configuration." "But with a goal of 0 bugs, how will your verification engineers know to focus on what's important for the project at this stage?" "They are intelligent people, I'm sure they'll know how to prioritize their work." "So decisions on which features need to be tested, at what resolution and quality level will be driven by each of the engineers? The same engineers who have a goal of 0-bugs." "OK. I see your point. But what are you suggesting, I ask them to aim low?" "No, not at all. What I suggest is looking at the goal from the project's perspective. When you tell your group to set a 0-bug goal, what you're telling them is 'do your best', but that's not necessarily helping the project to achieve the project goals." "What does a less aggressive goal mean?" "It means concentrating verification effort on the features that are the most important for achieving the goal. For example, if you choose that your goal is to sample the first silicon, then you will need to identify with the project management and marketing, what features in which configuration are required for that to happen. You need to focus the effort first and foremost to achieve the coverage within those constraints and to perform significant random testing of those features. That way the debug effort and the verification are being concentrated where it will make the most impact." "If I set the goal to be able to demo the TXH2000? Wouldn't that be setting the bar low?" "Not really, what it is doing is creating a framework for how project decisions are made. When you set your goal in-line with the overall project goals you ensure that you know what the project expects the outcome of your work will be. When conflicts arise, 'should I work on feature x or y?' or 'should we delay the tape-out to fix this bug' you will have a framework set up to answer these questions. "I can understand the alignment, but we still would like to have as clean design as possible for the first spin, since we need to follow the demo by delivering samples to customers." "Setting a goal, does not negate overshooting that goal. If you provide silicon which is production worthy on your first time around, that would be great, however if you focus your effort on making it meet production, and miss the demo target you haven't helped your company at all. Additionally, if the production-software isn't ready yet, or other components aren't aligned you won't be able to ship the project early anyway." Jane nodded her head in agreement. "So, could you give me an example of goals that you have set in your projects?" "On my last project we defined the goal as `being production worthy at the first spin.` The design had some minor changes and an additional interface, and we felt we had sufficient verification resources and time to achieve the goal despite the tight schedule. Because this was the goal of the project, when we caught a bug (quite late) that would not have allowed us to go to production, we stopped the tape-out and made the fix. The delay hurt our market position, but the project met its goal. On the project before,
we set our goal as 'Making the first spin be sample-able to our customers,
and the following spin be production-worthy.' The reason we added the
follow-on `the following spin be production-worthy` was because that also
was a framework for making decisions on the first spin. To ensure a reasonable
probability of the second iteration being production-worthy we felt that
all production features should be able to be tested in their basic mode
in a post-silicon testing. This would ensure that "How did you decide that one project could be ready the first time around, and the other would require an additional spin?" "It depends on
the complexity of the changes being made (or designed from scratch), how
many skilled verification and design engineers are available, the time-line
of the project (and its flexibility), and the project goals. Using these
constraints, the team works with the other stakeholders to try to reach
goals we are confident will be reached while remaining aligned to the
project." "Obviously it is challenging to get everyone to agree to the goals, and sometimes this gets a bit sticky. As a verification lead, if you believe the goal dictated is unrealistic, you need to be able to stand firm and state that within resource, time, and complexity constraints you do not have confidence of reaching the quality goal. This should be followed by a discussion on which features can be reduced, what extensions can be made to the time-line, which resources can be added or refinement of the goal. Standing firm on this, can be quite difficult. The alternative, committing to a goal you don't believe in, is much worse for your company and your career." "Josh, do you think you could you summarize that for me?" Jane said, hinting that she might have fallen asleep through the last few lines. "Sure no problem; The purpose of a verification-goal is to set a framework for decision making during a project. Arriving at a realistic goal, in-line with the project's needs helps align the teams' expectations from one another. Setting a lofty goal, which you don't really expect to reach, can be devastating to the overall program." "So when can you tell my boss about this? I don't think he'll like it." Josh leaned over and whispered into her ear, "When we share our salaries, I'll be glad to share your work"... Design right. Verify right. Ace Verification |
||||
|
|
||||
About
Us | Careers | Contact
Us
Copyright ©
2006 Ace Verification Corp. All rights reserved.
