User Tools
Sidebar
project-wiki:design_reviews:opportunity_development_review
Table of Contents
Opportunity Development Review
Opportunity Development approval is granted after a successful review of a high-quality, transferable set of requirements and top-level plans to meet those requirements. This information is captured in a set of artifacts. The quality and transferability are evaluated by a careful assessment of the materials submitted by the team, followed by the Opportunity Development design review.
At the review, the team will provide the following:
- A presentation no longer than five minutes that includes the following:
- A summary of the Requirements Matrix, Software Requirements Specification or other Requirement Specification
- A demonstration of the prototypes used to understand the problem, communicate with the market, communicate with the sponsor, get up to speed on relevant technology, or otherwise understand the challenge, the market requirements, and the relevant performance measures
- Project Milestones Table
- The Team Charter
Following the presentation, the reviewers and the team will discuss the quality and transferability of the decisions captured in the submitted artifacts. Suggestions for improvement will be considered. The objective of the review is to ensure the team has a solid understanding of the design challenge and any relevant technology in order to support effective and appropriate concept development.
Artifact Submission for Opportunity Development
The artifacts submitted for the Opportunity Development review should answer the questions, “What are the design objectives and the complete product requirements? What are the schedule and resources allocated to achieve these objectives?” and, “How have you validated that the product requirements and Primary Artifacts are likely to be feasible and truly represent the market's and the sponsor's desires?” To answer these questions, the team should submit a pdf file containing the following artifacts:
- Primary Artifacts:
- Project Background. A brief description of the sponsor, the design challenges, and the project that allows the reader to understand what you are tasked with doing and why it is important.
- System Requirements Matrix with sections A–D completed, organized according to the Goal Pyramid on pages 230–31 of Product Development. You may find the template found on Box under Class Documents/Templates to be helpful in preparing your Requirements Matrix.
Be sure that your Requirements Matrix is consistent with the guidelines given on pages 262--63, 274--75, and 276--79 of Product Development.
For some products, especially software-only products, it may be better to use a Software Requirements Specification instead of a Requirements Matrix. Please consult with your coach and your pod instructor if you feel this applies to your team. - A Project Milestones Table that describes the stages of development to be completed, the deliverables required for each stage, the intended completion date for each stage, and the anticipated budget for each stage. It should also list other significant milestones that will help you reach your stage review readiness. This table should include significant project milestones specific to your project (i.e., not just the dates of Capstone reviews).
- Requirement validation results indicating the sponsor's and/or market's acceptance of the final requirements as listed in the Requirements Matrix. Refer to other Primary Artifacts and Supporting Artifacts as necessary.
- Supporting Artifacts should include one or more of the following, or other similar artifacts:
- Competitive and/or market benchmarking procedures and results
- Definition of Market Representatives
- Personas and/or storyboards
- White papers or other technical reports that relate to the project
- Reports on prototypes used to help the team, the market, and the sponsor better understand the challenges and goals of the project. If no prototypes are used during Opportunity Development, it is difficult or impossible to obtain high-quality results.
- Validation procedures used to obtain market feedback on completed requirements
- Results of requirement validation procedures used with market representatives
- Statement from sponsor indicating specific agreement with all of the market requirements, performance measures, and ideal values.
NOTE: Validation requires that the team obtain specific information from
the sponsor and/or the market that indicates agreement with the requirements as written. The team cannot provide validation with
only their own judgment.
Evaluation
The submitted artifacts will be assessed for how clearly and completely they answer the questions above. The requirements and validation included in the artifacts will also be assessed for their quality.
The rubrics used in the evaluation is shown below. There are different rubrics for hardware or mixed projects and software projects. Discuss with your pod instructor which rubrics you will use.
The scoresheets used by the evaluators are available on Box for hardware or mixed projects and for software projects.
Evaluation rubric for Opportunity Development stage
| Element | Item(weight) | Excellent (9-10) | Good (7-8) | Poor (0-6) |
|---|---|---|---|---|
| Primary Artifacts – reviewers evaluate all | Project Background (0.5) | Clearly and succintly presents the background of the project such that a technically literate reader with no knowledge of the project can understand the importance and the challenges of the project. | Presents the background of the project sufficiently to motivate reading the other artifacts. | Is confusing or hard to read. Provides little sense of importance of the project or technical challenges that may exist. Hard to see why anyone would want to do the project. |
| Project Objective Statement (0.5) | Clearly and succinctly describes the scope, schedule, and resources for the project. Is thoughtful and specific to the project. | Describes the scope, schedule, and resources for the project. Is generic or otherwise not carefully thought out. | Is missing one or more of the essential items. Shows little or no understanding of the problem. | |
| System Requirements Matrix or other Requirements Specification (2) | Matrix contains Market Requirements, Performance Measures, Requirement-Measurement Relationships, and Ideal Values. Meets all of the criteria for the requirements matrix information found in Table 4.1 or Table A.2 of the textbook. If used, an alternate Requirement Specification document is professionally and thoughtfully prepared and thouroughly captures the requirements for the project. Matrix or other specification is under effective revision control. Artifact has been checked by someone other than the author. Artifact is included in vector or text (rather than bitmap) form so it can be zoomed up and read easily. All elements of specification are thoughtful and appear correct. | Matrix or other specification document is mostly well done, but falls short of the listed criteria in one or two areas. Some elements of the specification appear to be superficial. Commonly we would see lots of subjective or arbitrary-scale performance measures in this range; this shows the team has not really seriously thought about how to measure performance. One or two necessary requirements are missing. Revision control is superficial. | Matrix or other specification document has some serious weaknesses that raise questions about its utility. In the opinion of the reviewer, the market requirements are a poor reflection of the actual market desires. In the opinion of the reviewer, the requirement specification is not sufficient to ensure successful project completion even if all of the listed specifications are met. Revision control is missing. | |
| Project Milestone Table (0.5) | Table contains appropriate milestones for the project, including all required reviews and project-specific milestones that will help the project move forward. Table is thoughtfully and appropriately prepared. Effectively under revision control. | Table contains milestones for all required reviews. Table is not specific to the particular design project. Revision control is superficial. | Table is missing, or is very superficial. Does not include all the required reviews. Revision control is missing. | |
| Requirements Validation Results (1.5) | Appropriate efforts to validate the performance measures with the market and/or the sponsor, and the requirements matrix or other requirement specifications with the sponsor are clearly presented. There is clear evidence that specific feedback has been obtained showing the desirability of the requirements. The artifact contains test procedures and results (or links to supporting artifacts with test procedures and results) that clearly show the data obtained and the procedures used to obtain the data. The artifact is properly under revision control. | Report summarizes efforts to obtain feedback from the market and/or sponsor concerning the requirements. The evidence for external validation is present, but not fully compelling. Revision control is superficial. | Report is missing, or is primarily focused on team evaluation of the the requirements, rather than on external evaluation by the market and/or sponsor. Revision control is missing. | |
| Supporting Artifacts – reviewers spot check | Market interaction artifacts (1.5) | Artifacts reporting on interactions with the market, such as market surveys, focus groups, or interviews (including both procedures and results) are carefully and clearly presented. Enough detail is given to allow a third-party to repeat the interactions. Artifacts are under effective revision control. | Artifacts reporting on interactions with the market, such as market surveys, focus groups, or interviews (including both procedures and results) are presented. Enough detail is given to allow a third-party to repeat the interactions by using some judgment. Revision control is superficial. | Artifacts reporting on interactions with the market, such as market surveys, focus groups, or interviews (including both procedures and results) are missing or poorly presented. Little or no detail about the market interactions is given. Revision control is missing. |
| Prototyping artifacts (1.5) | Artifacts reporting on the making and using of prototypes to communicate with the market and sponsor and/or get up to speed on relevant technology are carefully and clearly presented. Enough detail is given to allow a third-party to repeat the use (but not the creation) of the prototypes. Prototypes are appropriate for their intended use. Prototypes have helped the team get real about the project. Artifacts are under effective revision control. | Artifacts on prototypes are presented. Prototypes are somewhat appropriate for the intended use. Prototypes seem to have reasonable purposes. A third party could recreate the use of the prototypes with the use of some engineering judgment. Revision control is superficial. | No artifacts on making and using prototypes are presented. If used, prototypes are used in inappropriate ways. Prototypes seem to have little purpose related to understanding or advancing the design. Little or no information is given about how the prototypes are used. Revision control is missing. | |
| Validation Artifacts (1.5) | Detailed artifacts showing methods and results of obtaining validation from the market and/or the sponsor, if needed to support the requirements validation results, are clearly and carefully presented. Artifacts are under effective revision control. | The evidence for external validation is present, but not fully compelling. Arifacts are somewhat unclear; significant judgment is required to interpret them. Revision control is superficial. | Necessary artifacts are missing, or are primarily focused on team evaluation of the the requirements, rather than on external evaluation by the market and/or sponsor. Revision control is missing. | |
| Formatting (0.5) | Is carefully and thoughtfully formatted. Bookmarks and/or hyperlinks are provided to help the reader navigate the document. Is free from grammatical errors. If printed reports are desired by sponsor, complies with printed report formatting guidelines. | Mostly complies with formatting requirements. One or two documents need to be fixed to comply with formatting requirements. Has a few grammatical errors. | Shows little or no evidence of following formatting requirements. Haphazard formatting, difficult to read. Has many grammatical errors. | |
| Rubric revision 2.5 made from ScoreSheetMaker revision 1.5 | ||||
Evaluation rubric for Opportunity Development stage (Software Projects)
| Element | Item(weight) | Excellent (9-10) | Good (7-8) | Poor (0-6) |
|---|---|---|---|---|
| Primary Artifacts – reviewers evaluate all | Project Background (0.5) | Clearly and succintly presents the background of the project such that a technically literate reader with no knowledge of the project can understand the importance and the challenges of the project. | Presents the background of the project sufficiently to motivate reading the other artifacts. | Is confusing or hard to read. Provides little sense of importance of the project or technical challenges that may exist. Hard to see why anyone would want to do the project. |
| Project Objective Statement (0.5) | Clearly and succinctly describes the scope, schedule, and resources for the project. Is thoughtful and specific to the project. | Describes the scope, schedule, and resources for the project. Is generic or otherwise not carefully thought out. | Is missing one or more of the essential items. Shows little or no understanding of the problem. | |
| Software Requirements Specification (SRS) (2) (2) | Software Requirements Specification is professionally and thoughtfully prepared and thoroughly captures the requirements for the project. Requirements are clearly written. SRS is under effective revision control. Artifact has been checked by someone other than the author. All elements of specification are thoughtful and appear correct. | SRS is mostly well done but falls short of the listed criteria in one or two areas. Some elements of the specification appear to be superficial. Lacks some of the necessary requirements to ensure project success. Revision control is superficial. | SRS has some serious weaknesses that raise questions about its utility. In the opinion of the reviewer, the market requirements are a poor reflection of the actual market desires. In the opinion of the reviewer, the requirement specification is not sufficient to ensure successful project completion even if all of the listed specifications are met. Revision control is missing. | |
| Project Milestone Table (0.5) | Table contains appropriate milestones for the project, including all required reviews and project-specific milestones that will help the project move forward. Table is thoughtfully and appropriately prepared and may be a high-resolution, readable image from team’s project management software. Effectively under revision control. | Table contains milestones for all required reviews. Table is not specific to the project and only contains Capstone assignment deadlines. Revision control is superficial. | Table is missing or is very superficial. Does not include all the required reviews. Revision control is missing. | |
| Requirements Validation Results (1.5) | Appropriate efforts to validate the requirements with the market and/or the sponsor. There is clear evidence that specific feedback has been obtained showing the desirability of the requirements. The artifact contains test procedures and results (or links to supporting artifacts with test procedures and results) that clearly show the data obtained and the procedures used to obtain the data. The artifact is properly under revision control. | Report summarizes efforts to obtain feedback from the market and/or sponsor concerning the requirements. The evidence for external validation is present, but not fully compelling. Revision control is superficial. | Report is missing or is primarily focused on team evaluation of the the requirements, rather than on external evaluation by the market and/or sponsor. Revision control is missing. | |
| Supporting Artifacts – reviewers spot check | Market interaction artifacts (1.5) | Artifacts reporting on interactions with the market, such as market surveys, focus groups, or interviews (including both procedures and results) are carefully and clearly presented. Enough detail is given to allow a third-party to repeat the interactions. Artifacts are under effective revision control. | Artifacts reporting on interactions with the market, such as market surveys, focus groups, or interviews (including both procedures and results) are presented. Enough detail is given to allow a third-party to repeat the interactions by using some judgment. Revision control is superficial. | Artifacts reporting on interactions with the market, such as market surveys, focus groups, or interviews (including both procedures and results) are missing or poorly presented. Little or no detail about the market interactions is given. Revision control is missing. |
| Prototyping artifacts (1.5) | Artifacts reporting on the making and using of prototypes to communicate with the market and sponsor and/or get up to speed on relevant technology are carefully and clearly presented. Enough detail is given to allow a third-party to repeat the use (but not the creation) of the prototypes. Prototypes are appropriate for their intended use. Prototypes have helped the team get real about the project. Artifacts are under effective revision control. | Artifacts on prototypes are presented. Prototypes are somewhat appropriate for the intended use. Prototypes seem to have reasonable purposes. A third party could recreate the use of the prototypes with the use of some engineering judgment. Revision control is superficial. | No artifacts on making and using prototypes are presented. If used, prototypes are used in inappropriate ways. Prototypes seem to have little purpose related to understanding or advancing the design. Little or no information is given about how the prototypes are used. Revision control is missing. | |
| Validation Artifacts (1.5) | Detailed artifacts showing methods and results of obtaining validation from the market and/or the sponsor, if needed to support the requirements validation results, are clearly and carefully presented. Artifacts are under effective revision control. | The evidence for external validation is present, but not fully compelling. Arifacts are somewhat unclear; significant judgment is required to interpret them. Revision control is superficial. | Necessary artifacts are missing or are primarily focused on team evaluation of the the requirements, rather than on external evaluation by the market and/or sponsor. Revision control is missing. | |
| Formatting (0.5) | Is carefully and thoughtfully formatted. Bookmarks and/or hyperlinks are provided to help the reader navigate the document. Is free from grammatical errors. If printed reports are desired by sponsor, complies with printed report formatting guidelines. | Mostly complies with formatting requirements. One or two documents need to be fixed to comply with formatting requirements. Has a few grammatical errors. | Shows little or no evidence of following formatting requirements. Haphazard formatting, difficult to read. Has many grammatical errors. | |
| Rubric revision 0.4 made from ScoreSheetMaker revision 1.5 | ||||
Mechanics
The mechanics of Opportunity Development reviews are as follows:
- No later than 10:00 AM three class days (not counting weekends or holidays) ahead of the scheduled review, the team submits the artifact package, as described above. The submission is done from the Capstone dashboard. The review coaches and the pod instructor will receive an email when the submission is completed. If the submission is late, the evaluating coaches will not provide a grade for the initial submission.
- No later than 8:00 AM one class day (not counting weekends or holidays) ahead of the scheduled review, the two coaches who are assigned to review the team will complete their grading and written evaluation of the artifact package. These evaluations will be available for download from the Capstone dashboard. If the initial submission was late, grades will not be available, and the written evaluations may not be available before the review.
- The team prepares an oral presentation no more than five minutes in length that includes the project objective statement and discusses why the team believes they are ready for stage approval, including responses to the evaluator comments where appropriate.
- At the review, the team will make their brief presentation and present prototypes used to understand, generate and/or validate requirements.
- The reviewers, the team, and the coach will discuss the answers to the questions as contained in the artifacts and the presentation for 18 minutes. Weaknesses in the answers or the artifacts will be explored, and recommendations for improvement will be given.
- After consulting with the review coaches, the instructor will spend two minutes to communicate the results of the review to the team. Possible results are:
- Stage approval is granted unconditionally.
- Stage approval is conditionally granted, subject to the team making some required revisions. The team must clear the revisions with their pod instructor before approval is granted. The team presents the revised submission directly to the pod instructor for approval. After receiving approval, the team submits the revised submission on the Capstone dashboard.
- Stage approval is denied. The team will need to improve the design and/or the evaluation, schedule a new review, and return to step 1.
- When the team has received stage approval, they proceed to Architecture Development, beginning with the Concept Review.
Formatting
The Opportunity Development submission should be formatted according to the instructions in Artifact Formatting.
If your sponsor requires a bound printed report, follow the additional instructions in Formatting Printed Reports.
project-wiki/design_reviews/opportunity_development_review.txt · Last modified: 2023/08/19 21:11 by bdj2
Except where otherwise noted, content on this wiki is licensed under the following license: Copyright Brigham Young University, all rights reserved
