Design Reviews
Design reviews are an integral part of the product design process. It is a step in the process where a team is able to present and check their product ideas and prototypes against the requirements that have been developed, to see if they are on the right track. It's also a good time to check project scope, as well as get directed to useful resources.
The Challenge requires teams to complete four design reviews: two peer reviews with another teams, and two remote mentor reviews. The requirements and suggestions for the peer reviews are presented below. Presentation requirements are strict, but specific content may vary depending on the team and their progress. General tips on documentation and presentation are given in the Documentation and Presentation section of the course.
All design reviews should be student-led. Team coaches should not be primary presenters in any design review.
Peer Reviews
Barring any exceptional circumstances, these should be done live over a video call, or in person if possible. Peer reviews can be done with more than one other team at the same time to get additional feedback (e.g. three or four teams arrange a video conference and present to each other in turn). If there are multiple teams in the school school/organization, it is preferred that those teams do not do their official peer reviews only with each other (though they should certainly practice with each other, if possible).
Teams must arrange their own peer reviews. We recommend using the Challenge Discord #looking-for-peer-reviewers channel to arrange peer review.
All peer review presentations should be followed by a Q&A period for the presenting team. Presenting teams should designate a scribe who is prepared to write down reviewers' comments.
Remote Mentor Reviews
Because of the number of teams in the Challenge, remote mentor reviews are asynchronous. Teams will record a video that shows their presentation, the speaker(s), and any product demonstrations. Feedback will be given by remote mentors. Challenge staff will try to match mentors and projects with the most appropriate expertise when possible.
Time Zones
All due dates are 11:59:59 PM (23:59:59) Anywhere on Earth (AoE). It means that if every place on Earth has passed the given date, it is past the due date. AoE is UTC-12:00.
Extensions
You may attempt to submit any design review past the due date, though the following consequences apply:
Peer reviews: you may find it difficult to find a team to review with you
Mentor reviews: you will not be guaranteed to receive feedback
Final event: your project will not be considered for the final event
Design review guidelines are below. Expand each section to see the specific guidelines.
White Gate - Preliminary Peer Review (due by February 2nd, 2025)
The purpose of this peer review is to share your team's initial progress with at least one other team. At this point, we nominally expect that teams have done the following:
found a co-designer to work with
conducted an interview with the co-designer (and/or caretakers, family, etc, as appropriate)
come up with a few initial ideas, and perhaps run them by the co-designer, and/or done some low-fidelity prototyping testing.
These steps correspond to the Introduction and most of the Design Processes sections of the course.
Preliminary prototyping often looks like paper prototypes (for user interfaces), or prototypes made of basic materials like cardboard, clay, and duct tape (to check measurements, fit, and movement). These generally should not look like complete products - if they do, you've likely over-engineered for this stage of the product design cycle.
Not every team will be quite at this point - some will be further ahead, and some will need a bit more time. That is okay! For this review, go down the list of requirements as much as you are able, keeping in mind that you are trying to present where your team is at so that you can get external feedback and help.
Presenter Guidelines
Presentation is 7-10 minutes long
Introduce your co-designer (Who are they? What do they like to do? What challenges would they to tackle? Are there any other relevant pieces of information, such as living situation, family, caretaking, profession, etc?)
Discuss the results of your interview – user needs statements, product requirements, any down-selection and justification
Single-sentence product objective
Discuss product risks
Give a visual representation of your product (e.g. drawing, CAD, flow diagram, etc - see FAQ below about visual representations of your co-designer)
Preliminary prototype objective (Which risks or uncertainties were you trying to test/resolve?)
Preliminary prototype picture/video demonstration
Results of testing
Next steps
Officially Supported Teams: Submit the White Gate design review form (provided in Discord, under #design_review_checkoffs) immediately after finishing this design review.
Reviewer Guidelines
If you are a student on another team watching a peer review, you are a reviewer. Here are some things to keep in mind.
Do you have a sense of who the co-designer is as a person?
Did the presenting team conduct an appropriate interview and obtain a set of user needs and product requirements?
Do you have a sense of the direction of the project, and what the team is trying to accomplish? Does it seem appropriate given the co-designer interview?
How is the scope of the project, particularly given the skills of the team? Is it too ambitious? Too simple? (You may need to ask about the team's skillset and resources to answer this question.)
Are the risks and methods of addressing the risks appropriate for this setting?
Did the preliminary prototype appropriately check the initial design ideas for the product? Are there changes they should be making that they didn't mention?
Are there tasks or features that can be simplified or eliminated without affecting the team’s ability to meet the user needs?
Are there resources that the team might not be taking advantage of that you can point them to?
Orange Gate - Preliminary Mentor Review (due by February 9th, 2025)
The purpose of this mentor review is similar to the White Gate review, but should incorporate any revisions and testing made since the initial peer review (White Gate). This review is also a prerequisite for requesting funding from Beaver Works, which also involves providing a bill of materials for your next iteration. Beaver Works will provide up to $125 of project funding based on team needs, though teams may supplement with their own fundraising, as needed.
Presenter Guidelines
Presentation is 7-10 minutes long
Introduce your co-designer (Who are they? What do they like to do? What challenges would they to tackle? Are there any other relevant pieces of information, such as living situation, family, caretaking, profession, etc?)
Discuss the results of your interview – user needs statements, product requirements, any down-selection and justification
Single-sentence product objective
Discuss product risks
Give a visual representation of your product (e.g. drawing, CAD, flow diagram, etc - see FAQ below about visual representations of your co-designer)
Lo-fi prototype objective (Which risks or uncertainties were you trying to test/resolve?)
Lo-fi prototype picture/video demonstration
Results of testing
Bill of materials (if applicable)
Next steps
Submission Guidelines (Officially Supported Teams)
Submit the Orange Gate design review form (provided in Discord, under #design_review_checkoffs), by giving us a link to a YouTube video of your presentation.
Your link may be public or unlisted (but not private).
Make sure that your video follows the rules about co-designer information (in the FAQ below). In particular, make sure that your video only contains personally-identifiable information about your co-designer if they have filled out a media release form (for pictures and video) and you have asked for their permission (for all information).
CRE[AT]E mentors and staff will provide feedback for these reviews asynchronously.
Blue Gate - Critical Peer Review (due by March 9th, 2025)
The purpose of this peer review is to share your team's progress with at least one other team. At this point, we nominally expect that teams have gone through at least two design iterations, getting feedback from their co-designers at each step.
Hi-fidelity prototypes can start to look like final products, and should be moving away from notional or stand-in components. For example, most, if not all hardware components should be made of sturdier materials than cardboard and duct tape, and software wireframes and paper prototypes should be giving way to real software demonstrations. That does not mean all components have to be in a complete state, and having some remaining in a lo-fi state is okay if you haven't gotten to those yet.
Not every team will be quite at this point - some will be further ahead, and some will need a bit more time. That is okay! For this review, go down the list of requirements as much as you are able, keeping in mind that you are trying to present where your team is at so that you can get external feedback and help.
Presenter Guidelines
Go through the guidelines on clear presentation from the course, if you haven't done so already.
Presentation is 7-10 minutes long
Re-introduce your co-designer (Who are they? What do they like to do? What challenges would they to tackle? Are there any other relevant pieces of information, such as living situation, family, caretaking, profession, etc?)
Provide a single-sentence product objective
Briefly discuss your initial idea and previous prototypes, focusing on what you learned from feedback, and any changes you made to your product's direction or requirements.
Discuss any risks you have resolved in previous iterations, and any remaining risks. Remember that user rejection is a risk.
Give a visual representation of your product through its iterations (e.g. drawing, CAD, flow diagram, etc - see FAQ below about visual representations of your co-designer)
If it is important to show your product in relation to a user (e.g. if it is a wearable device or mounted near/on a user), you should include the a figure representing the user and/or related body part(s).
Hi-fi prototype objectives (Which risks or uncertainties were you trying to test/resolve?)
Hi-fi prototype picture/video demonstration
Results of testing
Next steps
Remember that while this is called "high-fidelity" prototype, what that means will vary from team to team, and typically does not represent a finished product
Officially Supported Teams: Submit the Blue Gate design review form (provided in Discord, under #design_review_checkoffs) immediately after finishing this design review.
Reviewer Guidelines
If you are a student on another team watching a peer review, you are a reviewer. Here are some things to keep in mind.
Do you have a sense of who the co-designer is as a person?
Did the presenting team clearly articulate their product's goals and how they will achieve them?
How has the team managed product risk? Did the previous iterations of the product help the team figure things out?
Are appropriate changes being made from iteration to iteration? Are there changes they should be making that they didn't mention?
Are there resources that the team might not be taking advantage of that you can point them to?
Brown Gate - Critical Mentor Review (due by March 16th, 2025)
The purpose of this mentor review is similar to the Blue Gate review. This review is due at the same time as the Blue Gate due to time constraints for review before the final event.
Presenter Guidelines
Same as Blue Gate
Submission Guidelines (Officially Supported Teams)
Submit the Blue Gate design review form (provided in Discord, under #design_review_checkoffs), by giving us a link to a YouTube video of your presentation.
Your link may be public or unlisted (but not private).
Make sure that your video follows the rules about co-designer information (in the FAQ below). In particular, make sure that your video only contains personally-identifiable information about your co-designer if they have filled out a media release form (for pictures and video) and you have asked for their permission (for all information).
CRE[AT]E mentors and staff will provide feedback for these reviews asynchronously.
Black Gate - Final Event Submissions (due by April 13th, 2025 - no extensions)
There are two parts to the final event submission: a document and a video. The Final Event materials are due approximately one week before the event itself to allow for some down-selection for Challenge finalists.
Part 1: Document
The final prototype is a more refined version of your product that has gone through some iterations of design, test, and re-design. It might just be what your co-designer needed, or you may have a ways to go (so it may only be "final" for the purposes of this class). We'd like to know how you made it, and how others could make it as well! Also, please include pictures and videos whenever possible. Videos in the document should be GIFs, since they will auto-play when the document is viewed.
Who is your co-designer?
In your own words, tell us about your co-designer and what you learned from interviewing. And not just about their impairment - who are they as a person? Include anything that your co-designer would be comfortable sharing. Some of these things you may be able to answer before your interview, and others are things you should try to gather information on. Some suggestions:
introductions
living situation
daily routines
hobbies
occupation
any relation to you or others in the community
impairments or AT-related needs
solutions they have already tried
Needs and Requirements
co-designer needs statements
product requirements (derived from needs)
Concept Generation and Selection
Give us a narrative of your brainstorming! Tell us about the following:
How did you clarify the problem?
What did your external search and brainstorming look like? Any particularly notable ideas that you wanted to keep or ended up discarding?
How did you select the concept with which you'll move forward?
Please also tell us briefly about your design iterations, how you went from earlier to more sophisticated prototypes, and what design elements you were testing in earlier iterations.
Tell Us About Your Product
Bill of Materials - What are you using to build your final prototype? Include quantities, order links, and cost breakdown of any purchased materials.
Software Tools - For software projects, please include any existing software frameworks, tools, and languages you used.
"Technical Drawing" - Can be a CAD model, a wireframe, a hand-drawn sketch, a fabric pattern, a wiring diagram, and more! Include multiple views and detailed specifications and/or constraints (sizes, shapes, resistances, weights, etc) when possible. If you're using something like an obj or stl file (anything to be read by specific software), please also include an image of the item(s).
Build Instructions - How would another maker build this? Include any specific pain points you noticed, or any particularly sensitive parts of the design (things that were strongly affected by slight changes). See these Instructables articles on Nico's Full Body Oven Mitt and the Bom Bidet as examples. For software components, in place of a "how to build," please include a full README file that tells the reader what frameworks and languages you used, how to navigate your code, and how to execute your software. Pointers to your GitHub repositories are preferable for software (make sure they're public).
Test Results
Test Plan - What was your prototype test plan? What can you test by yourself, with your co-designer, or with others? Are there specific times or places that you need to pay attention to for testing?
Testing Pictures/Video - If your tester(s) are okay with it, document your tests with pictures and video! Always be sure to ask for permission (and submit a media release form) first, though. If you cannot provide a picture with your co-designer (or it is difficult to do so), please approximate a demonstration of your product to the best of your ability with a member of your team.
Testing Results - What happened? What did you learn? Detail any quantitative ("The plastic sleeve was 2 cm too long") and qualitative ("My co-designer had to strain her neck to reach the spot where her head was supposed to go") observations, as well as any direct user feedback.
Reflections
We'd like you to think about a few things after going through your final prototype.
Future Improvements - What did you learn from this iteration? What worked, and what needs to be tweaked or entirely redesigned? What would you do if you had more time and/or resources?
Scalability - We're doing individual co-design in this course, but can you tell us something about how scalable your design is? How specific is your solution to your co-designer and/or their environment? Are there other people it might help? How do you think you'd go about making 100, 1,000, or 10,000 copies of this? It's perfectly fine if the answer is that your product is completely unscalable and is super-specific to your co-designer. Tell us why, and congratulations on tackling a problem that would likely have otherwise been ignored!
Design Process Reflection - What would you do differently if you had to go through this process again? This part is focused on the process, rather than anything about your product that you learned along the way.
Part 2: Video
Go through the guidelines on clear presentation from the course, if you haven't done so already.
Presentation is 4-5 minutes long (this is a strict time limit)
Briefly introduce yourselves, including where you're from and what school or organization your team is part of, if relevant.
Introduce your co-designer (Who are they? What do they like to do? What challenges would they to tackle? Are there any other relevant pieces of information, such as living situation, family, caretaking, profession, etc?)
Provide a single-sentence product objective
Briefly step through your iterations in 1-2 slides, focusing on any major changes you may have made between low and high fidelity versions, or changes in product direction.
Focus your presentation on your final product, giving us a visual representation and a demonstration of how it works. Include your videos of it in use, by your co-designer if possible.
Briefly describe any future plans you have for your product.
End by telling us the top 2-3 things your team learned during the CRE[AT]E Challenge.
You will also be asked to upload a thumbnail (cover image) for your video for when we upload it to YouTube (after the end of the Challenge).
Finalists should be prepared to take 2-3 minutes of questions live after videos video during the final event. Teams will be notified if they are finalists shortly before the event.
Optional: Two-slide snapshot
Teams may optionally submit a two-slide "snapshot" of their projects, which will be shown during breaks in the final event. These slides will only be shown for a few seconds at a time, so we recommend photos-focused slides with relatively few words. Please include your team number/name. Possible ideas on what to include: your product "hero shot," product testing photos, team photo, single-sentence product objective.
Snapshots must be provided as a 16:9 aspect ratio PowerPoint slide.
Submission Instructions
Black Gate submission will be via Google Form, with the video and optional snapshot uploaded onto a separate Dropbox folder. Instructions have been sent to registered teams. If you have not previously registered with the CRE[AT]E Challenge, but would like to submit a project, please contact us at bwsi-create-challenge@mit.edu as soon as possible and we will send you instructions.
Design Review FAQ
Are design review checkoffs required for the Challenge?
Yes.
Where do I find the checkoff forms?
Checkoff form links are provided on Edly.
Does everyone on a team need to present?
Not everyone on a team needs to present at every review, though we would prefer that everyone on each team present at some point during the Challenge.
Can we do our reviews early?
Sure! Teams may want to do reviews early because of scheduling reasons, or because they are requesting funding from Beaver Works. If you are planning to do a peer review early, please arrange with another team. If you are planning to do a mentor review early, please just submit it as usual.
Can my team have an extension on our design review?
Sure! You may attempt to submit any design review past the due date. However, the following consequences apply:
Peer reviews: you will find it difficult to find a team to review with you
Mentor reviews: you will not be guaranteed to receive feedback
Final event: your project will not be considered for the final event, and may not make it onto the site after the Challenge
Please manage your time accordingly.
Help! I need to find another team to do a peer review!
We recommend using the Challenge Discord #looking-for-peer-reviewers channel to arrange peer review. If all else fails, ping @mentor and @staff and we'll see what we can do.
How should we do visual representations?
Visual representations are very important, as they help ensure that you and your reviewers have the same idea of what you are doing. Here are some suggestions, depending on what you are presenting:
Co-designer challenges - photos, including of the environment (with permission and co-designer media sign-off!); cartoon storyboards, diagrams; see Idea to Ink
Mechanical products - hand-drawn diagrams; CAD drawings (from multiple views); photos; include measurements when possible
Electrical products - block diagrams of system components and interfaces; circuit diagrams; photos
Fabric/wearable products - hand-drawn pictures/diagrams; cutting layouts and patterns; photos, including while the product is being worn
Software products - block diagrams of system components and interfaces; user interaction flow diagrams; paper prototypes; screenshots
How should we handle pictures, videos, and other information about co-designers?
If your presentations or any later documentation contains identifiable pictures or video of your co-designer (meaning someone can tell who that person is), your co-designer (or their parent or guardian) must fill out a Media Release form. Regardless of whether or not your presentations contain identifiable information, you should ask your co-designer what information they are comfortable being public. Specific things that people may have varying levels of comfort with include:
names (e.g. full names, first names only, first name last initial, first and last initial, a complete pseudonym, etc)
locations
schools, workplaces, or professions
any medical diagnoses
photographs and videos, particularly anything including their faces or other identifying features
Visual representations are important to the design process, especially as your product matures, so if your co-designer does not want any photos of them at all, you will have to be creative in how you present your work. Ideas include pictures without your co-designer, sketches of their use case, having a team member stand in for demonstrations as feasible, etc. This is not an easy thing to deal with, even outside the context of this course, to please do your best, remember to check with your co-designer, and ask course staff if you need any help.
If co-designers want to fill out a media release form (required if you are submitting videos or photographs that include identifying information, or other identifying information in text), please have them do so here. They will also need your team and coach's information.
Why are the design reviews "gates" and why do they have colors?
Beaver Works AT courses have a tradition of calling design reviews and documentation assignments "gates" and giving them a theme. Previous themes have included Middle Earth locations and dairy products.
Why are the design reviews called "preliminary" and "critical"?
Preliminary and critical design reviews (PDR and CDR) are terms borrowed from Systems Development Life Cycle and the way it is usually carried out in projects done for the United States Government. PDR and CDR (among other types of reviews) are commonly used by medical device developers (with the US Food and Drug Administration), space systems development (by NASA), and the US Department of Defense.
The common theme among all these (and what we want you to exercise) is to have independent outside experts (your peers and your remote mentors) examine your project's problem, approach, risks, and constraints, and ensure that you are on a good path towards project completion, providing suggestions, feedback, and possibly redirection as needed.