Teaching - Spring 2019
Software Testing, Quality Assurance, and Maintenance - ECE653
Overview
ECE653: Software Testing, Quality Assurance, and Maintenance will provide an introduction to software testing and quality assurance techniques. The students will learn a wide spectrum of techniques and tools that can be used to improve and evaluate software quality ranging from mature testing methodologies to cutting edge automated verification algorithms. Topics to be covered include: coverage criteria (graph, data-flow, and logic coverage), symbolic execution (static, dynamic, concolic), constraint solving (SMT), inductive invariants, automatic deductive verification, automatic invariant synthesis, and Software Model Checking.
Contact
Lectures are held Tuesdays from 11:30 to 14:20 in E7 5353. There are no lab or tutorial slots.
My office hours are by appointment and will be held in EIT 4007. I will usually also be available immediately after lectures. See contact details.
Course material, announcements, and submissions will be handled through Learn.
Begin all email subjects with [ECE653].
Try not to leave your questions until the last minute.
Teaching assistant
TBD.
Course content
Prerequisite
Graduate standing in the ECE department at Waterloo.
The course will include programming assignments in Java/C/Python. Background in Compilers and Logic is useful, but is not required.
Prereq: ECE 650 or 750 Tpc 26 or instructor consent.
Coreq:ECE 650 or 750 Tpc 26 or instructor consent.
Antireq: CS 447, 647, ECE 453, SE 465.
Material
There is no required textbook that needs to be bought. Lecture slides, lecture notes, and reading material will be made available on Learn.
Schedule
All lecture material will be available through Learn.
This is a tentative schedule that will get adapted during the term. Check Learn for updates.
| Week | Date | Class |
|---|---|---|
| 1 | May 7 | Admin; Introduction; Fault, Error, Failure |
| 2 | May 14 | Foundations: Syntax and Semantics |
| 3 | May 21 | Structural Coverage |
| 4 | May 28 | Data-flow Coverage; Logic Coverage; Z3 SMT Solver |
| 5 | June 4 | Quiz 1; Symbolic Execution; Dynamic Symbolic Execution |
| 6 | June 11 | Semantics of Symbolic Execution; Propositional Logic |
| 7 | June 18 | SAT Solving; First Order Logic |
| 8 | June 25 | SMT; Automatic Exploit Generation; Axiomatic Semantics |
| 9 | July 9 | Axiomatic Semantics; Dafny |
| 10 | July 16 | Quiz 2; Dafny: Lemmas; Dafny: VCGen |
| 11 | July 23 | Verification Condition Generation; Dafny: Arrays; Dafny: Sorting |
| 12 | July 30 | Formal Method Tools |
The dates for the quizzes might change. Dates will be finalized at least a week before the respective quiz.
Grading
Grades may be curved or adjusted at the instructor’s discretion.
Assignments: 30%
Quizzes: 20%
Final Exam: 50%
All assignments and quizzes will be returned in class, during office hours, or through LEARN.
You must pass the final exam to pass the course. You must pass the weighted average of quizzes and final to pass the course. The final grade will be computed as follows:
def grade(assignments, quizzes, final):
normal = 0.3 * assignments + 0.2 * quizzes + 0.5 * final
weighted = (0.2 * quizzes + 0.5 * final) / 0.7
if final < 50:
return min(normal, final)
elif weighted < 50:
return min(normal, weighted)
else:
return normal
If you missed the quizzes (and had permissions from the instructor according to the Course Policy), the final grade will be computed as follows:
def grade(assignments, quizzes, final):
normal = 0.3 * assignments + 0.7 * final
if final < 50:
return min(normal, final)
else:
return normal
Course Policies
By registering for this class, students agree to the following class policies:
All work turned in will be that of the individual student unless stated otherwise. Violations would result in zero credit to all students concerned. Policy 71 will be followed for any discovered cases of plagiarism.
You have 2 days of lateness to use on assignment submissions throughout the term. Each day you hand in an assignment late consumes one of the days of lateness. If you consume all of your late days, assignments that are still late will get 0 marks. You can only hand in an assignment up to the time all assignments are returned. Missed assignments get 0 marks. For example, you may hand in A1 two days late and A2 on time, or you can hand in A1 one day late and A2 one day late.
If you miss a quiz, you will receive 0 marks for the quiz. If you have a legitimate reason (at the discretion of the instructor) that you cannot take a quiz, and obtain permission from the instructor a week in advance, the percentage for the quiz may be shifted to the final. No alternative quiz time will be provided.
Official outline
This is the high-level outline provided by the ECE department; this course will follow the general guideline, but will be adjusted according to your feedback, interests, and experience.
Introduces students to systematic testing of software systems. Software verification, reviews, metrics, quality assurance, and prediction of software reliability and availability. Students are expected to have programming experience with reading and writing code for large projects.
Policies
-
In order to maintain a culture of academic integrity, members of the University of Waterloo community are expected to promote honesty, trust, fairness, respect and responsibility. [See the academic integrity site for more information.]
-
Turnitin.com: Text matching software (Turnitin®) will be used to screen assignments in this course. Turnitin® is used to verify that all materials and sources in assignments are documented. Students’ submissions are stored on a U.S. server, therefore students must be given an alternative (e.g., scaffolded assignment or annotated bibliography), if they are concerned about their privacy and/or security. Students will be given due notice, in the first week of the term and/or at the time assignment details are provided, about arrangements and alternatives for the use of Turnitin® in this course. It is the responsibility of the student to notify the instructor if they, in the first week of term or at the time assignment details are provided, wish to submit the alternate assignment.
-
A student who believes that a decision affecting some aspect of his/her university life has been unfair or unreasonable may have grounds for initiating a grievance. Read Policy 70, Student Petitions and Grievances, Section 4.
-
When in doubt please be certain to contact the department’s administrative assistant who will provide further assistance.
-
A student is expected to know what constitutes academic integrity to avoid committing an academic offence, and to take responsibility for his/her actions.
-
A student who is unsure whether an action constitutes an offence, or who needs help in learning how to avoid offences (e.g., plagiarism, cheating) or about "rules" for group work/collaboration should seek guidance from the course instructor, academic advisor, or the undergraduate Associate Dean.
-
For information on categories of offences and types of penalties, students should refer to Policy 71, Student Discipline.
-
For typical penalties check Guidelines for the Assessment of Penalties.
-
A decision made or penalty imposed under Policy 70 (Student Petitions and Grievances) (other than a petition) or Policy 71 (Student Discipline) may be appealed if there is a ground.
-
A student who believes he/she has a ground for an appeal should refer to Policy 72, Student Appeals.
-
AccessAbility Services, located in Needles Hall, Room 1132, collaborates with all academic departments to arrange appropriate accommodations for students with disabilities without compromising the academic integrity of the curriculum. If you require academic accommodations to lessen the impact of your disability, please register with the AccessAbility Services at the beginning of each academic term.
Acknowledgments
Thanks to Arie Gurfinkel for sharing his course material from previous iterations of the course.
Temporary Slide Download
For students that do not have access to Learn yet.
PDF version for easier printing (if you absolutely have to) or if you prefer looking at PDFs.