HCI 460

Spring 2016

Craig Miller

Weekly Notes

Assignments

Project 4
Comparison Study and Report
Due Sunday June 5 before 11:30 PM

Overview

For this assignment, your group will conduct a simple comparison study and report its findings.

Requirements and Guidelines

Objective

Unlike your usability test, your group will focus on only one objective. Ideally, the objective should only require a test with two conditions and one dependent variable. The dependent variable should be quantitative (e.g. error count, timing or sum of user ratings). While not required, you will likely choose an objective that addresses a design decision early in the development process, which allows for a simple paper-and-pencil experiment.

Method

The method should be short and simple and directly address the study's objective. Ideally, the process will last only a few minutes for each session and can be conducted by one experimenter. This way, each person in the group could run 10 participants within an hour and possibly yield a total of 20-30 data points. At minimum, your group needs to produce 8 data points per condition, although more is preferred. Because your study involves human participants, you will need to obtain informed consent.

The study may have a between-groups design or a within-groups design. For a between-groups design, make sure that each experimenter runs the same number for each condition. For a within-groups design, make sure that the conditions are properly counter-balanced.

Statistical Analysis

Statistical tests are required. Because of the simplicity of the study, a T-test should suffice in most cases. Talk to me if you need advice or help in choosing and conducting your analysis. Also, you should report confidence intervals for the dependent variable's average(s).

If you choose to use a subjective user rating for your dependent variable, you are encouraged to use one of the standard scales discussed previously in class.

The Report Format

Rubin and Chisnell don't provide any specific format for a comparison study. Here are some guidelines for putting your report together.

Executive Summary

Create a short paragraph, perhaps just two or three sentences, that describes your findings.

Purpose and test objective

Provide a paragraph that presents the background motivation for the comparison test, followed by the test objective.

Method Section

This section presents your procedure for conducting your study. The description should be specific enough so that others could read your report and successfully replicate the procedure. You may want to create subsections for describing the materials and participants used in the study.

Because your test will use human participants, you will need to ensure informed consent. Make sure you describe your procedure in the method section and supplement it with whatever notice you give to your participants.

Results Section

This section provides a written summary of your results, including the statistical outcome and possibly a table or graph. Raw data usually is not presented in the results section. For your report, provide the raw data in the appendix.

Findings and Recommendations

This section should provide an interpretation of the results. If the statistical analysis failed to show any significance, provide possible reasons why the results were not significant. Remember that a failure to produce a statistically significant effect does not necessarily mean that no real effect exists. Never, ever use the word prove to summarize a finding based on a statistical analysis.

Submission

Submit your report and materials using the D2L web site. PDF is my preferred format, but I will accept any commonly available format (e.g. Word, HTML). Again, provide a short description of how the individual members in your group contributed to conducting the test and writing the report.

Grading

A D2L rubric will be used to assess the report.