Collecting free-response data via computers: Effects of technology on responses to hypothetical scenarios

This study compared pen-and-paper and computer administrations of free-response instruments. Two-hundred-and-fifty-seven subjects were assigned to either a computer or pen-and-paper condition and were administered two free-response instruments, one a diagnostic for message design logic, the other a hypothetical scenario depicting a group decision-making situation. Findings revealed no difference in proportion of cases assigned to each level of message design across mode of administration. However, contrary to predictions, women were not more likely to be classified at the rhetorical level of message design. Although not statistically significant due to low power, examination of message design within mode of administration revealed that more women from the pen-and-paper administration were classified as rhetoricals than men, but more men were classified at that level than women in the computer administration. Some differences in message content from the group situation were found across mode of administration or by gender within mode. Discussion addresses the cognitive processes associated with message production in mediated circumstances as well as gender differences in attitudes toward computing.