Wednesday, July 16, 2008

Online survey response rates - effect of more time

The graph below is a comparison of response rates to an online course evaluation in a college at Washington State University. Faculty had hypothesized that the response rate was limited by the amount of time the survey was open to students so the time available was varied in two administrations:

Fall 2007 11/23 to 12/22 (29 days) 10582 possible respondents
Spring 2008 3/31 - 5/5 (35 days) 9216 possible responents

The x-axis in the graph below is time, but normalized to the % of time the survey was open.

The y axis is normalized also, to the number of responses possible, based on course enrollment data, i.e., Y is response rate.

Figure 1 (click to enlarge)
The Fall 2007 survey ran to completion (reached an asymptote) where the spring one was (maybe) still rising at the cut off date. Fall and Spring total response rates are very similar, suggesting that more time when the survey is open has little impact on total response rate. So, contrary to what faculty hypothesized, the same overall response rate was achieved in a longer surveying window. This aligns with other data we have on response rates -- there is some other factor governing response rate that is not yet identified.

Its interesting to note that you can see waves in the spring data, as if faculty exhorted students on Monday and got another increment of response.

No comments: