September 22, 2008

Science is hard

Posted in Uncategorized at 7:13 pm by carolynresearch

That is, the logistics are hard.

I’m doing a human subjects study, looking at how novices use formal design tools. If I lived in a perfect world, I’d have 100 students, carefully selected to represent the higher and lower achievers in classes, and having taken or not taken every relevant class. I’d have 10-20 hours with everyone. I’d be able to pay them all what their time is worth.

Instead, I am ignoring marks because I don’t want to have an issue with ethics review. I’m down to 5 hours, max, with maybe 20 students, and that will involve me repeating sessions many times. If I had 100 students, I’d run out of funded study time before I could transcribe all of the sessions, unless I inlisted friends and family. I’m sure that would be an ethics violation, too. I’m giving away one cool toy, everyone else gets no compensation.

I’m sure that the students who volunteer for something like this are a bit skewed towards the higher achievers, the ones who are interested in the new and the cool technologies. Maybe that’s not a major issue, but it’s something else to explain away.

When I’ve chatted with my officemates about software engineering research, some of them have expressed frustration at the state of the practice, how unscientific the studies published are. It frustrates me too…I know that I won’t be entirely happy with my research when it’s finished. I’ll try and figure out all of its weaknesses, but logistically, I’ll never finish if I try to address them all.

Time to go look for free times in common among my volunteers, and send another “nice nag” to the ones who volunteered once and never got back with free times or a retraction. Then off to another class to pitch my study.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: