October 20, 2008

Are requirements tests?

Posted in Uncategorized at 8:32 pm by carolynresearch

I had a chat with a couple of my officemates earlier this afternoon. One said that he though that requirements should be turned immediately into tests, and that any tools which keep requirements around are useless. Well, he’s prone to making sweeping statements, but I think that he’s got a lot of company in this opinion. I’ve heard it a lot from agile proponents, but also from others: requirements aren’t really anything until they’re operationalized as tests.

I kind of hesitate to agree. If software needs to work on a certain platform, I guess you could say that the appropriate test is to run all tests on that platform. And if a function needs to update a database from a form, you could do a typical data run. Theoretically, one could test single process non-reactive code with all possible inputs at all possible points, but even that is difficult. Even a simple numerical bit of code can have a huge state space, if, say, it takes any appropriate matrix and does SVD or something. I marked an assignment where the students had to take a graph as input, and determine if it had any cycles. The space of possible graphs is pretty large, and the pathological cases will depend on the exact mistaken algorithm chosen. If this was part of a real-time garbage collection function, it would be even harder.

Then there’s the fuzzy, hard to specify requirements. The communications must be secure. The interface must be usable. The software can’t crash, ever.

The more I think about it, the more I believe requirements aren’t just tests. I think there’s a Dijkstra quote about testing not revealing the absense of bugs, only their presense. I don’t know if I’d go that far, and say the only possible solution is through proving programs correct. After all, the hard part might just be translating natural language into something that can be proven or disproven.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: