Recently, I had a discussion with a friend about Testing, and I thought I might take a moment to talk about my opinions on Testing. The first thing to say, is that we are restricting what is being tested to Software, Testing may exist in other disciplines, but I'd like to say it's not as distinct as it is in Software Engineering.
So what is the purpose of testing? To find bugs. Some think it is to prove a program is bug-free, but tests can still fail to find the flaws or bugs of a program. Such cases can be given examples such as memory leaks, buffer overflows or even just slightly unexpected behaviour. If you want to prove a program is bug free, you have to do it mathematically, but even then that is commonly an incomplete proof too because you place faith in the case that the operating system, compiler, interpreter or other overhead software systems your code relies on are also either proven or fully tested. It's for this reason that there are good reasons not to use the C standard library at times, and it's also for this reason that code can produce unexpected outputs. Sometimes this output is even shifted by the very architecture you use, as there are hardware differences too.
So, Software has multiple testing methods, the most common is just a lot of print statements, to confirm which line or variable contents. This is certainly useful, but tends to be left redundant with superior debugging tools like GDB, which lets you see variable contents and step through code line by line. It's effective for building an understanding, but beyond debugging, this kind of test does very little to confirm the presence of bugs as it is commonly used for debugging, and understanding the problems that lie in it.
An alternative lies in something called Unit Tests. A unit test is simply described as a line of code that asserts that a statement holds true, and if it isn't then there is an error or bug in the code that lies in that unit of code. The paradox of this, is you need to prove, test or simply have religious faith your code for the unit tests is good and the other problems described above for testing don't arise.
Unit tests, being just code makes automating tests a lot easier, but still errors or issues can arise. Lets take the case of a double, a double is simply put a representation of a Real number. This means it's has a decimal or fractional part and an integer part. Because of this, arithmetic with Doubles can end up being slightly imprecise. Usually this lack of precision isn't an issue, but for some domains like Finance, or Missile Trajectory calculations, this can be a huge problem, and it's for this reason Doubles are usually represented instead by Integers where possible. For unit tests, the question is how does one define equality between Doubles, like 2.00000001 and 2.0. The answer is to define a tolerance of imprecision for their equality, but this just provides an example of the issues that arise from using code to test code.
Unit tests also have another problem, which is the amount of code overhead you have to write for your code. It tends to be a lot of code, and it often tests normal cases and error cases to make sure both pass for each unit of code you have written. It's worth noting at this point, a unit tends to be just a function(Method is basically the same). The overhead in code is enough that sometimes writing the test code can be a bit pointless and redundant, as I personally consider a failed test to be a test which doesn't find a bug. The reason for this being that tests are to find bugs, not prove bugs aren't there; though there is a good made case for them being to prove bugs aren't there as you can't truly be certain there is a bug there or not. This is often conveyed in the discussion of software correctness being basically impossible to prove.
The final thing I want to talk about is TDD, an abbreviation for Test Driven Development. The basic idea, is to write the tests, then write the code to pass those tests. The flaws in this arise when you consider the code overhead in tests with how they are often times more code than the actual implementation code will be. The other problem lies in asking yourself the question of, how can you be 100% or even 90% certain what the tests you need to meet are, before you've written the code? This question is risen as tests are to find bugs in code, but you have no code to find bugs in. Additionally, how can you know the full intricacies of a problem before you have written the code? When you come to writing the code, chances are, you will modify your tests to fit your code as well and meet in the middle.
Because of the problems in code overhead and the silliness in having to know the full breadth of your problem, I personally dislike TDD in general, but I acknowledge its usefulness in critical systems such as health systems or traffic infrastructure or the like. For the vast majority of software, this is not necessary. This isn't to say I think Unit tests are bad, I think they are actually very useful, and provide a lot more comfort and trust that your code does what it says on the tin. All in all, the conclusion you can take from this, is that the Air traffic controller for your Air-plane is unlikely to be proven bug-free, and you could be the unlucky individual to be in the conditions for the bugs to arise.
| <Latest> | <Next> | <List> | <Previous> | <Oldest> |