Every developer or software engineer understands debugging very well. It is arguable that majority of their time goes into debugging. To a programmer, a bug has mastered the art of getting into your nerves, especially at critical times. There is only one time, when a bug in your code is a good thing.
So, in the world of coding, programmers and developers have created best practices for testing and debugging for any code written in any language. That’s what we will be reviewing today.
We won’t be going into any specific language or level of experience. This is a general review of principles that are applicable in debugging and testing any software, functions, and code.
Let’s get right into it.
Imagine it’s that time of the year when you take ‘coding breaks’, and go for a holiday. This time, you decided to go camping. You are in the kitchen preparing your favorite meal or soup. You have every ingredient and spices for the recipe you downloaded. Since you saw the guy on YouTube make the meal, this should be a walk in the park. Right? Since you’re camping flies, spiders, and roaches are all over and keep falling in your meal. This means that before you serve your meal, you have to make sure it has no insects. You have to remove the bugs from your meal, meaning debug it
.
Debugging is eliminating bugs. In programming, you will debug a lot. A study by Cambridge University claims that on average, a developer uses half the time debugging. That’s alot.
The term “bug” is used to a limited extent to designate any fault or trouble in the connections or working of electric apparatus. Hawkin’s New Catechism of Electricity, 1896
Going back to our cooking analogy, what can we do to make sure there are no bugs in your soup/meal?
defensive programming
. You try your best to write clean code.testing
where you actively stir your soup hoping to catch any bug inside.dangerous programming
(not recommended).what could go wrong?
and writing a piece of code that will run if it actually goes wrong
.After writing a piece of code that does a specific task, you should test it to make sure it works or actually__does it__. Consider a car-maker. Before selling any car to actual customers in the market, the government requires the car maker and other third parties to test the car to ensure it is safe to drive. Similarly, programmers test their code to make sure it works as expected.
Testing is a form of validation. Validation helps discover problems in a program. If no problems are found, it means your program’s correctness is high. Other forms of validation other than testing include code reviews and verification.
Even with the best validation, it’s very hard to achieve perfect quality in software. Here are some typical residual defect rates (bugs left over after the software has shipped) per kloc (one thousand lines of source code):
- 1 - 10 defects/kloc: Typical industry software.
- 0.1 - 1 defects/kloc: High-quality validation. The Java libraries might achieve his level of correctness.
- 0.01 - 0.1 defects/kloc: The very best, safety-critical validation. NASA and companies like Praxis can achieve this level.
This can be discouraging for large systems. For example, if you have shipped a million lines of typical industry source code (1 defect/kloc), it means you missed 1000 bugs!
Testing involves using the function’s or program’s specifications to come up with a set of inputs and outputs and running your code against them. The goal is to find out if the program functions as expected.
While testing, assertions are used to ensure that the expected performance is achieved.
For example, consider a function that should return True
when provided with a positive integer. To test this function, we can give it a positive integer as input and assert that the output is True
.
Also, we can assert that the output is not false
, or provide a non-positive integer, say zero or a negative number and assert that the output is not true
.
Therefore, the basic idea in testing your code is coming up with a set of inputs and outputs, and asserting that the code performs as expected in the light of those inputs and outputs.
If unexpected behavior is observed during testing, it means the program or code is buggy and should be edited to handle such cases.
Unit testing is validating a single piece of program such as testing a single function in the program.
Ideally, a well-tested program will have written tests for each individual module within it. A Unit Test is a test that validates an individual module such as a function, or a Class in isolation.
Testing modules in isolation makes debugging an application easy. If a unit test fails, it means that the bug is inside the module, rather than everywhere else in the program. If a module passes the test, it means that the bug is not within the code in that module, thus no need to debug the code inside it. This is the real power of unit tests.
Regression testing involves validating that a piece of code works correctly after new features or functionalities have been included. It involves ensuring that functionalities that had been tested before the new feature, functionality, or edits still work correctly.
The two common aspects of regression testing include adding test for bugs as you find them and catching reintroduced errors that were previously fixed.
Adding tests for bugs as you find them: When a bug is discovered, a regression test is created to reproduce the bug. This test is then added to the test suite to ensure that the bug does not reappear in the future. Regression testing helps to prevent the recurrence of known issues.
Catching reintroduced errors: As the software evolves and new features are added or existing code is modified, there is a risk of inadvertently reintroducing bugs or breaking previously fixed functionality. Regression tests help identify such regressions, allowing developers to rectify them before releasing the software.
This type of testing involves validating that the various components of the program interact and function as expected. It involves testing and asserting the program works correctly as a whole.
While unit tests are focused on individual components of the program, the integration tests focus on ensuring that multiple units work together as a whole.
Verifying if the overall program works: Integration testing evaluates the functioning of different components when combined into a cohesive system. It assesses the communication, data flow, and compatibility between modules, subsystems, or services.
Common mistake is Rushing to do integration tests: Integration testing is often performed towards the later stages of the software development process, closer to the release. Due to time constraints or development delays, there is sometimes a tendency to rush through integration testing, potentially leading to overlooking certain issues. However, it is crucial to allocate sufficient time and resources for comprehensive integration testing to ensure the overall system’s reliability and stability.
This far, we have looked at the different ways testing is achieved. But how do you decide how you will write the tests to your program? On what basis do you test a function, or a class, or the entire program? Do you test based on the specification, the way code is implemented, or maybe follow your gut? How does the gut even know what to do?
Just a reminder, a specification is the description of the function’s behavior, that is, the types of parameters, type of return value, and constraints and relationships between them.
Programmers approach testing from two notable approaches: black box and white box testing. Let’s explore both of them.
Black box testing refers to creating tests that are derived entirely from the specification of the function or the program. This means that the test cases are developed by looking at the definition of the function through its specification.
The objective of black box testing is to identify defects or inconsistencies between the expected behavior and the actual behavior of the software. It ensures that the software meets the specified requirements, works correctly with various inputs, and produces the desired outputs.
In this test approach, the tests are designed without looking at the code. Testers design test cases based on functional specifications, user stories, or use cases, and execute them to validate the software’s behavior.
There are two advantages we can derive from this approach. First, testing can be done by someone other than the implementer to avoid some implementer biases. Also,testing can be reused if implementation changes. This is the main motivation behind Test Driven Development (TDD)
White box testing involves looking at how the code is implemented to guide the design of test suites. All the possibilities that can occur based on the code implementation are considered during the design of a test suite.
Testers come-up test cases based on the internal structure of the software, including code branches, loops, and data structures. They may perform techniques like code coverage analysis
, unit testing, and debugging to validate the software’s correctness and efficiency.
This methodology has some significant drawbacks. First, it is easy to miss some paths or possibilities because some functions can have numerous pathways. Consider loops, how many paths do you need to consider and how many are you likely to miss?
Another drawback is that some loops can require tests going through them arbitrary amount of times.
Some tricks in dealing with white box testing is to ensure that tests cover all possible branches of the loop conditional. While dealing with for loops, ensure that you test when the loop is not entered, when the loop is only executed once, and when the body of the loop executes more than once. Similar case applies while working with while loops by including all cases that include the loop exit in your test suite.
Condition | test-1 | test-2 | test-3 | test-4 |
---|---|---|---|---|
username | True | False | False | True |
Password | False | False | True | True |
Home screen | False | False | False | True |
The decision table testing is very effective for evaluation of code that integrates business logic. 4. **State transition testing** - It is a software testing technique that focuses on testing the behavior of a system or software application as it transitions between different states. It is particularly useful for systems that have distinct states and where the behavior depends on the current state and the transitions between states.
<details>
<summary>To understand better, let us use an illustrative analogy of the ATM.</summary>
Initially, the ATM is in the `state` of idleness that is just displaying a default screen. This state changes when an ATM card is entered. Entering the ATM card triggers an `event` that changes the initial state. Each user action, triggers an event, that triggers a system event. For example, when the user enters their PIN, the ATM initiates a system event that verifies the PIN is correct. Therefore, the _state transition testing_ involves testing the system while in different states.</details>
These are just some approaches you can use in testing your code. Others you should definitely check out include:
Generally, this introduction material has covered debugging, and testing. With these concepts, you should have a significant understanding of software debugging and testing. The next recommended steps is to familiarize yourself with language-specific testing environments and the best practices in testing.
If you are using PHP, you should check out PHPUNIT or Pest which are testing tools to help you create and run tests on your code and applications. For python check out Pytest or unittest. For Ruby, check out RSpec and Cucumber.
I hope you had fun reading about debugging and software testing. I am Francis Njuguna - (github profile) and my goal is to share information about programming, technology, agriculture, businesses, people, and education. Be sure to subscribe to my Newsletter and I will keep you in the loop with juicy stuff.
Bye for now, Cheers.