As software systems multiply and mature through iteration, there is a need to maintain quality and consistency in delivery. To this end, testing should assume an increasingly large role in the digital organization’s software development process.
Software tests can be designed using many different patterns, however most tests fit into one of three categories for an experience:
- Unit Tests - Unit tests are designed to test a discrete unit of work using one or more scenarios. Units of work can be represented as functions, logic blocks or other areas of code. Unit tests are generally executed within a very small context.
- Integration Tests - Integration tests are designed to test a software system process through multiple tiers within the system. Integration tests test the ability of a system’s components to work together to produce a predictable result. Integration tests generally execute within a larger context.
- Interface Tests - Interface tests are designed to test the functionality of a user interface. Interface tests are generally the output of some other design tool which produces a scripted test for a given user interface. Interface tests test the ability of a user interface to correctly adapt to user input and produce reliable outputs given a specific set of inputs.
Running suites of software tests is the job of a test runner. Test runners are software tools available in various types and capabilities depending on requirements. Test runners can execute Unit, Integration and Interface tests, reporting results to whomever is executing the test runner.
Manually running software tests can be a laborious and time consuming task. There is a correlation between a software system and the test needed to support that system. Therefore, as software systems become increasingly large and complex, so do the suites of tests needed in support of the system. Additionally, as software systems within the digital organization multiply, so do the suites of tests.
These two realities represent a formidable challenge to the digital organization. Manually managing test infrastructure across projects is not a scalable solution.
This element supports the Continuous Integration element. While Continuous Integration can still provide feedback on the build process, it is most valuable when paired with an automated test suite. In these cases, the Continuous Integration server manages the test runner and reports on successes and failures.
Test Automation means consistently running tests at key points in the software development process, and establishing and maintaining a consistent process for when and how to execute tests. It requires quickly communicating test results across project members and key organizational contacts. These are all distinctive results of an efficient organizational testing strategy. Achieving and maintaining these results requires automation.
Testing automation allows the digital organization to mitigate the “human factor” and establish strict software controls over how and when testing occurs within a project. Using automation, the digital organization can establish processes and procedures for testing and use automation to enforce them across projects. In addition to organizational efficiencies, test automation can greatly reduce the number of defects and the costs associated with finding and fixing those defects.