Automated Test Creation

58 team-automated-test-creation

It’s imperative that the software-driven organization develops and maintains a robust testing solution capable of scaling with the needs of the organization. Automation should be the cornerstone of organizational testing solutions.

Explore More.

Related Mindset:

Continuous Delivery




Testable code created as part of a completed user story


An automated test suite to continually validate the condition of the experience against requirements

Software testing is a critical component in the software-driven digital organization.

As software systems multiply and mature through iteration, there is a need to maintain quality and consistency in delivery. To this end, testing should assume an increasingly large role in the digital organization’s software development process.

Testing Categories

Software tests can be designed using many different patterns, however most tests fit into one of three categories for an experience:

  • Unit Tests - Unit tests are designed to test a discrete unit of work using one or more scenarios. Units of work can be represented as functions, logic blocks or other areas of code. Unit tests are generally executed within a very small context.
  • Integration Tests - Integration tests are designed to test a software system process through multiple tiers within the system. Integration tests test the ability of a system’s components to work together to produce a predictable result. Integration tests generally execute within a larger context.
  • Interface Tests - Interface tests are designed to test the functionality of a user interface. Interface tests are generally the output of some other design tool which produces a scripted test for a given user interface. Interface tests test the ability of a user interface to correctly adapt to user input and produce reliable outputs given a specific set of inputs.

Running Tests

Running suites of software tests is the job of a test runner. Test runners are software tools available in various types and capabilities depending on requirements. Test runners can execute Unit, Integration and Interface tests, reporting results to whomever is executing the test runner.

Manually running software tests can be a laborious and time consuming task. There is a correlation between a software system and the test needed to support that system. Therefore, as software systems become increasingly large and complex, so do the suites of tests needed in support of the system. Additionally, as software systems within the digital organization multiply, so do the suites of tests.

These two realities represent a formidable challenge to the digital organization. Manually managing test infrastructure across projects is not a scalable solution.

Continuous Integration

This element supports the Continuous Integration element. While Continuous Integration can still provide feedback on the build process, it is most valuable when paired with an automated test suite. In these cases, the Continuous Integration server manages the test runner and reports on successes and failures.

Test Automation

Test Automation means consistently running tests at key points in the software development process, and establishing and maintaining a consistent process for when and how to execute tests. It requires quickly communicating test results across project members and key organizational contacts. These are all distinctive results of an efficient organizational testing strategy. Achieving and maintaining these results requires automation.

Testing automation allows the digital organization to mitigate the “human factor” and establish strict software controls over how and when testing occurs within a project. Using automation, the digital organization can establish processes and procedures for testing and use automation to enforce them across projects. In addition to organizational efficiencies, test automation can greatly reduce the number of defects and the costs associated with finding and fixing those defects.

Common Pitfalls

There are many different viewpoints to automated testing and how the tests are created. Organizations should avoid the following situations:

  • Developing automated tests as a separate effort from feature development - The most efficient manner of creating automated tests is to have test creation as a part of the ‘Definition of Done’ for a user story. In this manner, the development team that implemented the story is also responsible for incorporating the tests. In most cases, there should be testing for each of the acceptance criteria within the user story.
  • Decreasing test development to increase velocity - While most organizations agree that a robust test suite is a valuable asset, many of those organizations will at least entertain the idea of limiting test development for a period of time to increase user story velocity. In these cases, the health of the application will suffer and lead to decreased future velocity due to defect and stability issues.
  • Focusing on a single type of testing - As mentioned, there are multiple types of testing for an experience. While some testing is better than none, an organization should embrace all three types of testing to ensure the experience is working as intended.
  • Testing on an environment that is fundamentally different from production - Developers and testers should have access to an environment that mirrors production so that test results from one environment can be considered validation for all environments.