01What it is
An automated test is a small program that uses the application and checks the result. It might create an invoice with three line items and check the total, sign in as a client and check that only that client's tickets appear, or enter a wrong password and check that the login refuses it.
A person could do all of this by hand, and before a release someone often does. The difference is speed and repetition. A test suite of several hundred checks runs in a few minutes, the same way every time, as often as needed. Each new feature adds its own tests, so the suite grows with the application and keeps checking features that were finished long ago.
02Three kinds of test
Tests are usually sorted by how much of the application they cover at once:
- Unit tests check one small piece on its own, such as the function that calculates VAT or formats a date. They run in milliseconds, and a project typically has many of them.
- Integration tests check that several pieces work together, for example that saving a ticket writes the right rows to the database and records who made the change.
- End-to-end tests drive a real browser through the application like a user: open the page, click, type, submit, read what appears. They are the slowest and closest to real use, so they are kept for the most important paths, such as signing in or sending an invoice.
03Continuous integration
Tests only help if they actually run. Continuous integration, CI for short, makes sure they do. Every time a developer proposes a change as a pull request, a separate server takes the code, installs it from scratch and runs a fixed sequence of checks. This sequence is called the pipeline.
A typical pipeline checks more than the tests:
- The type check confirms that every value has the kind of data the code expects. In TypeScript projects this catches many mistakes before any test runs.
- The style check, often called linting, enforces the team's rules on how code is written and flags common errors.
- The tests run: unit, integration and, where set up, end-to-end.
- The build compiles the whole application as it would be deployed. If it does not build, it cannot go live.
04What happens when a check fails
The result appears on the pull request as a green tick or a red cross, with a link to the exact step and message. The repository is set up so that a change with a red cross cannot be merged. The author fixes the problem, pushes the correction, and the pipeline runs again.
This turns quality into a rule that applies to everyone. Nobody has to remember to run the tests, and a change that breaks an existing feature is stopped before it reaches the people using the application.
05An example from a business application
Take a client portal where customers file support tickets and see their invoices. A developer adds a filter so staff can show tickets by priority. The new code works, and the filter looks right in the browser.
In the pipeline, one older integration test fails. It checks that a customer signed in to the portal sees only their own company's tickets. The new filter queried the database without the company condition, so with a certain combination of filters a customer could have seen another company's tickets. The red cross stops the merge, the developer adds the missing condition, and the test passes.
Without the test, this would have been found by a customer, or not at all. With it, the mistake cost a few minutes.
06Alternatives and limits
The alternative to automated tests is manual testing: a person follows a checklist before each release. It is useful for judging how something feels to use and for catching what nobody thought to test. It is slow, it gets skipped under time pressure, and it rarely covers features that seem unrelated to the change.
Automated tests have limits too. They only check what someone wrote a test for, and a test can itself be wrong. A green pipeline means the known requirements still hold. It does not prove the software is free of errors. That is why we combine tests with a review of every change and with you trying new features before they count as done.
Popular CI services include GitHub Actions, GitLab CI and Jenkins. They differ in where they run and how they are configured. The idea is the same in all of them.
07What to check when you buy software
Tests are easy to promise and hard to see from outside. These questions help:
- Is there a test suite, and does it run automatically on every change?
- Can a change be merged when the tests fail? The answer should be no.
- Are the most important paths covered: sign-in, permissions, money?
- Will the tests and the pipeline configuration be handed over with the code, so they keep running after the project ends?
08How we use it
Our CI runs on GitHub Actions. For every pull request it checks the types, the code style and the tests, and whether the application builds. A change with a failing check cannot be merged.
Each assignment lists what must be true when it is finished, and those points become tests. Changes to sign-in, permissions or passwords also get a separate security review before they are merged. The tests and the pipeline are part of the code you receive at handover, so they keep running in your own GitHub account.
Frequently asked questions
Do tests make a project more expensive?
Writing them takes time during the build. They save more than that later, because mistakes are found within minutes of being made, and changes to an existing application can be made without re-checking everything by hand.
Does a green pipeline mean there are no bugs?
No. It means everything that has a test still works. Bugs in situations nobody wrote a test for are still possible, which is why every change is also reviewed and you try new features yourself.
Do I need to do anything to keep the tests running?
No. The pipeline runs on its own in your GitHub account for every proposed change. The usual monthly free allowance of GitHub Actions is enough for most small applications.
Can I see the test results?
Yes. Each pull request on GitHub shows a green tick or a red cross for every check, and you can open the details without any programming knowledge.
Will you still test things by hand?
Yes. We click through every new feature before it goes to you, and you try it before it counts as finished. Automated tests cover repetition. People judge whether something is right to use.