Tests and Coverage in Bash

Why Test Bash Scripts?

Bash scripts are a powerful way to automate tasks, but unfortunately, most developers skip creating test cases or measuring coverage for them—something that's crucial, especially when making refactors. Imagine changing a variable name and not knowing if your release scripts still work as expected. In this article, we’ll dive into why Bash testing and coverage are essential and show you how to use Bats (Bash Automated Testing System) and kcov to ensure your scripts remain reliable and maintainable.

Testing your Bash scripts helps to:

• Prevent regressions when modifying code.
• Ensure scripts behave correctly under different scenarios.
• Improve maintainability and confidence in deployments.
• Catch unexpected failures before they reach production.
• Validate script performance with different inputs and system environments.

---

Setting Up Bats for Bash Testing

Bats is a lightweight, TAP-compliant testing framework for Bash.

Installation

For most systems, you can install Bats using:

# Using Homebrew (MacOS/Linux)
brew install bats-core

# Using a package manager (Debian-based Linux)
sudo apt install bats

# Clone and install manually
git clone https://github.com/bats-core/bats-core.git
cd bats-core
./install.sh /usr/local

Writing Your First Test

Create a test script, e.g., test_example.bats:

#!/usr/bin/env bats

@test "Check if hello function outputs 'Hello, World!'" {
  run bash -c 'echo "Hello, World!"'
  [ "$status" -eq 0 ]
  [ "$output" == "Hello, World!" ]
}

Run the test with:

bats test_example.bats

An the output will be:

✔ Check if hello function outputs 'Hello, World!

Example: Testing a Script

Suppose we have a script, greet.sh, that takes a name as input:

#!/bin/bash

if [ -z "$1" ]; then
  echo "Usage: $0 <name>"
  exit 1
fi

echo "Hello, $1!"

A test for this script:

#!/usr/bin/env bats

@test "Greet with a name" {
  run bash greet.sh John
  [ "$status" -eq 0 ]
  [ "$output" == "Hello, John!" ]
}

@test "Missing name argument" {
  run bash greet.sh
  [ "$status" -eq 1 ]
  [[ "$output" == *"Usage:"* ]]
}

Run the test with:

bats test_example.bats

An the output will be:

✔ Greet with a name
✔ Missing name argument

---

Measuring Coverage with kcov

kcov is a tool that generates test coverage reports for Bash scripts by tracking which lines of code are executed during tests.

Installation

# On Debian-based systems
sudo apt install kcov

# On MacOS (via Homebrew)
brew install kcov

Running kcov with Bats

Create a directory for the coverage output and run kcov:

mkdir coverage
kcov coverage bats test_example.bats

This generates an HTML report in coverage/. Open coverage/index.html to view detailed coverage statistics.

Improving Coverage

To ensure comprehensive coverage:

• Write tests covering all branches of conditional statements.
• Test scripts with different system environments.
• Include tests for error handling and unexpected inputs.
• Ensure coverage for loop conditions and edge cases.
• Simulate different user inputs and permissions levels.

Example: Ensuring Full Coverage

If a script has different exit codes based on input, tests should validate all possible cases:

#!/bin/bash
if [ -z "$1" ]; then
  echo "Error: No argument provided"
  exit 1
elif [[ "$1" =~ ^[0-9]+$ ]]; then
  echo "Valid number input"
  exit 0
else
  echo "Invalid input"
  exit 2
fi

A comprehensive test suite for this script:

#!/usr/bin/env bats

@test "No argument provided" {
  run bash script.sh
  [ "$status" -eq 1 ]
  [[ "$output" == *"Error: No argument provided"* ]]
}

@test "Valid numeric input" {
  run bash script.sh 42
  [ "$status" -eq 0 ]
  [[ "$output" == *"Valid number input"* ]]
}

@test "Invalid input" {
  run bash script.sh "hello"
  [ "$status" -eq 2 ]
  [[ "$output" == *"Invalid input"* ]]
}

Expanding Coverage Analysis

Beyond basic execution, kcov also allows analyzing:

• Unreachable code detection: Identify parts of your script that are never executed.
• Line-by-line execution reports: Determine which areas need additional test cases.
• Integration with CI/CD pipelines: Automate coverage tracking in GitHub Actions, GitLab CI, or Jenkins.
• Performance impact: Detect inefficient loops or conditions by examining execution flow.

Automating Coverage Reporting

To generate and upload reports automatically:

kcov --coveralls-id=$COVERALLS_REPO_TOKEN coverage/ script.sh

This integrates with Coveralls to visualize test coverage over time, ensuring continuous improvement.

By leveraging kcov, you gain deeper insight into test effectiveness, helping you write more robust and reliable Bash scripts.