GTest Example Repo

This is a simple example repo illustrating how to use GTest to test C++ code. As
a demonstration I use a simple class that holds two values and can perform
addition, multiplication, subtraction, and division of said values.

GTest
(googletest) is a
cross-platform testing framework developed by Google to facilitate all sorts of
testing for C++. The provided documentation
(primer,
advanced)
for the framework is quite detailed and easy to follow and furthermore, it is
used enough that you can find additional solutions from external resources such
as stack overflow (when was the last time you hit
the SO landing page?).

Note that this is not an exhaustive example, there are many additional features
from GTest not show here. For example:

global test suite SetUp and TearDown
multithreaded testing
typed tests (similar to parametric tests, designed for templates)
exception/failure testing
etc.

Requirements

This has been developed and tested under Ubuntu 16.04 and the instructions are
for the same, it’s possible you may need to slightly modify commands for your
setup.

GCC

We use GCC (specifically g++) to build the C source code. This is likely
already installed, but if not it is easy to install with sudo apt install g++.

gcov

We use the gcov utility to generate the coverage reports. This is included
with GCC so no extra installation is required.

lcov

For human readable HTML reports we use the lcov utility on top of the
standard gcov. Install with sudo apt install lcov.

CMake

We use cmake to facilitate building the test suites. We don’t need the very
latest version so you can simply install with sudo apt install cmake.

GTest

First we get the GTest source code:

sudo apt install libgtest-dev

GTest doesn’t come precompiled unfortunately, so we need to build and install
the source code:

cd /usr/src/gtest          # Move to source dir
sudo cmake CMakeLists.txt  # Collect everything with cmake
sudo make                  # Comnpile source
sudo cp *.a /usr/lib       # Copy built libraries to lib dir

pre-comit (optional)

This repo also demonstrates the use of C++ pre-commit hooks. First install the
utilities themselves:

clang-format, for auto formatting the code:
sudo apt install clang-format
cpplint, for code style: pip install cpplint
cppcheck, for checking syntax: sudo apt install cppcheck

Next setup pre-commit itself

Install pre-commit: pip install pre-commit
From the root directory of this repo, setup pre-commit:
pre-commit install --install-hooks
Run the hooks: pre-commit run --all-files

Any further commits will also trigger these utilities to run automatically.

Usage

Manually

It can be difficult to cleanup CMake builds sometimes, so it’s recommended to
create a separate build directory and work from there.

From the root directory of this repo (gtest-example), create your build
directory and move there:
```
mkdir _build && cd _build
```
Run CMake to setup the Makefile:
```
cmake ..
```
CMake uses the CMakeLists.txt file in the root directory, hence passing
the parent directory (..) to cmake.
CMake will have created a Makefile for you. You can now build the test
executables:
```
make
```
You will now have several executables named run*Tests, where each
executable is associated with a test suite (discussed below). Try running a
test suite, for example:
```
./runSimpleTests
```

You can easily cleanup build artifacts by removing the entire build directory.
For example, from the root directory:

rm -r _build

It is possible to run coverage manually, but it is somewhat tedious. Take a look
at the details in the Script section below as well as the script itself for a
some more information.

Script

There is a build_run.sh script which builds and runs all tests. After
running tests, a summary of the functional tests (pass/fail) and code coverage
is printed at the end of the script:

./build_run.sh

HTML reports

If we include the -report switch with the script, it will also generate HTML
reports using lcov and drop the HTML files in the BUILD_DIR directory
as <exe name>-html (e.g. runTests-html).

./build_run.sh -report

View the reports by opening the appropriate <exe name>-html/index.html in
your browser.

Clean

The script can also accept a -clean flag to remove any generated files. If
the clean flag is provided, all other flags are ignored and the script will only
execute the cleaning portion of the code, nothing else.

./build_run.sh -clean

Details

The script creates an out-of-source directory in which to build and run tests,
as this is much easier to cleanup since cmake does not provide any clean
logic. Inside that directory (BUILD_DIR), the tests are built using
cmake followed by make. Next, each of the test executables is run and
it’s pass/fail status logged in the EXE array. If you are manually checking
a specific tests, you can easily run make <exe name> (after running cmake .. in the BUILD_DIR), to build the single executable.

Code coverage info is stored in the same directory as the compiled object files.
Each test has it’s build files (and coverage files) in
$BUILD_DIR/CMakeFiles/<exe name>.dir (e.g.
_test-build/CmakeFiles/runTests.dir). We move into the respective object
directories, and run gcov to print the simple text coverage reports to the
terminal. This command is piped into a series of awk commands which extract
the coverage for our easy_math.cpp implementation. The coverage value is
logged in the COV array. Coverage can be inspected manually by running the
gcov *.gcno command in the appropriate object directory.

Next a summary is printed, pulling results logged in the EXE and COV
arrays.

If the -report flag is set, the script will re-enter the object directories
($BUILD_DIR/CMakeFiles/<exe name>.dir) and run lcov and genhtml to
create the human readable HTML reports. This can similarly be done manually for
individual tests by entering the appropriate object directory and running lcov --capture --directory . --output-file <exe name>.info to grab the gcov
info (note, gcov must be run prior to lcov). Then run genhtml <exe name>.info --output-directory <exe name>-html to create the HTML to be opened
in the browser.

Test Suites

A brief description of each test suite.

Simple Tests

Executable: runSimpleTests

This test suite implements simple, explicit tests that do not use any additional infrastructure or reusable code.

Failing Tests

Executable: runFailingTests

This test suite is simply to demonstrate the difference between the built in
ASSERT_* and EXPECT_* checks. Using EXPECT_* will prompt a
non-fatal failure. That is to say any remaining code in the test case will
still be executed. Conversely, ASSERT_* checks will cause a fatal failure
and the test case is abandoned immediately. In general it makes sense to use
EXPECT_ unless there are specific circumstances that warrant abandoning a
test mid execution.

Fixture Tests

Executable: runFixtureTests

This test suite demonstrates the use of the simple Test test fixture. Our
test fixture, MathsTest is defined in the fixture_tests.h header. The
fixture just creates an instance of our EasyMaths class with some hardcoded
values to avoid having to duplicate this setup for each of our test cases. In
the cpp file we TEST_F to pull our MathsTest fixture from the
header. The fixture name incidentally becomes the name of our test suite. Within
this test suite we define 4 tests, one for each of our member functions. Since
we have access to our local test variables a and b, we can easily access
these to calculate an expected value in the test case itself. When running the
test suite, notice the output illustrating that the SetUp and TearDown
functions from the fixture are automatically run before and after the test case
respectively.

Parametric Tests

Executable: runParamTests

Similar to the fixture tests, but using the parametric TestWithParam test
fixture. SweepTest is defined in the parametric_tests.h header and is
identical to to the one used in the fixture tests except we now have a
parameter to be assigned. The TestWithParam class only accepts a single
parameter, but we can easily add additional parameters by making the single
available test parameter a tuple and unpacking that tuple during the test setup
or execution. In the cpp file we now use TEST_P to pull our
SweepTest parametric fixture. We also must create a list of parameters with
which to run tests by using INSTANTIATE_TEST_CASE_P. These parameter sets
are given a name and then tied to the parametric fixture/test suite (i.e.
SweepTest). You can have any number of parameter sets for a given test suite
but beware if no values are assigned there is no warning or error and the
tests will simply be skipped. Parameters can be explicitly specified using the
Values wrapper, or combinations of parameters can be exhaustively combined
by using the Combine wrapper.

General GTest Notes

You don’t need to write a main entry point to run GTest. If you link
gtest_main during your compile GTest automatically handles this. If you
have special circumstances, it is still possible to write your own main
though.
You typically will want to use the EXPECT_* checks from GTest, but
ASSERT_* checks are available if circumstances warrant it.
If you are comparing floating point numbers, use the built in
EXPECT_FLOAT_* which automatically handles floating point discrepancies.
You can pipe a custom error message into the built in checks which can be
useful for debugging. The built in checks also do a reasonable job of
providing context to failures. The check arguments should passed in expected
first, then your value in questions, for example
EXPECT_EQ(expected_val, val_under_test) << "Custom message!".
Take a look at the output of the failing tests to see this in play:
```
/home/ben/git/gtest-example/test/src/failing_tests.cpp Failure
Value of: pass
  Actual: false
Expected: true
>> Expected non-fatal failure!
```
The TEST* macros accept a fixture/suite name and a test case name. The
fixture/suite name can and should be reused, but the test case name must be
unique. For example TEST_F(FixtureName, TestCaseName){}.

TODO

Add some functional tests
- Mock things with FFF (GMock can only do classes, not free functions)
Add back templating and run TYPED_TEST