A lightweight and colourful test framework.

Alcotest exposes a simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run. See the manpage for details.

The API documentation can be found here. For information on contributing to Alcotest, see CONTRIBUTING.md.

A simple example (taken from examples/simple.ml):

Generated by the following test suite specification:

(* A module with functions to test *)
module To_test = struct
  let lowercase = String.lowercase_ascii
  let capitalize = String.capitalize_ascii
  let str_concat = String.concat ""
  let list_concat = List.append
end

(* The tests *)
let test_lowercase () =
  Alcotest.(check string) "same string" "hello!" (To_test.lowercase "hELLO!")

let test_capitalize () =
  Alcotest.(check string) "same string" "World." (To_test.capitalize "world.")

let test_str_concat () =
  Alcotest.(check string) "same string" "foobar" (To_test.str_concat ["foo"; "bar"])

let test_list_concat () =
  Alcotest.(check (list int)) "same lists" [1; 2; 3] (To_test.list_concat [1] [2; 3])

(* Run it *)
let () =
  let open Alcotest in
  run "Utils" [
      "string-case", [
          test_case "Lower case"     `Quick test_lowercase;
          test_case "Capitalization" `Quick test_capitalize;
        ];
      "string-concat", [ test_case "String mashing" `Quick test_str_concat  ];
      "list-concat",   [ test_case "List mashing"   `Slow  test_list_concat ];
    ]

The result is a self-contained binary which displays the test results. Use dune exec examples/simple.exe -- --help to see the runtime options.

Here's an example of a of failing test suite:

By default, only the first failing test log is printed to the console (and all test logs are captured on disk). Pass --show-errors to print all error messages.

Add (alcotest :with-test) to the depends stanza of your dune-project file, or "alcotest" {with-test} to your opam file. Use the with-test package variable to declare your tests opam dependencies. Call opam to install them:

$ opam install --deps-only --with-test .

You can then declare your test and link with Alcotest: (test (libraries alcotest …) …), and run your tests:

$ dune runtest

You can filter which tests to run by supplying a regular expression matching the names of the tests to execute, or by passing a regular expression and a comma-separated list of test numbers (or ranges of test numbers, e.g. 2,4..9):

$ ./simple.exe test '.*concat*'
Testing `Utils'.
This run has ID `6EK5G4SO'.

  [SKIP]        string-case            0   Lower case.
  [SKIP]        string-case            1   Capitalization.
  [OK]          string-concat          0   String mashing.
  [OK]          list-concat            0   List mashing.

Full test results in `~/project/_build/_tests/run/Utils/20261106-230254-6EK5G4SO'.
Test Successful in 0.001s. 2 tests run.

$ ./simple.exe test 'string-case' '1..3'
Testing `Utils'.
This run has ID `YSMUHQ8K'.

  [SKIP]        string-case            0   Lower case.
  [OK]          string-case            1   Capitalization.
  [SKIP]        string-concat          0   String mashing.
  [SKIP]        list-concat            0   List mashing.

Full test results in `~/project/_build/_tests/run/Utils/20261106-230255-YSMUHQ8K'.
Test Successful in 0.001s. 1 test run.

Note that you cannot filter by test case name (i.e. Lower case or Capitalization), you must filter by test name & number instead.

See the examples directory for more examples.

Alcotest.group nests test cases. They run in place, and are reported and selected under a dotted name:

Alcotest.run "suite"
  [ suite "storage"
      [ test "read" read;
        group ~tags:`Slow "backend" [ test "git" git ];
        test "write" write ] ]

$ ./test.exe test '^storage\.backend'

In general you should use `Quick tests: tests that are ran on any invocations of the test suite. You should only use `Slow tests for stress tests that are ran only on occasion (typically before a release or after a major change). These slow tests can be suppressed by passing the -q flag on the command line, e.g.:

$ ./test.exe -q # run only the quick tests
$ ./test.exe    # run quick and slow tests

A test says when it runs, and the command line sets up the context it runs in. The context is a set of tags that starts with quick and slow: -q removes slow, and --tags or ALCOTEST_TAGS add tags, or remove them with -.

open Alcotest

test_case "plain" `Quick f                       (* the default context *)
test_case "big" `Slow f                          (* the default context, not -q *)
test_case "smoke" `Always f                      (* always *)
test_case "fetch" (`Tag "ci") f                  (* only when asked for *)
test_case "tls" Tag.(v "ci" || v "release") f    (* when asked for either *)
test_case "dump" Tag.(slow && v "release") f     (* release, not with -q *)
test_case "local" Tag.(not (v "ci")) f           (* unless asked for ci *)

$ ./test.exe                         # plain, big, smoke, local
$ ./test.exe -q                      # plain, smoke, local
$ ./test.exe --tags=ci               # also fetch and tls, no local
$ ./test.exe --tags=release -q       # plain, tls, smoke, local, no dump
$ ./test.exe -q --tags=slow          # -q does not win: also big
$ ./test.exe --tags=ci,-quick,-slow  # fetch, tls, smoke
$ ALCOTEST_TAGS=ci dune test --force
$ ./test.exe tags                    # the tags these tests use

--tags replaces ALCOTEST_TAGS. Tags passed to --tags must be used by a test in that executable, or it stops with an error. ALCOTEST_TAGS applies to a whole dune workspace, where many executables do not use a tag, so it does not check them. An executable whose tests the tags all disable exits 0 with a warning naming the tags they use, and list shows each condition other than `Quick and `Slow.

With --junit, or ALCOTEST_JUNIT=1 in the environment, each run also writes a JUnit XML report, for CI systems such as GitLab, Jenkins or CircleCI. This is experimental: the location and contents of the report may change. The reports of all runs are in the same directory:

<log directory>/junit/<name>-<time>-<id>.xml

<name> is the name given to run. The log directory is _build/_tests of the dune workspace under dune test, or the one given by -o.

Each test has the file and line where it is defined, or where it failed. A test written as a plain tuple has no location: use test or test_case. The <testsuite> has the file that calls run.

As in other JUnit reports, a test that fails a check or calls fail is a <failure>, and a test that raises any other exception, such as failwith or assert false, is an <error>. Both make the run fail; CI systems count them separately.

The report is written at the end of the run, or when the process exits early, in which case it marks the running test as an <error>. Nothing is written if the process is killed.

After ALCOTEST_JUNIT=1 dune test, the reports are in _build/_tests/junit. Upload them even when tests fail. A CI job usually starts from a clean _build; otherwise the pattern also matches the reports of earlier runs. A test that dune does not run again, because its result is cached, writes no report: use dune test --force to run them all. Dune does not run a test again when only ALCOTEST_JUNIT changes.

GitLab:

test:
  variables:
    ALCOTEST_JUNIT: "1"
  script:
    - dune test
  artifacts:
    when: always
    reports:
      junit: _build/_tests/junit/*.xml

GitHub Actions, with action-junit-report:

- run: opam exec -- dune test
  env:
    ALCOTEST_JUNIT: 1
- uses: mikepenz/action-junit-report@v6
  if: success() || failure()
  with:
    report_paths: _build/_tests/junit/*.xml

CircleCI:

- run:
    command: dune test
    environment:
      ALCOTEST_JUNIT: 1
- store_test_results:
    path: _build/_tests/junit

Jenkins:

// after a `dune test` with ALCOTEST_JUNIT=1 in its environment
junit '_build/_tests/junit/*.xml'

Locally, with ALCOTEST_JUNIT=1 exported, the reports of earlier runs stay in _build/_tests/junit, so a JUnit tool such as junit2html can show how each test did across runs:

junit2html --report-matrix history.html _build/_tests/junit/*.xml

In most cases, the base tests are unit -> unit functions. However, it is also possible to pass an extra option to all the test functions by using 'a -> unit, where 'a is the type of the extra parameter.

In order to do this, you need to specify how this extra parameter is read on the command-line, by providing a Cmdliner term for command-line arguments which explains how to parse and serialize values of type 'a (note: do not use positional arguments, only optional arguments are supported).

For instance:

let test_nice i = Alcotest.(check int) "Is it a nice integer?" i 42

let int =
  let doc = "What is your preferred number?" in
  Cmdliner.Arg.(required & opt (some int) None & info ["n"] ~doc ~docv:"NUM")

let () =
  Alcotest.run_with_args "foo" int [
    "all", ["nice", `Quick, test_nice]
  ]

Will generate test.exe such that:

$ ./test.exe test
Usage: test.exe test [--help] [OPTION]… [NAME_REGEX] [TESTCASES]
test.exe: required option -n is missing

$ ./test.exe test -n 42
Testing `foo'.
This run has ID `GC4AOLNM'.

  [OK]          all          0   nice.

Full test results in `~/project/_build/_tests/run/foo/20261106-230624-GC4AOLNM'.
Test Successful in 0.000s. 1 test run.

Alcotest provides an Alcotest_lwt module that you could use to wrap Lwt test cases. The basic idea is that instead of providing a test function in the form unit -> unit, you provide one with the type Lwt_switch.t -> unit -> unit Lwt.t, and Alcotest_lwt.run returns a promise for you to run with Lwt_main.run.

However, there are a couple of extra features:

• If an async exception occurs, it will cancel your test case for you and fail it (rather than exiting the process).

• You get given a switch, which will be turned off when the test case finishes (or fails). You can use that to free up any resources.

For instance:

let free () = print_endline "freeing all resources"; Lwt.return ()

let test_lwt switch () =
  Lwt_switch.add_hook (Some switch) free;
  Lwt.async (fun () -> failwith "All is broken");
  Lwt_unix.sleep 10.

let () =
  Lwt_main.run @@ Alcotest_lwt.run "foo" [
    "all", [
      Alcotest_lwt.test_case "one" `Quick test_lwt
    ]
  ]

Will generate:

$ ./test.exe
Testing `foo'.
This run has ID `BSCZ4E0B'.

> [FAIL]        all          0   one.

┌──────────────────────────────────────────────────────────────────────────────┐
│ [FAIL]        all          0   one.                                          │
└──────────────────────────────────────────────────────────────────────────────┘
freeing all resources
File "test.ml", line 11, character 6:
[failure] All is broken
          Raised at Stdlib.failwith in file "stdlib.ml", line 29, characters 17-33
          Called from Lwt.Concurrent_composition.async in file "src/core/lwt.ml", line 2544, characters 10-14

Logs saved to `~/project/_build/_tests/run/foo/20261106-230256-BSCZ4E0B/all.000.output'.
 ──────────────────────────────────────────────────────────────────────────────

Full test results in `~/project/_build/_tests/run/foo/20261106-230256-BSCZ4E0B'.
1 failure! in 0.000s. 1 test run.

The README is pretty clear about that:

Alcotest is a lightweight and colourful test framework.

Alcotest is the only testing framework using colors!

More seriously, Alcotest is similar to ounit but it fixes a few of the problems found in that library:

• Alcotest has a nicer output, it is easier to see what failed and what succeeded and to read the log outputs of the failed tests;

• Alcotest uses combinators to define pretty-printers and comparators between the things to test.

Other nice tools doing different kind of testing also exist:

• qcheck does random generation and property testing (e.g. Quick Check);

• crowbar and bun are similar to qcheck, but use compiler-directed randomness, i.e. they take advantage of the AFL support the OCaml compiler;

• ppx_inline_tests allows to write tests in the same file as your source-code; they will be run only in a special mode of compilation.