metabase/hawk

Hawk

It watches your code like a hawk! You like tests, right? Then run them with our state-of-the-art Clojure test runner.

;;; this is not necessarily up to date; use the latest SHA in GitHub
{io.github.metabase/hawk {:sha "8813c2b0969e9f78025c267af0488c96792b35d1"}}

Hawk is a Clojure-CLI friendly wrapper around Eftest with some extra features and opinionated behavior. It started out as the Metabase test runner, but we spun it out so we can use it in other places too.

Example deps.edn config

{:aliases
 {:test
  {:extra-paths ["test"]
   :extra-deps  {io.github.metabase/hawk {:sha "ca1775da999ed066947bd37ca5710167f4adecaa"}}
   :exec-fn     mb.hawk.core/find-and-run-tests-cli}}}

Leiningen-style test selection

You can run tests against a single namespace or directory, or one test specifically, by passing :only [argument]:

Arguments to clojure -X are read in as EDN; for things other than plain symbols or numbers you usually need to wrap them in single quotes in your shell. Our test runner uses this argument to determine where to look for tests. Here's how different EDN forms are interpreted as our test runner:

Example commands:

Checking to make sure things don't happen during initialization

You can use mb.hawk.init/assert-tests-are-not-initializing to make sure things that shouldn't be happening as a side-effect of loading namespaces, such as initializing a database, are not happening where they shouldn't be.

(ns my.namespace
  (:require
   [mb.hawk.init]))

(defn initialize-database! []
  (mb.hawk.init/assert-tests-are-not-initializing "Don't initialize the database in a top-level form!")
  ...)

Fancy JUnit Output

Hawk automatically generates JUnit output using bespoke JUnit output code that prints diffs using humane-test-output. JUnit output is automatically output to target/junit, and only in :cli/ci mode. Not currently configurable! Submit a PR if you want to output it somewhere else.

Parallel Tests

Unlike Eftest, parallelization in Hawk tests is opt-in. This is mostly a byproduct of it beginning life as the Metabase test runner. All tests are ran synchronously unless they are given ^:parallel metadata (either the test itself, or the namespace).

Hawk includes mb.hawk.parallel/assert-test-is-not-parallel, which you can use to make sure things that shouldn't be ran in parallel tests are not:

(ns my.namespace
  (require [mb.hawk.parallel]))

(defn do-with-something-redefined [thunk]
  (mb.hawk.parallel/assert-test-is-not-parallel "Don't use do-with-something-redefined inside parallel tests!")
  (with-redefs [something something-else]
    (thunk)))

Run tests from the REPL

Run tests from the REPL the same way the CLI will run them:

(mb.hawk.core/find-and-run-tests-repl {:only ['my.namespace-test]})

Additional is assertion types

• re=: checks whether a string is equal to a regular expression
• partial=: like = but only compares stuff (using clojure.data/diff) that's in expected. Anything else is ignored.
• =?: see Approximately Equal

Test modes:

The Hawk test runner can run in one of three modes.

The mode is determined as follows:

1. If an explicit :mode is passed to the options map (e.g. :exec-args or CLI args passed to clojure -X), it is used;

2. Otherwise, if the env var HAWK_MODE or Java system property hawk.mode is specified, it is used;

3. Otherwise, if the env var CI or system property ci is set, :cli/ci will be used;

4. If you use mb.hawk.core/find-and-run-tests-cli as your :exec-fn, :cli/local will be used;

5. If you run tests from the REPL with mb.hawk.core/find-and-run-tests-repl, :repl will be used.

Matching Namespace Patterns

Tell the test runner to only run tests against certain namespaces with :namespace-pattern:

;; only run tests against namespaces that start with `my-project` and end with `test`
{:aliases
 {:test
  {:exec-fn    mb.hawk.core/find-and-run-tests-cli
   :exec-args {:namespace-pattern "^my-project.*test$"}}}}

Excluding directories

:exclude-directories passed in the options map will tell Hawk not to look for tests in those directories. This only works for directories on your classpath, i.e. things included in :paths! If you need something more sophisticated, please submit a PR.

{:aliases
 {:test
  {:exec-fn   mb.hawk.core/find-and-run-tests-cli
   :exec-args {:exclude-directories ["src" "resources" "shared/src"]}}}}

Skipping namespaces or vars with tags

You can optionally exclude tests in namespaces with certain tags by specifying the :exclude-tags option:

{:aliases
 {:test
  {:exec-fn   mb.hawk.core/find-and-run-tests-cli
   :exec-args {:exclude-tags [:my-project/skip-namespace]}}}}

or

clj -X:test :exclude-tags '[:my-project/skip-namespace]'

And adding it to namespaces like

(ns ^:my-project/skip-namespace my.namespace
  ...)

Skipping vars with options

Sometimes you don't want to modify source to skip a few tests. You can easily specify to skip individual tests with:

clj -X:dev:test :only metabase.util.queue-test :ignored '{:vars ["metabase.util.queue-test/bounded-transfer-queue-test" "metabase.util.queue-test/take-batch-test"]}'

This allows for exact matching on var names to prevent tests. A typical example might be a test that is failing due to a leap year. A typical fix might be someone prepares a patch, gets it through CI, and then all jobs hoping to pass CI must rebase on this change. An alternative is that some central store is updated with

{"ignored": {"vars": ["some.ns/date-test"]}}

And then all of CI can carry on while you fix this. Once fixed, remove this var from the central store and continue.

Only running tests against namespaces or vars with tags

The opposite of :exclude-tags -- you can only run tests against a certain set of tags with :only-tags. If multiple :only-tags are specified, only namespaces or vars that have all of those tags will be run.

:only-tags can be combined with :only and/or :exclude-tags.

clj -X:test :only-tags [:my-project/e2e-test]

will only run tests in namespaces like

(ns ^:my-project/e2e-test my-namespace
  ...)

or ones individually marked :my-project/e2e-test like

(deftest ^:my-project/e2e-test my-test
  ...)

Whole-Suite Hooks

You can specify hooks to run before or after the entire test suite runs like so:

(methodical/defmethod mb.hawk.hooks/before-run ::my-hook
  [_options]
  (do-something-before-test-suite-starts!))

(methodical/defmethod mb.hawk.hooks/after-run ::my-hook
  [_options]
  (do-cleanup-when-test-suite-finishes!))

options are the same options passed to the test runner as a whole, i.e. a combination of those specified in your deps.edn aliases as well as additional command-line options.

The dispatch value is not particularly important -- one hook will run for each dispatch value -- but you should probably make it a namespaced keyword to avoid conflicts, and give it a docstring so people know why it's there. The order the hooks are run in is indeterminate. The docstrings for before-run and after-run are updated automatically as new hooks are added; you can check it to see which hooks are in use. Note that hooks will not be ran unless the namespace they live in is loaded; this may be affected by :only options passed to the test runner.

Return values of methods are ignored; they are done purely for side effects.

Partitioning tests

You can divide a test suite into multiple partitions using the :partition/total and :partition/index keys. This is an easy way to speed up CI by diving large test suites into multiple jobs.

clj -X:test '{:partition/total 10, :partition/index 8}'
...
Running tests in partition 9 of 10 (575 tests of 5753)...
Finding tests took 46.6 s.
Running 575 tests
...

:partition/index is zero-based, e.g. if you have ten partitions (:partiton/total 10) then the first partition is 0 and the last is 9.

Tests are partitioned at the var (deftest) level after all tests are found the usual way, but all tests in any given namespace will always be split into the same partition. All namespaces that would be loaded if you were running the entire test suite are still loaded. Partitions are split as evenly as possible, but tests are guaranteed to be split deterministically into exactly the number of partitions you asked for.

Additional options

All other options are passed directly to Eftest; refer to its documentation for more information.

clj -X:test '{:fail-fast? true}'

License

Copyright © 2019-2026 James Reeves, 2023-2026 Metabase, Inc.

Distributed under the Eclipse Public License either version 2.0 or (at your option) any later version.