Testing¶
Ensuring that code changes work with new Elasticsearch versions, elasticsearch-py Python module
versions, and even new Python versions can be daunting. I’ve tried to make it easy to verify that
changes will work.
Setup Testing¶
Since nose testing basically died somewhere during Curator’s early days, a new testing
framework has become necessary. This is where pytest comes in.
Install pytest¶
From where your git cloned or forked repository is, you need to install Curator and its
dependencies, including for testing:
pip install -U '.[test]'
Manually install testing dependencies¶
These are indicated in pyproject.toml in the [project.optional-dependencies] subsection.
An example is listed below:
[project.optional-dependencies]
test = [
"requests",
"pytest >=7.2.1",
"pytest-cov",
]
These may change with time, and this document not be updated, so double check dependencies here before running the following:
pip install -U requests pytest pytest-cov
It should be simpler to run the regular method, but if you have some reason to do this manually, those are the steps.
Elasticsearch as a testing dependency¶
Warning
Not using a dedicated instance or instances for testing will result in deleted data! The tests perform setup and teardown functions which will delete anything in your cluster between each test.
Important
Integration tests will at least require Elasticsearch running on http://127.0.0.1:9200 or
TEST_ES_SERVER being set. The few tests that require a remote cluster to be configured will
need REMOTE_ES_SERVER to be set as well.
I will not cover how to install Elasticsearch locally in this document. It can be done, but it is much easier to use Docker containers instead.
If you host a dedicated instance somewhere else (and it must be unsecured for testing), you can specify this as an environment variable:
TEST_ES_SERVER="http://10.0.0.1:9201" \
pytest --cov=curator --cov-report html:cov_html
Additionally, four tests will be skipped if no value for REMOTE_ES_SERVER is provided.
TEST_ES_SERVER="http://10.0.0.1:9201" \
REMOTE_ES_SERVER="http://10.0.0.2:9201" \
pytest --cov=curator --cov-report html:cov_html
The REMOTE_ES_SERVER must be a separate instance altogether, and the main instance must
whitelist that instance for reindexing operations. If that sounds complicated, you’re not wrong.
There are remedies for this, and Curator comes with the necessary tools
Using Docker¶
Fortunately, Curator provides an out-of-the-box, ready to go set of scripts for setting up not only one container, but both containers necessary for testing the remote reindex functionality.
Warning
Do not use anything but create.sh and destroy.sh, or edit the Dockerfile.tmpl or
small.options files unless you’re actively trying to improve these scripts. These keep the
Elasticsearch containers lean. Do examine create.sh to see which Elasticsearch startup
flags are being used.
Create Docker containers for testing¶
Replace X.Y.Z with an Elasticsearch version:
$ cd /path/to/curator_code/docker_test/scripts
$ ./create.sh X.Y.Z
Docker image curator_estest:8.6.1 not found. Building from Dockerfile...
...
Waiting for Elasticsearch instances to become available...
This will create both Docker containers, and will print out the REMOTE_ES_SERVER line to use:
Please select one of these environment variables to prepend your 'pytest' run:
REMOTE_ES_SERVER="http://10.0.0.2:9201"
Clean up Docker containers used for testing¶
Note
The container names curator8-es-local and curator8-es-remote are hard coded in both
scripts so that destroy.sh will clean up exactly what create.sh made.
$ cd /path/to/curator_code/docker_test/scripts
$ ./destroy.sh
curator8-es-local
curator8-es-remote
curator8-es-local
curator8-es-remote
Cleanup complete.
The repo directory¶
/path/to/curator_code/docker_test/repo will be created by create.sh and deleted by
destroy.sh. This is used for snapshot testing and will only ever contain a few files. Anything
snapshotted there temporarily is cleaned by the teardown between tests.
Running Tests¶
Using pytest¶
Using the value of REMOTE_ES_SERVER you got from create.sh, or your own “remote”
Elasticsearch instance, testing is as simple as running:
Note
All of these examples presume that you are at the base directory of Curator’s code such that
the tests direcory is visible.
REMOTE_ES_SERVER="http://10.0.0.2:9201" pytest
Generating coverage reports¶
$ REMOTE_ES_SERVER="http://10.0.0.2:9201" pytest --cov=curator
............................................................................ [ 12%]
............................................................................ [ 24%]
............................................................................ [ 36%]
............................................................................ [ 48%]
............................................................................ [ 60%]
............................................................................ [ 72%]
............................................................................ [ 84%]
............................................................................ [ 96%]
........................ [100%]
---------- coverage: platform darwin, python 3.11.1-final-0 ----------
Name Stmts Miss Cover
------------------------------------------------------------
curator/__init__.py 10 0 100%
curator/_version.py 1 0 100%
curator/actions/__init__.py 14 0 100%
...
curator/validators/schemacheck.py 42 0 100%
------------------------------------------------------------
TOTAL 4023 1018 75%
475 passed in 4.92s
Generating an HTML coverage report¶
$ REMOTE_ES_SERVER="http://10.0.0.2:9201" pytest --cov=curator --cov-reporthtml:cov_html
............................................................................ [ 12%]
............................................................................ [ 24%]
............................................................................ [ 36%]
............................................................................ [ 48%]
............................................................................ [ 60%]
............................................................................ [ 72%]
............................................................................ [ 84%]
............................................................................ [ 96%]
........................ [100%]
---------- coverage: platform darwin, python 3.11.1-final-0 ----------
Coverage HTML written to dir cov_html
475 passed in 5.24s
At this point, you can view /path/to/curator_code/cov_html/index.html in your web browser. On
macOS, this is as simple as running:
$ open cov_html.index.html
It will open in your default browser.
Testing only unit tests¶
As unit tests do not require a remote Elasticsearch instance, adding the REMOTE_ES_SERVER
environment variable is unnecessary:
$ pytest tests/unit
You can also add --cov=curator and/or --cov=curator html:cov_html options.
Testing only integration tests¶
Most integration tests do not require a remote Elasticsearch instance, so adding the
REMOTE_ES_SERVER environment variable is unnecessary. Having a functional instance of
Elasticsearch at http://127.0.0.1:9200 or the TEST_ES_SERVER environment variable set is
required.
$ pytest tests/integration
You can also add --cov=curator and/or --cov=curator html:cov_html options.
This will result in 4 skipped tests:
$ pytest tests/integration
.......................................................................... [ 47%]
...............................s.s...ss................................... [ 94%]
......... [100%]
============================ short test summary info =============================
SKIPPED [1] tests/integration/test_reindex.py:110: REMOTE_ES_SERVER is not defined
SKIPPED [1] tests/integration/test_reindex.py:275: REMOTE_ES_SERVER is not defined
SKIPPED [1] tests/integration/test_reindex.py:157: REMOTE_ES_SERVER is not defined
SKIPPED [1] tests/integration/test_reindex.py:206: REMOTE_ES_SERVER is not defined
153 passed, 4 skipped, 7 warnings in 217.76s (0:03:37)
You can see the s in the test output. The message for each skipped test also clearly explains
that REMOTE_ES_SERVER is undefined. If you were to run this with REMOTE_ES_SERVER, it
would clear up the skipped tests.
Running specific tests¶
These examples are all derived from unit tests, but the same formatting applies to integration
tests as well. The path for those will just be tests/integration/test_file.py.
Important
Integration tests will at least require Elasticsearch running on http://127.0.0.1:9200 or
TEST_ES_SERVER being set. The few tests that require a remote cluster to be configured will
need REMOTE_ES_SERVER to be set as well.
Testing all tests within a given file¶
This will test every method of every class in test_file.py
$ pytest tests/unit/test_file.py
................................................... [100%]
51 passed in 0.32s
Testing all tests within a given class¶
This will test every method of class TestClass in test_file.py
$ pytest tests/unit/test_file.py::TestClass
.............. [100%]
14 passed in 0.35s
Testing one test within a given class¶
This will test method test_method of class TestClass in test_file.py
$ pytest tests/unit/test_file.py::TestClass::test_method
. [100%]
1 passed in 0.31s