Copyright (C) 2008-2023 Free Software Foundation, Inc. See the end of the file for license conditions. This directory contains files intended to test various aspects of Emacs's functionality. Please help add tests! See the file file-organization.org for the details of the directory structure and file-naming conventions. For tests in the manual/ subdirectory, look there for separate README files, or look for instructions in the test files themselves. Emacs uses ERT, Emacs Lisp Regression Testing, for testing. See (info "(ert)") or https://www.gnu.org/software/emacs/manual/html_node/ert/ for more information on writing and running tests. Tests could be tagged by the developer. In this test directory, the following tags are recognized: * :expensive-test The test needs a serious amount of time to run. It is not intended to run on a regular basis by users. Instead, it runs on demand only, or during regression tests. * :nativecomp The test runs only if Emacs is configured with Lisp native compiler support. * :unstable The test is under development. It shall run on demand only. The Makefile sets the environment variable $EMACS_TEST_DIRECTORY, which points to this directory. This environment variable does not exist when the tests are run outside make. The Makefile supports the following targets: * make check Run all tests as defined in the directory. Expensive and unstable tests are suppressed. The result of the tests for .el is stored in .log. * make check-maybe Like "make check", but run only the tests for files which have unresolved prerequisites. * make check-expensive Like "make check", but run also the tests marked as expensive. * make check-all Like "make check", but run all tests. * make check- Like "make check", but run only the tests in test//*.el. is a relative directory path, which has replaced "/" by "-", like in "check-src" or "check-lisp-net". * make -or- make .log Run all tests declared in .el. This includes expensive tests. In the former case the output is shown on the terminal, in the latter case the output is written to .log. could be either a relative file name like "lisp/files-tests", or a package name like "files-tests". ERT offers selectors, which make it possible to filter out which test cases shall run. The make variable $(SELECTOR) gives you a simple mean to use your own selectors. The ERT manual describes how selectors are constructed, see (info "(ert)Test Selectors") or https://www.gnu.org/software/emacs/manual/html_node/ert/Test-Selectors.html You could use predefined selectors of the Makefile. "make SELECTOR='$(SELECTOR_DEFAULT)'" runs all tests for .el except the tests tagged as expensive or unstable. Other predefined selectors are $(SELECTOR_EXPENSIVE) (run all tests except unstable ones) and $(SELECTOR_ALL) (run all tests). If your test file contains the tests "test-foo", "test2-foo" and "test-foo-remote", and you want to run only the former two tests, you could use a selector regexp (note that the "$" needs to be doubled to protect against "make" variable expansion): make SELECTOR='"foo$$"' In case you want to use the symbol name of a test as selector, you can use it directly: make SELECTOR='test-foo-remote' Note that although the test files are always compiled (unless they set no-byte-compile), the source files will be run when expensive or unstable tests are involved, to give nicer backtraces. To run the compiled version of a test use make TEST_LOAD_EL=no ... Some tests might take long time to run. In order to summarize the tests with the longest duration, call make SUMMARIZE_TESTS= ... The backtrace of failing tests are truncated to the default value of 'ert-batch-backtrace-right-margin'. To see more of the backtrace, use make TEST_BACKTRACE_LINE_LENGTH= ... The tests are run in batch mode by default; sometimes it's useful to get precisely the same environment but run in interactive mode for debugging. To do that, use make TEST_INTERACTIVE=yes ... By default, ERT test failure summaries are quite brief in batch mode--only the names of the failed tests are listed. If the $EMACS_TEST_VERBOSE environment variable is set and non-empty, the failure summaries will also include the data from the failing test. If the $EMACS_TEST_JUNIT_REPORT environment variable is set to a file name, a JUnit test report is generated under this name. Some of the tests require a remote temporary directory (autorevert-tests.el, dnd-tests.el, eglot-tests.el, filenotify-tests.el, shadowfile-tests.el and tramp-tests.el). Per default, a mock-up connection method is used (this might not be possible when running on MS Windows). If you want to test a real remote connection, set $REMOTE_TEMPORARY_FILE_DIRECTORY to a suitable value in order to overwrite the default value: env REMOTE_TEMPORARY_FILE_DIRECTORY=/ssh:host:/tmp make ... There are also continuous integration tests on (see admin/notes/hydra) and (see admin/notes/emba). Both environments provide an environment variable, which could be used to determine, whether the tests run in one of these test environments. $EMACS_HYDRA_CI indicates the hydra environment, and $EMACS_EMBA_CI indicates the emba environment, respectively. If tests on these premises take too long, and it is needed to create a core dump for further analysis, the environment variable $EMACS_TEST_TIMEOUT could set a limit (in seconds) when this shall happen. (Also, see etc/compilation.txt for compilation mode font lock tests and etc/grep.txt for grep mode font lock tests.) This file is part of GNU Emacs. GNU Emacs is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. GNU Emacs is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with GNU Emacs. If not, see .