Run Python Tests Skill
Run end-to-end Python tests after making changes to verify correctness.
Arguments
- No arguments: Run all Python tests
<filename>: Run all Python tests in the specified file<filename>:<test_name>: Run a specific Python test in the specified file<filename 1> <filename 2>: Run all Python tests in the specified files
Arguments provided: $ARGUMENTS
Instructions
Test Timeout
Some tests take longer to run than others. The TEST_TIMEOUT parameter controls how long each test is allowed to run before being terminated.
- Quick verification (preferred): Pass
TEST_TIMEOUT=20to get fast feedback. Tests that exceed this timeout will be terminated. - Full test run: Omit
TEST_TIMEOUTto let tests run with the default timeout (300 seconds).
Always start with a quick verification, using TEST_TIMEOUT=20. Faster feedback loops lead to faster iteration.
If there are timeouts during a quick verification run, check if the timed-out tests are relevant to the current task:
- If you can determine relevance autonomously (e.g., the test name clearly relates to the code you changed), re-run those specific tests without a timeout.
- If you cannot determine relevance, ask the user whether the timed-out tests should be re-run without a timeout.
All Tests
./build.sh RUN_PYTEST ENABLE_ASSERT=1 TEST_TIMEOUT=20All Tests From A Specific File
./build.sh RUN_PYTEST ENABLE_ASSERT=1 TEST_TIMEOUT=20 TEST="<filename without extension>"For example:
./build.sh RUN_PYTEST ENABLE_ASSERT=1 TEST_TIMEOUT=20 TEST="test_crash"for running tests from tests/pytests/test_crash.py.
All Tests From Multiple Files
./build.sh RUN_PYTEST ENABLE_ASSERT=1 TEST_TIMEOUT=20 TEST="<filename 1> <filename 2>"For example:
./build.sh RUN_PYTEST ENABLE_ASSERT=1 TEST_TIMEOUT=20 TEST="test_crash test_gc"for running tests from tests/pytests/test_crash.py and tests/pytests/test_gc.py.
Specific Test From A Specific File
./build.sh RUN_PYTEST ENABLE_ASSERT=1 TEST_TIMEOUT=20 TEST="<filename without extension>:<test_name>"For example:
./build.sh RUN_PYTEST ENABLE_ASSERT=1 TEST_TIMEOUT=20 TEST="test_crash:test_query_thread_crash"for running the test_query_thread_crash test from tests/pytests/test_crash.py.
Interpreting The Test Output
For each failed test, you'll see an error message with details about the failure, as seen *from the Python test runner*. Each failed test will also have an associated log file, located under tests/pytests/logs. The name of the log file changes with every test run, but it's included in the output of the test runner.
Report
After running the tests, put together a report:
- Number of tests passed
- Number of tests failed
- Number of tests skipped
- For each failing test:
- The error message reported in the test output, on the Python side - The stack trace from the server logs for the Redis server: - If the panic is in Rust code, include the Rust panic message and the Rust backtrace (from the # search_rust_backtrace section) - If the crash is in C code, include just the C backtrace. - Path to the log file, relative to the root of the repository