(I mean for Test driver (software) framework, not to test a device driver.)
Goal:
Test any cmdline/GUI programs at process granularity, on a local machine. (support or scale-out in cloud is not required)
Sort of like Selenium or TestCafe in web testing, but for native programs.
Usage Cases:
- Spawn a non-interactive cmdline program with arguments.
- Check its exit code/signal for success.
- Filter its stdout/stderr to decide test success.
- Read output files to decide test success.
- Spawn a interactive GUI program with arguments and a domain socket fd.
- Wait (check) its status by reading live stdout/stderr stream and/or read from IPC.
- Send window system events and/or IPC packets to control its behavior.
- Check outputs as 1. does.
- Linux first. Windows support is nice-to-have, or find another framework.
- There is no complex dependencies between tests, as long as serial or parallel run can be controlled (at per-suite or per-run basis).
Expected features on Linux:
- Spawn any program and catch exit code/signal/stdout and other files.
- Spawn or run code for test prepare and clean-up/verification.
- Catch spawned program’s stdout/stderr lively as output.
- Two-way communication with spawned program through packet native IPC. (e.g. AF_UNIX SOCK_DGRAM)
- Send window system (x11, wayland) event to spawned program as input.
- Optionally support windows (with IPC, win32 WM/input events), or find another framework.
Alternative
Find a cross-platform sub-process framework to handle the complex details of sub-process, TTY and IPC handling. So that I could easily build a test driver on-top of it.
Workflow frameworks like chromium recipe-py seems way too convoluted (and under documented) for the test driver use case.
){kind=link}