Context
In software development we often use a build pipeline to perform a series of checks on a proposed change. The checks commonly include linting, type checking and unit tests, but can include things like checking for contract changes, performance reports, accessibility reports
Build pipelines will prevent a change from proceeding if any of these checks are failing.
What I’m not trying to solve
Build pipeline tools like BuildKite, Github Actions, Gitlab etc usually provide dashboards about pipeline behaviour, such as number of builds, percentage that are failing etc.
It is common to see tools provide reports on to a given pull request. For example, here’s a PR that has a report on test timings.
What I’m looking for – Aggregated reporting on specific issues
Often there’ll be a particular kind of error that we want to be aware of. A common one is flakey tests.
Or there might be a particular lint rule that we’re phasing in, so we’re allowing builds to pass, and we want to see a metrics of how many instances of that warning are existing over time, hopefully seeing it decline.
For this, we want our individual steps of the build pipeline to provide configurable data payloads to some aggregator, and then query the aggregator for dashboards etc.
I’m aware that tools like Buildkite etc might already provide such functionality, in which case what I’m looking for is a link to their documentation/the name of the concept in their terms.
