307d893549
* Add flaky test reporting and issue creation for Blocks This commit introduces the functionality for identifying and reporting flaky tests within our CI workflow. - The reporter captures flaky test results and saves them to `flaky-tests` directory. - If a test fails initially but passes upon retries, it is marked as flaky and a corresponding report is generated. - The workflow creates GitHub issues for flaky tests, aiding in better visibility and tracking of these intermittent issues. 1. **Workflow Changes**: - **Archive flaky test reports**: Adds a step to archive flaky test reports generated during the test runs. - **Merge flaky test reports**: Introduces a new step to merge all flaky test reports from different shards. - **Create GitHub issues for flaky tests**: Implements a job that uses the `@wordpress/report-flaky-tests` package to create GitHub issues for detected flaky tests when a PR is not in draft state and flaky test reports are present. 2. **New Flaky Test Reporter**: - Adds `flaky-tests-reporter.ts` that defines a custom Playwright reporter for identifying and recording flaky tests. 3. **Playwright Configuration**: - Updates `playwright.config.ts` to include the new flaky tests reporter in the list of reporters when running in CI. 4. **Demo Tests**: - Adds `demo.spec.ts` containing sample tests to validate the flaky test reporting functionality. Implementing this feature will help us track and address flaky tests more efficiently, ensuring more stable and reliable test suites. * Remove draft condition for testing * 'Merge failures artifacts' job shouldn't touch flaky test artifacts * Try fixing if condition * Add required permissions * Let's try providing all the permissions * Let's try providing all the permissions * Create flaky tests issues when PR is merged with trunk * Update if condition * Run action for specific paths only * Add changefile(s) from automation for the following project(s): woocommerce-blocks * Remove Demo file * Remove Demo file * Use commit hash to use exact version of the script * Replace label with team: Kirigami & Origami --------- Co-authored-by: github-actions <github-actions@github.com> |
||
|---|---|---|
| .. | ||
| woo-ai | ||
| woocommerce | ||
| woocommerce-admin | ||
| woocommerce-beta-tester | ||
| woocommerce-blocks | ||