This PR is the second attempt to speed up monorepo installation by removing autoloading optimization setting from the configuration and performing the optimization where it is appropriate (composer post-install hooks and when building releases).
In the installation context, the composer post-install hook will execute a background task to dump the optimized autoloading, while the installation process completes other tasks.
In this PR we ar relocating playwright downloads caching into the monorepo action (behind input parameter). That consolidates monorepo-related caching and also reduces the time fetching the cache action in external workflows.
This PR drops composer autoloader optimization for the WooCommerce plugin to speed up the build process. The autoloader still gets optimized in `plugins/woocommerce/bin/build-zip.sh` file.
* Tooling: cache pnp deps
* Tooling: try fixing cache errors
* Tooling: try to parallelize js builds a bit
* Tooling: try to parallelize js builds a bit
* Tooling: try to parallelize js builds a bit
* Tooling: try to parallelize js builds a bit
* Tooling: try to parallelize js builds a bit
* Tooling: try to parallelize js builds a bit
* Tooling: try to parallelize js builds a bit
* Tooling: try to parallelize js builds a bit
* Tooling: try to parallelize js builds a bit
* Tooling: added pre-build steps to solve concurrency issues
* Tooling: limit workers
* Tooling: revert to original build command
* Tooling: playwright related tweaks
* Tooling: minor tweaks.
* Tooling: fix caching path.
* Tooling: trigger php builds to test deps caching.
* Tooling: trigger php builds to test deps caching.
* Tooling: lock caching to monorepo setup, so it doesn't slow down build steps.
* Tooling: lock Playwright cache to build, as build is pre-requisite for running tests.
* Tooling: added caching build-folders.
* Tooling: added caching build-folders.
* Tooling: added caching build-folders.
* Revert "Tooling: added caching build-folders."
This reverts commit c57b9dff13.
* Tooling: added caching build-folders.
* Tooling: cleanup sweep.
* Tooling: cleanup sweep.
* Tooling: added a changelog entry.
* Tooling: cleanup.
* Tooling: try speedup live branches build.
* Tooling: try speedup live branches build.
* Tooling: cleanup.
* Tooling: try speedup live branches build.
* Tooling: try speedup live branches build.
* * Use run_attempt in the artifact name
* Remove the Allure report steps
* Test change to trigger tests
* Only run the publish report step if there is an artifact-id
* Only run the publish report step if there is an artifact-id
* Revert test changes
* Also condition the deprecated publish report step
* Fix api tests results path
* Add changelog
* Run Core API tests - HPOS disabled on pull_request
* Update the artifacts name pattern to avoid conflicts
* Revert pull_request trigger
* Add a call to a new report workflow in woocommerce-test-reports repo
* Test change to trigger tests
* Update report title
* Test change to trigger tests
* Check REPORT_TITLE
* Add quotes on all arguments
* Update reporting configuration to all core e2e jobs
* Add changelog
* Use GITHUB_HEAD_REF for pull_request
* Set REPORT_TITLE for workflow dispatch
* Updated trigger from daily-e2e to daily-checks
Co-authored-by: Jon Lane <jon.lane@automattic.com>
Co-authored-by: Bart Kalisz <bartlomiej.kalisz@gmail.com>
Co-authored-by: Adrian Moldovan <3854374+adimoldovan@users.noreply.github.com>
* Fix the report name sent to woocommerce-test-reports workflow
* Add test type
* Test change
* Remove test type
* Merge artifacts to keep allure-results
* Overwrite artifacts if one already exists with the same name
* Test changes to trigger tests
* Fix allure-results and .state paths
* Filter tests for faster feedback
* Debug artifacts content
* Use name instead of pattern
* Use name instead of pattern
* Don't use an intermediate path
* Use artifact name and path instead of only pattern
* Use artifact name and path instead of only pattern
* Use artifact name and path instead of only pattern
* Remove testMatch filter to run all tests
* Revert test change
* Add changelog
* Deprecate pr-smoke-test.yml
* Add configs for each environment
* Add scripts
* Add arguments to playwright command
* Update pnpm scripts and existing e2e projects
* Fix shellcheck warning
* Update test env start for Gutenberg stable run
* Add changelog
* Remove unused var
* Set the USE_WP_ENV var
* Configure daily e2e jobs
* Rename some steps
* Add release checks jobs and deprecate smoke-test-release.yml
* Deprecate smoke-test-release.yml
* Add tests-release-checks.yml to trigger ci.yml on push to tags
* Add the pull_request trigger to test the workflow
* Remove the pull_request trigger to test the workflow
* Add the pull_request trigger to test the workflow
* Add the tag input in ci.yml
* Remove import and use core reference
* Only build if build-plugin is true
* Use release-artifact instead
* Split setup-monorepo step
* Use tag: 'nightly' to test
* Use inputs.tag directly in the wp-env setup step
* Add script to override the wp-env config
* Run the env config script in ci
* Set the artifact name
* Fix workflow syntax
* Add changelog
* Validate RELEASE_TAG and ARTIFACT_NAME
* Use real ref type and name
* Use real ref type and name
* Use real ref type and name
* Use github context instead
* Test with 9.0.0-beta.2
* Add push to tags trigger directly in ci.yml
* Configure one job to run with nightly-checks
* Remove empty lines
* Format events array one element per line
* Format events array one element per line
* Add flaky test reporting and issue creation for Blocks
This commit introduces the functionality for identifying and reporting flaky tests within our CI workflow.
- The reporter captures flaky test results and saves them to `flaky-tests` directory.
- If a test fails initially but passes upon retries, it is marked as flaky and a corresponding report is generated.
- The workflow creates GitHub issues for flaky tests, aiding in better visibility and tracking of these intermittent issues.
1. **Workflow Changes**:
- **Archive flaky test reports**: Adds a step to archive flaky test reports generated during the test runs.
- **Merge flaky test reports**: Introduces a new step to merge all flaky test reports from different shards.
- **Create GitHub issues for flaky tests**: Implements a job that uses the `@wordpress/report-flaky-tests` package to create GitHub issues for detected flaky tests when a PR is not in draft state and flaky test reports are present.
2. **New Flaky Test Reporter**:
- Adds `flaky-tests-reporter.ts` that defines a custom Playwright reporter for identifying and recording flaky tests.
3. **Playwright Configuration**:
- Updates `playwright.config.ts` to include the new flaky tests reporter in the list of reporters when running in CI.
4. **Demo Tests**:
- Adds `demo.spec.ts` containing sample tests to validate the flaky test reporting functionality.
Implementing this feature will help us track and address flaky tests more efficiently, ensuring more stable and reliable test suites.
* Remove draft condition for testing
* 'Merge failures artifacts' job shouldn't touch flaky test artifacts
* Try fixing if condition
* Add required permissions
* Let's try providing all the permissions
* Let's try providing all the permissions
* Create flaky tests issues when PR is merged with trunk
* Update if condition
* Run action for specific paths only
* Add changefile(s) from automation for the following project(s): woocommerce-blocks
* Remove Demo file
* Remove Demo file
* Use commit hash to use exact version of the script
* Replace label with team: Kirigami & Origami
---------
Co-authored-by: github-actions <github-actions@github.com>
* Deprecate pr-smoke-test.yml
* Add configs for each environment
* Add scripts
* Add arguments to playwright command
* Update pnpm scripts and existing e2e projects
* Fix shellcheck warning
* Update test env start for Gutenberg stable run
* Add changelog
* Remove unused var
* Set the USE_WP_ENV var
* Configure daily e2e jobs
* Rename some steps
* Update PR template and auto change log
* Ignore .github md files in linting
* Rebuilt monorepo-utils to solve conflict
* Updated regex pattern to include uppercase X
* Move the changelog entry not required checkbox on top of the description.
* Return the comment even if shouldAutomateNoChangelog is false
* Updated tests
* Rebuilt monorepo-utils
---------
Co-authored-by: Adrian Moldovan <3854374+adimoldovan@users.noreply.github.com>
* Add the slack-test-report util
* Error if the required GitHub context variables are not set
* Error if the required SLACK_CHANNEL env var is not set
* Use a more generic message to be able to report on jobs that are not tests, like build
* Add the alert-on-failure job in ci
* Run for pull_request to test the job
* Setup monorepo first
* Force a linting error
* Better logging
* Use inputs.trigger as report name if set
* Force an e2e test failure
* Set the commit message in the notification
* Use INPUT_TRIGGER for all notifications
* Revert test changes
* Add condition on github.event_name for lint jobs
* Update a file to trigger a lint job
* Update condition to run only on pull_request
* Revert test change
* Always run the Slack notification step
* Add pull_request test trigger
* Update report step
* Remove branch filter
* Try with push event and branch filter
* Pass secrets to ci.yml
* Update testIgnore pattern for Gutenberg project
* Add changelog
* Remove the test push trigger
* One more test run
* Removed push test trigger
* Add workflow_call event
* Add tests-daily-run.yml
* Remove workflow version
* Update input value
* Remove secrets from caller workflow
* Rename job
* Check context
* Check context
* Check workflow_call inputs
* Override github.event_name with inputs.trigger
* Fix input name
* Add config for daily e2e tests with Gutenberg
* Update env commands for Gutenberg run
* Add disableHpos env variable in ci-utils
* Revert the trigger to push
* Revert the env variable setting in ci-jobs
* Revert the env variable setting in ci-jobs
* Rebuild utils
* Update test env start command
* Define the Gutenberg Playwright project
* Use path relative to workspace root
* Update events for test projects
* Update events for test projects
* Add changelogs
* Remove the e2e tests with Gutenberg tests from daily runs
* Set the schedule and workflow_dispatch triggers
* Add --project default flag to only run the default tests in CI
* Renamed snapshots to match the new Playwright project name
* Add disableHpos test env variable
* Update events for HPOS disabled tests
* Add pull_request event to test the env
* Remove the pull_request event
* Add changelog
* Make the disableHpos var a boolean
* Also set the env in ci.yml
* Remove console log statement
* Print woocommerce_custom_orders_table_enabled
* Run test setup script again to check output
* Capture env setup output and print it
* Get woocommerce_custom_orders_table_enabled value in test env script
* Get more details on HPOS setting
* Remove the DISABLE_HPOS logic from test env setup script
* Use DISABLE_HPOS instead of ENABLE_HPOS everywhere
* Turn off no-console and jest/no-standalone-expect eslint rules for api core tests
* Test API tests with HPOS disabled
* Fix eslint error
* Clean up debug statements
* Remove pull_request test trigger
jonathansadowski
Vladimir Reznichenko
Adrian Moldovan
Jonathan Lane
Ron Rennick
Manish Menaria
Sam Seay
Bart Kalisz
Moon