Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance tests: Make theme versions consistent cross-env #50905

Merged
merged 7 commits into from
Jun 19, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .wp-env.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@
"wp-content/plugins/gutenberg": ".",
"wp-content/mu-plugins": "./packages/e2e-tests/mu-plugins",
"wp-content/plugins/gutenberg-test-plugins": "./packages/e2e-tests/plugins",
"wp-content/themes/gutenberg-test-themes": "./test/gutenberg-test-themes"
"wp-content/themes/gutenberg-test-themes": "./test/gutenberg-test-themes",
"wp-content/themes/gutenberg-test-themes/twentytwentyone": "https://downloads.wordpress.org/theme/twentytwentyone.1.7.zip",
"wp-content/themes/gutenberg-test-themes/twentytwentythree": "https://downloads.wordpress.org/theme/twentytwentythree.1.0.zip"
}
}
}
Expand Down
11 changes: 9 additions & 2 deletions bin/plugin/commands/performance.js
Original file line number Diff line number Diff line change
Expand Up @@ -337,8 +337,6 @@ async function runPerformanceTests( branches, options ) {
performanceTestDirectory,
'test/emptytheme'
),
'https://downloads.wordpress.org/theme/twentytwentyone.1.7.zip',
oandregal marked this conversation as resolved.
Show resolved Hide resolved
'https://downloads.wordpress.org/theme/twentytwentythree.1.0.zip',
],
env: {
tests: {
Expand All @@ -352,6 +350,15 @@ async function runPerformanceTests( branches, options ) {
performanceTestDirectory,
'packages/e2e-tests/plugins'
),
'wp-content/themes/gutenberg-test-themes':
path.join(
performanceTestDirectory,
'test/gutenberg-test-themes'
),
'wp-content/themes/gutenberg-test-themes/twentytwentyone':
'https://downloads.wordpress.org/theme/twentytwentyone.1.7.zip',
'wp-content/themes/gutenberg-test-themes/twentytwentythree':
'https://downloads.wordpress.org/theme/twentytwentythree.1.0.zip',
},
},
},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,12 @@ describe( 'Front End Performance', () => {
};

beforeAll( async () => {
await activateTheme( 'twentytwentythree' );
await activateTheme( 'gutenberg-test-themes/twentytwentythree' );
Copy link
Member

@oandregal oandregal May 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noting here what I've observed, so it's documented somewhere.

This PR maps the specific versions in use within the gutenberg-test-themes folder in the theme root (wp-content/themes) of the testing WordPress environment. As a result, the environment has two instances of the TT1 and TT3 themes: one whose stylesheet is twentytwentyone (latest version) and another with stylesheet gutenberg-test-themes/twentytwentyone (the bundled version, here's at 1.7 for TT1).

You can confirm this by going to the "Appearance > Themes" page (inspect the "activate" button of each instance to know its stylesheet):

Captura de ecrã de 2023-05-25 10-03-53

Also by asking docker: docker exec -it `docker ps -f name=tests-wordpress_1 -q` ls -lat /var/www/html/wp-content/themes.

Copy link
Member Author

@WunderBart WunderBart May 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for noting this down! 🙇 Just wanted to add that we can check the available test env themes also via the following wp-cli command:

npx wp-env run tests-cli 'wp theme list'

For this branch, it should return the following:

+-----------------------------------------+----------+--------+---------+
| name                                    | status   | update | version |
+-----------------------------------------+----------+--------+---------+
| emptytheme                              | inactive | none   | 1.0     |
| gutenberg-test-themes/emptyhybrid       | inactive | none   | 1.0     |
| gutenberg-test-themes/style-variations  | inactive | none   | 1.0     |
| gutenberg-test-themes/twentytwentyone   | inactive | none   | 1.7     |
| gutenberg-test-themes/twentytwentythree | active   | none   | 1.0     |
| twentyeleven                            | inactive | none   | 4.3     |
| twentyfifteen                           | inactive | none   | 3.4     |
| twentyfourteen                          | inactive | none   | 3.6     |
| twentynineteen                          | inactive | none   | 2.5     |
| twentyseventeen                         | inactive | none   | 3.2     |
| twentysixteen                           | inactive | none   | 2.9     |
| twentyten                               | inactive | none   | 3.8     |
| twentythirteen                          | inactive | none   | 3.8     |
| twentytwelve                            | inactive | none   | 3.9     |
| twentytwenty                            | inactive | none   | 2.2     |
| twentytwentyone                         | inactive | none   | 1.8     |
| twentytwentythree                       | inactive | none   | 1.1     |
| twentytwentytwo                         | inactive | none   | 1.4     |
+-----------------------------------------+----------+--------+---------+

As a comparison, we can list the themes used in dev env, which should not return the gutenberg-test-themes/* ones, for example:

> npx wp-env run cli 'wp theme list'

+-------------------+----------+--------+---------+
| name              | status   | update | version |
+-------------------+----------+--------+---------+
| emptytheme        | inactive | none   | 1.0     |
| twentyeleven      | inactive | none   | 4.3     |
| twentyfifteen     | inactive | none   | 3.4     |
| twentyfourteen    | inactive | none   | 3.6     |
| twentynineteen    | inactive | none   | 2.5     |
| twentyseventeen   | inactive | none   | 3.2     |
| twentysixteen     | inactive | none   | 2.9     |
| twentyten         | inactive | none   | 3.8     |
| twentythirteen    | inactive | none   | 3.8     |
| twentytwelve      | inactive | none   | 3.9     |
| twentytwenty      | inactive | none   | 2.2     |
| twentytwentyone   | inactive | none   | 1.8     |
| twentytwentythree | active   | none   | 1.1     |
| twentytwentytwo   | inactive | none   | 1.4     |
+-------------------+----------+--------+---------+

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Brilliant. Thanks for the tip!

await logout();
} );

afterAll( async () => {
saveResultsFile( __filename, results );
await activateTheme( 'twentytwentyone' );
} );

it( 'Report TTFB, LCP, and LCP-TTFB', async () => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ describe( 'Front End Performance', () => {
};

beforeAll( async () => {
await activateTheme( 'twentytwentyone' );
await activateTheme( 'gutenberg-test-themes/twentytwentyone' );
await logout();
} );

Expand Down
6 changes: 6 additions & 0 deletions packages/e2e-tests/specs/performance/post-editor.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import path from 'path';
* WordPress dependencies
*/
import {
activateTheme,
createNewPost,
saveDraft,
insertBlock,
Expand Down Expand Up @@ -83,6 +84,11 @@ describe( 'Post Editor Performance', () => {

let traceResults;

beforeAll( async () => {
// See https://github.com/WordPress/gutenberg/pull/50905/files#r1209014677;
await activateTheme( 'gutenberg-test-themes/twentytwentyone' );
} );

afterAll( async () => {
saveResultsFile( __filename, results );
deleteFile( traceFilePath );
Expand Down
1 change: 0 additions & 1 deletion packages/e2e-tests/specs/performance/site-editor.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,6 @@ describe( 'Site Editor Performance', () => {
saveResultsFile( __filename, results );
await deleteAllTemplates( 'wp_template' );
await deleteAllTemplates( 'wp_template_part' );
await activateTheme( 'twentytwentyone' );
} );

// Number of loading measurements to take.
Expand Down