Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance tests: Make theme versions consistent cross-env #50905

Merged
merged 7 commits into from
Jun 19, 2023
Merged

Performance tests: Make theme versions consistent cross-env #50905

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
b315dd1
Upload [email protected] and [email protected] to test themes
WunderBart May 24, 2023
ebdefe0
Use uploaded themes in performance tests
WunderBart May 24, 2023
a96092c
Be explicit about which themes we use for perf tests
WunderBart May 24, 2023
af5d1c9
Map zip URLs instead of uploading
WunderBart May 25, 2023
6980769
Remove bundled themes
WunderBart May 25, 2023
80ed46e
Use fixed twentytwentyone for post editor perf specs
WunderBart Jun 19, 2023
2a18422
Merge remote-tracking branch 'origin' into refactor/upload-perf-test-…
WunderBart Jun 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions bin/plugin/commands/performance.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -337,8 +337,6 @@ async function runPerformanceTests( branches, options ) {
performanceTestDirectory,
'test/emptytheme'
),
'https://downloads.wordpress.org/theme/twentytwentyone.1.7.zip',
oandregal marked this conversation as resolved.
Show resolved Hide resolved
'https://downloads.wordpress.org/theme/twentytwentythree.1.0.zip',
],
env: {
tests: {
Expand All @@ -352,6 +350,11 @@ async function runPerformanceTests( branches, options ) {
performanceTestDirectory,
'packages/e2e-tests/plugins'
),
'wp-content/themes/gutenberg-test-themes':
path.join(
performanceTestDirectory,
'test/gutenberg-test-themes'
),
},
},
},
Expand Down
3 changes: 1 addition & 2 deletions packages/e2e-tests/specs/performance/front-end-block-theme.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,12 @@ describe( 'Front End Performance', () => {
};

beforeAll( async () => {
await activateTheme( 'twentytwentythree' );
await activateTheme( 'gutenberg-test-themes/twentytwentythree' );
Copy link
Member

@oandregal oandregal May 25, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noting here what I've observed, so it's documented somewhere.

This PR maps the specific versions in use within the gutenberg-test-themes folder in the theme root (wp-content/themes) of the testing WordPress environment. As a result, the environment has two instances of the TT1 and TT3 themes: one whose stylesheet is twentytwentyone (latest version) and another with stylesheet gutenberg-test-themes/twentytwentyone (the bundled version, here's at 1.7 for TT1).

You can confirm this by going to the "Appearance > Themes" page (inspect the "activate" button of each instance to know its stylesheet):

Captura de ecrã de 2023-05-25 10-03-53

Also by asking docker: docker exec -it `docker ps -f name=tests-wordpress_1 -q` ls -lat /var/www/html/wp-content/themes.

Copy link
Member Author

@WunderBart WunderBart May 25, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for noting this down! 🙇 Just wanted to add that we can check the available test env themes also via the following wp-cli command:

npx wp-env run tests-cli 'wp theme list'

For this branch, it should return the following:

+-----------------------------------------+----------+--------+---------+
| name                                    | status   | update | version |
+-----------------------------------------+----------+--------+---------+
| emptytheme                              | inactive | none   | 1.0     |
| gutenberg-test-themes/emptyhybrid       | inactive | none   | 1.0     |
| gutenberg-test-themes/style-variations  | inactive | none   | 1.0     |
| gutenberg-test-themes/twentytwentyone   | inactive | none   | 1.7     |
| gutenberg-test-themes/twentytwentythree | active   | none   | 1.0     |
| twentyeleven                            | inactive | none   | 4.3     |
| twentyfifteen                           | inactive | none   | 3.4     |
| twentyfourteen                          | inactive | none   | 3.6     |
| twentynineteen                          | inactive | none   | 2.5     |
| twentyseventeen                         | inactive | none   | 3.2     |
| twentysixteen                           | inactive | none   | 2.9     |
| twentyten                               | inactive | none   | 3.8     |
| twentythirteen                          | inactive | none   | 3.8     |
| twentytwelve                            | inactive | none   | 3.9     |
| twentytwenty                            | inactive | none   | 2.2     |
| twentytwentyone                         | inactive | none   | 1.8     |
| twentytwentythree                       | inactive | none   | 1.1     |
| twentytwentytwo                         | inactive | none   | 1.4     |
+-----------------------------------------+----------+--------+---------+

As a comparison, we can list the themes used in dev env, which should not return the gutenberg-test-themes/* ones, for example:

> npx wp-env run cli 'wp theme list'

+-------------------+----------+--------+---------+
| name              | status   | update | version |
+-------------------+----------+--------+---------+
| emptytheme        | inactive | none   | 1.0     |
| twentyeleven      | inactive | none   | 4.3     |
| twentyfifteen     | inactive | none   | 3.4     |
| twentyfourteen    | inactive | none   | 3.6     |
| twentynineteen    | inactive | none   | 2.5     |
| twentyseventeen   | inactive | none   | 3.2     |
| twentysixteen     | inactive | none   | 2.9     |
| twentyten         | inactive | none   | 3.8     |
| twentythirteen    | inactive | none   | 3.8     |
| twentytwelve      | inactive | none   | 3.9     |
| twentytwenty      | inactive | none   | 2.2     |
| twentytwentyone   | inactive | none   | 1.8     |
| twentytwentythree | active   | none   | 1.1     |
| twentytwentytwo   | inactive | none   | 1.4     |
+-------------------+----------+--------+---------+

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Brilliant. Thanks for the tip!

await logout();
} );

afterAll( async () => {
saveResultsFile( __filename, results );
await activateTheme( 'twentytwentyone' );
} );

it( 'Report TTFB, LCP, and LCP-TTFB', async () => {
Expand Down
2 changes: 1 addition & 1 deletion packages/e2e-tests/specs/performance/front-end-classic-theme.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ describe( 'Front End Performance', () => {
};

beforeAll( async () => {
await activateTheme( 'twentytwentyone' );
await activateTheme( 'gutenberg-test-themes/twentytwentyone' );
await logout();
} );

Expand Down
5 changes: 5 additions & 0 deletions packages/e2e-tests/specs/performance/post-editor.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import path from 'path';
* WordPress dependencies
*/
import {
activateTheme,
createNewPost,
saveDraft,
insertBlock,
Expand Down Expand Up @@ -83,6 +84,10 @@ describe( 'Post Editor Performance', () => {

let traceResults;

beforeAll( async () => {
await activateTheme( 'emptytheme' );
WunderBart marked this conversation as resolved.
Show resolved Hide resolved
} );

afterAll( async () => {
saveResultsFile( __filename, results );
deleteFile( traceFilePath );
Expand Down
27 changes: 27 additions & 0 deletions test/gutenberg-test-themes/twentytwentyone/404.php
View file
Open in desktop
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we could instead commit the zip binary and use that in wp-env so that we don't have to generate this giant diff?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, turns out we can! 😄 I tried passing the zip URL into mapped folder and it worked:

{
	"core": "WordPress/WordPress",
	"plugins": [ "." ],
	"themes": [ "./test/emptytheme" ],
	"env": {
		"tests": {
			"mappings": {
				"wp-content/plugins/gutenberg": ".",
				"wp-content/mu-plugins": "./packages/e2e-tests/mu-plugins",
				"wp-content/plugins/gutenberg-test-plugins": "./packages/e2e-tests/plugins",
				"wp-content/themes/gutenberg-test-themes": "./test/gutenberg-test-themes",
				"wp-content/themes/gutenberg-test-themes/twentytwentyone": "https://downloads.wordpress.org/theme/twentytwentyone.1.7.zip",
				"wp-content/themes/gutenberg-test-themes/twentytwentythree": "https://downloads.wordpress.org/theme/twentytwentythree.1.0.zip"
			}
		}
	}
}

It looks like it has to be a proper URL, though. Local path to a zip file does not work, so we'll need to rely on the availability of those versions. Having said that, I guess it's a good thing - if certain versions become unavailable then we probably should update as well. 😄

I'll make an update.

Copy link
Member Author

@WunderBart WunderBart May 25, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in af5d1c9.

/cc @oandregal

Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
<?php
/**
* The template for displaying 404 pages (not found)
*
* @link https://codex.wordpress.org/Creating_an_Error_404_Page
*
* @package WordPress
* @subpackage Twenty_Twenty_One
* @since Twenty Twenty-One 1.0
*/

get_header();
?>

<header class="page-header alignwide">
<h1 class="page-title"><?php esc_html_e( 'Nothing here', 'twentytwentyone' ); ?></h1>
</header><!-- .page-header -->

<div class="error-404 not-found default-max-width">
<div class="page-content">
<p><?php esc_html_e( 'It looks like nothing was found at this location. Maybe try a search?', 'twentytwentyone' ); ?></p>
<?php get_search_form(); ?>
</div><!-- .page-content -->
</div><!-- .error-404 -->

<?php
get_footer();
38 changes: 38 additions & 0 deletions test/gutenberg-test-themes/twentytwentyone/archive.php
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
<?php
/**
* The template for displaying archive pages
*
* @link https://developer.wordpress.org/themes/basics/template-hierarchy/
*
* @package WordPress
* @subpackage Twenty_Twenty_One
* @since Twenty Twenty-One 1.0
*/

get_header();

$description = get_the_archive_description();
?>

<?php if ( have_posts() ) : ?>

<header class="page-header alignwide">
<?php the_archive_title( '<h1 class="page-title">', '</h1>' ); ?>
<?php if ( $description ) : ?>
<div class="archive-description"><?php echo wp_kses_post( wpautop( $description ) ); ?></div>
<?php endif; ?>
</header><!-- .page-header -->

<?php while ( have_posts() ) : ?>
<?php the_post(); ?>
<?php get_template_part( 'template-parts/content/content', get_theme_mod( 'display_excerpt_or_full_post', 'excerpt' ) ); ?>
<?php endwhile; ?>

<?php twenty_twenty_one_the_posts_navigation(); ?>

<?php else : ?>
<?php get_template_part( 'template-parts/content/content-none' ); ?>
<?php endif; ?>

<?php
get_footer();
5 changes: 5 additions & 0 deletions test/gutenberg-test-themes/twentytwentyone/assets/css/custom-color-overrides.css
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
/**
* Custom Color Overrides
*
* This file is automatically populated if the user chooses custom colors in the Customizer.
*/
Loading