Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trying out yarn #39121

Closed
wants to merge 24 commits into from
Closed

Trying out yarn #39121

wants to merge 24 commits into from

Conversation

blowery
Copy link
Contributor

@blowery blowery commented Jan 29, 2020

Try out yarn!

  • ditch the npm package-lock
  • get a yarn install to actually work (needs Drop engines requirement color-studio#23, currently bypassing by running yarn --ignore-engines)
  • turn on workspaces
  • update file: refs in various package.json's to semver refs based on current monorepo package versions
  • update circle build
  • update docker build
  • get e2e tests working - likely need to turn it into a workspace
  • update peerDependency warnings - this probably involve adding the necessarydevDependencies to each of the workspaces (will do this in a future PR to keep this PR readable)
  • check whether assumptions still hold true in dep scripts
  • update renovate?
  • remove usage of --ignore-engines
  • decide if we want to enforce yarn-deduplicate
  • update lerna config to use yarn and workspaces
  • ...

@matticbot
Copy link
Contributor

@jameslnewell
Copy link
Contributor

jameslnewell commented Jan 29, 2020

While running npm start I was getting errors with wpcom trying to use the symlinked package in package/wpcom.js/src/index.js. It looks like this package is basically a historical copy as there's no file: references and no prepare script to build the files.

The make command in its package.json didn't produce workable output so I've moved the whole package from packages/wpcom.js/ to legacy/wpcom.js so yarn will install the published version and avoid symlinking the local version.

@matticbot
Copy link
Contributor

matticbot commented Jan 29, 2020

Here is how your PR affects size of JS and CSS bundles shipped to the user's browser:

App Entrypoints (~18 bytes removed 📉 [gzipped])

name                   parsed_size           gzip_size
entry-gutenboarding         +256 B  (+0.0%)      -58 B  (-0.0%)
entry-main                   +90 B  (+0.0%)      +22 B  (+0.0%)
entry-login                  +90 B  (+0.0%)      +25 B  (+0.0%)
entry-domains-landing        +56 B  (+0.0%)       -7 B  (-0.0%)

Common code that is always downloaded and parsed every time the app is loaded, no matter which route is used.

Sections (~4819 bytes removed 📉 [gzipped])

name              parsed_size           gzip_size
checkout             -22328 B  (-1.7%)    -4332 B  (-1.3%)
devdocs               +1863 B  (+1.2%)       +3 B  (+0.0%)
stats                   -37 B  (-0.0%)      -37 B  (-0.0%)
settings-writing        -37 B  (-0.0%)      -22 B  (-0.0%)
post-editor             -37 B  (-0.0%)       +8 B  (+0.0%)
help                    -37 B  (-0.0%)      -25 B  (-0.0%)
concierge               -37 B  (-0.0%)     -147 B  (-0.2%)
backups                 -37 B  (-0.0%)     -140 B  (-0.2%)
activity                -37 B  (-0.0%)      -58 B  (-0.1%)
comments                -36 B  (-0.0%)      -45 B  (-0.0%)
woocommerce             +22 B  (+0.0%)      -12 B  (-0.0%)
purchases               +22 B  (+0.0%)      -12 B  (-0.0%)

Sections contain code specific for a given set of routes. Is downloaded and parsed only when a particular route is navigated to.

Async-loaded Components (~83040 bytes added 📈 [gzipped])

name                                   parsed_size           gzip_size
async-load-design-playground              +98266 B  (+5.8%)   +27747 B  (+7.1%)
async-load-design                         +98266 B  (+5.6%)   +27747 B  (+6.8%)
async-load-design-blocks                  +98250 B  (+3.5%)   +27707 B  (+4.2%)
async-load-lib-happychat-connection         +270 B  (+0.4%)       +9 B  (+0.1%)
async-load-signup-steps-clone-point          -37 B  (-0.0%)      -23 B  (-0.1%)
async-load-blocks-inline-help-popover        -37 B  (-0.0%)     -116 B  (-0.2%)
async-load-blocks-calendar-popover           -36 B  (-0.0%)      -31 B  (-0.1%)

React components that are loaded lazily, when a certain part of UI is displayed for the first time.

Legend

What is parsed and gzip size?

Parsed Size: Uncompressed size of the JS and CSS files. This much code needs to be parsed and stored in memory.
Gzip Size: Compressed size of the JS and CSS files. This much data needs to be downloaded over network.

Generated by performance advisor bot at iscalypsofastyet.com.

@@ -119,7 +119,7 @@
"path-browserify": "1.0.0",
"percentage-regex": "3.0.0",
"phone": "2.4.2",
"photon": "file:../packages/photon",
"photon": "^3.0.0",
Copy link
Contributor

@jameslnewell jameslnewell Jan 29, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably want to use ^ syntax everywhere there was a file: reference - otherwise everytime a local package is bumped we'll have to go and update all the references to it so yarn continues to symlink the workspace instead of installing the specific version from the registry.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 what happens if local version is bumped v4 in this case? Would we then want to use * instead?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah for private packages like client/* it might be a good idea to encourage the developer who makes a change to also update any usages across the codebase.

In packages published to the registry, * would have unintended affects.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a very tricky subject. If we don't force version sync across all the monrepo (ie. update all usages of a package when it changes version), we may end up in this scenario:

  • ./packages/packageA depends on [email protected]
  • ./packages/packageB is bumped to 2.0.0, with breaking changes
  • Dependency in ./packages/packageA is not updated.

That will make ./packages/packageA code incompatible with ./packages/packageB, which will be a source of confusion and a pain to debug.

Also, consumers of packageA and packageB may end up with two copies: [email protected] (because trans dep from packageA) and [email protected] (because they are on the latest version).

One of the downsides is that updating a package will have a "cascade" effect that may end up updating a ton of packages, therefore requiring a lot of releases. And by extension, adding extra work for consumers of our packages if they want to be up to date.

Copy link
Contributor

@jameslnewell jameslnewell Mar 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm all for adding a version constraint that enforces workspaces are always symlinked (the goal of adding ^ was to keep it symlinked for longer). I tried yarn@2 with its constraints in mind but the ecosystem isn't stable enough yet. Maybe we can use manypkg to enforce that for now?

@blowery
Copy link
Contributor Author

blowery commented Jan 29, 2020

@jsnajdr and @sgomes are working on getting wpcom.js into packages properly in some other PRs. There's also #39139 coming in that will likely follow the path same kind of path.

package.json Outdated Show resolved Hide resolved
@jameslnewell
Copy link
Contributor

jameslnewell commented Jan 30, 2020

The remaining CI failure:

  Module <rootDir>/node_modules/@wordpress/jest-preset-default/scripts/setup-globals.js in the setupFiles option was not found.
         <rootDir> is: /home/circleci/wp-calypso/apps/full-site-editing

Is due to @wordpress/jest-preset-default containing fixed paths instead of relying on the traditional NodeJS resolution algorithm e.g. https://github.com/WordPress/gutenberg/blob/master/packages/jest-preset-default/jest-preset.js#L8

Converting the paths to use require.resolve gets things working. I'll work on a PR.

Edit: WordPress/gutenberg#19957

@blowery
Copy link
Contributor Author

blowery commented Jan 30, 2020

yarn-deduplicate seems like a good idea. We could enforce it via a post-install script perhaps? Not sure how that will tie in with Renovate though...

@jameslnewell jameslnewell force-pushed the try/yarn-package-manager branch from 443b52a to 66d84cf Compare February 9, 2020 21:46
@blowery blowery marked this pull request as ready for review February 11, 2020 13:10
@blowery blowery requested review from a team as code owners February 11, 2020 13:10
@blowery
Copy link
Contributor Author

blowery commented Feb 11, 2020

fse just moved to dotcom-fse

@blowery blowery force-pushed the try/yarn-package-manager branch from 48dc7be to 7522e25 Compare February 11, 2020 15:10
@blowery
Copy link
Contributor Author

blowery commented Feb 11, 2020

rebased this and force-pushed it @jameslnewell

@blowery
Copy link
Contributor Author

blowery commented Feb 11, 2020

whoa

Tests passed!

@blowery blowery added [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. and removed [Status] In Progress labels Feb 11, 2020
@blowery blowery force-pushed the try/yarn-package-manager branch from e90f00e to 5a9aebb Compare February 11, 2020 16:51
lerna.json Outdated Show resolved Hide resolved
.circleci/config.yml Outdated Show resolved Hide resolved
@blowery blowery force-pushed the try/yarn-package-manager branch from ac0401e to a6071b9 Compare February 11, 2020 21:23
.circleci/config.yml Outdated Show resolved Hide resolved
blowery and others added 21 commits April 7, 2020 09:06
Inital moves to yarn workspaces
- add workspaces to root package.json
- move to actual semver versions for monorepo packages instead of file: refs

replaced file: with local package version, updated npm run and circleci

update docker

ignore engines

fix install

moved wpcom.js

remove npm lock file, fix deps file, update wpcom dep

use ^ wherever there was a file: reference

update @automattic/color-studio to version that doesn't spec engines

resolve a single instance of @emotion/core

remove --ignore-engines

update lerna config to teach it about yarn and workspaces

add e2e tests to workspaces

cleanup after rebase

updated @wordpress/jest-preset-default

resolve delete conflicts

Inital moves to yarn workspaces
- add workspaces to root package.json
- move to actual semver versions for monorepo packages instead of file: refs

replaced file: with local package version, updated npm run and circleci

ignore engines

moved wpcom.js

remove npm lock file, fix deps file, update wpcom dep

update @automattic/color-studio to version that doesn't spec engines

resolve a single instance of @emotion/core

remove --ignore-engines

add e2e tests to workspaces

cleanup after rebase

updated @wordpress/jest-preset-default

updated lock

use immutable instead of frozen-lockfile

remove package-lock.json from lerna ignores

remove unnecessary -- when running typecheck

spec the yarn version we want

tell renovate to run yarn-deduplicate --fewer after each update run

loosen yarn version to match up with what circle ci has

npm->yarn for various steps and tasks

move back to --frozen-lockfile, --immutable is a 2.0 thing

Update docs/lockfile.md

Co-Authored-By: Marin Atanasov <[email protected]>

Update docs/monorepo.md

Co-Authored-By: Marin Atanasov <[email protected]>

update lockfile docs

update install script

fix grammar in lockfile docs

remove redundant --

update npm text and commands in various locations

fix missed conflicts

bring back package-lock.json for e2e tests

cleanup after rebase

update new references to npm to use yarn

fix file: reference

fix lock file and usage of npm

resolve conflict

fix typings issues after rebase due to multiple versions of a package
@scinos scinos force-pushed the try/yarn-package-manager branch from ce82ff1 to 673880f Compare April 7, 2020 07:09
@scinos scinos mentioned this pull request Apr 8, 2020
11 tasks
@scinos
Copy link
Contributor

scinos commented Apr 17, 2020

Merged as part of #41140

@scinos scinos closed this Apr 17, 2020
@matticbot matticbot removed the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label Apr 17, 2020
@scinos scinos deleted the try/yarn-package-manager branch September 11, 2020 07:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants