Improve the build integration with create react app

CONTRIBUTING.md

@@ -45,49 +45,15 @@ to enable debugging in VS code:
 
 There are also `React Developer Tools` available on the chrome web store to improve inspection of React within Chrome.
 
-### Cannot debug imported packages
-
-Unfortunately this doesn't successfully debug imported packages.
-You can set a breakpoint in the UI code, and step into an imported function --
-but the debugger shows the wrong location (line number) in the imported module.
-The problem is some combination of create react app (or webpack), typescript, and monorepo).
-
-This solution might probably work, but I haven't tried it:
-
-- https://github.com/minheq/monorepo-cra-source-map
-
-Other relevant links might be:
-
-- https://stackoverflow.com/questions/42708484/what-is-the-module-package-json-field-for
-- https://www.typescriptlang.org/docs/handbook/project-references.html
-- https://stackoverflow.com/questions/59867657/create-react-app-typescript-in-monorepo-code-sharing
-- https://github.com/webpack/webpack/issues/4674
-
 ## Project configuration
 
 The project configuration is based on `monorepo-demo` --
-for more details see [Creating the repo](MONOREPO.md#creating-the-repo).
-
-After you get it from GitHub, run `yarn install` and `yarn build` before you try to run it.
-
-## Temporary bug and its work-around
-
-The `yarn watch` command doesn't work on my machine --
-it works correctly in the `monorepo-demo` project, but on my machine it hangs after displaying this message:
-
-```
-app-react: Starting the development server...
-shared-lib: 11:50:53 PM - Found 0 errors. Watching for file changes.
-```
+for more details see:
 
-IMO the `package.json` files are the same --
-and the difference is that this project uses a newer version of `create-react-apps`, and a newer version of TypeScript, for no important reason.
+- [How I initially created the repo](MONOREPO.md#how-i-initially-created-the-repo)
+- [Better integration with Create React App](MONOREPO.md#better-integration-with-create-react-app).
 
-I think that's caused by this bug -- https://github.com/facebook/create-react-app/pull/8700 -- whose fix was merged into master two days ago and is not yet released.
-
-I could revert to an earlier version while waiting for the fix, but won't -- will instead update `create-react-apps` when a fix is released.
-
-The temporary work-around, instead of running `yarn watch`, is to run `yarn watch-lib` and `yarn watch-ui` in two different command windows.
+After you get it from GitHub, run `yarn install` and `yarn build` before you try to run it.
 
 ## Watch during development
 

MONOREPO.md

@@ -2,7 +2,7 @@ This is what is called a "monorepo" -- several projects/packages are contained i
 
 For more information, search the web using words like "`monorepo`", "`lerna`", "`yarn workspace`".
 
-## Creating the repo
+## How I initially created the repo
 
 I used the following commands to create the initial repo.
 
@@ -21,7 +21,13 @@ I used the following commands to create the initial repo.
     "packages/*"
   ]
   ```
-- `lerna import` to import an existing monolithic repository
+- `lerna import` to import an existing monolithic CRA repository
+
+  Other sites (see for example
+  [here](https://github.com/facebook/create-react-app/issues/1333#issuecomment-384978840))
+  warn than running `create-react-app` within a monorepo
+  would interfere with the monorepo's hoisting.
+
 - Edit [`package.json`](./package.json) to add `scripts`,
   which are implemented using `lerna run`,
   which invoke scripts either in all packages or scoped to specific packages.
@@ -54,28 +60,79 @@ Some example projects to imitate:
 
 The configuration of this repo is based on the first of these i.e. `monorepo-demo`.
 
-## Package names
+## Better integration with Create React App
 
-In theory the packages names might be scoped, with names like `@monorepo/ui-react` or `@cwellsx/ui-react`.
+There's an outstanding CRA issue:
+[Support Lerna and/or Yarn Workspaces - Issue #1333](https://github.com/facebook/create-react-app/issues/1333)
 
-In fact the packages are not for reuse in other projects outside this monorepo,
-so they will not be uploaded to npmjs.com -- and so it doesn't matter that the package names
-may conflict with some already-published package.
+My first attempt (above) was mostly successful:
 
-If this is changed in future to add a scope, beware that various configuration files and script parameters sometimes need a package name, sometimes a directory name.
+- Component packages are built separately/previously
+- Component packages are then available to and included into the CRA app, just like any other dependency would be in `node_modules`
+
+Unfortunately I found that, when I debugged the CRA app, the debugger
+couldn't set breakpoints in nor properly step into the TypeScript in component packages.
+
+That might (I'm not sure) be because WebPack is transpiling
+the `node_modules` which makes their source maps out of date.
+
+So I looked for another way to do this and found it here:
+
+- https://github.com/viewstools/yarn-workspaces-cra-crna
+
+The steps are:
+
+- Define `module` fields in `package.json` to point to the uncompiled `*.ts` files
+
+  See also [What is the “module” package.json field for?](https://stackoverflow.com/questions/42708484/what-is-the-module-package-json-field-for)
+
+- Override [this line in `webpack.config.js`](https://github.com/facebook/create-react-app/blob/fa648daca1dedd97aec4fa3bae8752c4dcf37e6f/packages/react-scripts/config/webpack.config.js#L399) ...
+
+  ```
+  include: paths.appSrc,
+  ```
+
+  ... to make it, instead of one path, an array of all the paths which webpack should be willing to transpile.
+
+The are various ways to implement the change to `webpack.config.js`:
+
+- Using `craco`
+
+  https://github.com/facebook/create-react-app/issues/1333#issuecomment-587415796
+
+- Using `customize-cra`:
+
+  https://github.com/facebook/create-react-app/issues/1333#issuecomment-593667643
+
+- Using `react-app-rewired`:
+
+  https://github.com/viewstools/yarn-workspaces-cra-crna  
+  https://github.com/viewstools/yarn-workspaces-cra-crna/blob/master/react-app-rewire-yarn-workspaces/index.js
 
-## Module type
+- Forking the whole `create-react-app` repository:
 
-The `tsconfig.shared.json` specifies
+  https://github.com/bradfordlemley/create-react-app
 
-```
-"module": "commonjs"
-```
+Anyway I did it using `react-app-rewired` as described in
+https://github.com/viewstools/yarn-workspaces-cra-crna
 
-Other projects often specify `"module": "esnext"` which works because Babel converts it to old-style modules under the hood.
+I needed to do something else as well: because the TypeScript source is in a `src` subdirectory
+I had to change the `import` statements in the app to import from
+`client\src` instead of from `client`.
+Ideally I should have been able to, instead, fix this with
+`baseUrl` and `paths` in the `tsconfig.json` however these
+are not supported by CRA.
 
-In this repository the `prebuild-data` project is built and run and Node.js and isn't processed by Babel,
-so we specify the old-style modules using the TypeScript option.
+Given this new method it's sufficient to run `yarn build` in the `ui-react` project --
+its build will now also build all its dependencies.
+It's no longer necessary to build or pre-build the dependencies explicitly.
 
-Alternatively this wouldn't be needed if the project depended on using a relatively new version of Node.js -- see
-[How can I use an es6 import in node?](https://stackoverflow.com/a/45854500/49942)
+## Package names
+
+In theory the packages names might be scoped, with names like `@monorepo/ui-react` or `@cwellsx/ui-react`.
+
+In fact the packages are not for reuse in other projects outside this monorepo,
+so they will not be uploaded to npmjs.com -- and so it doesn't matter that the package names
+may conflict with some already-published package.
+
+If this is changed in future to add a scope, beware that various configuration files and script parameters sometimes need a package name, sometimes a directory name.

package.json

@@ -18,11 +18,9 @@
     "install": "lerna bootstrap",
     "clean": "yarn run unbuild && lerna run clean && rimraf node_modules",
     "test": "lerna run test",
-    "build": "lerna run build",
+    "build": "lerna run --scope ui-react build",
     "unbuild": "lerna run unbuild",
-    "watch-lib": "lerna run --parallel --ignore ui* watch",
-    "watch-ui": "lerna run --scope ui-react watch",
-    "watch": "lerna run --parallel watch",
+    "watch": "lerna run --scope ui-react watch",
     "show-dependencies": "show-dependencies.bat",
     "build-docs": "lerna run build-docs"
   }

packages/client/package.json

@@ -4,6 +4,7 @@
   "description": "The client-side API imported by the UI",
   "main": "./lib/index.js",
   "types": "./lib/index.d.ts",
+  "module": "./src/index.ts",
   "private": true,
   "dependencies": {
     "shared-lib": "0.1.0",

packages/server-data/package.json

@@ -4,6 +4,7 @@
   "description": "Mock data created by prebuild-data for the server",
   "main": "./lib/images.js",
   "types": "./lib/images.d.ts",
+  "module": "./src/images.ts",
   "private": true,
   "dependencies": {
     "shared-lib": "0.1.0"

packages/server-mock/package.json

@@ -4,6 +4,7 @@
   "description": "Implement a mock server",
   "main": "./lib/index.js",
   "types": "./lib/index.d.ts",
+  "module": "./src/index.ts",
   "private": true,
   "dependencies": {
     "shared-lib": "0.1.0",

packages/server-mock/src/index.ts

@@ -1,2 +1,3 @@
 export { loginUser } from "server";
-export { mockFetch, SimpleResponse } from "./mock";
+export { mockFetch } from "./mock";
+export type { SimpleResponse } from "./mock";

packages/server-types/package.json

@@ -4,6 +4,7 @@
   "description": "Type definitions used in the server and used by prebuild-data",
   "main": "./lib/index.js",
   "types": "./lib/index.d.ts",
+  "module": "./src/index.ts",
   "private": true,
   "dependencies": {
     "shared-lib": "0.1.0"

packages/server/package.json

@@ -4,6 +4,7 @@
   "description": "The server which supports the client-side API",
   "main": "./lib/index.js",
   "types": "./lib/index.d.ts",
+  "module": "./src/index.ts",
   "private": true,
   "dependencies": {
     "shared-lib": "0.1.0",

packages/shared-lib/package.json

@@ -4,6 +4,7 @@
   "description": "Define the data types shared between the client and the server",
   "main": "./lib/index.js",
   "types": "./lib/index.d.ts",
+  "module": "./src/index.ts",
   "private": true,
   "devDependencies": {
     "typescript": "^3.8.0"

packages/ui-react/.gitignore

@@ -22,6 +22,7 @@
 
 # production
 /build
+/build.*
 
 # misc
 .DS_Store

packages/ui-react/config-overrides.js

@@ -0,0 +1,42 @@
+const path = require("path");
+const fs = require("fs");
+
+module.exports = function override(config, env) {
+  // https://github.com/facebook/create-react-app/blob/a88a4c3af6b6b8557845f147604a098d2857a91a/packages/react-scripts/config/webpack.config.js#L356-L404
+  const tsConfig = config.module.rules[2].oneOf[1];
+
+  // apparently it works alright if you just add the whole packages dir
+  // but if that didn't work you might like to specify specific subdirectories of packages
+
+  const srcDir = path.resolve(__dirname, "./src");
+  const clientDir = path.resolve(__dirname, "../client/src");
+  const sharedLibDir = path.resolve(__dirname, "../shared-lib/src");
+  const serverMockDir = path.resolve(__dirname, "../server-mock/src");
+  const serverDir = path.resolve(__dirname, "../server/src");
+  const serverTypesDir = path.resolve(__dirname, "../server-types/src");
+  const serverDataDir1 = path.resolve(__dirname, "../server-data/src");
+  const serverDataDir2 = path.resolve(__dirname, "../server-data/json");
+
+  const packagesDir = path.resolve(__dirname, "..");
+  const packageDirs = fs
+    .readdirSync(packagesDir, { withFileTypes: true })
+    .filter((dirent) => dirent.isDirectory() && path.basename(__dirname) !== dirent.name)
+    .map((dirent) => path.join(packagesDir, dirent.name))
+    .map((dir) => (fs.existsSync(path.join(dir, "src")) ? path.join(dir, "src") : dir));
+
+  tsConfig.include = [
+    srcDir,
+    clientDir,
+    sharedLibDir,
+    serverMockDir,
+    serverDir,
+    serverTypesDir,
+    serverDataDir1,
+    serverDataDir2,
+  ];
+  // tsConfig.include = [srcDir, ...packageDirs];
+  tsConfig.include = [srcDir, packagesDir];
+  // console.log(tsConfig.include);
+
+  return config;
+};

packages/ui-react/package.json

@@ -24,20 +24,21 @@
     "react-dom": "^16.8.6",
     "react-router-dom": "^5.0.0",
     "react-scripts": "3.4.1",
+    "react-app-rewired": "^2.1.3",
     "typescript": "^3.8.0"
   },
   "devDependencies": {
     "copyfiles": "^2.2.0"
   },
   "scripts": {
     "build-docs": "rimraf \"../../docs/\" && copyfiles --up 1 \"build/**/*\" \"../../docs/\"",
-    "start": "react-scripts start",
-    "build": "react-scripts build",
+    "start": "react-app-rewired start",
+    "build": "react-app-rewired build",
     "test": "react-scripts test",
     "eject": "react-scripts eject",
     "express": "node server.js",
     "clean": "rimraf node_modules",
-    "watch": "react-scripts start"
+    "watch": "react-app-rewired start"
   },
   "eslintConfig": {
     "extends": "react-app"

packages/ui-react/src/App.tsx

@@ -1,4 +1,4 @@
-import { config, Route } from "client";
+import { config, Route } from "client/src";
 import React from "react";
 import * as ReactRouter from "react-router-dom";
 import "ui-assets/css/App.css";

packages/ui-react/src/components/EditorTags.tsx

@@ -1,4 +1,4 @@
-import { Data } from "client";
+import { Data } from "client/src";
 import React from "react";
 import "ui-assets/css/EditorTags.css";
 import { useSelectTags } from "../hooks";

packages/ui-react/src/components/SelectTagsState.ts

@@ -1,4 +1,4 @@
-import { Data } from "client";
+import { Data } from "client/src";
 
 // you could temporarily change this to enable logging, for debugging
 const isLogging = false;

packages/ui-react/src/components/ThrottledInput.tsx

@@ -1,4 +1,4 @@
-import { SearchInput } from "client";
+import { SearchInput } from "client/src";
 import React from "react";
 import { ErrorMessage } from "./ErrorMessage";
 import { useThrottledInput } from "../hooks";

packages/ui-react/src/components/Topbar.tsx

@@ -1,4 +1,4 @@
-import { config, Route } from "client";
+import { config, Route } from "client/src";
 import React from "react";
 import { NavLink } from "react-router-dom";
 import "ui-assets/css/Topbar.css";

packages/ui-react/src/components/getDiscussionSummary.tsx

@@ -1,4 +1,4 @@
-import { Data, Url } from "client";
+import { Data, Url } from "client/src";
 import React from "react";
 import { NavLink } from "react-router-dom";
 import type { KeyedItem } from "../layouts";

packages/ui-react/src/components/getDiscussionsSubtitle.tsx

@@ -1,4 +1,4 @@
-import { config, Data, Url } from "client";
+import { config, Data, Url } from "client/src";
 import React from "react";
 import { Link, NavLink } from "react-router-dom";
 

packages/ui-react/src/components/getImageSummary.tsx

@@ -1,4 +1,4 @@
-import { Data, toHtml, Url } from "client";
+import { Data, toHtml, Url } from "client/src";
 import React from "react";
 import { NavLink } from "react-router-dom";
 import type { KeyedItem } from "../layouts";

packages/ui-react/src/components/getMessage.tsx

@@ -1,4 +1,4 @@
-import { Data, toHtml } from "client";
+import { Data, toHtml } from "client/src";
 import React from "react";
 import type { KeyedItem } from "../layouts";
 import { getTags } from "./getTags";

packages/ui-react/src/components/getTagCount.tsx

@@ -1,4 +1,4 @@
-import { Data } from "client";
+import { Data } from "client/src";
 import React from "react";
 
 // called from getActivityContent (nested in User)

packages/ui-react/src/components/getTagLink.tsx

@@ -1,4 +1,4 @@
-import { Data, Url } from "client";
+import { Data, Url } from "client/src";
 import React from "react";
 import { Link } from "react-router-dom";
 

packages/ui-react/src/components/getTagSummary.tsx

@@ -1,4 +1,4 @@
-import { Data, Url } from "client";
+import { Data, Url } from "client/src";
 import React from "react";
 import { NavLink } from "react-router-dom";
 import type { KeyedItem } from "../layouts";

packages/ui-react/src/components/getTags.tsx

@@ -1,4 +1,4 @@
-import { Data } from "client";
+import { Data } from "client/src";
 import React from "react";
 import { getTagLink } from "./getTagLink";