Merge branch 'master' of https://github.com/eclipse-theia/theia into …

…bugfix/permit-close-before-load
eclipse-theia · Feb 17, 2022 · 870b600 · 870b600
2 parents 4a93e5d + ee716ac
commit 870b600
Show file tree

Hide file tree

Showing 411 changed files with 5,288 additions and 2,809 deletions.
diff --git a/.github/workflows/playwright.yml b/.github/workflows/playwright.yml
@@ -0,0 +1,48 @@
+name: Playwright Tests
+
+on:
+  push:
+    branches:
+      - master
+  workflow_dispatch:
+  pull_request:
+    branches:
+      - master
+  schedule:
+    - cron: "0 4 * * *" # Runs every day at 4am: https://docs.github.com/en/actions/reference/events-that-trigger-workflows#scheduled-events-schedule
+
+jobs:
+  build-and-test-playwright:
+    name: ubuntu-18.04,  Node.js 16.x
+
+    runs-on: ubuntu-18.04
+    timeout-minutes: 60
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v2
+
+      - name: Use Node.js "16.x"
+        uses: actions/setup-node@v1
+        with:
+          node-version: "16.x"
+          registry-url: "https://registry.npmjs.org"
+
+      - name: Use Python 3.x
+        uses: actions/setup-python@v2
+        with:
+          python-version: "3.x"
+
+      - name: Build
+        shell: bash
+        run: |
+          yarn --skip-integrity-check --network-timeout 100000
+          yarn browser build
+        env:
+          NODE_OPTIONS: --max_old_space_size=4096
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # https://github.com/microsoft/vscode-ripgrep/issues/9
+
+      - name: Test (playwright)
+        uses: GabrielBB/xvfb-action@v1
+        with:
+          run: yarn test:playwright
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -4,12 +4,25 @@
 
 [1.23.0 Milestone](https://github.com/eclipse-theia/theia/milestone/31)
 
+- [plugin-ext] add more detail to logging of backend and frontend start-up, especially in plugin management [#10407](https://github.com/eclipse-theia/theia/pull/10407) - Contributed on behalf of STMicroelectronics
+- [plugin] Added support for `vscode.CodeActionProvider.resolveCodeAction` [#10730](https://github.com/eclipse-theia/theia/pull/10730) - Contributed on behalf of STMicroelectronics
+
 <a name="breaking_changes_1.23.0">[Breaking Changes:](#breaking_changes_1.23.0)</a>
 
 - [plugin] Deprecated `PseudoTerminalOptions`. `ExternalTerminalOptions` should be used from now on instead. [#10683](https://github.com/eclipse-theia/theia/pull/10683) - Contributed on behalf of STMicroelectronics
 - [core] Removed method `attachGlobalShortcuts` from `ElectronMainApplication`. Attaching shortcuts in that way interfered with internal shortcuts. Use internal keybindings instead of global shortcuts. [#10704](https://github.com/eclipse-theia/theia/pull/10704)
+- [plugin-ext] function `logMeasurement` of `PluginDeployerImpl` class and browser class `HostedPluginSupport` is replaced by `measure` using the new `Stopwatch` API [#10407](https://github.com/eclipse-theia/theia/pull/10407)
+- [plugin-ext] the constructor of `BackendApplication` class no longer invokes the `initialize` method. Instead, the `@postConstruct configure` method now starts by calling `initialize` [#10407](https://github.com/eclipse-theia/theia/pull/10407)
+- [plugin] Added support for `vscode.window.createStatusBarItem` [#10754](https://github.com/eclipse-theia/theia/pull/10754) - Contributed on behalf of STMicroelectronics
+- [core] Replaced raw WebSocket transport with Socket.io protocol, changed internal APIs accordingly
+- [core] Removed all of our own custom HTTP Polling implementation
+- In order to cleanup the code base, the constructor signature of the following classes got changed in an API-breaking way:
+  - `ProblemWidget`
+  - `FileNavigatorWidget`
+  - `TerminalServer`
+  - `TimelineTreeWidget`
+  - `TypeHierarchyTreeWidget`
 - [core] Moved methods `attachReadyToShow`, `restoreMaximizedState`, `attachCloseListeners`, `handleStopRequest`, `checkSafeToStop`, `handleReload`, `reload` from `ElectronMainAPplication` into new class `TheiaElectronWindow`. [#10600](https://github.com/eclipse-theia/theia/pull/10600)
-
 ## v1.22.0 - 1/27/2022
 
 [1.22.0 Milestone](https://github.com/eclipse-theia/theia/milestone/30)
@@ -85,7 +98,7 @@
 - [plugin] added support for codicon icon references in view containers [#10491](https://github.com/eclipse-theia/theia/pull/10491)
 - [plugin] added support to set theme attributes in webviews [#10493](https://github.com/eclipse-theia/theia/pull/10493)
 - [plugin] fixed running plugin hosts on `electron` for `Windows` [#10518](https://github.com/eclipse-theia/theia/pull/10518)
-- [preferences] updated `AbstractResourcePreferenceProvider` to handle multiple preference settings in the same tick and handle open preference files. 
+- [preferences] updated `AbstractResourcePreferenceProvider` to handle multiple preference settings in the same tick and handle open preference files.
   It will save the file exactly once, and prompt the user if the file is dirty when a programmatic setting is attempted. [#7775](https://github.com/eclipse-theia/theia/pull/7775)
 - [preferences] added support for non-string enum values in schemas [#10511](https://github.com/eclipse-theia/theia/pull/10511)
 - [preferences] added support for rendering markdown descriptions [#10431](https://github.com/eclipse-theia/theia/pull/10431)
@@ -109,11 +122,11 @@
 - [plugin] changed return type of `WebviewThemeDataProvider.getActiveTheme()` to `Theme` instead of `WebviewThemeType` [#10493](https://github.com/eclipse-theia/theia/pull/10493)
 - [plugin] removed the application prop `resolveSystemPlugins`, builtin plugins should now be resolved at build time [#10353](https://github.com/eclipse-theia/theia/pull/10353)
 - [plugin] renamed `WebviewThemeData.activeTheme` to `activeThemeType` [#10493](https://github.com/eclipse-theia/theia/pull/10493)
-- [preferences] removed `PreferenceProvider#pendingChanges` field. It was previously set unreliably and caused race conditions. 
-  If a `PreferenceProvider` needs a mechanism for deferring the resolution of `PreferenceProvider#setPreference`, it should implement its own system. 
+- [preferences] removed `PreferenceProvider#pendingChanges` field. It was previously set unreliably and caused race conditions.
+  If a `PreferenceProvider` needs a mechanism for deferring the resolution of `PreferenceProvider#setPreference`, it should implement its own system.
   See PR for example implementation in `AbstractResourcePreferenceProvider`. [#7775](https://github.com/eclipse-theia/theia/pull/7775)
 - [terminal] removed deprecated `activateTerminal` method in favor of `open`. [#10529](https://github.com/eclipse-theia/theia/pull/10529)
-- [webpack] Source maps for the frontend renamed from `webpack://[namespace]/[resource-filename]...` to `webpack:///[resource-path]?[loaders]` where `resource-path` is the path to 
+- [webpack] Source maps for the frontend renamed from `webpack://[namespace]/[resource-filename]...` to `webpack:///[resource-path]?[loaders]` where `resource-path` is the path to
   the file relative to your application package's root [#10480](https://github.com/eclipse-theia/theia/pull/10480)
 
 ## v1.20.0 - 11/25/2021

diff --git a/README.md b/README.md
@@ -90,6 +90,7 @@ Read below how to engage with Theia community:
   - [API Integration Testing](doc/api-testing.md)
   - [Coding Guidelines](https://github.com/eclipse-theia/theia/wiki/Coding-Guidelines)
   - [Code Organization](https://github.com/eclipse-theia/theia/wiki/Code-Organization)
+  - [Plugin and VSCode API](doc/Plugin-API.md)
 
 ## License
 

diff --git a/configs/base.tsconfig.json b/configs/base.tsconfig.json
@@ -5,6 +5,7 @@
     "declaration": true,
     "declarationMap": true,
     "noImplicitAny": true,
+    "noImplicitOverride": true,
     "noEmitOnError": false,
     "noImplicitThis": true,
     "noUnusedLocals": true,

diff --git a/configs/errors.eslintrc.json b/configs/errors.eslintrc.json
@@ -131,6 +131,7 @@
         }
       }
     ],
+    "@theia/localization-check": "error",
     "@theia/no-src-import": "error",
     "@theia/runtime-import-check": "error",
     "@theia/shared-dependencies": "error",

diff --git a/configs/typedoc.json b/configs/typedoc.json
@@ -1,15 +1,16 @@
 {
   "emit": false,
   "exclude": [
-    "+(dev-packages|examples|typings|scripts)",
-    "packages/*/lib",
-    "**/node_modules",
+    "../+(dev-packages|examples|typings|scripts)/**",
+    "../packages/*/lib/**",
+    "**/node_modules/**",
     "**/*spec.ts"
   ],
   "excludeExternals": true,
   "external-modulemap": ".*\/packages\/([\\w\\-_]+)\/",
   "hideGenerator": true,
   "name": "Theia TypeDoc",
-  "out": "gh-pages/docs/next",
-  "readme": "README.md"
+  "out": "../gh-pages/docs/next",
+  "readme": "../README.md",
+  "entryPointStrategy": "expand"
 }
diff --git a/dev-packages/application-manager/package.json b/dev-packages/application-manager/package.json
@@ -43,7 +43,7 @@
     "compression-webpack-plugin": "^9.0.0",
     "copy-webpack-plugin": "^8.1.1",
     "css-loader": "^6.2.0",
-    "electron-rebuild": "^1.11.0",
+    "electron-rebuild": "^3.2.7",
     "fs-extra": "^4.0.2",
     "ignore-loader": "^0.1.2",
     "less": "^3.0.3",

diff --git a/dev-packages/application-manager/src/generator/webpack-generator.ts b/dev-packages/application-manager/src/generator/webpack-generator.ts
@@ -56,8 +56,8 @@ export class WebpackGenerator extends AbstractGenerator {
 const path = require('path');
 const webpack = require('webpack');
 const yargs = require('yargs');
-const CopyWebpackPlugin = require('copy-webpack-plugin');
-const CircularDependencyPlugin = require('circular-dependency-plugin');
+${this.ifMonaco(() => `const CopyWebpackPlugin = require('copy-webpack-plugin');
+`)}const CircularDependencyPlugin = require('circular-dependency-plugin');
 const CompressionPlugin = require('compression-webpack-plugin')
 
 const outputPath = path.resolve(__dirname, 'lib');
@@ -75,13 +75,13 @@ const development = mode === 'development';${this.ifMonaco(() => `
 const monacoEditorCorePath = development ? '${this.resolve('@theia/monaco-editor-core', 'dev/vs')}' : '${this.resolve('@theia/monaco-editor-core', 'min/vs')}';`)}
 
 const plugins = [
-    new CopyWebpackPlugin({
-        patterns: [${this.ifMonaco(() => `{
+    ${this.ifMonaco(() => `new CopyWebpackPlugin({
+        patterns: [{
             from: monacoEditorCorePath,
             to: 'vs'
-        }`)}]
+        }]
     }),
-    new webpack.ProvidePlugin({
+    `)}new webpack.ProvidePlugin({
         // the Buffer class doesn't exist in the browser but some dependencies rely on it
         Buffer: ['buffer', 'Buffer']
     })

diff --git a/dev-packages/application-package/package.json b/dev-packages/application-package/package.json
@@ -33,10 +33,10 @@
     "@types/request": "^2.0.3",
     "@types/semver": "^5.4.0",
     "@types/write-json-file": "^2.2.1",
-    "changes-stream": "^2.2.0",
     "deepmerge": "^4.2.2",
     "fs-extra": "^4.0.2",
     "is-electron": "^2.1.0",
+    "nano": "^9.0.5",
     "request": "^2.82.0",
     "semver": "^5.4.1",
     "write-json-file": "^2.2.0"

diff --git a/dev-packages/application-package/src/npm-registry.ts b/dev-packages/application-package/src/npm-registry.ts
@@ -16,7 +16,7 @@
 
 /* eslint-disable @typescript-eslint/no-explicit-any */
 import * as request from 'request';
-const ChangesStream = require('changes-stream');
+import * as nano from 'nano';
 import { NpmRegistryProps } from './application-props';
 
 export interface IChangeStream {
@@ -92,7 +92,7 @@ export class NpmRegistry {
     readonly props: NpmRegistryProps = { ...NpmRegistryProps.DEFAULT };
     protected readonly options: NpmRegistryOptions;
 
-    protected changes: undefined | IChangeStream;
+    protected changes?: nano.ChangesReaderScope;
     protected readonly index = new Map<string, Promise<ViewResult>>();
 
     constructor(options?: Partial<NpmRegistryOptions>) {
@@ -115,13 +115,11 @@ export class NpmRegistry {
         this.index.clear();
         if (this.options.watchChanges && this.props.registry === NpmRegistryProps.DEFAULT.registry) {
             if (this.changes) {
-                this.changes.destroy();
+                this.changes.stop();
             }
-            // invalidate index with NPM registry web hooks
-            // see: https://github.com/npm/registry-follower-tutorial
-            const db = 'https://replicate.npmjs.com';
-            this.changes = new ChangesStream({ db }) as IChangeStream;
-            this.changes.on('data', change => this.invalidate(change.id));
+            // Invalidate index with NPM registry web hooks
+            this.changes = nano('https://replicate.npmjs.com').use('registry').changesReader;
+            this.changes.get({}).on('change', change => this.invalidate(change.id));
         }
     }
     protected invalidate(name: string): void {

diff --git a/dev-packages/cli/package.json b/dev-packages/cli/package.json
@@ -36,12 +36,9 @@
     "@theia/localization-manager": "1.22.1",
     "@theia/ovsx-client": "1.22.1",
     "@types/chai": "^4.2.7",
-    "@types/mkdirp": "^0.5.2",
     "@types/mocha": "^5.2.7",
     "@types/node-fetch": "^2.5.7",
     "@types/puppeteer": "^2.0.0",
-    "@types/requestretry": "^1.12.3",
-    "@types/tar": "^4.0.3",
     "chai": "^4.2.0",
     "chalk": "4.0.0",
     "decompress": "^4.2.1",
@@ -52,18 +49,13 @@
     "puppeteer": "^2.0.0",
     "puppeteer-to-istanbul": "^1.2.2",
     "temp": "^0.9.1",
-    "unzipper": "^0.9.11",
     "yargs": "^15.3.1"
   },
   "devDependencies": {
     "@types/chai": "^4.2.7",
-    "@types/mkdirp": "^0.5.2",
     "@types/mocha": "^5.2.7",
     "@types/node-fetch": "^2.5.7",
     "@types/proxy-from-env": "^1.0.1",
-    "@types/puppeteer": "^2.0.0",
-    "@types/requestretry": "^1.12.3",
-    "@types/tar": "^4.0.3",
-    "@types/unzipper": "^0.9.2"
+    "@types/puppeteer": "^2.0.0"
   }
 }
diff --git a/dev-packages/cli/src/theia.ts b/dev-packages/cli/src/theia.ts
@@ -25,7 +25,7 @@ import * as ffmpeg from '@theia/ffmpeg';
 import checkHoisted from './check-hoisting';
 import downloadPlugins from './download-plugins';
 import runTest from './run-test';
-import { extract } from '@theia/localization-manager';
+import { LocalizationManager, extract } from '@theia/localization-manager';
 
 process.on('unhandledRejection', (reason, promise) => {
     throw reason;
@@ -110,6 +110,7 @@ async function theiaCli(): Promise<void> {
     // affecting the global `yargs` instance used by the CLI.
     const { appTarget } = defineCommonOptions(yargsFactory()).help(false).parse();
     const manager = new ApplicationPackageManager({ projectPath, appTarget });
+    const localizationManager = new LocalizationManager();
     const { target } = manager.pck;
     defineCommonOptions(yargs)
         .command<{
@@ -228,6 +229,46 @@ async function theiaCli(): Promise<void> {
                 await downloadPlugins({ packed });
             },
         })
+        .command<{
+            freeApi?: boolean,
+            deeplKey: string,
+            file: string,
+            languages: string[],
+            sourceLanguage?: string
+        }>({
+            command: 'nls-localize [languages...]',
+            describe: 'Localize json files using the DeepL API',
+            builder: {
+                'file': {
+                    alias: 'f',
+                    describe: 'The source file which should be translated',
+                    demandOption: true
+                },
+                'deepl-key': {
+                    alias: 'k',
+                    describe: 'DeepL key used for API access. See https://www.deepl.com/docs-api for more information',
+                    demandOption: true
+                },
+                'free-api': {
+                    describe: 'Indicates whether the specified DeepL API key belongs to the free API',
+                    boolean: true,
+                    default: false,
+                },
+                'source-language': {
+                    alias: 's',
+                    describe: 'The source language of the translation file'
+                }
+            },
+            handler: async ({ freeApi, deeplKey, file, sourceLanguage, languages = [] }) => {
+                await localizationManager.localize({
+                    sourceFile: file,
+                    freeApi: freeApi ?? true,
+                    authKey: deeplKey,
+                    targetLanguages: languages,
+                    sourceLanguage
+                });
+            }
+        })
         .command<{
             root: string,
             output: string,

diff --git a/dev-packages/localization-manager/README.md b/dev-packages/localization-manager/README.md
@@ -12,8 +12,45 @@
 
 ## Description
 
-The `@theia/localization-manager` package is used easily create localizations of Theia and Theia extensions for different languages. 
-Its main use case is to extract localization keys and default values from `nls.localize` calls within the codebase.
+The `@theia/localization-manager` package is used easily create localizations of Theia and Theia extensions for different languages. It has two main use cases.
+
+First, it allows to extract localization keys and default values from `nls.localize` calls within the codebase using the `nls-extract` Theia-CLI command. Take this code for example:
+
+```ts
+const hi = nls.localize('greetings/hi', 'Hello');
+const bye = nls.localize('greetings/bye', 'Bye');
+```
+
+It will be converted into this JSON file (`nls.json`):
+
+```json
+{
+  "greetings": {
+    "hi": "Hello",
+    "bye": "Bye"
+  }
+}
+```
+
+Afterwards, any manual or automatic translation approach can be used to translate this file into other languages. These JSON files are supposed to be picked up by `LocalizationContribution`s.
+
+Additionally, Theia provides a simple way to translate the generated JSON files out of the box using the [DeepL API](https://www.deepl.com/docs-api). For this, a [DeepL free or pro account](https://www.deepl.com/pro) is needed. Using the `nls-localize` command of the Theia-CLI, a target file can be translated into different languages. For example, when calling the command using the previous JSON file with the `fr` (french) language, the following `nls.fr.json` file will be created in the same directory as the translation source:
+
+```json
+{
+  "greetings": {
+    "hi": "Bonjour",
+    "bye": "Au revoir"
+  }
+}
+```
+
+
+Only JSON entries without corresponding translations are translated using DeepL. This ensures that manual changes to the translated files aren't overwritten and only new translation entries are actually sent to DeepL.
+
+Use `theia nls-localize --help` for more information on how to use the command and supply DeepL API keys.
+
+For more information, see the [internationalization documentation](https://theia-ide.org/docs/i18n/).
 
 ## Additional Information
 

diff --git a/dev-packages/localization-manager/package.json b/dev-packages/localization-manager/package.json
@@ -29,7 +29,10 @@
     "watch": "theiaext watch"
   },
   "dependencies": {
+    "@types/bent": "^7.0.1",
     "@types/fs-extra": "^4.0.2",
+    "bent": "^7.1.0",
+    "chalk": "4.0.0",
     "deepmerge": "^4.2.2",
     "fs-extra": "^4.0.2",
     "glob": "^7.2.0",