Skip to content

Commit

Permalink
Adds ext dev (#520)
Browse files Browse the repository at this point in the history
* starts on extension emualtor commands

* further work on ext emulator

* Progress on emulator

* more logs

* Successfully emualted firestore counter! with some crednetial problems

* cleans up consoel.logs

* sets env during laod triggers

* format

* removes uneeded file changes

* removes uneeded file changes

* add ext:dev:emulators:exec

* formats

* DRYing up exec

* reorganizing code and making initial fixes suggested by Sam

* more fixes

* added ext:emualtor flags

* adds directory climbing to find an extension.yaml file

* adds unit tests for triggerHelper

* adds unit tests for paramHelper

* typo

* adds unit tests for paramHelper

* adds unit tests for paramHelper

* pr fixes

* typo and unused import

* pr fixes

* add sourceDirectory detection

* formats

* pr fixes

* moving some flags around;

* default sourcedirectory to functions

* adds param validation

* style fix

* adds tests for param validation

* adds regex validation and tests

* emulator-> emulators

* pr fixes;

* adds log for missing trigger

* formatted

* removes unneeded import of _

* slightly better error message

* Add "firebase ext:dev:init" command (#506)

* formatting;

* fixing package-lock

* pass predefined triggers to FunctionEmulatorRuntime so they are always available at runtime

* remove commented out code;

* removes newline

* format/

* predefinedTriggers->extensiontriggers

* use project provided by --project flag

* remove source directory line (#510)

* Update WELCOME.md for ext:dev:init (#509)

* adds validation for multiselect and select params

* typo fix

* also reject empty string for reqd params

* moves validation utils to extensionHelper, adds validateSpec, adds unit tests

* copy fixes, and adds an extra check for param type

* pr fixes and extra tests

* remove hardcoded variable;

* remove hardcoded variable;

* remove hardcoded variables

* bug fix and remove console log

* add runtime field to ext spec (#514)

* add runtime field to ext spec

* use getProjectId instead of options.project

* typo fix in test description

* differentiate javascript/typescript welcome messages (#513)

* differentiate javascript/typescript welcome messages

* adds support for --test-config

* Update src/extensions/emulator/optionsHelper.ts

Co-Authored-By: rachelsaunders <[email protected]>

* Update src/extensions/emulator/optionsHelper.ts

Co-Authored-By: rachelsaunders <[email protected]>

* Update src/extensions/emulator/optionsHelper.ts

Co-Authored-By: rachelsaunders <[email protected]>

* Update src/extensions/emulator/optionsHelper.ts

Co-Authored-By: rachelsaunders <[email protected]>

* use JSON.parse instead;

* finalizing copy

* Update src/extensions/emulator/optionsHelper.ts

Co-Authored-By: rachelsaunders <[email protected]>

* Update src/extensions/emulator/optionsHelper.ts

Co-Authored-By: rachelsaunders <[email protected]>

* Update src/extensions/emulator/optionsHelper.ts

Co-Authored-By: rachelsaunders <[email protected]>

* Validate all params instead of stopping on a failed param (#518)

* check all params instead of stopping on a failed param

* copy fix

* Update src/test/extensions/askUserForParam.spec.ts

Co-Authored-By: rachelsaunders <[email protected]>

* Update src/extensions/askUserForParam.ts

Co-Authored-By: rachelsaunders <[email protected]>

* Update src/test/extensions/askUserForParam.spec.ts

Co-Authored-By: rachelsaunders <[email protected]>

Co-authored-by: rachelsaunders <[email protected]>

* adds validation that default value passes validationRegex (#517)

* Add extdev preview and hide ext:dev commands behind it (#519)

* add extdev preview and hide ext:dev commands behind it

* formats;

* add ext to previews

* formats

* formats

* formats

* hide local  functionality from non preview users

* more accurate error messages

* formats

* missing space

Co-authored-by: Lauren Long <[email protected]>
Co-authored-by: Tina Liang <[email protected]>
Co-authored-by: rachelsaunders <[email protected]>
  • Loading branch information
4 people authored Mar 12, 2020
1 parent 2605a98 commit 18802d0
Show file tree
Hide file tree
Showing 54 changed files with 2,638 additions and 263 deletions.
409 changes: 361 additions & 48 deletions package-lock.json

Large diffs are not rendered by default.

3 changes: 3 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@
},
"dependencies": {
"@google-cloud/pubsub": "^1.1.5",
"@types/js-yaml": "^3.12.2",
"JSONStream": "^1.2.1",
"archiver": "^3.0.0",
"body-parser": "^1.19.0",
Expand All @@ -87,6 +88,7 @@
"google-auto-auth": "^0.10.1",
"google-gax": "~1.12.0",
"inquirer": "~6.3.1",
"js-yaml": "^3.13.1",
"jsonschema": "^1.0.2",
"jsonwebtoken": "^8.2.1",
"lodash": "^4.17.14",
Expand Down Expand Up @@ -123,6 +125,7 @@
"@types/fs-extra": "^5.0.5",
"@types/glob": "^7.1.1",
"@types/inquirer": "^6.0.3",
"@types/js-yaml": "^3.12.1",
"@types/lodash": "^4.14.136",
"@types/marked": "^0.6.5",
"@types/mocha": "^5.2.5",
Expand Down
4 changes: 4 additions & 0 deletions src/api.js
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,10 @@ var api = {
"https://runtimeconfig.googleapis.com"
),
storageOrigin: utils.envOverride("FIREBASE_STORAGE_URL", "https://storage.googleapis.com"),
firebaseStorageOrigin: utils.envOverride(
"FIREBASE_FIREBASESTORAGE_URL",
"https://firebasestorage.googleapis.com"
),
hostingApiOrigin: utils.envOverride(
"FIREBASE_HOSTING_API_URL",
"https://firebasehosting.googleapis.com"
Expand Down
2 changes: 1 addition & 1 deletion src/archiveDirectory.js
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ const _zipDirectory = (sourceDirectory, tempFile, options) => {
.readdirRecursive({ path: sourceDirectory, ignore: options.ignore })
.catch((err) => {
if (err.code === "ENOENT") {
return utils.reject(`Could not read directory "${sourceDirectory}"`, { origional: err });
return utils.reject(`Could not read directory "${sourceDirectory}"`, { original: err });
}
throw err;
})
Expand Down
108 changes: 1 addition & 107 deletions src/commands/emulators-exec.ts
Original file line number Diff line number Diff line change
@@ -1,89 +1,5 @@
import * as childProcess from "child_process";
import { StdioOptions } from "child_process";
import * as clc from "cli-color";

import { Command } from "../command";
import { Emulators } from "../emulator/types";
import { FirebaseError } from "../error";
import * as utils from "../utils";
import * as logger from "../logger";
import * as controller from "../emulator/controller";
import { DatabaseEmulator } from "../emulator/databaseEmulator";
import { EmulatorRegistry } from "../emulator/registry";
import { FirestoreEmulator } from "../emulator/firestoreEmulator";
import * as commandUtils from "../emulator/commandUtils";
import * as getProjectId from "../getProjectId";
import { EmulatorHub } from "../emulator/hub";

async function runScript(script: string, extraEnv: Record<string, string>): Promise<number> {
utils.logBullet(`Running script: ${clc.bold(script)}`);

const env: NodeJS.ProcessEnv = { ...process.env, ...extraEnv };

const databaseInstance = EmulatorRegistry.get(Emulators.DATABASE);
if (databaseInstance) {
const info = databaseInstance.getInfo();
const address = `${info.host}:${info.port}`;
env[DatabaseEmulator.DATABASE_EMULATOR_ENV] = address;
}

const firestoreInstance = EmulatorRegistry.get(Emulators.FIRESTORE);
if (firestoreInstance) {
const info = firestoreInstance.getInfo();
const address = `${info.host}:${info.port}`;

env[FirestoreEmulator.FIRESTORE_EMULATOR_ENV] = address;
env[FirestoreEmulator.FIRESTORE_EMULATOR_ENV_ALT] = address;
}

const hubInstance = EmulatorRegistry.get(Emulators.HUB);
if (hubInstance) {
const info = hubInstance.getInfo();
const address = `${info.host}:${info.port}`;
env[EmulatorHub.EMULATOR_HUB_ENV] = address;
}

const proc = childProcess.spawn(script, {
stdio: ["inherit", "inherit", "inherit"] as StdioOptions,
shell: true,
windowsHide: true,
env,
});

logger.debug(`Running ${script} with environment ${JSON.stringify(env)}`);

return new Promise((resolve, reject) => {
proc.on("error", (err: any) => {
utils.logWarning(`There was an error running the script: ${JSON.stringify(err)}`);
reject();
});

// Due to the async nature of the node child_process library, sometimes
// we can get the "exit" callback before all "data" has been read from
// from the script's output streams. To make the logs look cleaner, we
// add a short delay before resolving/rejecting this promise after an
// exit.
const exitDelayMs = 500;
proc.once("exit", (code, signal) => {
if (signal) {
utils.logWarning(`Script exited with signal: ${signal}`);
setTimeout(reject, exitDelayMs);
return;
}

const exitCode = code || 0;
if (code === 0) {
utils.logSuccess(`Script exited successfully (code 0)`);
} else {
utils.logWarning(`Script exited unsuccessfully (code ${code})`);
}

setTimeout(() => {
resolve(exitCode);
}, exitDelayMs);
});
});
}

module.exports = new Command("emulators:exec <script>")
.before(commandUtils.beforeEmulatorCommand)
Expand All @@ -93,26 +9,4 @@ module.exports = new Command("emulators:exec <script>")
.option(commandUtils.FLAG_ONLY, commandUtils.DESC_ONLY)
.option(commandUtils.FLAG_INSPECT_FUNCTIONS, commandUtils.DESC_INSPECT_FUNCTIONS)
.option(commandUtils.FLAG_IMPORT, commandUtils.DESC_IMPORT)
.action(async (script: string, options: any) => {
const projectId = getProjectId(options, true);
const extraEnv: Record<string, string> = {};
if (projectId) {
extraEnv.GCLOUD_PROJECT = projectId;
}
let exitCode = 0;
try {
await controller.startAll(options, /* noGui = */ true);
exitCode = await runScript(script, extraEnv);
} catch (e) {
logger.debug("Error in emulators:exec", e);
throw e;
} finally {
await controller.cleanShutdown();
}

if (exitCode !== 0) {
throw new FirebaseError(`Script "${clc.bold(script)}" exited with code ${exitCode}`, {
exit: exitCode,
});
}
});
.action(commandUtils.emulatorExec);
15 changes: 15 additions & 0 deletions src/commands/ext-dev-emulators-exec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
import { Command } from "../command";

import * as commandUtils from "../emulator/commandUtils";
import * as optionsHelper from "../extensions/emulator/optionsHelper";

module.exports = new Command("ext:dev:emulators:exec <script>")
.description("emulate an extension, run a test script, then shut down the emulators")
.option(commandUtils.FLAG_INSPECT_FUNCTIONS, commandUtils.DESC_INSPECT_FUNCTIONS)
.option(commandUtils.FLAG_TEST_CONFIG, commandUtils.DESC_TEST_CONFIG)
.option(commandUtils.FLAG_TEST_PARAMS, commandUtils.DESC_TEST_PARAMS)
.action(async (script: string, options: any) => {
const emulatorOptions = await optionsHelper.buildOptions(options);
commandUtils.beforeEmulatorCommand(emulatorOptions);
await commandUtils.emulatorExec(script, emulatorOptions);
});
37 changes: 37 additions & 0 deletions src/commands/ext-dev-emulators-start.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
import { Command } from "../command";
import * as controller from "../emulator/controller";
import * as commandUtils from "../emulator/commandUtils";
import * as optionsHelper from "../extensions/emulator/optionsHelper";
import * as utils from "../utils";
import { FirebaseError } from "../error";

module.exports = new Command("ext:dev:emulators:start")
.description("start the local Firebase extension emulator")
.option(commandUtils.FLAG_INSPECT_FUNCTIONS, commandUtils.DESC_INSPECT_FUNCTIONS)
.option(commandUtils.FLAG_TEST_CONFIG, commandUtils.DESC_TEST_CONFIG)
.option(commandUtils.FLAG_TEST_PARAMS, commandUtils.DESC_TEST_PARAMS)
.action(async (options: any) => {
const emulatorOptions = await optionsHelper.buildOptions(options);
try {
commandUtils.beforeEmulatorCommand(emulatorOptions);
await controller.startAll(emulatorOptions);
} catch (e) {
await controller.cleanShutdown();
if (!(e instanceof FirebaseError)) {
throw new FirebaseError("Error in ext:dev:emulator:start", e);
}
throw e;
}

utils.logSuccess("All emulators started, it is now safe to connect.");

// Hang until explicitly killed
await new Promise((res, rej) => {
process.on("SIGINT", () => {
controller
.cleanShutdown()
.then(res)
.catch(res);
});
});
});
170 changes: 170 additions & 0 deletions src/commands/ext-dev-init.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
import * as fs from "fs";
import * as path from "path";
import * as marked from "marked";
import TerminalRenderer = require("marked-terminal");

import { Command } from "../command";
import * as Config from "../config";
import { FirebaseError } from "../error";
import { promptOnce } from "../prompt";
import * as logger from "../logger";
import * as npmDependencies from "../init/features/functions/npm-dependencies";
marked.setOptions({
renderer: new TerminalRenderer(),
});

const TEMPLATE_ROOT = path.resolve(__dirname, "../../templates/extensions/");
const FUNCTIONS_ROOT = path.resolve(__dirname, "../../templates/init/functions/");

const EXT_SPEC_TEMPLATE = fs.readFileSync(path.join(TEMPLATE_ROOT, "extension.yaml"), "utf8");
const PREINSTALL_TEMPLATE = fs.readFileSync(path.join(TEMPLATE_ROOT, "PREINSTALL.md"), "utf8");
const POSTINSTALL_TEMPLATE = fs.readFileSync(path.join(TEMPLATE_ROOT, "POSTINSTALL.md"), "utf8");

/**
* Sets up Typescript boilerplate code for new extension
* @param {Config} config configuration options
*/
async function typescriptSelected(config: Config): Promise<void> {
const packageLintingTemplate = fs.readFileSync(
path.join(TEMPLATE_ROOT, "typescript", "package.lint.json"),
"utf8"
);
const packageNoLintingTemplate = fs.readFileSync(
path.join(TEMPLATE_ROOT, "typescript", "package.nolint.json"),
"utf8"
);
const tsconfigTemplate = fs.readFileSync(
path.join(TEMPLATE_ROOT, "typescript", "tsconfig.json"),
"utf8"
);
const indexTemplate = fs.readFileSync(path.join(TEMPLATE_ROOT, "typescript", "index.ts"), "utf8");
const gitignoreTemplate = fs.readFileSync(
path.join(TEMPLATE_ROOT, "typescript", "_gitignore"),
"utf8"
);
const tslintTemplate = fs.readFileSync(
path.join(FUNCTIONS_ROOT, "typescript", "tslint.json"),
"utf8"
);

const lint = await promptOnce({
name: "lint",
type: "confirm",
message: "Do you want to use TSLint to catch probable bugs and enforce style?",
default: true,
});

await config.askWriteProjectFile("extension.yaml", EXT_SPEC_TEMPLATE);
await config.askWriteProjectFile("PREINSTALL.md", PREINSTALL_TEMPLATE);
await config.askWriteProjectFile("POSTINSTALL.md", POSTINSTALL_TEMPLATE);
await config.askWriteProjectFile("functions/src/index.ts", indexTemplate);
if (lint) {
await config.askWriteProjectFile("functions/package.json", packageLintingTemplate);
await config.askWriteProjectFile("functions/tslint.json", tslintTemplate);
} else {
await config.askWriteProjectFile("functions/package.json", packageNoLintingTemplate);
}
await config.askWriteProjectFile("functions/tsconfig.json", tsconfigTemplate);
await config.askWriteProjectFile("functions/.gitignore", gitignoreTemplate);
}

/**
* Sets up Javascript boilerplate code for new extension
* @param {Config} config configuration options
*/
async function javascriptSelected(config: Config): Promise<void> {
const indexTemplate = fs.readFileSync(path.join(TEMPLATE_ROOT, "javascript", "index.js"), "utf8");
const packageLintingTemplate = fs.readFileSync(
path.join(TEMPLATE_ROOT, "javascript", "package.lint.json"),
"utf8"
);
const packageNoLintingTemplate = fs.readFileSync(
path.join(TEMPLATE_ROOT, "javascript", "package.nolint.json"),
"utf8"
);
const gitignoreTemplate = fs.readFileSync(
path.join(TEMPLATE_ROOT, "javascript", "_gitignore"),
"utf8"
);
const eslintTemplate = fs.readFileSync(
path.join(FUNCTIONS_ROOT, "javascript", "eslint.json"),
"utf8"
);

const lint = await promptOnce({
name: "lint",
type: "confirm",
message: "Do you want to use ESLint to catch probable bugs and enforce style?",
default: false,
});

await config.askWriteProjectFile("extension.yaml", EXT_SPEC_TEMPLATE);
await config.askWriteProjectFile("PREINSTALL.md", PREINSTALL_TEMPLATE);
await config.askWriteProjectFile("POSTINSTALL.md", POSTINSTALL_TEMPLATE);
await config.askWriteProjectFile("functions/index.js", indexTemplate);
if (lint) {
await config.askWriteProjectFile("functions/package.json", packageLintingTemplate);
await config.askWriteProjectFile("functions/.eslintrc.json", eslintTemplate);
} else {
await config.askWriteProjectFile("functions/package.json", packageNoLintingTemplate);
}
await config.askWriteProjectFile("functions/.gitignore", gitignoreTemplate);
}

/**
* Command for setting up boilerplate code for a new extension.
*/
export default new Command("ext:dev:init")
.description("initialize files for writing an extension in the current directory")
.action(async (options: any) => {
const cwd = options.cwd || process.cwd();
const config = new Config({}, { projectDir: cwd, cwd: cwd });

try {
const lang = await promptOnce({
type: "list",
name: "language",
message:
"What language would you like to use to write the Cloud Functions for your Extension?",
default: "javascript",
choices: [
{
name: "JavaScript",
value: "javascript",
},
{
name: "TypeScript",
value: "typescript",
},
],
});
switch (lang) {
case "javascript": {
await javascriptSelected(config);
break;
}
case "typescript": {
await typescriptSelected(config);
break;
}
default: {
throw new FirebaseError(`${lang} is not supported.`);
}
}

await npmDependencies.askInstallDependencies({}, config);

const welcome = fs.readFileSync(path.join(TEMPLATE_ROOT, lang, "WELCOME.md"), "utf8");
return logger.info("\n" + marked(welcome));
} catch (err) {
if (!(err instanceof FirebaseError)) {
throw new FirebaseError(
`Error occurred when initializing files for new extension: ${err.message}`,
{
original: err,
}
);
}
throw err;
}
});
Loading

0 comments on commit 18802d0

Please sign in to comment.