- About Builder
- Builder Installation
- Builder Syntax
- Include Files
- Advanced Builder Usage
- Testing
- License
Builder combines a preprocessor with an expression language and advanced imports.
There are a number of ways in which you can install Builder depending on how you plan to integrate it into your workflow. Once installed on your computer, you can use it to process your Squirrel application and factory firmware before you transfer the code to an impCentral™ Device Group.
Note The Electric Imp VS Code extension already incorporates Builder and can be used to upload code to Device Groups. If you are using the VS Code extension, there is no need install Builder separately to take advantage of its features.
You can use Builder to pull the contents of separate code files into your main source code files. These additional files might contain library code that you make use of across a number of different products, or they might contain confidential data which you don’t want to keep inside source code files that are managed through a software version control system.
You tell Builder which files to import, and where within your main source code they should be inserted, by using the @include
command. Builder is able to access additional files that are stored on your computer or held remotely on an external resource (eg. HTTP/HTTPs server or a repository).
While Builder can be used to insert code this way, it can be used in far more sophisticated ways thanks to its integrated expression processor and programming logic. For example, if you need to generate multiple versions of your application firmware for versions of your product which make use of different imp modules, you can use Builder’s conditional execution features, variables and loops to pull your various code components together at build time and output files that are ready to be transferred to impCentral.
To speed up the process, files that are stored remotely which are not expected to change between builds can be cached for quick re-use. Builder's reproducible artifacts feature makes it possible to store references to all files and variables, so that builds can be re-created for future debugging.
For details on the commands that Builder offers, please see the Directives section. This is part of the Builder Syntax section, which also describes how Builder commands are structured.
Builder requires Node.js 18.14.2 and above. It can be installed and used by two ways:
- As an npm command line tool
- As an npm library.
Install Builder:
npm install -g Builder
Now use Builder’s pleasebuild
command to configure the newly installed utility:
pleasebuild [-l] [-D<variable> <value>]
[--github-token <token>] [--azure-user <username> --azure-token <token>]
[--bitbucket-server-addr <address>] [--bitbucket-server-user <username> --bitbucket-server-token <token>]
[--lib <path_to_file>] [--use-remote-relative-includes] [--suppress-duplicate-includes-warning]
[--cache] [--clear-cache] [--cache-exclude-list <path_to_file>]
[--save-dependencies [<path_to_file>]] [--use-dependencies [<path_to_file>]]
[--save-directives [<path_to_file>]] [--use-directives [<path_to_file>]]
<input_file>
where <input_file>
is the path to source file which should be preprocessed and the other options are:
Option | Synonym | Mandatory? | Value Required? | Description |
---|---|---|---|---|
-l | No | No | Generates line control statements. For a more detailed explanation, please read this GCC page | |
-D<variable> | No | Yes | Defines a variable. May be specified several times to define multiple variables | |
--github-token | No | Yes | A GitHub personal access token. Password is not supported. | |
--azure-user | No | Yes | An Azure Repos username. | |
--azure-token | No | Yes | An Azure Repos personal access token. Should be specified if the --azure-user option is specified |
|
--bitbucket-server-addr | No | Yes | A Bitbucket Server address. E.g., https://bitbucket-srv.itd.example.com . Note: this option is mandatory to include files from Bitbucket Server |
|
--bitbucket-server-user | No | Yes | A Bitbucket Server username. | |
--bitbucket-server-token | No | Yes | A Bitbucket Server personal access token or password (not recommended). Should be specified if the --bitbucket-server-user option is specified. |
|
--lib | --libs | No | Yes | Include the specified JavaScript file(s) as a library. May be specified several times to include multiple libraries. The provided value may specify a concrete file or a directory (all files from the directory will be included). The value may contain wildcards (all matched files will be included) |
--use-remote-relative-includes | No | No | Interpret every non-absolute path in the @include and @include once directives as relative to the location of the source file where it is mentioned. See the Include Files section |
|
--suppress-duplicate-includes-warning | --suppress-duplicate | No | No | Do not show a warning if a source file with the same content was included multiple times from different locations and this results in code duplication |
--cache | -c | No | No | Turn on caching for all files included from remote resources. See the Caching Remote Files section. This option is ignored if the --save-dependencies option is specified (the cache is turned off for all files in this case). If the --use-dependencies option is specified the cache is turned off for the files referenced in the dependency file and is turned on for all other remote files |
--clear-cache | No | No | Clear the cache before Builder starts running. See the Caching Remote Files section | |
--cache-exclude-list | No | Yes | Set the path to the file that lists resources which should not be cached. the Caching Remote Files section | |
--save-dependencies | No | No | Save references to the required repository files in the specified file. If a file name is not specified, the dependencies.json file in the local directory is used. See ‘Reproducible Artifacts’ |
|
--use-dependencies | No | No | Use the specified file to set which repository files are required. If a file name is not specified, the dependencies.json file in the local directory is used. See ‘Reproducible Artifacts’. |
|
--save-directives | No | No | Save Builder variable definitions in the specified file. If a file name is not specified, the directives.json file in the local directory is used. See ‘Reproducible Artifacts’ |
|
--use-directives | No | No | Use Builder variable definitions from the specified file. If a file name is not specified, the directives.json file in the local directory is used. See ‘Reproducible Artifacts’ |
Install Builder:
npm i --save Builder
Now instantiate, configure and execute Builder from within your source code. For example:
const Builder = require('Builder');
const builder = new Builder();
// Specify whether you need line control statements. See the "-l" CLI option.
builder.machine.generateLineControlStatements = <true|false>;
// Cache all files included from remote sources. See the "--cache" CLI option.
builder.machine.useCache = <true|false>;
// Set GitHub credentials. See the "--github-token" CLI option.
builder.machine.readers.github.token = "<PERSONAL_ACCESS_TOKEN>";
// Set Azure Repos credentials. See the "--azure-user" and "--azure-token" CLI options.
builder.machine.readers.azureRepos.username = "<USERNAME>";
builder.machine.readers.azureRepos.token = "<ACCESS_TOKEN>";
// Set Bitbucket Server address and credentials. See the "--bitbucket-server-*" CLI options.
builder.machine.readers.bitbucketSrv.serverAddr = "<ADDRESS>";
builder.machine.readers.bitbucketSrv.username = "<USERNAME>";
builder.machine.readers.bitbucketSrv.token = "<PASSWORD_OR_ACCESS_TOKEN>";
// Path to the file that lists the resources which should be excluded from caching.
// See the "--cache-exclude-list" CLI option.
builder.machine.excludeList = "<PATH_TO_FILE>";
// Interpret a non-absolute path in includes as relative to the current file.
// See the "--use-remote-relative-includes" CLI option.
builder.machine.remoteRelativeIncludes = <true|false>;
// Suppress warning about duplicate includes.
// See the "--suppress-duplicate-includes-warning" CLI option.
builder.machine.suppressDupWarning = <true|false>;
// See the "--save-dependencies" CLI option.
builder.machine.dependenciesSaveFile = <false|"PATH_TO_FILE">;
// See the "--use-dependencies" CLI option.
builder.machine.dependenciesUseFile = <false|"PATH_TO_FILE">;
// See the "--save-directives" CLI option.
builder.machine.directivesSaveFile = <false|"PATH_TO_FILE">;
// See the "--use-directives" CLI option.
builder.machine.directivesUseFile = <false|"PATH_TO_FILE">;
const inputFile = "PATH_TO_YOUR_INPUT_FILE";
// Set the directory of the input file as one of search dirs.
// path.resolve() is used to get the absolute path (requires the path nodejs module)
builder.machine.readers.file.inputFileDir = path.resolve(inputFile);
const result = builder.machine.execute(`@include "${inputFile}"`);
console.log(result);
To understand Builder configuration, please review this source code.
The following value types are supported in expressions:
- numbers (eg.
1
,1E6
,1e-6
,1.567
) - strings (eg.
"abc"
,'abc'
) - booleans (eg.
true
,false
) null
Builder supports the following operators:
||
&&
==
!=
<
>
<=
>=
+
-
*
/
%
+
-
!
Membership of an object is expressed using any of the following expressions:
somevar.member
somevar["member"]
([1, 2, 3])[1]
Builder provides the standard ternary operator (?:
) for evaluating basic conditions:
<condition> ? <if_condition_true> : <if_condition_false>
Variables can be used in Builder expressions. Variable names can contain $
, _
, latin letters and digits, however they must not start with a digit. Variables can be defined in the following ways:
- Builder's
@set
directive. - Your computer's environment variables.
- Pass the option
-D<variable name> <variable value>
to Builder’spleasebuild
command.
All undefined variables are evaluated as null
.
When resolving a variable’s value:
- Builder looks for its definition among the command line parameters (as
-D<variable name> <variable value>
) passed to thepleasebuild
command. - If no such variable definition is found, the code is scanned for
@set
directive statements preceding the variable usage. - If no variable definitions are found in the previous steps, Builder looks in the host environment variables.
There is no special predicate required to make use of environment variables. Builder looks in the host environment variables to try and resolve the expressions if no command line or local variables have been set.
For example, on a Mac:
server.log("Host home path is @{HOME}");
will print the home directory path of the current user of the system where Builder was executed.
Environment variables differ based on OS. If you wish to use environment variables with Builder, a quick internet search will give you details on how to list the variables currently available on your system and also how to set new variables.
Builder provides the following pre-defined variables:
-
__LINE__
— The line number (relative to the file in which this variable appears). For example:Hi from line @{__LINE__}!
-
__FILE__
— The name of the file in which this variable appears. For example:Hi from file @{__FILE__}!
-
__PATH__
— The absolute path (not including file name) to the file where this variable appears. Can contain a URL for remote includes. For example:Hi from file @{__PATH__}!
-
__REPO_PREFIX__
— The root of the repository resource which is being processed. Is an internal variable of Builder. For example:github:electricimp/Builder
-
__REPO_REF__
— The git reference (branch name or tag) of the repository resource which is being processed. Is an internal variable of Builder. -
__URL_ROOT__
— The root of the remote weblink resource which is being processed. Is an internal variable of Builder. For example:
https://example.com http://example.com:8080
__URL_PATH__
— The path to the file on the remote weblink resource which is being processed, relative to the root of resource. Is an internal variable of Builder.
Builder has two directives @while
and @repeat
for managing loops. Inside these loops the following variables are available:
loop.index
— 0-indexed iteration counterloop.iteration
— 1-indexed iteration counter
Usage examples for these variables can be found in the @while
and @repeat
directive examples.
Builder provides the following helper functions:
defined(<variable_name>)
— returnstrue
if a variable is defined,false
otherwise.include(<source>)
— includes external source.escape(<value>)
— escapes special characters in string (\b
,\f
,\n
,\r
,\t
,\
,'
,"
).base64(<value>)
— encodes value as base64.min(<numbers>)
— returns a number equal to the lowest number in the supplied list.max(<numbers>)
— returns a number equal to the highest number in the supplied list.abs(<number>)
— returns the absolute value of the supplied number.
Builder also comes with some string functions, based on the JavaScript methods of the same names. These functions are available under the namespace S
. The first argument of each function is always the string to be operated on. For documentation on the remaining arguments, please see ‘JavaScript String Methods’.
S.concat()
S.endsWith()
S.includes()
S.repeat()
S.split()
S.startsWith()
S.substr()
S.substring()
S.toLowerCase()
S.toUpperCase()
S.trim()
S.trimLeft()
S.trimRight()
Lines starting with @
followed by space or a line break are treated as comments and not added to the output. For example:
@ This is a Builder comment and will not appear in the output
Any text following //
and extending to the end of the line will be ignored by Builder* and will not appear in the result output. For example:
@set SOME_STRING = "my string" // This is a Builder comment that will not appear in output
All of Builder’s directives start with the @
symbol. Don’t include a space or line break between the @
and the required directive’s name as this will be interpreted as a comment.
This directive evaluates the expression enclosed by braces ({
and }
) and inserts the result into the output. The enclosure can be the value of a named variable, an expression or a macro.
@{<variable:identifier>} @{<expression>} @{macro(a, b, c)}
The line:
The result is: @{123 * 456}.
results in the following output:
The result is: 56088.
This directive assigns a value or the value of an expression to a variable. Variables are defined in a global context. A value can be any supported type or function.
@set <variable:identifier> <value:expression> @set <variable:identifier> = <value:expression>
In this example, we define a number of variables using @set
, then use @{...}
to create squirrel log messages with those variables. If the following lines are added to your source code:
@ Set Builder global variables
@set SOME_INT = 10
@set SOME_STRING = "my string"
@set BOOL_VAL = (12 > 4)
@
@ Use a Builder function to set a global variable
@set MIN_INT min(1, 2, 3)
@
// Use Builder global variables in squirrel log messages
server.log(@{SOME_INT});
server.log("@{SOME_STRING}");
server.log(@{BOOL_VAL});
server.log(@{MIN_INT});
then during processing, Builder will output:
// Use Builder global variables in squirrel log messages
server.log(10);
server.log("my string");
server.log(true);
server.log(1);
This directive defines a code block with its own parameters. Macros are declared in a global scope. Macro parameters are only available within the macro scope and override global variables with the same name (but do not affect them).
@macro <name>(<arguments>) <body> @endmacro
Note @end
can be used in place of @endmacro
if you prefer.
Macros can be used either inline with the @{...}
directive, or with the @include
directive. When macros are used inline no line-control statements are generated for the output inside the macro scope and trailing newlines are trimmed from the macro output.
@{macro(a, b ,c)} @include macro(a, b ,c)
Define a macro and use the @{...}
directive to create a multi-line string to log in squirrel:
@ Define a macro
@macro some_macro(a, b, c)
Hello, @{a}!
Roses are @{b},
And violets are @{c}
@end
@
// Use an inline Builder macro to create a multi-line string
poem <- @"@{some_macro("username", "red", "blue")}";
server.log(poem);
This results in the following output:
// Use an inline Builder macro to create a multi-line string
poem <- @" Hello, username!
Roses are red,
And violets are blue";
server.log(poem);
Define a macro and use it with @include
to create a multi-line string to log in squirrel:
@ Define a macro
@macro some_macro(a, b, c)
Hello, @{a}!
Roses are @{b},
And violets are @{c}
@end
@
// Use Builder include and macro directives to create a multi-line squirrel string
poem <- @"
@include some_macro("username", "red", "blue")
";
server.log(poem);
This results in the following output:
// Use Builder include and macro directives to create a multi-line squirrel string
poem <- @"
Hello, username!
Roses are red,
And violets are blue
";
server.log(poem);
Use a function to configure and use a macro with an optional parameter.
@ Define a macro with an optional parameter
@macro some_macro(a, b, c)
Hello, @{a}!
Roses are @{b},
And violets are @{defined(c) ? c : "of undefined color"}.
@end
@
// Use Builder include and macro directives to create a multi-line squirrel string
poem <- @"
@include some_macro("username", "red")
";
server.log(poem);
This results in the following output:
// Use Builder include and macro directives to create a multi-line squirrel string
poem <- @"
Hello, username!
Roses are red,
And violets are of undefined color.
";
server.log(poem);
This directive can be used to include local files, files from remote resources or macros.
@include <source:expression>
-
For a
@macro
:@include some_macro("username", 123)
-
For a local file:
@include "somepath/somefile.ext"
-
For a file from remote resource:
@include "someresource/somepath/somefile.ext"
The @ character must not be present in the path and name of the file which is being included.
Please see the Include Files section for information about include formats, file searching rules, supported remote resources, examples and other details.
This directive acts just like @include
but has no effect if the specified source has already been included. However, macros are always included.
@include once <source:expression>
This directive invokes a loop which only ends when specified conditions are met. You can access Builder’s loop variables within @while
loops.
@while <test:expression> // 0-based iteration counter: @{loop.index} // 1-based iteration counter: @{loop.iteration} @endwhile
Note @end
may be used in place of @endwhile
if you prefer.
The following lines, when added to your source code:
@set myvar = 12
@while myvar > 9
@set myvar = myvar - 1
var: @{myvar}
loop.index: @{loop.index}
loop.iteration: @{loop.iteration}
@end
will output:
var: 11
loop.index: 0
loop.iteration: 1
var: 10
loop.index: 1
loop.iteration: 2
var: 9
loop.index: 2
loop.iteration: 3
This directive invokes a loop that repeats for a certain number of iterations. You can access Builder’s loop variables within @repeat
loops.
@repeat <times:expression> // 0-based iteration counter: @{loop.index} // 1-based iteration counter: @{loop.iteration} @endrepeat
Note @end
may be used in place of @endrepeat
if you prefer.
The following lines, when added to your source code:
@repeat 3
loop.index: @{loop.index}
loop.iteration: @{loop.iteration}
@end
will output:
loop.index: 0
loop.iteration: 1
loop.index: 1
loop.iteration: 2
loop.index: 2
loop.iteration: 3
This directive provides conditional branching.
@if <test:expression> // Consequent code @elseif <test:expression> // else if #1 code @else // Alternative code @endif
Note @end
may be used in place of @endif
if you prefer.
@if __FILE__ == 'abc.ext'
// include something
@elseif __FILE__ == 'def.ext'
// include something else
@else
// do something completely different
@endif
This directive simply emits an error.
@error <message:expression>
@if PLATFORM == "platform1"
// platform 1 code
@elseif PLATFORM == "platform2"
// platform 2 code
@elseif PLATFORM == "platform3"
// platform 3 code
@else
@error "Platform " + PLATFORM + " is unsupported"
@endif
This directive simply emits a warning.
@warning <message:expression>
@if PLATFORM == "platform1"
// platform 1 code
@elseif PLATFORM == "platform2"
// platform 2 code
@elseif PLATFORM == "platform3"
// platform 3 code
@else
@warning "Building for default platform"
// default platform code
@endif
The filter operator, |
, allows you to pass a value through any of the supported functions.
@{<expression> | <filter>}
This is equivalent to:
@{<filter>(<expression>)}
// Include external HTML piped through the escape processing function
a = "@{include('index.html')|escape}"
// Include an external binary piped through the base64 encoder function
b = "@{include('file.bin')|base64}"
This section contains detailed information about usage of the @include
and @include once
directives to include local or remote files.
Local file is a file on the same system where Builder is running.
Remote file is a file on a remote resource (eg. a repository or a HTTP/HTTPs server). See the Remote Include Files section for the list of supported remote resources.
The file mentioned in the @include
or @include once
directives may contain a relative or an absolute path.
@include "https://example.com/somefile.ext" // absolute path to remote file
@include once "github:electricimp/Promise/promise.class.nut" // absolute path to remote file
@include "c:\somefolder\somefile.ext" // absolute path to local file
@include once "./somefile.ext" // relative path
@include "somefile.ext" // relative path
The next section clarifies how Builder searches the included file.
Processed file is a file currently being processed by Builder, it can be local or remote, it contains the @include
or @include once
directive(s).
Path to the processed file is an absolute path to the processed file.
Path in the include is the path mentioned in the @include
or @include once
directive, it can be an absolute path to local file, an absolute path to remote file or a relative path.
Full path to the include file is an absolute path to the file which will be included by Builder.
The below rules define how Builder searches the included file:
If the path in the include is an absolute path to the remote file, it is considered as the final path to the include file. If the file is not found there, Builder reports an error.
@include "https://example.com/somefile.ext" // absolute path to remote file
@include once "github:electricimp/Promise/promise.class.nut" // absolute path to remote file
If the path in the include is an absolute path to the local file, it is considered as the final path to the include file. If the file is not found there, Builder reports an error.
@include "c:\somefolder\somefile.ext" // absolute path to local file
@include "/home/user/someuser/somefolder/somefile.ext" // absolute path to local file. But see the note below.
Note: Under some conditions a path in the include which starts from the "/" symbol is processed by the special rule described below.
If
- the first symbol of the path in the include is "/"
- and the processed file is a remote file (ie. the file on a remote resource)
- and the
--use-remote-relative-includes
option is specified
then the final path to the include file is a concatenation of the root of remote resource and the path in the include. If the file is not found there, Builder reports an error. See the Remote Include Files section for the root definition of the supported remote resources.
The processed file: "https://example.com/folderA/folderB/somefile.nut"
It contains:
@include "/folderC/anotherfile.nut"
The final path to the include: "https://example.com/folderC/anotherfile.nut"
The processed file: "github:someuser/somerepo/folderA/folderB/somefile.nut@develop"
It contains:
@include "/folderC/anotherfile.nut"
The final path to the include: "github:someuser/somerepo/folderC/anotherfile.nut@develop"
If
- the path in the include is a relative path
- and the processed file is a remote file (ie. the file on a remote resource)
- and the
--use-remote-relative-includes
option is specified
then the final path to the include file is a concatenation of the path to the processed file and the path in the include. If the file is not found there, Builder reports an error.
The processed file: "https://example.com/folderA/folderB/somefile.nut"
It contains:
@include "anotherfile.nut"
The final path to the include: "https://example.com/folderA/folderB/anotherfile.nut"
The processed file: "github:someuser/somerepo/folderA/folderB/[email protected]"
It contains:
@include "folderC/anotherfile.nut"
The final path to the include: "github:someuser/somerepo/folderA/folderB/folderC/[email protected]"
If
- the path in the include is a relative path
- and the processed file is a local file (or it is a remote file but the
--use-remote-relative-includes
option is not specified)
then Builder makes the following steps to find the include file:
- Only if the processed file is a local file, the final path to the include file is a concatenation of the path to the processed file and the path in the include. If the file is not found there, moves to the next step.
- The final path to the include file is a concatenation of the path to the file specified as the
<input_file>
parameter of thepleasebuild
command and the path in the include. If the file is not found there, moves to the next step. Note: If you use Builder as a library and want this search step to work, make sure you specify the input file path (builder.machine.readers.file.inputFileDir
variable) in your source code within the Builder initialization routine. See the Library Installation section to learn more. - The final path to the include file is a concatenation of the path to the directory from where the
pleasebuild
command has been called and the path in the include. If the file is not found there, Builder reports an error.
The processed file: "/home/user/someuser/folderA/somefile.ext"
It contains:
@include "folderX/anotherfile.nut"
The "pleasebuild" command is called from: "/home/user/someuser/folderB"
With the <input_file> parameter: "folderC/initialfile.nut"
The anotherfile.nut file will be searched sequentially in the following locations:
1. "/home/user/someuser/folderA/folderX/"
2. "/home/user/someuser/folderB/folderC/folderX/"
3. "/home/user/someuser/folderB/folderX/"
Builder supports file includes from the following types of remote resources:
For includes from weblinks.
@include "http://<server>[:<port>]/<path>"
@include "https://<server>[:<port>]/<path>"
@include once "http://<server>[:<port>]/<path>"
@include once "https://<server>[:<port>]/<path>"
where:
server
is the host name or IP address.port
is the port number.path
is the path to file.
@include "https://example.com/somefile.ext"
@include once "http://example.com/folderA/folderB/somefile.nut"
http://<server>:<port>
https://<server>:<port>
For includes from github.com.
@include "github:<user>/<repo>/<path>[@<ref>]"
@include once "github:<user>/<repo>/<path>[@<ref>]"
where:
user
is the user/organization name.repo
is the repository name.path
is the path to file.ref
is the git reference (branch name or tag, defaults to master).
@include "github:electricimp/Promise/promise.class.nut" // head of the default branch
@include "github:electricimp/Promise/promise.class.nut@develop" // head of the develop branch
@include once "github:electricimp/Promise/[email protected]" // tag v3.0.1
github:<user>/<repo>
Authentication is optional. It is required to access private repositories only. But please note that when you use authentication, the GitHub API provides much higher rate limits.
For the authentication you need to provide:
- a GitHub username (
--github-user
option) - a GitHub personal access token or a password, which is less secure and not recommended (
--github-token
option).
For includes from Bitbucket Server. Note, this is not the same as Bitbucket Cloud, includes from Bitbucket.org are not supported.
Builder can work with only one Bitbucket server at a time. Its address must be specified (--bitbucket-server-addr
option). Server version 5.3.0 or above is supported.
@include "bitbucket-server:<project>/<repo>/<path>[@<ref>]"
@include "bitbucket-server:~<user>/<repo>/<path>[@<ref>]"
@include once "bitbucket-server:<project>/<repo>/<path>[@<ref>]"
@include once "bitbucket-server:~<user>/<repo>/<path>[@<ref>]"
where:
project
is the project name. Should be used to include files from project repositories.user
is the user name. Should be used to include files from personal repositories. Note, the user name must be prepended with~
. Eg, for the user name John it should be~john
.repo
is the repository name.path
is the path to file.ref
is the git reference (branch name or tag, defaults to master).
@include "bitbucket-server:Tools/Promise/promise.class.nut" // head of the default branch
@include once "bitbucket-server:Tools/Promise/promise.class.nut@develop" // head of the develop branch
@include "bitbucket-server:~john/Promise/[email protected]" // tag v3.0.1
bitbucket-server:<project>/<repo>
bitbucket-server:~<user>/<repo>
Authentication is optional. It is required to access private repositories only.
For the authentication you need to provide:
- a Bitbucket Server username (
--bitbucket-server-user
option) - a Bitbucket Server personal access token or a password, which is less secure and not recommended (
--bitbucket-server-token
option).
For includes from Azure Repos.
@include "git-azure-repos:<org>/<project>/<repo>/<path>[@<ref>]"
@include once "git-azure-repos:<org>/<project>/<repo>/<path>[@<ref>]"
where:
org
is the organization name.project
is the project name.repo
is the repository name.path
is the path to file.ref
is the git reference (branch name or tag, defaults to master).
@include once "git-azure-repos:org/project/repo/path/some.class.nut" // head of the default branch
@include "git-azure-repos:org/project/repo/path/some.class.nut@develop" // head of the develop branch
@include "git-azure-repos:org/project/repo/path/[email protected]" // tag v3.0.1
git-azure-repos:<org>/<project>/<repo>
Authentication is optional. It is required to access private repositories only.
For the authentication you need to provide:
- an Azure Repos username (
--azure-user
option) - an Azure Repos personal access token (
--azure-token
option).
For includes from Git repositories hosted locally. Note, even as files are local, Builder considers them as files from a remote resource when searching the file to include.
@include "git-local:<path>[@<ref>]"
@include once "git-local:<path>[@<ref>]"
where:
path
is the path to file.ref
is the git reference (branch name, tag or commit, defaults to current branch which the local git repository is set to).
@include once "git-local:/path/to/repo/and/file/some.class.nut" // head of the default branch
@include once "git-local:/path/to/repo/and/file/some.class.nut@develop" // head of the develop branch
@include "git-local:/path/to/repo/and/file/[email protected]" // tag v3.0.1
@include "git-local:/path/to/repo/and/file/some.class.nut@c13c59e96f3f6a37f75f9e520d0fdc5591e0ba83" // concrete commit
Is determined by Builder.
- Caching is not applicable to local Git files.
- If there are uncommitted changes, they will not be seen by Builder. Hence
@include "git-local:<path>"
(without<ref>
) is not interchangeable with@include "<path>"
. - There are local and remote branches in Git. If you want to create a local remote-tracking branch from a remote branch, you can use, for example, the
git checkout <remote_branch_name>
command. For more info, see the Git docs. - To include a remote git branch, you should specify the name of the remote repository in the
ref
part of the directive. For example:
@include "git-local:/path/to/repo/and/file/some.class.nut@develop" // include local git branch
@include "git-local:/path/to/repo/and/file/some.class.nut@origin/develop" // include remote git branch from the "origin" remote repository
To reduce compilation time, Builder can optionally cache files included from a remote resource. If the file cache is enabled, remote files are cached locally in the .builder-cache
directory. Cached resources expire and are automatically invalidated 24 hours after their addition to the cache.
To turn the cache on, pass the --cache
or -c
option to Builder. If this option is not specified, Builder will not use the file cache even if the cached data exists and is valid — Builder will continue to query remote resources on every execution.
To reset the cache, use both the --cache
and the --clear-cache
options.
If a resource should never be cached, it needs to be added to the exclude-list.builder
file (see the example below). You can use wildcard characters to mask file names.
Note: Non-identical paths, even if they point to absolutely the same location, are treated as different paths. Eg., if you include /a/b/1.nut
, it becomes cached, and then you include /a/b/../b/1.nut
, it will not be found in the cache.
Pattern matching syntax is a similar to that of .gitignore. A string is a wildcard pattern if it contains '?
' or '*
' characters. Empty strings or strings that starts with '#
' are ignored.
A '?
' symbol matches any single character. For example, bo?t.js
matches boot.js
and boat.js
, but doesn't match bot.js
.
A '*
' matches any string, that is limited by slashes, including the empty string. For example, /foo/*ar
matches /foo/bar
, /foo/ar
and /foo/foo-bar
, but doesn't match /foo/get/bar
or /foo/bar/get
.
Two consecutive asterisks **
in patterns matched against full pathname may have special meaning:
-
A leading
**
followed by a slash means match in all directories. For example,**/foo
matches file or directoryfoo
anywhere, the same as patternfoo
.**/foo/bar
matches file or directorybar
anywhere that is directly under directoryfoo
. -
A trailing
/**
matches everything inside. For example,abc/**
matches all files inside directoryabc
. -
A slash followed by two consecutive asterisks then a slash matches zero or more directories. For example,
a/**/b
matchesa/b
,a/x/b
,a/x/y/b
and so on. -
Other consecutive asterisks are considered invalid.
# Avoid caching a specific file
github:electricimp/MessageManager/MessageManager.lib.nut
# Exclude all electricimp repos
github:electicimp/**
# Exclude all tagged files or files from the specific branches from the cache
github:*/**/*@*
It is possible to save the versions of all remote files which are used during the current run of Builder, and reuse exactly that versions later. This is applicable for files from repositories only.
See the Repository Files: Dependencies section for the details.
To specify a proxy that should be used when you are including files from remote resources, set the environment variables HTTP_PROXY
/http_proxy
and/or HTTPS_PROXY
/https_proxy
for HTTP and HTTPS protocols respectively.
For example, to operate through a proxy running at IP address 192.168.10.2
on port 3128
for HTTP requests, you should set the environment variable: HTTP_PROXY='http://192.168.10.2:3128'
. All of Builder’s HTTP requests will now go through the proxy.
This section contains information that will help you work with Builder more effectively, but may not be needed for more basic Builder tasks.
It is possible to save the build configuration data used for preprocessing a source file in order to create an identical source file again later with that saved configuration. Builder variable definitions are saved in a ‘directives.json’ file, and references to the concrete versions of repository files and libraries are stored in a ‘dependencies.json’ file.
The --save-directives [<path_to_file>]
and --use-directives [<path_to_file>]
options are used to, respectively, save and reuse Builder variable definitions. The definitions are saved in a JSON file. If a file name is not specified, the directives.json
file in the local directory is used. These options are processed the similar way as the --save-dependencies
and --use-dependencies
options, above.
When the --use-directives [<path_to_file>]
option is used, the saved Builder variable definitions are merged with definitions specified by -D<variable> <value>
options.
A typical directives.json
file looks like this:
{
"Variable0": "value0",
"Variable1": "value1"
}
--save-dependencies [<path_to_file>]
and --use-dependencies [<path_to_file>]
options are used to save and to reuse, respectively, references to concrete versions of repository files and libraries. The references are saved in a JSON file. If a file name is not specified, the dependencies.json
file in the local directory is used. Every reference consists of repository file URL and:
- Git Blob ID (Git Blob SHA) — for GitHub files
Note It is possible to obtain the Git Blob ID of a GitHub file using the following git command:git hash-object <path_to_file>
- Git Commit ID (Git Commit SHA) — for Bitbucket Server, Azure Repos and Git Local files
For more information, please see the Git Manual and the Git API.
These options are processed the following way:
- If only
--save-dependencies [<path_to_file>]
is specified, the references to all source files retrieved from repositories are saved in the provided JSON file (ordependencies.json
). - If only
--use-dependencies [<path_to_file>]
is specified, the source files from repositories are retrieved using the references read from the provided JSON file (ordependencies.json
). - If both
--save-dependencies [<path_to_file>]
and--use-dependencies [<path_to_file>]
are specified, then:- The source files from repositories are retrieved using the references read from the JSON file passed to the
--use-dependencies
option (ordependencies.json
). - If the source code contains @includes for files from repositories which have not yet been retrieved, they are retrieved now.
- Builder performs the preprocessing operation.
- References to all source files retrieved from repositories are saved in the JSON file passed to the
--save-dependencies
option (ordependencies.json
).
- The source files from repositories are retrieved using the references read from the JSON file passed to the
Note If --save-dependencies
is specified, the --cache
option is ignored. If --use-dependencies
is specified, the --cache
option does not affect the files referenced in the dependency file.
A typical dependencies.json
file looks like this:
[
[
"github:ProjectA/repositoryA/fileA",
"2ff017dc92e826ad184f9cdeadd1a2446f8d6032"
],
[
"github:ProjectB/repositoryB/fileB",
"a01b64f9ce764f226f52c6b9364396d4a8bd550b"
],
[
"bitbucket-server:projectC/repositoryC/fileC",
"4bc4024f1f2ad99e8bd2ade73d151912e031d1f5"
],
[
"git-azure-repos:org/projectD/repositoryD/fileD",
"d1ccee9ed6e250c6d5e1f052107125659d3ba9d0"
],
[
"git-local:/path/to/repo/and/fileD",
"c13c59e96f3f6a37f75f9e520d0fdc5591e0ba83"
]
]
Note: Non-identical paths, even if they point to absolutely the same location, are treated as different paths. Eg., if you have saved dependencies for /a/b/1.nut
and then you include /a/b/../b/1.nut
- the dependencies will not be applied to this file.
Builder can accept JavaScript libraries to add functionality to its global namespace. The library should export an object, the properties of which will be merged into the global namespace. For example, to include a function, upper(), to convert strings to uppercase, define your library file like so:
module.exports = {
upper: (s) => s.toUpperCase()
};
Now include the function within directives in your input file:
@{upper("warning:")}
@{upper(include("warning.txt"))}
Finally, run Builder with the option --lib path/to/your/lib/file
.
Functions called by Builder will be called with their this argument set to a Builder context object. Within the context object, Builder variables like __FILE__
, functions like max()
, and other included library functions will be made available at the top level. Variables defined in your input code with @macro
or @set
will be available under the key globals.
Ignoring the binding of this may cause unexpected behavior, for example when calling methods on objects. Take the following example library:
class MyClass {
constructor(str) {
this._str = str;
}
getStr() {
return this._str;
}
}
myObject = MyClass("my text");
module.exports = {
myObject
};
Attempting to use this library with the directive @{myObject.getStr()}
will not deliver the expected behavior because this in getStr() will be set to a Builder context object and not to myObject. When calling class methods ensure they have been bound to the correct value of this:
class MyClass {
constructor(str) {
this._str = str;
}
getStr() {
return this._str;
}
}
myObject = MyClass("my text");
module.exports = {
getStr: myObject.getStr.bind(myObject)
};
When running tests locally, please test on both Windows and macOS. All environment variables are optional. However, if you are working with @includes
from GitHub and do not provide GitHub credentials, rate limits imposed by GitHub may cause test failures. The default for SPEC_LOGLEVEL
is error
.
npm install
SPEC_LOGLEVEL=<debug|info|warning|error>
SPEC_GITHUB_TOKEN=<GitHub personal access token>
npm test
Note 1: The standard set of tests doesn't include Bitbucket Server, Azure Repos and Git Local (but see the Note 2) integration testing. To run Bitbucket Server, Azure Repos or Git Local tests, please see the sections below.
Note 2: The standard set of tests uses Git Local in several tests for testing the overall behavior of Builder. They require the SPEC_GIT_LOCAL_REPO_PATH
variable to be set (see the Git Local section below). These are optional tests so they will be skipped if that variable is not set.
Note 3: There are several tests that require access to the root of the filesystem (or to the root of the disk C:
on Windows). They will create/remove there a directory named builder_test_g2e5r6uh
, so please make sure you don't have some important data in such directory.
Prerequisites:
- A running instance of Bitbucket Server
- A clone of Builder repo placed on this server
- If the server / repo are not public, an account (username and password / token) with permissions to access the repo
npm install
SPEC_LOGLEVEL=<debug|info|warning|error>
# E.g., "https://bitbucket-srv.itd.example.com"
SPEC_BITBUCKET_SERVER_ADDRESS=<Bitbucket Server address>
# Format: "<project>/<repo>". E.g., "myProj/BuilderClone"
# If the repo belongs to a user (not to a project), the format is: "~<user>/<repo>". E.g., "~john/BuilderClone"
SPEC_BITBUCKET_SERVER_REPO_PATH=<Path to the cloned Builder repo on the server>
SPEC_BITBUCKET_SERVER_USERNAME=<Bitbucket Server username>
SPEC_BITBUCKET_SERVER_TOKEN=<Bitbucket Server password/access token>
npm run test:bitbucket-server
Prerequisites:
- A clone of Builder repo placed at Azure Repos
- An account (username and token) with permissions to access the repo
npm install
SPEC_LOGLEVEL=<debug|info|warning|error>
# Format: "<org>/<project>/<repo>". E.g., "myOrg/myProj/BuilderClone"
SPEC_AZURE_REPOS_REPO_PATH=<Path to the cloned Builder repo at the Azure Repos>
SPEC_AZURE_REPOS_USERNAME=<Azure Repos username>
SPEC_AZURE_REPOS_TOKEN=<Azure Repos access token>
npm run test:azure-repos
Prerequisites:
- A clone of Builder repo placed locally
npm install
SPEC_LOGLEVEL=<debug|info|warning|error>
SPEC_GIT_LOCAL_REPO_PATH=<Path to the root of the cloned Builder repo>
npm run test:git-local
Builder is licensed under the MIT License.