Filesystem Guide

Filesystem Guide

Emscripten allows you to set up a virtual filesystem that points to preloaded data, as well as virtual devices that can read and write data.

There are two basic ways to use the filesystem:

• Package some files with your build. You can just tell emcc to package a directory or a set of files, and those files will be accessible from the compiled code normally, using fopen etc.
• Manually use the FS API from JavaScript. This lets you create, modify etc. files in a more manual manner.

For concrete examples of using the filesystem API, see the automatic tests in tests/runner.py. Search for FS. for find relevant tests, for example test_files. For specific examples of how to embed files that can be read from compiled C/C++, see for example the OpenJPEG test.

Packaging files

The simplest thing to do is just tell emcc to package files for you,

emcc file.cpp -o file.html --preload-file asset_dir

That command will generate file.html, and alongside it file.data which will contain all the files in asset_dir/. You can then distribute your application with just those two files.

Assets can also be embedded directly into the html file:

emcc file.cpp -o file.html --embed-file asset_dir

When this is done with a relative path, the prefixes will remain the same in the file system. For example:

emcc file.cpp -o file.html --embed-file ../../asset_dir

Files will all be prefixed with ../../asset_dir/. To change this behavior, call emcc as follows:

emcc file.cpp -o file.html --embed-file ../../assets@/

This will package the files at the root of the file system.

You can also run the file packager manually, tools/file_packager.py. See docs in that file. It generates a .data file and .js file, which contains the manual commands to utilize the data file (emcc --preload just bundles that code in your normal generated output). You can then load that JavaScript before loading your main compiled code.

• Note that this lets you do the file packaging without running emcc to compile your code, the two processes are separated this way.
• Note also that you can load multiple datafiles. Just run the file packager on each and load the .js outputs. See BananaBread for an example of this (cube2/js/game-setup.js).

Manually using the FS API

Note that all the configuration should be done before the main run() method is executed, typically by implementing Module.preRun, see Interacting-with-code.

Setting up standard I/O devices

Emscripten standard I/O works by going though the virtual /dev/stdin, /dev/stdout and /dev/stderr devices. You can set them up using your own I/O functions by calling FS.init(input_callback, output_callback, error_callback) (all arguments optional).

• The input callback will be called with no parameters whenever the program attempts to read from stdin. It should return an ASCII character code when data is available, or null when it isn't.
• The output callback will be called with an ASCII character code whenever the program writes to stdout. It may also be called with null to flush the output.
• The error callback is similar to the output one, except it is called when data is written to stderr.

If any of the callbacks throws an exception, it will be caught and handled as if the device malfunctioned.

By default:

• stdin will read from the terminal in command line engines and use window.prompt() in browsers (in both cases, with line buffering).
• stdout will use a print function if one such is defined, printing to the terminal in command line engines and to the browser console in browsers that have a console (again, line-buffered).
• stderr will use the same output function as stdout.

Setting up files and folders

The FS object provides several methods to create files, folders, devices and symbolic links:

• FS.createFolder(parent, name, canRead, canWrite): Creates a single empty folder and returns a reference to it.

  • (string|object) parent: The parent folder, either as a path (e.g. '/usr/lib') or an object previously returned from a FS.createFolder() or FS.createPath() call.
  • string name: The name of the new folder.
  • bool canRead: Whether this folder should have read permissions set from the program's point of view.
  • bool canWrite: Whether this folder should have write permissions set from the program's point of view.

  Example:

    var home = FS.createFolder('/', 'home', true, false);
    FS.createFolder(home, 'user', true, true);
    FS.createFolder('/home', 'other-user', false, false);
  

• FS.createPath(parent, path, canRead, canWrite): Recursively creates a path and returns a reference to the innermost folder.

  • (string|object) parent: The parent folder, either as a path (e.g. '/usr/lib') or an object previously returned from a FS.createFolder() or FS.createPath() call.
  • string path: The path to the new folder. Any folders missing in this path will be created.
  • bool canRead: Whether the created folders should have read permissions set from the program's point of view.
  • bool canWrite: Whether the created folders should have write permissions set from the program's point of view.

  Example:

    FS.createPath('/', 'home/user1', true, false);
    FS.createPath('/', 'home/user2/Desktop', true, false);
  

• FS.createDataFile(parent, name, data, canRead, canWrite): Creates a file containing given data and returns a reference to it. A convenient way to set up a file for you is to call emcc with --preload-file. That will do everything necessary, including calling this function.

  • (string|object) parent: The parent folder, either as a path (e.g. '/usr/lib') or an object previously returned from a FS.createFolder() or FS.createPath() call.
  • string name: The name of the new file.
  • (string|array) data: The data that the new file will contain, either as a string or an array of bytes (integers in the [-128, 255] range).
  • bool canRead: Whether the file should have read permissions set from the program's point of view.
  • bool canWrite: Whether the file should have write permissions set from the program's point of view.

  Example:

    FS.createDataFile('/', 'foo', 'abc', true, false);
    FS.createDataFile('/', 'bar', [1, 2, 3], true, true);
  

• FS.createLazyFile(parent, name, url, canRead, canWrite): Creates a file that will be loaded lazily on first access from a given URL or local filesystem path, and returns a reference to it. WARNING: Firefox and Chrome have recently disabled synchronous binary XHRs, which means this cannot work for Javascript in regular HTML pages (but it works within WebWorkers). Instead, use createDataFile.

  • (string|object) parent: The parent folder, either as a path (e.g. '/usr/lib') or an object previously returned from a FS.createFolder() or FS.createPath() call.
  • string name: The name of the new file.
  • string url: In the browser, this is the URL whose contents will be returned when this file is accessed. In a command line engine, this will be the local (real) filesystem path from where the contents will be loaded. Note that writes to this file are virtual.
  • bool canRead: Whether the file should have read permissions set from the program's point of view.
  • bool canWrite: Whether the file should have write permissions set from the program's point of view.

  Example:

    FS.createLazyFile('/', 'foo', 'other/page.htm', true, false);
    FS.createLazyFile('/', 'bar', '/get_file.php?name=baz', true, true);
  

• FS.createPreloadedFile(parent, name, url, canRead, canWrite): Preloads a file asychronously. You should call this in preRun, and then run() will be delayed until all preloaded files are ready. This is how --preload-file works in emcc.

• FS.createLink(parent, name, target, canRead, canWrite): Creates a symbolic link and returns a reference to it.

  • (string|object) parent: The parent folder, either as a path (e.g. '/usr/lib') or an object previously returned from a FS.createFolder() or FS.createPath() call.
  • string name: The name of the link.
  • string target: The target of the link, a relative or absolute path. The path does not need to exist at the time the link is created.
  • bool canRead: Whether the link should have read permissions set from the program's point of view.
  • bool canWrite: Whether the link should have write permissions set from the program's point of view.

  Example:

    FS.createLink('/bin', 'g++', 'gcc' true, true);
    FS.createLink('/home/jack', 'log', '/var/log/prog.log', true, true);
    FS.createLink('/home/jack/Desktop', 'test', '../dev/prog/run.sh', true, true);
  

• FS.createDevice(parent, name, input, output): Creates a virtual device and returns a reference to it.

  • (string|object) parent: The parent folder, either as a path (e.g. '/usr/lib') or an object previously returned from a FS.createFolder() or FS.createPath() call.
  • string name: The name of the file representing the device.
  • function input: The function called when reading from the device. Should return a byte-sized integer if there's data, or null if there isn't. If output is specified, this can be null or undefined.
  • function output: The function called when writing to the device. Will be called with a byte-sizes integer to write data, or null to flush it (if flushing makes sense for the device). If input is specified, this can be null or undefined.

  Example:

    FS.createDevice('/dev', 'random', function() { return Math.floor(Math.random() * 256); });
    FS.createDevice('/dev/snd', 'pcm', null, playSound);
  

Deleting Files

• FS.deleteFile(path): Deletes the file, directory, link, or device.

  • (string) path: The absolute path of the file, directory, link, or device to delete (e.g. '/myDir/myFile' will delete myFile; myDir remains, of course).

Monitoring Read Files

It is important to only preload the files your app actually needs, to reduce download size and improve startup speed. There is an option to log all the actually used files during runtime, which you can use to figure out which files you app actually needs. To use it, define logOpenedFiles on the Module object. Module.printErr will then be called on each file that is read from, so you can define that function to log to a convenient place.

You can also look at FS.readFiles, which will be an object whose keys are all the files that were read from. This might be easier to use than logging.

Serializing and Restoring

The whole filesystem can be safely serialized and restored at any time by saving the values of FS.root and FS.nextInode, and replacing them with previously saved values, respectively. Keep in mind, however, that devices use functions, which are discarded on a normal JSON.stringify() call, so you might want to use a custom stringifier.

Issues

As mentioned above, Firefox and Chrome recently disabled synchronous typed array XMLHttpRequests, sadly. Asynchronous ones work, but synchronous code that is compiled needs to run synchronously. There are a few possible solutions to work around this:

• Do not do XHRs at all, instead preload the files. The easiest way to do that is to use --embed-file in emcc, it will bundle a file with the code, and prepare it in the filesystem API. For a browser environment, use --preload-file instead. Both are documented in emcc --help.
• If your compiled code can run in a web worker, then it can do synchronous typed array XHRs, browsers allow that.
• Closure compiler will minify the FS API code in -O2 and above. To write code that uses it, it must be optimized with the FS API code by closure. To do that, use emcc's --pre-js option, see emcc --help. There is also a subset of the FS API that is exported through closure, usable by things like Module['FS_createFolder'] (note _ instead of .), search for FS_createFolder in src/library.js to see which. If you use that API you do not need to optimize your code together.