Skip to content
This repository has been archived by the owner on Oct 27, 2020. It is now read-only.

custom write and read are much slower than the default #48

Open
GalmWing opened this issue Oct 31, 2018 · 3 comments
Open

custom write and read are much slower than the default #48

GalmWing opened this issue Oct 31, 2018 · 3 comments

Comments

@GalmWing
Copy link

I was trying to speed up the cache read/write, so I decided to store stuff in memory with node-cache, instead of in the filesystem (default behavior).

To my surprise, the build time is slower with in-memory cache than with filesystem, it's so slow even a build without cache-loader is faster.

node-cache has pretty much the same function signatures as the example provided in the docs here for Redis. so my implementation is basically carbon copy of that

const NodeCache = require( "node-cache" );
const myCache = new NodeCache();
const BUILD_CACHE_TIMEOUT = 24 * 3600; // 1 day
// Read data from database and parse them
function read(key, callback) {
  myCache.get(key, (err, result) => {
    if (err) {
      return callback(err);
    }

    if (!result) {
      return callback(new Error(`Key ${key} not found`));
    }

    try {
      let data = JSON.parse(result);
      callback(null, data);
    } catch (e) {
      callback(e);
    }
  });
}

// Write data to database under cacheKey
function write(key, data, callback) {
  myCache.set(key, JSON.stringify(data), BUILD_CACHE_TIMEOUT, callback);
}


...
module.exports = {
  // rest of config
  use: [
          {
            loader: "cache-loader",
            options: { read, write }
          },
          { loader: "babel-loader"},
          { loader: "ts-loader" }
        ]
}

My project is quite massive, hundreds of tsx files that need both babel and ts loaders (which is why I wanted to do caching).

Incremental build times:

  • Without cache-loader: 30s
  • With cache-loader and default read/write: 20s
  • With cache-loader and custom read/write to memory: 50s
@alexander-akait
Copy link
Member

Looks custom cache is more slow than original, what is problem?

@GalmWing
Copy link
Author

GalmWing commented Nov 2, 2018

The custom cache shouldn't be that slow. node-cache by itself (the module I'm using to cache in-memory) is very fast, and I'm not adding any overhead, cache-loader does also a json serialization with the default options. So I don't see why my custom read and write methods would make cache-loader be this slow

@alexander-akait
Copy link
Member

@GalmWing Feel free to investigate what is problem, should be not difficult

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants