Can you memoize as TaskEither?

[florianbepunkt]

How to effectively memoize a TaskEither?

I have an array with about 1000 elements. For each element I need to perform an api call, but there are about 2-5 different results across these 1000 elements. So I would like to memoize the api call, but I haven't found a way to do this yet.

Given the following code, the task either is lazily evaluated. How can I memoize the getApiResult function so the api gets only called once for every importId

import { pipe } from 'fp-ts/lib/function';
import * as TE from 'fp-ts/lib/TaskEither';
import * as ROA from 'fp-ts/ReadonlyArray';

type A = any;
type Transaction = { importId: string };


// how to memoize getApiResult?
const getApiResult = (importId: string): TE.TaskEither<Error, A> =>
  TE.tryCatch(
    () => {
      // calls some api, but I can't memoize the api function
    },
    () => new Error(''),
  );
  
const convert =
  (t: Transaction) =>
  (apiResult: A): TE.TaskEither<Error, string> => {
    throw new Error('Not implemented');
  };

const main = (transactions: readonly Transaction[]) =>
  pipe(
    transactions,
    ROA.map((t) => pipe(t.importId, getApiResult, TE.chain(convert(t)))),
  );

[mlegenhausen]

Why call getApiResult at all 1000 times? Why not reduce to all unique importIds before hand, then run your getApiResult function and then assign your results back to the associated transactions?

[florianbepunkt]

I have the same problem with multiple apis, called with different properties of a transaction. Instead of creating multiple sets of unique properties in advance, memoization seemed to be a cleaner approach, but I might be wrong.

[mlegenhausen]

IMHO if you can solve the problem with pure code you should go for it. It has the advantage of getting the best support from your type system and is easily testable. From my experience you will also learn a lot from your problem domain. Yes it is more explicit in your code but this a good thing else why bother with all the typings and manual error handling 😅

If you want to keep (mutable) state around you should go with @imcotton suggestions but keep in mind that you will have to manage your caching layer by yourself.

You can also take a look at this PR from @DenisFrezzato which suggests a memoizeF function for the fp-ts-contrib package.

[florianbepunkt]

Thank you to you both. I appreciate the help.

[imcotton]

IORef ¹ is easiest hack to place inside getApiResult.
StateReaderTaskEither ² is fp-ish way to go.

Footnotes

1. https://gcanti.github.io/fp-ts/modules/IORef.ts.html ↩

2. https://gcanti.github.io/fp-ts/modules/StateReaderTaskEither.ts.html ↩

[imcotton]

Memorization is a good concept, but keep in mind that internal memory may grow rapidly, necessarily the use of LRU caching comes into play which is another topic.

It's more like a hashing lookup table in your situation, where could use State Monad as a thin layer without touching the API.

Hereby the example: (btw it's State instead of StateReaderTaskEither, which is sufficient enough on its own)

import {
    eq as Eq,
    function as F,
    string as Str,
    option as O,
    state as S,
    readonlyMap as M,
    taskEither as TE,
} from 'fp-ts';



type A = unknown;
type Transaction = { importId: string };
type Cache = ReadonlyMap<Transaction, TE.TaskEither<Error, A>>;



declare const getApiResult: (id: string) => TE.TaskEither<Error, A>;
declare const convert: (tr: Transaction) => (res: A) => TE.TaskEither<Error, string>;



const eqTransaction = Eq.struct<Transaction>({ importId: Str.Eq });
const find = M.lookup(eqTransaction);
const update = M.upsertAt(eqTransaction);



const main = F.flow(
    S.traverseArray((tr: Transaction) => F.pipe(
        S.get<Cache>(),
        S.map(find(tr)),
        S.map(O.getOrElse(() => getApiResult(tr.importId))),
        S.chainFirst(v => S.modify(update(tr, v))),
        S.map(TE.chain(convert(tr))),
    )),
    S.map(TE.sequenceArray),
    S.evaluate(M.empty as Cache),
);