Http api

[florianbepunkt]

I would love to see a way to use the http api from @effect/platform with a lambda fn fronted by api gateway. Currently we use middy and their http router in our code base for some microservices.

[florianbepunkt]

To be more precise, I believe two functionalities could be beneficial, although maybe out of scope for this package (especially the latter).

(1) A function similar to HttpApiBuilder.toWebHandler and https://github.com/floydspace/effect-aws/tree/main/packages/lambda that will take an Http api and returns a handler function that takes an APIGatewayProxyEvent as input and produces an APIGatewayProxyResult as output

(2) It should be possible to take the http api definition and derive a CDK api gateway from it that can be deployed, similar to OpenAPI cdk constructs.

[floydspace]

hi @florianbepunkt
I think anything related to aws coulb be part of this monorepo, I'm not against it.
I know @AMar4enko had implemented something like what you describe in the (2), I believe this one.
I personally did not use httpapi yet, still in my list, I will explore it, but if you are welcome to contribute if you already have implementation in mind, if not at least you can draft some interface of usage, which would be a helpful starting point.

[florianbepunkt]

@floydspace Thanks for the link. This is an interesting proof of concept.

I believe the CDK construct is the more difficult part. We have several APIs, where we have an OpenAPI spec that uses entities defined by Effect Schema. This OpenAPI spec file is used to create a rest api CDK construct, similar to what the poc does. The problem, which would also apply to the poc you have linked: Effect produces JSON schema v7 while AWS api gateway can only handle draft4. With slightly more complex schemas you will unfortunately run into problems :/ Therefore we had to use a custom JSON schema generator to create draft4 JSON schemas. I opened an issue here: AMar4enko/effect-http-api-gateway#1 Maybe AMar4enko has came up with a way to solve this.

I'm not sure whether using OpenAPI is the right approach. If you define the API in effect, you are already doing the validation there. So while there is probably no harm in double validating requests, it is not necessary. Plus you get the limitations mentioned above.

With regards to (1):

I believe something around the following lines should work (untested):

import { DateTime, Effect, Layer, Schema as S } from "effect";
import {
  HttpApi,
  HttpApiBuilder,
  HttpApiEndpoint,
  HttpApiGroup,
  HttpApp,
  HttpServer,
  type HttpRouter,
} from "@effect/platform";
import type { Router } from "@effect/platform/HttpApiBuilder";
import type { APIGatewayProxyEvent, APIGatewayProxyResult, Context } from "aws-lambda";

const eventToNativeRequest = (event: APIGatewayProxyEvent): Request => {
  const { httpMethod, headers, body, path, queryStringParameters } = event;

  // Construct the URL
  const protocol = headers["X-Forwarded-Proto"] || "https";
  const host = headers["Host"] || "localhost";
  const queryString = new URLSearchParams(
    (queryStringParameters as Record<string, string>) || {},
  ).toString();
  const url = `${protocol}://${host}${path}${queryString ? `?${queryString}` : ""}`;

  // Map headers to Headers object
  const requestHeaders = new Headers();
  for (const [key, value] of Object.entries(headers || {})) {
    if (value) {
      requestHeaders.append(key, value);
    }
  }

  // Return the Request object
  return new Request(url, {
    method: httpMethod,
    headers: requestHeaders,
    body: body || undefined,
  });
};

const fromNativeResponse = async (response: Response): Promise<APIGatewayProxyResult> => {
  const headers: { [header: string]: string } = {};

  response.headers.forEach((value, key) => {
    headers[key] = value;
  });

  const body = response.bodyUsed ? await response.text() : null;

  return {
    statusCode: response.status,
    headers,
    body: body || "",
    isBase64Encoded: false, // Assume the body is not Base64 encoded by default
  };
};

const makeApiLambda = <LA, LE>(
  layer: Layer.Layer<LA | HttpApi.Api | HttpRouter.HttpRouter.DefaultServices, LE>,
  options?: {
    readonly middleware?: (
      httpApp: HttpApp.Default,
    ) => HttpApp.Default<never, HttpApi.Api | Router | HttpRouter.HttpRouter.DefaultServices>;
    readonly memoMap?: Layer.MemoMap;
  },
) => {
  const { handler } = HttpApiBuilder.toWebHandler(layer, options);
  return async (event: APIGatewayProxyEvent, context: Context): Promise<APIGatewayProxyResult> => {
    const request = eventToNativeRequest(event);
    const response = await handler(request);
    return fromNativeResponse(response);
  };
};

// Usage example

class User extends S.Class<User>("User")({
  id: S.Number,
  name: S.String,
  createdAt: S.DateTimeUtc,
}) {}

class UsersApi extends HttpApiGroup.make("users").add(
  HttpApiEndpoint.get("findById", "/users/:id")
    .addSuccess(User)
    .setPath(
      S.Struct({
        id: S.NumberFromString,
      }),
    ),
) {}

class MyApi extends HttpApi.empty.add(UsersApi) {}

const UsersApiLive: Layer.Layer<HttpApiGroup.ApiGroup<"users">> = HttpApiBuilder.group(
  MyApi,
  "users",
  (handlers) =>
    handlers
      // the parameters & payload are passed to the handler function.
      .handle("findById", ({ path: { id } }) =>
        Effect.succeed(
          new User({
            id,
            name: "John Doe",
            createdAt: DateTime.unsafeNow(),
          }),
        ),
      ),
);

const MyApiLive: Layer.Layer<HttpApi.Api> = HttpApiBuilder.api(MyApi).pipe(Layer.provide(UsersApiLive));

const handler = makeApiLambda(Layer.mergeAll(MyApiLive, HttpServer.layerContext));

[florianbepunkt]

@floydspace I sketched a rough prototype: https://github.com/florianbepunkt/effect-aws-api

If you find the time, maybe you can have a look. I'm rather unsure how to approach / generalize this in a way that it helps others. The CDK construct make a lot of assumptions about a use case – some might make sense (lambdalith with http api), other might not.

Also there are some open questions I noted in the readme and the code.

[floydspace]

I dived a bit deeper to it and came up with approx the same implementation as yours,
the only without actually using the toWebHandler, but directly maping from httpApp

I will make a PR coming days, so you could review it if you don't mind @florianbepunkt .

[florianbepunkt]

@floydspace Yes, definitely. I ended up not using the toWebHandler (see a version of my implementation here: https://github.com/florianbepunkt/effect-aws-api)

Two points:

• I ended up with providing the raw lambda request and context as a service that can be yielded. Since I use cognate authorizers I needed to access the raw event request context in my auth middleware
• One of the last updates of @effect/platform (from 0.69.24 ❯ 0.69.31) blew everything up – now I get Error: Service not found: @effect/platform/HttpApiBuilder/Middleware for all request. So curious to see your take on it.

[floydspace]

@florianbepunkt here is a sneak preview what I drafted so far #86
I desided to copy some ideas from here, specifically the way how they map different lambda events to unified request object (still dirty and not really effectfull way)
but the core logic is pretty simple:

effect-aws/packages/lambda/src/ApiHandler.ts

Lines 18 to 55 in 41c88a7

	export const apiHandler: EffectHandler<
	AnyEvent,
	HttpApi.Api | HttpApiBuilder.Router | HttpRouter.HttpRouter.DefaultServices,
	never,
	unknown
	> = (event) =>
	Effect.gen(function* () {
	const app = yield* HttpApiBuilder.httpApp;
	const eventSource = getEventSource(event);
	const requestValues = eventSource.getRequest(event);
	const request = new Request(
	`http://${requestValues.remoteAddress}${requestValues.path}`,
	{
	method: requestValues.method,
	headers: requestValues.headers,
	body: requestValues.body,
	},
	);
	const response = yield* app.pipe(
	Effect.provideService(
	HttpServerRequest.HttpServerRequest,
	HttpServerRequest.fromWeb(request),
	),
	);
	return eventSource.getResponse({
	event,
	statusCode: response.status,
	body: new TextDecoder("utf-8").decode(
	(response.body as HttpBody.Uint8Array).body,
	),
	headers: response.headers,
	isBase64Encoded: false,
	});
	});

still testing it wiht my side project to identify some edge cases

[floydspace]

I ended up with providing the raw lambda request and context as a service that can be yielded. Since I use cognate authorizers I needed to access the raw event request context in my auth middleware

RawLambdaInput is a good point, might be a good candidate being as optional service to keep original HttpApiBuilder contract the same

One of the last updates of @effect/platform (from 0.69.24 ❯ 0.69.31) blew everything up – now I get Error: Service not found: @effect/platform/HttpApiBuilder/Middleware for all request. So curious to see your take on it.

I will check it. currently testing on v0.69.25

[florianbepunkt]

Nice work so far. I didn't know that there were so many possible request sources. Not sure how much it would benefit from a more "effect-idiomatic" style (not even sure if there is only one :D).

RawLambdaInput is a good point, might be a good candidate being as optional service to keep original HttpApiBuilder contract the same

Yes, it can be optional. I excluded it from the requirements that need to be passed. But since it's a given that we will always start from an AWS event, the lambda adapter can always inject it. It would be nice btw to make the type guards for the different source events part of the public API – I guess otherwise everyone that needs to access the raw event needs to duplicate the same logic.

One of the last updates of @effect/platform (from 0.69.24 ❯ 0.69.31) blew everything up

Just digged into it and fixed my PoC. You can also find CDK constructs in it for creating the API gateway. This is all extracted from a project, so not as generalized as I would like it to be.

Last thing: Since the lambda execution environment can get reused, it would be great to build up the dependencies/R only once, instead of building everything for each request (thinking of db connections here).

Just ping me for the PR, happy to review. Thank you for all your work for the serverless community, much appreciated.

[floydspace]

It would be nice btw to make the type guards for the different source events part of the public API

yes, I'm thinking to port the Powertools Parser (Zod) onto effect, so it will expose all the effect schemas for events and even implement effectful envelope pattern

Since the lambda execution environment can get reused, it would be great to build up the dependencies/R only once, instead of building everything for each request

that is covered in my draft, currently db layer can be configured globally and passed to makeApiLambda along with ApiLive and platform defaultLayer

Thank you for all your work for the serverless community, much appreciated

happy to be useful, thank you