Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mechanism for explicitly qualifying lifting #76

Closed
Tracked by #658
yoav-steinberg opened this issue Sep 8, 2022 · 31 comments · Fixed by #5935
Closed
Tracked by #658

Mechanism for explicitly qualifying lifting #76

yoav-steinberg opened this issue Sep 8, 2022 · 31 comments · Fixed by #5935
Assignees
Labels
🛠️ compiler Compiler 📐 language-design Language architecture

Comments

@yoav-steinberg
Copy link
Contributor

yoav-steinberg commented Sep 8, 2022

Workaround

Since it is possible to lift inflight closures (because a closure's "operation" is always "call me"), then, it is possible to work around the limitation we currently have by wrapping the operation in a closure and lifting this closure.

In the following example, selectBucket returns a cloud.Bucket and then the .put() operation cannot be qualified.

bring cloud;

let b1 = new cloud.Bucket() as "b1";
let b2 = new cloud.Bucket() as "b2";

let selectBucket = inflight (i: num) => {
  if i % 2 == 0 {
    return b1;
  } else {
    return b2;
  }
};

new cloud.Function(inflight () => {
  let bucket = selectBucket(1);
  bucket.put("hello.txt", "world");
// ^^^^ can't qualify
});

To overcome this limitation, we can change selectBucket to actually perform the operation:

bring cloud;

let b1 = new cloud.Bucket() as "b1";
let b2 = new cloud.Bucket() as "b2";

let putBucket = inflight (i: num, k: str, v: str) => {
  if i % 2 == 0 {
    b1.put(k, v);
  } else {
    b2.put(k, v);
  }
};

new cloud.Function(inflight () => {
  putBucket(1, "hello.txt", "world");
});

We need a syntax and implementation for the user to explicitly define rules describing how inflight code is going to use a resource.

The rules can be passed to the resource's capture implementation.

The most basic rule is obviously what inflight methods of the resource are being used. This can be used to set up permissions, for example. But we should also keep in mind that there might be information we want to pass describing possible ranges of arguments to these methods or regexes for valid argument values etc.

Example:

let b = cloud.Bucket();
cloud.Function((event) ~> { 
   // We should have a way to explicitly allow only write access to `b` and only access to the `"filename.dat"` key in `b`.
  b.upload("filename.dat", event.data);
});

In the future we should also have a way to automatically generate these rules based on the code.

@Chriscbr
Copy link
Contributor

Chriscbr commented Sep 10, 2022

An idea for the explicit capture syntax modified from https://github.com/monadahq/rfcs/blob/main/0044-winglang-requirements.md:

let bucket_per_region = {
  US: Bucket(region: 'us-east-1'),
  EU: Bucket(region: 'eu-west-2'),
};

struct MyEvent {
  data: str;
}

let handler = (event: MyEvent) ~> {
  let bucket = bucket_per_region.get(event.region);
  await bucket.upload('boom', event.data);
} captures [
  // user can provide very granular information
  { obj: bucket_per_region.US, calls: [{ method: "upload", arguments: ["boom", Type.Str] }] },
  { obj: bucket_per_region.EU, calls: [{ method: "upload", arguments: ["boom", Type.Str] }] },
  // OR user can just provide list of methods
  { obj: bucket_per_region.NA, methods: ["upload"] },
  { obj: bucket_per_region.NA, methods: ["upload"] },
];

I think for MVP it might be okay to start with just the list of method names - I feel like this is still enough to get useful feedback on the concept of "compiler automatically generates least privilege permissions between resources" from users.

@ekeren
Copy link

ekeren commented Sep 14, 2022

Imagine we 2 programmers in the team, bob and alice. Alice develops a function f_alice that accepts a parameter bucket

// this is wing code
function f_alice upload_file(bucket: cloud.Bucket){
    bucket.upload(foo, bla)
}

Bob wants to use this function as a lambda function

// this is wing code
import f_alice
bucket = cloud.Bucket()
queue = cloud.Queue()
inflight function f_bob(){
   f_alice(bucket) 
}
queue.add_consumer(cloud.Function(f_bob))

The first thing that comes to (my) mind, is that f_bob code is where the capture happens but it is Alice responsibility to define what are the permissions needed from bucket that is sent to f_alice, so if wing compiler is not able to handle the permissions implicitly Alice will need to explicitly mention them: (notice the requires statement)

// this is wing code
function f_alice upload_file(bucket: cloud.Bucket requires "upload"){
    bucket.upload(foo, bla)
}

I think we first need to agree who responsibility is it to define what the function binding requirements, I believe that in this example it is Alice

@ekeren
Copy link

ekeren commented Sep 14, 2022

A more relevant example if when Alice code is not written in wing

in this case, Alice is writing code in python:

import wing

def f_alice(bucket)
    bucket.upload(foo, bar)

# a non succinct option to export a python function to wing
wing.export(f_alice).withArg(0, wing.Bucket, wing.bind("upload"))

@ekeren
Copy link

ekeren commented Sep 18, 2022

@staycoolcall911 & @revitalbarletz FYI

@Chriscbr
Copy link
Contributor

Chriscbr commented Sep 22, 2022

@ekeren I think your example is interesting because it's possible that both functions may want to define information about their capturing.

// this is wing code
function f_alice(bucket: cloud.Bucket){
    bucket.upload(foo, bla)
}
// this is wing code
import f_alice
bucket = cloud.Bucket()
queue = cloud.Queue()
inflight function f_bob(){
   f_alice(bucket)
   bucket.download(bar, bloop)
}
queue.add_consumer(cloud.Function(f_bob))

f_alice needs others to know that it needs to be able to call "upload" to the bucket passed to it, and f_bob needs others to know that it needs to be able to call both "upload" and "download" on the bucket it references. (The capture only becomes "real" or "live" when cloud.Function(f_bob) is created - before then, f_bob is just some code with metadata.)

@Chriscbr
Copy link
Contributor

Chriscbr commented Sep 22, 2022

More syntax ideas...

bring cloud;

let outer = cloud.Bucket();
let handler = (inner: cloud.Bucket): number where {
  inner [upload]
  outer [download, list]
} ~> {
  if random() {
    mystery(inner)
  } else {
    special(outer);
  }
  return 42;
}

Another twist could be if you could specify the permissions you want to grant on the resource, rather than the methods you need to call. For example, we could have a convention in the language where all permission APIs must start with grant_, and then you might write:

let handler = (inner: cloud.Bucket): number where {
  inner { write },
  outer { read, write("*.txt") },
} ~> {
  ...
}

The compiler could then automatically add additional statements to whatever resources use handler:

let fn = cloud.Function(() ~> {
  handler(my_bucket);
  print("done!")
});

// FOR FREE (generated by compiler, hidden to user)
// my_bucket.grant_write(fn);
// outer.grant_read(fn);
// outer.grant_write(fn, "*.txt");

On second thought, I'm not sure designing this around permissions would really be a simpler experience for users. Plus, making permissions part of the type system feels like a weird mixing of responsibilities. Not sure though 🤷‍♂️

@yoav-steinberg
Copy link
Contributor Author

yoav-steinberg commented Sep 22, 2022

Another twist could be if you could specify the permissions you want to grant on the resource

I think we don't and probably never will know exactly what information we'll need to pass to the capture mechanism in the SDK for it to figure out exactly how to capture. We tend to assume these are permissions because that'll probably cover most use cases but it can be other things like credentials, protocol specifications... I think eventually we'll need a syntax where we pass the captured object (my_bucket) and a json describing how it is captured. The interpretation of the json is up to the resource implementation in the SDK. And currently it should support a list of used methods.

@eladb
Copy link
Contributor

eladb commented Sep 22, 2022

I agree that permissions are just one use case and we can't really be opinionated about what info we pass down to the resources when they are connected together.

In a sense they need a way to reflect on the AST of the capture site.

Maybe our language needs something like that.

(This reminds me of the idea to render a step functions workflow by reflecting on the AST).

@Chriscbr
Copy link
Contributor

Chriscbr commented Oct 11, 2022

One use case we've been discussing is:

  • "As a user, I would like to explicitly specify what resources and methods are used by my inflight function [because it can't be inferred by the compiler]"

Another use case we might want to consider is:

  • "As a user, I would like to explicitly specify what resources and methods are allowed in some context so that I can enforce permission boundaries at compile time"

There are some ways to achieve this with the existing language spec:

resource ReadonlyBucket extends cloud.Bucket {
    public override ~put() {
        throw new Error("Cannot put to a readonly bucket");
    }
}

But this isn't type safe, since you can pass a ReadonlyBucket anywhere a Bucket is expected, so you would only discover your application is trying to put objects in the ReadonlyBucket at runtime when errors are thrown.

Instead, maybe there's a way to specify restrictions at compile time. Pseudocode example:

constraint Readonly {
    resource: cloud.Bucket;
    methods: ["get", "list"]; // not allowing "put"
}

let handler = (bucket: @Readonly) ~> {
    print(bucket.get("data.json")); // OK
    print(bucket.put("data.json", stuff)); // compile error: `put` cannot be invoked on a @Readonly resource
};

The above syntax is totally half baked. But I wonder if there might be value in including capture-like constraints as part of our type system somehow. Does anyone have ideas for use cases for this?

@eladb
Copy link
Contributor

eladb commented Oct 13, 2022

This is getting interesting.

Before we dive into syntax, I think it would be useful to design the data model, then we can come up with the best syntax for it if we want to include it in our "header files" (currently seem like these are going to simply be JSII manifests with some extensions), in SDK code (maybe some docstring annotations) and in Wing code (e.g. to define explicit requirements and constraints).

So basically, we are talking about a way to describe how a resource is used within an inflight function. And when we say how it's used, in the object oriented world, it basically comes down to which methods are being called on it, and which arguments are passed to these methods (thinking of properties as sugar for get/set methods).

This description can have two reasons:

  1. For the producer (vendor) of a function to be able to give the inflight runtime environment sufficient information so the environment can support these activities (e.g. bind environment variables with runtime attributes such as address, or include the correct permissions in the runtime policy). Other uses may come up in the future.

  2. For the consumer of this inflight function to be able to express boundaries to the runtime environment so that vendors can't do stupid stuff (either maliciously or due to a bug). In that respect, the consumer (which is usually some compute resource like cloud.Function) expects that if an inflight function is used directly or indirectly and performs operations on the resource that break these constraints, I'd expect a compilation error.

I am starting to think that maybe the right mental model here is "policy", and then maybe it makes sense to get inspiration from an existing policy models such as AWS IAM or Kubernetes RBAC or [OPE].

For example, in AWS IAM policies, each statement includes: resource (arn), actions (a set of method names on the resource), principal (consuming role), effect (allow/deny) and conditions.

I think it is safe to ignore prinicpal here because the principal is basically the consuming resource (e.g. the cloud.Function).

Let's take an example:

let greet = (bucket: cloud.Bucket, msg: str) ~> {
  bucket.put("greeting.txt", msg);
};

let my_bucket = new cloud.Bucket();
let handler = () ~> {
  let s = "hello, world";
  greet(my_bucket, s);
  assert(my_bucket.get("greeting.txt") == s);

  my_bucket.put("hello.txt", "world");
};

new cloud.Function(handler);

If I work backwards from the ideal desired result here, I expect:

  1. The bucket object inside the inflight will be initialized with
    a runtime client for Bucket with a set() method, bound to
    this specific client.
  2. The IAM security policy of the execution role for this function
    to include a statement that allows an s3:PutObject operation
    to be performed with the key greeting.txt to this bucket.

So if I try to express this through the mental model of a policy, I'd say:

When the wing compiler compiles the greet() function, it generates a policy for it.

bucket:
  - method: put
    args:
      object_key: "greeting.txt"
      body: *
msg:
  - method: to_str

Do we need special treatment for resources or can avoid the distinction between resources and classes. In this case, I am using to_str() to represent a "read" operation from a string. This might be very useful for protecting sensitive information by not allowing certain functions to even read the contents.

Let's say it is possible to access this policy through greet.policy and basically access an query object model.

Alternatively policyof(greet)

assert(greet.policy["bucket"][0]["method"] == "set")
assert(greet.policy["bucket"][0]["args"]["body"] == "*")

Now, when the compiler compiles the handler inflight it generates a policy. This policy treats my_bucket as a captured symbol, but for all intents an purposes it is exactly like a hidden argument in the inflight call.

my_bucket:
  # from greet.policy.bucket
  - method: put
    args:
      object_key: "greeting.txt"
      body: *
      # v2 of wing will know this is 
      # "hello world", so it can be 
      # even be more specific

  # from "handler"
  - method: get
    args: 
     object_key: "greeting.txt"

  # from "handler"
  - method: put
    args: 
     object_key: "hello.txt"
     body: "world"

s:
  # from greet.policy.msg
  - method: to_str

Now, when cloud.Function is defined, the wing compiler
emits the captures stanza by inspecting handler.policy:

new cloud.Function(this, "Function", {
  captures: {
    my_bucket: {
      obj: my_bucket,
      policy: [{
        method: "put",
        args: {
          "object_key": "greeting.txt",
        }
      }, {
        method: "get",
        args: {
          "object_key": "greeting.txt",
        }
      }, {
        method: "put",
        args: {
          "object_key": "hello.txt",
          "body": "world"
        }
      }]
    }
  }
});

Then, bucket.capture() is called under the hood like this during capturing to invert the control and let the bucket decide how it should be captured.

for c in Object.values(props.captures) {
  c.obj.capture(this /* cloud.Function */, c.policy);
}

Now, the capture() method of the bucket can reflect on the policy and render the security policy accordingly as well as the runtime binding requirements.

More thoughts:

  • Can a policy of an inflight include additional environment attributes such as minimal memory/disk consumption or minimal timeout? Feels like a natural place to express this so consumers can act accordingly.

@ekeren
Copy link

ekeren commented Oct 13, 2022

@Chriscbr , regarding

To me, this suggests that there might be value in including capture-like constraints as part of our type system somehow.

When I was brainstorming about this with @yoav-steinberg, I went back to what @3p3r has said in the beginning about anonymous types inflight.

@yoav-steinberg
Copy link
Contributor Author

yoav-steinberg commented Oct 13, 2022

@eladb lets consider the following example and try to see if this fits your proposed data model:

let get_bucket = (primary_bucket: cloud.Bucket, secondary_bucket: cloud.Bucket): cloud.Bucket ~> {
  let in_maintenance = primary_bucket.get("maintenance");
  if in_maintenance == "1" { 
    return secondary_bucket;
  } else {
    return primary_bucket;
  }
};

let west_b = new cloud.Bucket();
let east_b = new cloud.Bucket();

let handler = () ~> {
  let b = get_bucket(west_b, east_b);
  b.put("hello.txt", "world");
};

new cloud.Function(handler);

get_bucket policy:

primary_bucket:
  - method: get
    args:
      object_key: "maintenance"

Note that we don't say anything about secondary_bucket, perhaps we need to add some returns section to the policy?

handler policy:

west_b:
  # from get_bucket.policy.primary_bucket
  - method: get
    args:
      object_key: "maintenance"

  # from "handler" - How can we figure this out!!!??
  - method: put
    args: 
     object_key: "hello.txt"
     body: "world"

east_b:
  # from "handler" - How can we figure this out!!!??
  - method: put
    args: 
     object_key: "hello.txt"
     body: "world"

@eladb
Copy link
Contributor

eladb commented Oct 13, 2022

@yoav-steinberg wrote:

@eladb lets consider the following example and try to see if this fits your proposed data model...

Yes, this won't work without some control-flow analysis or some explicit information from the user.

As you indicated, our current algorithm will synthesize the following policy for handler:

west_b:
  - method: get
    args:
      object_key: "maintenance"

For completeness, here's how this will happen:

The compiler needs to synthesize a requirements policy for handler based on the external inputs it has. In this case it only has two (implicitly captured) external inputs: west_b and east_b. For each input, the compiler will track it's usage in the function and will merge these statements into the handler's policy. In this case, the compiler can see that west_b is passed as primary_bucket to get_bucket so it will include the policy for get_bucket.policy.primary_bucket in the policy of handler for west_b. So far so good. Then, for east_b, the compiler sees that it is passed into get_bucket, but there is no entry for east_b in get_bucket's policy, so there is nothing to merge into the parent policy.

So we are now in a situation where this code will fail (in production) because the handler will try to call put() on one of these buckets and these requirements were not in the policy.

P.S. I am warming up to the term "requirements", as in get_bucket requires <policy-statement(s)>.

@eladb
Copy link
Contributor

eladb commented Oct 13, 2022

let west_b = new cloud.Bucket();
let east_b = new cloud.Bucket();

let foo = (bucket: cloud.Bucket) ~> {
  bucket.put("hello.txt", "world");
}; 

//-------------------
// foo's policy:
//-------------------
// bucket:
//  - method: put
//    args: 
//     object_key: "hello.txt"
//     body: "world"
//-------------------

let handler = () ~> {
  let b = west_b;
  foo(east_b);
  foo(b);
};

new cloud.Function(handler);

Iterate on all inputs (explicit or implicit=captures) and for each input:

  1. Search for all the method calls where this input is a parameter (or this).
  2. Query the method's policy statements for the statements associated with this input.
  3. Merge all statements into a single policy that becomes the handler's policy for this input.

Alternative algorithm:

  1. Find all calls to methods/functions inside the handler that have a policy associated with them.
  2. For each statement in each policy, replace the key with the assigned symbol from the inspected function.

In the case of handler, we have two calls to foo which has the following policy:

east_b:
  - method: put
    args: 
     object_key: "hello.txt"
     body: "world"
b:
  - method: put
    args: 
     object_key: "hello.txt"
     body: "world"

So now, we have a set of statements in the namespace of handler. Now we calculate all the "inputs" of handler (explicit and implicit), and associate each policy statement to the relevant input.

We are left with b as an unmatched resource and the compiler can yell at us and tell us that it needs some way to resolve b.

@Chriscbr
Copy link
Contributor

We are left with b as an unmatched resource and the compiler can yell at us and tell us that it needs some way to resolve b.

If we could reliably calculate the set of reaching definitions for b, then we could automatically match the policy to the correct resource (or set of candidate resources). But this is easier said than done, so we can leave out this kind of analysis out to start.

@eladb
Copy link
Contributor

eladb commented Oct 15, 2022

If we could reliably calculate the set of reaching definitions for b

Definitely something to consider as the compiler evolves, but as we discussed, since we need a way for users to resolve those ambiguities explicitly, doing this type of static analysis is not in the critical path.

@3p3r
Copy link
Contributor

3p3r commented Oct 15, 2022

I have to voice my concern over the direction this model is taking. I understand that the purpose of doing this, is to calculate minimum perms required to make a captured code work in cloud runtime.

But I have concerns about how to scale this. Every cloud has well over 200 resources. We are starting with a few, which are abstractions themselves. And everything seems to be designed around those few abstractions. From simulator to now permissions.

I am not seeing discussions or concerns on how to scale this eventually to hit all resources. It is beginning to feel like design is going from top to bottom instead of bottom to top.

We cannot maintain this polycon and permission model for that many resources without some sort of automation. Example: using iamfast to generate policies for AWS.

@3p3r
Copy link
Contributor

3p3r commented Oct 15, 2022

If all Wing can do is to generate minimum permissions required for its own set of limited polycons, we really haven’t solved anything in the current landscape. We made new problems and solved those new problems ourselves. There is no value proposition in it.

The problem of “how do I generate minimum permissions for polycons” does not exist. The problem is “how do I generate minimum permissions for cdk-tf L1 constructs”. Wing syntax is abstraction of what’s currently out there. We shouldn’t add new problems to it.

@3p3r
Copy link
Contributor

3p3r commented Oct 15, 2022

Another design concern is that: cloud SDK API is now being merged into Constructs API under polycons and their clients. Past a few tech demos, how does this scale? How do I consume the AWS SDK in my Lambda in Wing? If I can’t, then we introduced new problems, so what’s the point? If I can, then this permission model doesn’t make sense. Majority of cloud runtime logic is through access to cloud SDKs. We are abstracting both the SDK and CDK with no discussion on how this scales to cover all clouds and resources again.

@3p3r
Copy link
Contributor

3p3r commented Oct 15, 2022

A solution that comes to my mind that addresses this at a fundamental level is JSII-ifying major cloud SDKs through automation and instrumentation. Through this process we can inject permission metadata into the manifest and have the compiler simply query for those permissions during compilation. That way permissions are not tightly coupled with polycons anymore. Polycons can exist in their own world, permissions can also live on their own. Sounds terribly hard, but it’s not supposed to be easy to solve. That’s the value proposition. Anyone can make abstractions. But not many can address this at infrastructure level.

@ShaiBer
Copy link
Contributor

ShaiBer commented Oct 16, 2022

I get what you're saying @3p3r about solving problems we introduce and not ones users have now and about the scalability of our solution.
But I think that even if we add polycons and thereby add problems that we then solve, it is not a worthless effort because Polycons were added to solve other problems users have, and so if we manage to add them without taking something away from users then the net result is a better experience for them.
Also, what we're doing here is not the end game, it is building a mechanism that will allow us to have a functioning programming language people can use (even though they will need to define policies for captures) and start collect feedback from them on it. We can then take time to develop the more sophisticated solution that will generate minimal permissions automatically (in most cases) and improve the experience further.

@3p3r
Copy link
Contributor

3p3r commented Oct 16, 2022

What problem do Polycons address besides abstraction of multi cloud resources and respective SDK calls for Wing?

Here’s Wing’s original pitch: we made a language that eases the use of cloud, CDK, and constructs. Note that even in this pitch, we added something new, which is the syntax. But that’s okay because we solved an existing problem.

We are designing everything around Polycons and introducing hard dependency on Polycons in everything else:

  • Simulator? Works only with Polycons
  • Captures? Sorry. Only Polycons are supported.
  • Permissions? Guess what. Only Polycons!

What’s Wing doing here again? We’re building a language but are still stuck thinking in JavaScript domain. Emphasis is on making Polycons work both in and out of Wing more than how Wing internals should properly address the existing issue we are trying to solve.

It is not appetizing at all if, as a user, you tell me I need to write everything in a certain way (Polycon) inside a new syntax. I now have two problems to deal with.

This is what top to bottom design is. It’s not engineering. We’re just cruising to get a tech demo working. We’re designing ourselves into a corner so when the time comes to scale, non of this is usable.

Wing internal should be devoid of Polycons entirely. It should focus on CDK TF L1 constructs, what it actually compiles to. Not the abstractions we introduced.

@3p3r
Copy link
Contributor

3p3r commented Oct 16, 2022

Here's a simplified version of what I am asking, if it helps conveying my point across:

What's the endgame vision with this Polycon design, and how does it hit every resource in CDK-TF without manual intervention? How does it help solving the original problem with CDK?

Forget about the product and marketing aspects, just look at it from pure engineering POV. If we can't answer this as a team right now, we won't be able to answer it 6 months down the road either when we have accumulated so much code.

@Chriscbr
Copy link
Contributor

@3p3r I think you raise some valid concerns. To share one perspective, right now I think we're just using polycons as an "extension" of constructs that adds dependency injection to support multi-cloud applications. Wing's capture / resource binding mechanic is an independent feature, so I don't think it should require using polycons.

That said, I do feel like there's good reason for having some kind of policy abstraction to deliver on the "all clouds are equal" promise. For example, let's take this hypothetical code snippet where we're trying to use Wing's inflight capabilities with ordinary (preflight-only) constructs:

bring s3 from "@cdktf/provider-aws";
bring { S3Client, CopyObjectCommand } from "@aws-sdk/client-s3";

// ordinary constructs, not polycons
let srcBucket = s3.S3Bucket();
let targetBucket = s3.S3Bucket();

let handler = () ~> {
  let client = new S3Client();
  await client.send(new CopyObjectCommand({
    // let's assume the arn's aren't tokens for now
    Bucket: targetBucket.arn,
    CopySource: "${srcBucket.arn}/hello.txt",
    Key: "hello.txt",
  }));
};
let handlerFn = cloud.Function(handler);

My hope is something like this could be supported in Wing! But, because SDKs for resources are not standardized across clouds or providers [1], we can't reliably infer a list of operations to later be associated with the function signature.

We could dispatch to an external library like iamfast to produce a list of needed IAM permissions, but this would be placing a significant part of our value on a cloud specific tool. I think we should aim for a more cloud-agnostic architecture, even if it requires a little more work from library authors.

Going back to the example above, the only values that get captured from outside are resource ARNs, so the user would need to manually add permissions for this code to work:

handlerFn.addPolicyStatements([{
  	effect: ...
  	action: ...
  	resource: ...
}]);

This is all good and fine. But how do we provide the "automatic minimal policies" experience in userland - i.e. for resources outside of the Wing SDK?

One option could be to let the user model create a resource with an inflight method:

// extending an ordinary construct (this is still not a polycon)
resource MyBucket extends s3.S3Bucket {
  ~copyFrom(source: s3.S3Bucket, key: str) {
    let client = new S3Client();
    await client.send(new CopyObjectCommand({
      Bucket: this.arn,
      CopySource: "${source.arn}/{key}",
      Key: "{key}",
    }));
  }
}

Then when handler is updated to use copyFrom instead:

let handler = () ~> {
  await targetBucket.copyFrom(srcBucket, "hello.txt");
};

... the compiler can see "ok, we are capturing an inflight method named "copyFrom" on a Wing class, so I can annotate this handler's signature with a policy associating copyFrom with srcBucket and targetBucket."

The logic for automatically turning policies into actual resource permissions must also be part of a construct's API, but it's a small amount of boilerplate:

resource MyBucket extends s3.S3Bucket {
  ...

  // this API still doesn't feel right to me
  bind(compute: IInflightRunner, policy: cloud.Policy) {
    if !(compute instanceof aws.Function) {
      throw "unsupported - only supporting being captured by AWS lambdas for now";
    }
    if policy.methods.includes("copyFrom") {
      compute.addPolicyStatements(...);
    }
  }
}

A lot of this code is imagined up, so I'm curious what parts resonate and which don't. 🙂

[1] For example, some terraform resources might not have dedicated JavaScript SDKs, and could instead require you to hit HTTP endpoints perhaps?

@3p3r
Copy link
Contributor

3p3r commented Oct 16, 2022

Your first example actually perfectly explains why I am confused.

Going back to the example above, the only values that get captured from outside are resource ARNs, so the user would need to manually add permissions for this code to work.

But it does contain everything compiler needs to know about perms though. Compiler sees a "send" operation is being called on a client of type "S3Client" which is uploading to a construct of type "S3Bucket" with a known ARN. That's all the pieces needed to generate permissions right there. That's where my disconnect comes in. I see all the data in your basic example without Polycons and I am wondering why there needs to be another layer of abstractions on top of that, when we can just see underneath the abstraction with the compiler as well.

BTW I am not saying we should use iamfast. I am saying we should create software like iamfast that does this for all other clouds. We can take inspirations from iamfast, but I am not particularly fond of iamfast's design either.

mergify bot pushed a commit that referenced this issue Mar 15, 2023
Following up on #1682 to identify cases where we cannot infer which operations are performed on a captured resource.

See positive/negative tests for examples.

Rewrite the algorithm which analyzes the expressions captured by inflight methods so that it is able to identify more cases and emit errors when captures cannot be qualified (i.e. a resource is captured but we cannot determine which operations are performed on it without static analysis).

For each method, we identify all expressions that start with `this.xxx` and break them down into parts (using nested references). Then, we traverse the list of parts and split the expression into *preflight* and *inflight*. The preflight part is what we are capturing and the first inflight component qualifies which operations are performed on the captured object.

Reorganized capture tests into `resource_captures` (both under valid and invalid).

This does not address #76 but it explicitly identifies these cases. We will follow up at some point with a way to allow users to explicitly qualify the reference.

*By submitting this pull request, I confirm that my contribution is made under the terms of the [Monada Contribution License](https://docs.winglang.io/terms-and-policies/contribution-license.html)*.
@staycoolcall911 staycoolcall911 moved this from 🏗 In progress to 🤝 Backlog - handoff to owners in Wing Mar 21, 2023
@staycoolcall911 staycoolcall911 moved this from In Progress to Todo - p2 in Wing Language Roadmap Mar 21, 2023
@staycoolcall911
Copy link
Contributor

Following @eladb's #1449 - we decided to lower the priority for explicit capture permissions to p2

@eladb eladb removed their assignment Apr 11, 2023
skyrpex pushed a commit that referenced this issue Jun 21, 2023
@eladb eladb changed the title Mechanism for defining explicit capture permissions Mechanism for explicitly qualifying lifting Jul 4, 2023
@staycoolcall911 staycoolcall911 moved this from Todo - p2 to Todo - out of scope for beta in Wing Language Roadmap Aug 17, 2023
@staycoolcall911 staycoolcall911 moved this from Todo - out of scope for beta to Todo - p2 in Wing Language Roadmap Aug 17, 2023
@staycoolcall911 staycoolcall911 moved this from Todo - p2 to Todo - p1 in Wing Language Roadmap Aug 17, 2023
@staycoolcall911 staycoolcall911 added this to the KubeCon23 milestone Sep 14, 2023
@mergify mergify bot closed this as completed in #5935 Mar 19, 2024
mergify bot pushed a commit that referenced this issue Mar 19, 2024
…ts (#5935)

Fixes: #76
Creates a `lift` builtin function that can be used in inflight code to explicitly add lift qualifications to a method:
```wing
bring cloud;

let bucket = new cloud.Bucket();
bucket.addObject("k", "value");

let some_ops = ["put", "list"]; // We can define a list of ops in preflight code to be used when explicitly qualifying a lift

class Foo {
  pub inflight mehtod() {
    lift(bucket, some_ops); // Explicitly add some permissions to `bucket` using a preflight expression
    lift(bucket, ["delete"]); // Add more permissions to bucket using a literal
    log(bucket.get("k")); // Good old implicit qualification adds `get` permissions
    let b = bucket; // We can now use an inflight variable `b` to reference a preflight object `bucket`
    b.put("k2", "value2"); // We don't get a compiler error here, because explicit lifts are being used in the method disabling compiler qualification errors
    for k in b.list() { // `list` works on `bucket` because of explicit qualification and `b` references `bucket`
      log(k);
    }
    b.delete("k2"); // `delete` also works because of explicit qualification
    assert(bucket.tryGet("k2") == nil); `yay!`
  }
}

let foo = new Foo();

test "a test" {
  foo.mehtod();
}
```

## Checklist

- [x] Title matches [Winglang's style guide](https://www.winglang.io/contributing/start-here/pull_requests#how-are-pull-request-titles-formatted)
- [x] Description explains motivation and solution
- [x] Tests added (always)
- [x] Docs updated (only required for features)
- [ ] Added `pr/e2e-full` label if this feature requires end-to-end testing

*By submitting this pull request, I confirm that my contribution is made under the terms of the [Wing Cloud Contribution License](https://github.com/winglang/wing/blob/main/CONTRIBUTION_LICENSE.md)*.
@github-project-automation github-project-automation bot moved this from Todo - p1 to Done in Wing Language Roadmap Mar 19, 2024
@github-project-automation github-project-automation bot moved this from 🤝 Backlog - handoff to owners to ✅ Done in Wing Mar 19, 2024
@monadabot
Copy link
Contributor

Congrats! 🚀 This was released in Wing 0.61.17.

mergify bot pushed a commit that referenced this issue Apr 24, 2024
## Checklist

[rendered version](https://github.com/winglang/wing/blob/yoav/rfc-explicit_lift_qualification/docs/contributing/999-rfcs/2024-03-14-explicit-lift-qualification.md)

Related to #76, #5935


- [x] Title matches [Winglang's style guide](https://www.winglang.io/contributing/start-here/pull_requests#how-are-pull-request-titles-formatted)
- [x] Description explains motivation and solution
- [ ] Tests added (always)
- [ ] Docs updated (only required for features)
- [ ] Added `pr/e2e-full` label if this feature requires end-to-end testing

*By submitting this pull request, I confirm that my contribution is made under the terms of the [Wing Cloud Contribution License](https://github.com/winglang/wing/blob/main/CONTRIBUTION_LICENSE.md)*.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🛠️ compiler Compiler 📐 language-design Language architecture
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

8 participants