Skip to content

Commit

Permalink
Let users specify unique function keys using delay.MustRegister (#268)
Browse files Browse the repository at this point in the history
  • Loading branch information
KaylaNguyen authored Nov 5, 2021
1 parent 6d50fa8 commit ab4e2d3
Show file tree
Hide file tree
Showing 2 changed files with 316 additions and 244 deletions.
140 changes: 75 additions & 65 deletions v2/delay/delay.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,52 +3,29 @@
// license that can be found in the LICENSE file.

/*
Package delay provides a way to execute code outside the scope of a
user request by using the taskqueue API.
To declare a function that may be executed later, call Func
in a top-level assignment context, passing it an arbitrary string key
and a function whose first argument is of type context.Context.
The key is used to look up the function so it can be called later.
var laterFunc = delay.Func("key", myFunc)
It is also possible to use a function literal.
var laterFunc = delay.Func("key", func(c context.Context, x string) {
// ...
})
To call a function, invoke its Call method.
laterFunc.Call(c, "something")
A function may be called any number of times. If the function has any
return arguments, and the last one is of type error, the function may
return a non-nil error to signal that the function should be retried.
The arguments to functions may be of any type that is encodable by the gob
package. If an argument is of interface type, it is the client's responsibility
to register with the gob package whatever concrete type may be passed for that
argument; see http://golang.org/pkg/gob/#Register for details.
Any errors during initialization or execution of a function will be
logged to the application logs. Error logs that occur during initialization will
be associated with the request that invoked the Call method.
The state of a function invocation that has not yet successfully
executed is preserved by combining the file name in which it is declared
with the string key that was passed to the Func function. Updating an app
with pending function invocations should safe as long as the relevant
functions have the (filename, key) combination preserved. The filename is
parsed according to these rules:
* Paths in package main are shortened to just the file name (github.com/foo/foo.go -> foo.go)
* Paths are stripped to just package paths (/go/src/github.com/foo/bar.go -> github.com/foo/bar.go)
* Module versions are stripped (/go/pkg/mod/github.com/foo/[email protected]/baz.go -> github.com/foo/bar/baz.go)
There is some inherent risk of pending function invocations being lost during
an update that contains large changes. For example, switching from using GOPATH
to go.mod is a large change that may inadvertently cause file paths to change.
The delay package uses the Task Queue API to create tasks that call the
reserved application path "/_ah/queue/go/delay".
This path must not be marked as "login: required" in app.yaml;
it must be marked as "login: admin" or have no access restriction.
Package delay provides a way to execute code outside of the scope of
a user request by using the Task Queue API.
To use a deferred function, you must register the function to be
deferred as a top-level var. For example,
```
var laterFunc = delay.MustRegister("key", myFunc)
func myFunc(ctx context.Context, a, b string) {...}
```
You can also inline with a function literal:
```
var laterFunc = delay.MustRegister("key", func(ctx context.Context, a, b string) {...})
```
In the above example, "key" is a logical name for the function.
The key needs to be globally unique across your entire application.
To invoke the function in a deferred fashion, call the top-level item:
```
laterFunc(ctx, "aaa", "bbb")
```
This will queue a task and return quickly; the function will be actually
run in a new request. The delay package uses the Task Queue API to create
tasks that call the reserved application path "/_ah/queue/go/delay".
This path may only be marked as "login: admin" or have no access
restriction; it will fail if marked as "login: required".
*/
package delay // import "google.golang.org/appengine/v2/delay"

Expand Down Expand Up @@ -151,34 +128,72 @@ func fileKey(file string) (string, error) {
return modVersionPat.ReplaceAllString(file, ""), nil
}

// Func declares a new Function. The second argument must be a function with a
// first argument of type context.Context.
// This function must be called at program initialization time. That means it
// must be called in a global variable declaration or from an init function.
// This restriction is necessary because the instance that delays a function
// call may not be the one that executes it. Only the code executed at program
// initialization time is guaranteed to have been run by an instance before it
// receives a request.
// Func declares a new function that can be called in a deferred fashion.
// The second argument i must be a function with the first argument of
// type context.Context.
// To make the key globally unique, the SDK code will combine "key" with
// the filename of the file in which myFunc is defined
// (e.g., /some/path/myfile.go). This is convenient, but can lead to
// failed deferred tasks if you refactor your code, or change from
// GOPATH to go.mod, and then re-deploy with in-flight deferred tasks.
//
// This function Func must be called in a global scope to properly
// register the function with the framework.
//
// Deprecated: Use MustRegister instead.
func Func(key string, i interface{}) *Function {
f := &Function{fv: reflect.ValueOf(i)}

// Derive unique, somewhat stable key for this func.
_, file, _, _ := runtime.Caller(1)
fk, err := fileKey(file)
if err != nil {
// Not fatal, but log the error
stdlog.Printf("delay: %v", err)
}
f.key = fk + ":" + key
key = fk + ":" + key
f, err := registerFunction(key, i)
if err != nil {
return f
}
if old := funcs[f.key]; old != nil {
old.err = fmt.Errorf("multiple functions registered for %s", key)
}
funcs[f.key] = f
return f
}

// MustRegister declares a new function that can be called in a deferred fashion.
// The second argument i must be a function with the first argument of
// type context.Context.
// MustRegister requires the key to be globally unique.
//
// This function MustRegister must be called in a global scope to properly
// register the function with the framework.
// See the package notes above for more details.
func MustRegister(key string, i interface{}) *Function {
f, err := registerFunction(key, i)
if err != nil {
panic(err)
}

if old := funcs[f.key]; old != nil {
panic(fmt.Errorf("multiple functions registered for %q", key))
}
funcs[f.key] = f
return f
}

func registerFunction(key string, i interface{}) (*Function, error) {
f := &Function{fv: reflect.ValueOf(i)}
f.key = key

t := f.fv.Type()
if t.Kind() != reflect.Func {
f.err = errors.New("not a function")
return f
return f, f.err
}
if t.NumIn() == 0 || !isContext(t.In(0)) {
f.err = errFirstArg
return f
return f, errFirstArg
}

// Register the function's arguments with the gob package.
Expand All @@ -194,12 +209,7 @@ func Func(key string, i interface{}) *Function {
}
gob.Register(reflect.Zero(t.In(i)).Interface())
}

if old := funcs[f.key]; old != nil {
old.err = fmt.Errorf("multiple functions registered for %s in %s", key, file)
}
funcs[f.key] = f
return f
return f, nil
}

type invocation struct {
Expand Down
Loading

0 comments on commit ab4e2d3

Please sign in to comment.